1. Introduction

The Western United States has seen an increase in pinyon (Pinus spp.) and juniper (Juniperus spp.) since European settlement in the mid 1800's ^[1,2,3,4,5], becoming the third largest vegetation type in North America ^[6]. Pinyon/juniper has expanded its range from 3 million ha to 30 million ha in the last 150 years since westward expansion began to accelerate ^[7], due to livestock grazing and fire suppression ^{[8,9,10,11,12]}. Pinyon/juniper dominated landscapes have been found to impact water and nutrient cycles ^[13,14], plant community structure and composition ^{[15,16,17,18,19]}, fire regimes ^[7,20], habitat and forage availability for many wildlife species ^[15,21] and overall biodiversity ^[8,22,23]. As a result, removal of pinyon/juniper has become a priority in many areas to improve ranching opportunities through understory development and increase ecosystem stability ^{[8,24,25,26,27,28]}. Western juniper (Juniperus occidentalis) is a dominant tree species found within juniper ecosystems of the western Great Basin. Western juniper woodlands share similar structure and function to that of other pinyon/juniper forests of western North America and are the subject of this study.

Advances in remote sensing technology hold potential to facilitate monitoring of rangeland trends at a broad spatial scale, as remote sensing methods are typically less costly and more efficient for larger areas ^[29,30]. This efficiency results from reduced costs associated with labor, increased accessibility to remote areas, and saved time with computer processing (compared to on-the-ground sampling) ^[30]. Many land managers feel that traditional methods of point sampling on the ground as a means of obtaining forest metrics is outdated and are likely to be replaced by more modern, cost effective, and accurate methods that include high and very high resolution remotely sensed imagery ^[30,31,32]. Remote sensing can be valuable in examining long term vegetation trends (i.e., plant succession) ^[6,25], climate change ^[6,26], and as a predictive tool for rangeland management and monitoring ^[6,8]. An advantage to using remote sensing is the existence of panchromatic fine-scale satellite data that pre-dates current multi-spectral sensors, and aerial imagery that is available in some locations where satellite imagery is not available ^[6], allowing for long-term vegetation change detection.

Aerial imagery has been utilized to examine pinyon-juniper dominated landscapes and has been found effective to extract canopy cover and encroachment extent ^[5]. It is important to test and properly utilize emerging remote sensing and image classification technologies as they apply to rangeland management in order to verify their suitability to such applications ^[33]. Hulet et al. ^[34] found that object-based image analyses were effective in classifying vegetation structure. The objective of this study is to compare the accuracy of an object-based classification (using Feature Analyst^TM for ArcGIS 10.2®) with a supervised classification (using ENVI Feature Extraction®). Because of the discrepancy between the measurement of total juniper canopy cover versus canopy cover change over time, our goal was to identify the most effective method for detecting the accuracy of each approach. While some land managers are tasked with measuring total canopy cover, others want to determine percent change in order to evaluate range trends. Therefore, methods are needed that can be used to identify classification accuracy considering both applications. Additionally, we intend to demonstrate the level of accuracy possible in measuring change detection using remote sensing with panchromatic imagery.

---

2. Methods

---

2.1. Study site description

The study area is located in southeastern Oregon, U.S.A on the Roaring Springs Ranch (centroid at 118°45'15.435" W, 42°33'14.25" N). This region has experienced significant encroachment and infill by western juniper over the past century due primarily to change in grazing and fire occurrence and management. Unlike much of western landscapes, this area lacks the co-dominance with pinyon species. Juniper removal has occurred within the study area since 1995, which covers approximately 28.5 hectares of juniper-dominated woodland and consists of western juniper, bare rock, bare soil, and a diversity of shrub and herbaceous species. Average annual precipitation is approximately 320 mm. Dominant vegetation includes a diversity of native shrubs (Artemesia tridentata, Purshia tridentata), grasses, (Pseudoregeneria spicata, Festuca idahoensis, Poa secunda, Achnatherum thurberianum) and forbs (Balsamorhiza sagittata, Crepis acuminata, Lupinus spp) ^[23]. Our research showed no evidence of fires in the vicinity of our study area during or between the years of data acquisition.

Aerial imagery acquired for analyzing changes in juniper cover of time consisted of 1995 panchromatic, 1-meter Digital Orthophoto Quads (DOQ) and a 2011 color composite (4-band), 1-meter resolution, NAIP (National Agriculture Imagery Program, Orthoimagery) image produced by the United States Geological Survey (USGS). Both 1995 and 2011 imagery were chosen to determine the effects of juniper removal treatments that have occurred over that time period. The extent of each downloaded image ranged across the entire state of Oregon but was then extracted to cover only the study area. This reduced both file size and processing time ^[29]. There was no detectable cloud cover in the imagery used in this study. Based on field surveys, there were no detectible impacts to juniper trees resulting from fire, cavitation, defoliation, drought, or seasonal variability.

NAIP imagery is acquired on a 5 year cycle (since 2003) or a 3 year cycle (since 2008). Imagery is acquired by aerial photography during the growing season as a way to measure agricultural growth, but is useful in rangeland management applications as well ^[35]. It is useful because of its spatial extent, low cost, and relatively high spatial resolution (1 meter). In addition, it is acquired in the same season, reducing the need to consider seasonal changes between images. However, our Oregon imagery provided some spectral discrepancies that require noting. First, 4-band (RGB, NIR) imagery for Oregon was not available until 2009, requiring use of lesser spectral resolution for the 1995 imagery. Secondly, part of our image experienced a frame variation, causing differences in panchromatic pixel values in our 1995 image along a straight line. This was corrected by separating the image at the line of the frame change and running the analysis twice.

---

2.2. Object-based classification

In an object-based classification, pixels in the remotely sensed image undergo a pre-classification categorization in which like-pixels are categorized together into an "object", which are then classified according to the specified training sites ^[10]. Feature Analyst^TM for ArcGIS 10.2® was used to classify both the 1995 panchromatic and 2011 4-band images. Training sites were drawn by hand to outline individual juniper trees, which were determined by their shape, size, and location. Although somewhat objective, training sites in western juniper landscapes are best constructed using a heads-up method and therefore require a certain amount of human judgement (36). Through trial and error we determined the ideal training features by experimenting with the minimum pixel aggregation, resampling factor, and kernel size. We then re-ran the analysis after removing incorrectly classified features. This was repeated until manual inspection yielded a classification that we felt had high accuracy and subsequent attempts at adding and correcting training features provided little to no change. The results were then smoothed by removing any features smaller than 9 pixels (Figure 1).

---

2.3. Pixel-based classification

A pixel-based classification is much simpler than an object-based classification and selects a class for each pixel by running a pixel-by-pixel analysis and comparing the spectral value of each value to the average of those of the pixels within the training sites ^[10]. A supervised pixel-based classification was performed on both the 1995 panchromatic image and the 2011 4-band image using ENVI® 5.2. Training sites were selected to produce high accuracy classification results determined by manual inspection and classified into two classes: (1) juniper and (2) bare ground/not juniper. Results were smoothed using a majority filter with a kernal size of 3 and a minimum aggregate of 9 pixels for each image (Figure 1).

---

2.4. Accuracy assessment

An accuracy assessment was performed using 100 random points (generated by ArcMap) and a Kappa Statistic was calculated comparing the results to the original image. Each of the 100 points were assigned "juniper" or "not juniper" in the 1995 image, the 2011 image, the 1995 classification file, and the 2011 classification file based on the individual pixel that the point was located in. In many cases, comparing the 1995 panchromatic image to the 2011 4-band image proved useful in determining which pixels were juniper (as opposed to rock, riparian areas, or aspen, which are more difficult to distinguish in panchromatic images). The assigned class (juniper vs not juniper) for the classification files were then compared to the assigned class of the original image and determined to be a "hit" or a "miss", which was then used in our calculations of user's accuracy, producer's accuracy, and kappa statistic. In addition, we calculated the area of the classified features using ArcMap 10.3 and determined percentage of the total study area that was classified into the respective categories.

---

3. Results

Pixel-based classification using ENVI detected juniper at a systematically higher rate than the object-based classification using Feature Analyst for both the 1995 and 2011 imagery, with total estimates of tree canopy cover area being nearly four-times larger using the pixel-based classification (Table 1). Strictly comparing total canopy cover area with the study area showed approximately a 76% decrease in juniper cover from 1995-2011 (Table 1).

Our accuracy assessment showed near 100% accuracy for overall accuracy (dividing the total correct by the total number), producer's accuracy (error of omission), user's accuracy (error of commission), and kappa statistic (measurement for how effective the classification method is compared to a random classification) for the 2011 imagery (Table 2, ^[37]). 1995 imagery generally showed higher accuracy for detecting bare ground (94% producer's accuracy vs 80% producer's accuracy using pixel-based classification, 96% user's accuracy vs 70% user's accuracy using pixel-based classification, 100% vs 40% producer's accuracy using object-based classification), except for the case of the user's accuracy using the object-based classification (100% detecting juniper and 90% detecting bare ground, Table 2).

---

4. Discussion

Both methods of classification proved to be reasonably effective at detecting western juniper, especially at low densities as in our 2011 image (Table 2). The high kappa statistics, categorized as demonstrating "substantial agreement" for the 1995 classification and "almost perfect agreement" in the 2011 classification ^[38], indicate high overall accuracy for both classification techniques with both images. However, it should be noted that our results were possibly inflated due to the extremely low density of trees in the 2011 image. The results are atypical of large study areas, but more representative of smaller study areas such as ours where lower densities of trees are more common. Additionally, the low resolution images may have resulted in the misidentification of smaller trees not detectable in the imagery from either or both years.

According to our results, Feature Analyst would be better suited for extracting more accurate canopy cover metrics because of the lack of over-classification of pixels seen with the pixel-based classification (Table 1). Similar research suggests that classifications based on image segmentation (such as identifying individual juniper trees) is more accurately achieved through object-based classification, especially as pixel size continues to decrease ^[39]. Although this is not a novel concept, it does demonstrate its usefulness in measuring juniper cover in western rangelands. This is especially useful in analyzing juxtaposition of patchiness and its effects on hydrologic functions, nutrient cycling, and habitat availability ^[37,40,41]. However, the overall consistency of over-classification of juniper in the pixel-based supervised classification makes it suitable for monitoring changes over time as an index, evidenced by the fact that both methods detected approximately a 76% decrease in juniper cover (Table 1). Even occasional inaccuracies in remotely sensed data is typically preferable to ground measurements that can be both expensive and time consuming ^[42].

Increased heterogeneity in the landscape creates difficulties in both classification methods. Our study area consisted of juniper, grasses, wet and dry bare ground, and exposed rock. Classifications made for both images were less effective at differentiating juniper from bare rock. One major weakness with the panchromatic image is its inability to differentiate different tree species ^[6,43]. It was useful in our accuracy assessment to compare the 1995 panchromatic image with the 2011 4-band image and publicly available high-resolution imagery to aid in determining which pixels contained juniper. Manual correction will need to be performed on any classification that is performed in less than ideal areas, potentially increasing both time and labor costs.

Using archived NAIP remotely sensed imagery is effective at determining change over time; however, its accuracy is difficult to verify as obtaining ground truth points is dependent on recorded information that may not be available. One limitation to our method is that accuracy was determined using the same imagery used in the classification, which is less accurate than having the ability to use ground truth points ^[44,45]. Due to this difficulty, it is impossible to precisely determine sources and extent of error. However, our data reinforce the fact that historical NAIP imagery can be used effectively if no other ancillary data is available.

---

5. Conclusion

Our results indicate that object-based classification is preferable to pixel-based classification for monitoring encroachment of western juniper. However, both methods can be used to document range trend statistics over time in a management setting ^[41], taking into account the cost of the software and skill level of the user ^[29].

Ancillary data used in connection with ground truth points provide valuable information of ground-truth characteristics of a remotely sensed image. In an ideal situation, ground truth data would be provided in the form of ground vegetation measurements at the time of the acquisition of the remotely sensed image. However, this data was unavailable in this case because of the backward-looking nature of the study, as there were no ground truth points available for 1995 and 2011 and no method for recreating them, a common issue in modern rangeland trend monitoring using remote sensing. This forced us to compare the classification results to the classified image, giving us an accuracy of the classification, but not necessarily the accuracy of the classification to the ground truth. Because of potential inaccuracies in performing an accuracy assessment without ground truth points, we advise that land managers create and document consistent ground truth points for use in an accuracy assessment to be used each year they perform an image-based classification.

---

Acknowledgements

We express gratitude to the Roaring Springs Ranch for allowing the use of the ranch to conduct this study and to fund this project. We appreciate the use of computers and software of the Geospatial Habitat Analysis Lab at Brigham Young University. We also thank the reviewers of this manuscript for their thoughtful comments and suggestions.

---

Conflict of interest

All authors declare no conflict of interest in this paper.

---