IMAGINE Subpixel Classifier™
Quantitative Evaluation
Classification Performance & Accuracy

In a controlled field experiment, we worked together with a partner to create a multispectral data set to quantitatively evaluate the classification accuracy of IMAGINE Subpixel Classifier and to compare its performance to that of a traditional classifier.

In the experiment, IMAGINE Subpixel Classifier correctly classified 93% or 388 of 416 material-containing pixels and significantly outperformed the traditional classifier, correctly detecting 190% more of the material-containing pixels, with a lower number of errors.

Only IMAGINE Subpixel Classifier reported the amount of material in each pixel classified.


Figure 1. IMAGINE Subpixel Classifier detected 190% more of the material-containing pixels than did the Maximum Likelihood Classifier, with fewer errors.

Set Up
We laid out 188 panels in a flat grid pattern on a field of grass. The panels ranged in size from 100% to 5% of an image pixel. Each panel could fall within as many as four image pixels; we deliberately chose not to align the image grid with the panel grid.

We imaged the panels with NASA’s Calibrated Airborne Multispectral Scanner (CAMS) sensor to simulate the spectral bands of Landsat Thematic Mapper (TM). We flew the sensor on a Lear jet and used the six CAMS spectral bands that are identical in position and width to the six reflective TM bands.

By using CAMS, we produced 2.5 m2 pixels, permitting us to more easily construct and deploy pixel-sized and smaller panels. Thus, we could (1) develop and test a statistically large set of spectrally identical subpixel targets, and (2) produce a data set where known subpixel occurrences of the material could be located and the relative amount of material in the pixels determined for accuracy assessment.

Results

IMAGINE Subpixel Classifier correctly classified 87% or 164 of the 188 panels and misclassified only five pixels. The traditional Maximum Likelihood Classifier correctly classified 48% or 91 of the 188 panels and misclassified nine pixels.

When using six TM bands, we expect IMAGINE Subpixel Classifier to enable us to classify materials occupying more than 20% of a pixel. IMAGINE Subpixel Classifier detected 100% of all panels that were larger than 20% the size of a pixel. IMAGINE Subpixel Classifier classified all panels that were larger than 10% the size of a pixel with an accuracy of 90% or higher. IMAGINE Subpixel Classifier even successfully classified 25% of the panels that were only 5% the size of a pixel.

In contrast, the traditional classifier detected 100% of only those panels that occupied 100% of a pixel. The detection level dropped rapidly--only 65% of the panels 50% the size of a pixel and 5% of the panels 10% the size of a pixel.

At the pixel level, IMAGINE Subpixel Classifier correctly classified 93% or 388 of the 416 pixels that contained panel, whereas the traditional classifier correctly classified only 32% of these pixels.

Material Pixel Fraction (MPF)
IMAGINE Subpixel Classifier has the unique ability to estimate and report the amount of the material of interest contained in each pixel. The fraction of material of interest in each pixel is called the Material Pixel Fraction.

IMAGINE Subpixel Classifier groups all of the pixels in an image as either containing the material or not, and groups all of the pixels that contain the material into MPF classes based on how much of the material each contains. Each MPF class is assigned a color and, together with the number of pixels in each MPF class, is displayed as a legend, as shown in Figure 2.
Accuracy Assessment

In the experiment, we next calculated the MPF for each detected pixel and compared the results with IMAGINE Subpixel Classifier reported MPF. The very strong correlation (correlation coefficient r = .90) between manual measurement of MPF and IMAGINE Subpixel Classifier MPF accuracy is representative of the accuracy observed in other applications of IMAGINE Subpixel Classifier.

We assumed that the fraction of the material of interest in a pixel could be independently estimated from the brightness of the pixel in one of the CAMS image planes. The panels were bright relative to the grassy background, particularly at the visible wavelengths. And since the panels were opaque and did not transmit any background radiance through them, we assumed a linear mixing model.

In particular, the pixel digital number (DN) value in an image plane should range from the signature DN for pixels containing 100% material of interest to a background-characteristic DN value for pixels containing less than a detectable fraction of panel material. DN values between these extremes should be generally linearly proportional to the fraction of material of interest in the pixel.

We selected CAMS Band 4 (0.63-0.69m m), which covers the same wavelength range as Landsat TM Band 3, for our analysis. The panel signature had a Band 3 DN value of 97.61, corresponding to an MPF of 100%. A representative sample of twenty grassy background pixels had a mean DN value of 47.45 (standard deviation = 4.99), corresponding to an MPF of 10% or less.

We used the regression, MPF (percent) = 1.79 x (DN - 47.45) + 10 (1), to independently determine the MPF for each pixel in the image that was identified by the IMAGINE Subpixel Classifier as containing panel material. We then compared the independently calculated MPF to the MPF class reported by the Subpixel Classifier to assess the accuracy of the reported fraction.

Conclusion
When classifying a specific material, IMAGINE Subpixel Classifier has distinct advantages over traditional classifiers. As the field experiment demonstrates, IMAGINE Subpixel Classifier is superior in its ability to classify a specific material and unique in its ability to quantify the material at the subpixel level.

Although this experiment involved the classification of material high in contrast to the background, we have seen similar levels of contrasting performance in non-controlled environments to which IMAGINE Subpixel Classifier has been applied.

Summary
Both traditional and IMAGINE Subpixel Classifier are essential image processing tools, whether used separately or in conjunction with one another.

You should consider using a traditional classifier when your material of interest is composed of a spectrally varied range of materials that must be considered as a single classification unit. An example would be a "forest" class that would contain the spectral diversity of different tree species and conditions.

You should consider using IMAGINE Subpixel Classifier when you are looking for a specific material that is spectrally distinct from other materials in complex environments and/or smaller than the sensor’s spatial resolution. For example, you would use IMAGINE Subpixel Classifier to classify an individual tree species or condition within the forest, but not the "forest."

Look to IMAGINE Subpixel Classifier when you wish to:

· Classify objects smaller than the spatial resolution of the sensor, as little
as 20% of a pixel. Also useful with high spatial resolution sensors such as IKONOS
and QuickBird

· Discriminate specific materials in mixed pixels

· Report the fraction of material present in each pixel classified