## Outlier modeling for spectral data reduction |

JOSA A, Vol. 31, Issue 7, pp. 1445-1452 (2014)

http://dx.doi.org/10.1364/JOSAA.31.001445

Enhanced HTML Acrobat PDF (862 KB)

### Abstract

The spectra in spectral reflectance datasets tend to be quite correlated and therefore they can be represented more compactly using standard techniques such as principal components analysis (PCA) as part of a lossy compression strategy. However, the presence of outlier spectra can often increase the overall error of the reconstructed spectra. This paper introduces a new outlier modeling (OM) method that detects, clusters, and separately models outliers with their own set of basis vectors. Outliers are defined in terms of the robust Mahalanobis distance using the fast minimum covariance determinant algorithm as a robust estimator of the multivariate mean and covariance from which it is computed. After removing the outliers from the main dataset, the performance of PCA on the remaining data improves significantly; however, since outlier spectra are a part of the image, they cannot simply be ignored. The solution is to cluster the outliers into a small number of clusters and then model each cluster separately using its own cluster-specific PCA-derived bases. Tests show that OM leads to lower spectral reconstruction errors of reflectance spectra in terms of both normalized RMS and goodness of fit.

© 2014 Optical Society of America

**OCIS Codes**

(300.6170) Spectroscopy : Spectra

(110.4234) Imaging systems : Multispectral and hyperspectral imaging

(110.3010) Imaging systems : Image reconstruction techniques

**ToC Category:**

Imaging Systems

**History**

Original Manuscript: January 24, 2014

Revised Manuscript: April 1, 2014

Manuscript Accepted: April 23, 2014

Published: June 12, 2014

**Citation**

Farnaz Agahian and Brian Funt, "Outlier modeling for spectral data reduction," J. Opt. Soc. Am. A **31**, 1445-1452 (2014)

http://www.opticsinfobase.org/josaa/abstract.cfm?URI=josaa-31-7-1445

Sort: Year | Journal | Reset

### References

- F. H. Imai, M. R. Rosen, and R. S. Berns, “Multispectral imaging of van Gogh’s self portrait at the National Gallery of Art,” in Proc. PICS: Image Processing, Image Quality, Image Capture Systems Conference (IS&T) (2001), pp. 185–189.
- R. S. Berns, “The science of digitizing paintings for color-accurate image archives: a review,” J. Imaging Sci. Technol. 4, 305–325 (2001).
- Y. Zhao, L. A. Taplin, M. Nezamabadi, and R. S. Berns, “Using matrix R method in the multispectral image archives,” in Proceedings of the 10th Congress of the International Colour Association (AIC) (2005), pp. 469–472.
- H. Maitre, F. Schmitt, J. Crettez, Y. Wu, and J. Y. Hardeberg, “Spectrophotometric image analysis of fine art paintings,” in Proceedings of the IS&T/SID Color Imaging Conference (1996), pp. 50–53.
- M. Yamaguchi, T. Teraji, K. Ohsawa, T. Uchiyama, H. Motomura, Y. Murakami, and N. Ohyama, “Color image reproduction based on the multispectral and multiprimary imaging: experimental evaluation,” Proc. SPIE 4663, 15–26 (2001). [CrossRef]
- M. Okuyama, N. Tsumura, and Y. Miyake, “Evaluating a multispectral imaging system for mapping pigments in human skin,” Opt. Rev. 10, 580–584 (2003). [CrossRef]
- A. Kaarna, P. Zemcik, H. Kalviainen, and J. Parkkinen, “Multispectral image compression,” in Proceedings of the 14th International Conference on Pattern Recognition (1998), pp. 1264–1267.
- Q. Du and J. E. Fowler, “Hyperspectral image compression using JPEG2000 and principal component analysis,” IEEE Geosci. Remote Sens. Lett. 4, 201–205 (2007).
- B. Penna, T. Tillo, E. Magli, and G. Olmo, “Hyperspectral image compression employing a model of anomalous pixels,” IEEE Geosci. Remote Sens. Lett. 4, 664–668 (2007). [CrossRef]
- D. Y. Tzeng and R. S. Berns, “A review of principal component analysis and its applications to color technology,” Color Res. Appl. 30, 84–98 (2005). [CrossRef]
- F. Agahian, S. A. Amirshahi, and S. H. Amirshahi, “Reconstruction of reflectance spectra using weighted principal component analysis,” Color Res. Appl. 33, 360–371 (2008). [CrossRef]
- A. Garcia-Beltran, J. L. Nieves, J. Hernandez-Andres, and J. Romero, “Linear bases for spectral reflectance functions of acrylic paints,” Color Res. Appl. 23, 39–45 (1998). [CrossRef]
- F. Ayala, J. F. Echavarri, and P. Renet, “Use of three tristimulus values from surface reflectance spectra to calculate the principal components to reconstruct these spectra by using only three eigenvector,” J. Opt. Soc. Am. A. 23, 2020–2026 (2006). [CrossRef]
- I. T. Jolliffe, Principal Component Analysis, 2nd ed., Springer Series in Statistics (Springer-Verlag, 2002).
- K. Pearson, “On lines and planes of closest fit to systems of points in space,” Philos. Mag. 2(11), 559–572 (1901). [CrossRef]
- H. Hotelling, “Analysis of a complex of statistical variables into principal components,” J. Educ. Psychol. 24, 417–444 (1933).
- R. A. Johnson and D. W. Wichern, Applied Multivariate Statistical Analysis (Prentice-Hall, 1992).
- P. Filzmoser, “A multivariate outlier detection method,” in Proceedings of International Conference on Computer Data Analysis and Modeling (2004), pp. 18–22.
- P. J. Rousseeuw and K. Van Driessen, “A Fast algorithm for the minimum covariance determinant estimator,” Technometrics. 41, 212–223 (1999).
- P. J. Rousseeuw, “Least median of squares regression,” J. Am. Stat. Assoc. 79, 871–880 (1984).
- LIBRA: a MATLAB Library for Robust Analysis (2006), http://users.jyu.fi/~samiayr/DM/demot/LIBRA_Contents.pdf . http://wis.kuleuven.be/stat/robust/software .
- S. Hordley, G. Finlayson, and P. Morovic, “A multispectral image database and an application to image rendering across illumination,” in Proceedings of Third International Conference on Image and Graphics (2004), pp. 394–397.
- F. Yasuma, T. Mitsunaga, D. Iso, and S. K. Nayar, “Generalized assorted pixel camera: post-capture control of resolution, dynamic range and spectrum,” (Department of Computer Science, Columbia University, 2008).
- Eastern Finland Spectral Image Database, University of Eastern Finland, Spectral Color Research Group, https://www.uef.fi/spectral/spectral-database .
- MATLAB (b), and System Identification Toolbox, Release 2013a, The MathWorks, Inc., Natick, Massachusetts, United States.
- R. Vidal, “A tutorial on subspace clustering,” IEEE Signal Process. Mag. 28(2), 52–68 (2011). [CrossRef]
- J. Hernandez-Andres, J. Romero, A. García-Beltrán, and J. L. Nieves, “Testing linear models on spectral daylight measurements,” Appl. Opt. 37, 971–977 (1998). [CrossRef]

## Cited By |
Alert me when this paper is cited |

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

« Previous Article | Next Article »

OSA is a member of CrossRef.