Abstract

In multivariate regression analysis of spectroscopy data, spectral preprocessing is often performed to reduce unwanted background information (offsets, sloped baselines) or accentuate absorption features in intrinsically overlapping bands. These procedures, also known as pretreatments, are commonly smoothing operations or derivatives. While such operations are often useful in reducing the number of latent variables of the actual decomposition and lowering residual error, they also run the risk of misleading the practitioner into accepting calibration equations that are poorly adapted to samples outside of the calibration. The current study developed a graphical method to examine this effect on partial least squares (PLS) regression calibrations of near-infrared (NIR) reflection spectra of ground wheat meal with two analytes, protein content and sodium dodecyl sulfate sedimentation (SDS) volume (an indicator of the quantity of the gluten proteins that contribute to strong doughs). These two properties were chosen because of their differing abilities to be modeled by NIR spectroscopy: excellent for protein content, fair for SDS sedimentation volume. To further demonstrate the potential pitfalls of preprocessing, an artificial component, a randomly generated value, was included in PLS regression trials. Savitzky–Golay (digital filter) smoothing, first-derivative, and second-derivative preprocess functions (5 to 25 centrally symmetric convolution points, derived from quadratic polynomials) were applied to PLS calibrations of 1 to 15 factors. The results demonstrated the danger of an over reliance on preprocessing when (1) the number of samples used in a multivariate calibration is low (<50), (2) the spectral response of the analyte is weak, and (3) the goodness of the calibration is based on the coefficient of determination (<i>R</i>²) rather than a term based on residual error. The graphical method has application to the evaluation of other preprocess functions and various types of spectroscopy data.

Analysis of optical spectra by Fourier methods: filtering and least-squares regression in reciprocal space

D. E. Aspnes and H. Arwin
J. Opt. Soc. Am. 73(12) 1759-1764 (1983)

Partial least squares regression calculation for quantitative analysis of metals submerged in water measured using laser-induced breakdown spectroscopy

Tomoko Takahashi, Blair Thornton, Takumi Sato, Toshihiko Ohki, Koichi Ohki, and Tetsuo Sakka
Appl. Opt. 57(20) 5872-5883 (2018)

Acidity measurement of iron ore powders using laser-induced breakdown spectroscopy with partial least squares regression

Z.Q. Hao, C.M. Li, M. Shen, X.Y. Yang, K.H. Li, L.B. Guo, X.Y. Li, Y.F. Lu, and X.Y. Zeng
Opt. Express 23(6) 7795-7801 (2015)

Error analysis of the spectral shift for partial least squares models in Raman spectroscopy

Haiyi Bian and Jing Gao
Opt. Express 26(7) 8016-8027 (2018)

Improved Savitzky–Golay-method-based fluorescence subtraction algorithm for rapid recovery of Raman spectra

Kun Chen, Hongyuan Zhang, Haoyun Wei, and Yan Li
Appl. Opt. 53(24) 5559-5569 (2014)