OSA's Digital Library

Optics Express

Optics Express

  • Editor: C. Martijin de Sterke
  • Vol. 15, Iss. 9 — Apr. 30, 2007
  • pp: 5531–5536
« Show journal navigation

Improved reflectance reconstruction for multispectral imaging by combining different techniques

Hui-Liang Shen, John H. Xin, and Si-Jie Shao  »View Author Affiliations


Optics Express, Vol. 15, Issue 9, pp. 5531-5536 (2007)
http://dx.doi.org/10.1364/OE.15.005531


View Full Text Article

Acrobat PDF (106 KB)





Browse Journals / Lookup Meetings

Browse by Journal and Year


   


Lookup Conference Papers

Close Browse Journals / Lookup Meetings

Article Tools

Share
Citations

Abstract

In multispectral imaging system, one of the most important tasks is to accurately reconstruct the spectral reflectance from system responses. We propose such a new method by combing three most frequently used techniques, i.e., wiener estimation, pseudo-inverse, and finite-dimensional modeling. The weightings of these techniques are calculated by minimizing the combined standard deviation of both spectral errors and colorimetric errors. Experimental results show that, in terms of color difference error, the performance of the proposed method is better than those of the three techniques. It is found that the simple averaging of the reflectance estimates of these three techniques can also yield good color accuracy.

© 2007 Optical Society of America

1. Introduction

Nowadays multispectral imaging techniques have been intensively studied for the applications of digital recording, archiving, and display. One of the most important tasks of multispectral imaging is to accurately recover the spectral reflectances of the object surfaces from system responses [1–7

1. J. Y. Hardeberg, “Acquisition and reproduction of color images: colorimetric and multispectral approaches,” Ph.D. dissertation (Ecole Nationale Superieure des Telecommunications, 1999).

]. In the literature, three spectral characterization techniques, i.e., wiener estimation [2

2. H. L. Shen and J. H. Xin, “Spectral characterization of a color scanner by adaptive estimation,” J. Opt. Soc. Am. A 21, 1125–1130 (2004). [CrossRef]

, 3

3. H. Haneishi, T. Hasegawa, A. Hosoi, Y. Yokoyama, N. Tsumura, and Y. Miyake, “System design for accurately estimating the spectral reflectance of art paintings,” Appl. Opt. 39, 6621–6632 (2000). [CrossRef]

], pseudo-inverse [1

1. J. Y. Hardeberg, “Acquisition and reproduction of color images: colorimetric and multispectral approaches,” Ph.D. dissertation (Ecole Nationale Superieure des Telecommunications, 1999).

], and finite-dimensional modeling [4–7

4. V. Cheung, S. Westland, C. Li, J. Hardeberg, and D. Connah, “Characterization of trichromatic color cameras by using a new multispectral imaging technique,” J. Opt. Soc. Am. A 22, 1231–1240 (2005). [CrossRef]

] have been widely adopted for reflectance estimation. In the wiener estimation and finite-dimensional modeling, the spectral sensitivity or responsivity needs to be instrumentally measured or mathematically recovered. The pseudo-inverse technique computes the transform between system responses and reflectance without any a priori knowledge about the imaging system.

To meet the rigorous requirement in industrial applications such as color measurement and color quality control, researchers are still working for new spectral characterization techniques. This study tries to investigate whether the accuracy of spectral reconstruction can be improved by combining the currently available techniques. In the color constancy area, Cardei and Funt found that better illumination estimation could be obtained by combing the results of different color constancy algorithms [8

8. V. Cardei and B. Funt, “Committee-based color constancy,” in Proc. IS&T/SID Seventh Color Imaging Conf: Color Science, Systems, and Applications (Society for Imaging Science and Technology, Virginia, 1999), pp.311–313.

]. Based on statistical analysis, Stokman and Gevers investigated the optimal strategy of color model selection and fusion to improve the discriminative power of image feature detection algorithms [9

9. H. Stokman and T. Gevers, “Selection and fusion of color models for image feature detection,” IEEE Trans. Pattern Anal. Mach. Intell. 29, 371–381 (2007). [CrossRef] [PubMed]

]. This study is mainly inspired by these two works.

2. Formulation of multispectral imaging process

When the response of the multispectral imaging system is proportional to the intensity of light entering the camera, the response vector v of L channels can be formulated as [2

2. H. L. Shen and J. H. Xin, “Spectral characterization of a color scanner by adaptive estimation,” J. Opt. Soc. Am. A 21, 1125–1130 (2004). [CrossRef]

]:

vL×1=ML×NrN×1+bL×1+nL×1,
(1)

where the subscript N=31 denotes the number of samples in the visible wavelength range, r denotes the spectral reflectance, M represents the spectral responsivity incorporating the spectral power distribution of lighting source, the spectral transmittances of narrowband filters, and the spectral sensitivity of digital camera. b denotes the bias response vector caused by camera dark current, and n denotes zero-mean imaging noise. It is noted that if the system does not behave linearly, the optoelectronic conversion function needs to be further considered [2

2. H. L. Shen and J. H. Xin, “Spectral characterization of a color scanner by adaptive estimation,” J. Opt. Soc. Am. A 21, 1125–1130 (2004). [CrossRef]

, 4

4. V. Cheung, S. Westland, C. Li, J. Hardeberg, and D. Connah, “Characterization of trichromatic color cameras by using a new multispectral imaging technique,” J. Opt. Soc. Am. A 22, 1231–1240 (2005). [CrossRef]

].

In spectral characterization of imaging system, it is feasible to mathematically recover the spectral responsivity M and bias b from training color samples, subject to the constraint of non-negativeness [10

10. K. Barnard and B. Funt, “Camera characterization for color research,” Color Res. Appl. 27, 152–163 (2002). [CrossRef]

, 2

2. H. L. Shen and J. H. Xin, “Spectral characterization of a color scanner by adaptive estimation,” J. Opt. Soc. Am. A 21, 1125–1130 (2004). [CrossRef]

]. It is noted that, as the filters used in this study are narrow-banded, the condition of smoothness is not needed.

3. Three techniques for spectral characterization of multispectral imaging

3.1 Wiener estimation

In general, the purpose of spectral characterization is to estimate the reflectance vector r̂ from the response vector u=v-b through an N×L matrix W, such that

. r̂=Wu.
(2)

In wiener estimation [2

2. H. L. Shen and J. H. Xin, “Spectral characterization of a color scanner by adaptive estimation,” J. Opt. Soc. Am. A 21, 1125–1130 (2004). [CrossRef]

, 3

3. H. Haneishi, T. Hasegawa, A. Hosoi, Y. Yokoyama, N. Tsumura, and Y. Miyake, “System design for accurately estimating the spectral reflectance of art paintings,” Appl. Opt. 39, 6621–6632 (2000). [CrossRef]

, 11

11. W. K. Pratt, Digital Image Processing, 2nd ed. (Wiley, New York, 1991).

], W is calculated by using the spectral responsivity M:

W=KrMT(MKrMT+Kn)1,
(3)

where superscript T denotes transpose, K r denotes the N×N covariance matrix of r, and K n denotes the covariance matrix of noise. In this study, K n=0 is assumed.

3.2 Pseudo-inverse

In the pseudo-inverse technique [1

1. J. Y. Hardeberg, “Acquisition and reproduction of color images: colorimetric and multispectral approaches,” Ph.D. dissertation (Ecole Nationale Superieure des Telecommunications, 1999).

], the transform matrix W is directly solved as

W=RU+=RUT(UUT),
(4)

where the superscript + denotes pseudo-inverse, R denotes the matrix of reflectance vector r, and U denotes the matrix of response vector u.

3.3 Finite-dimensional modeling

Due to the smooth property of spectral reflectance, r can always be represented by the linear combination of J(<N) basis functions b j [12

12. L. Maloney, “Evaluation of linear models of surface spectral reflectance with small numbers of parameters,” J. Opt. Soc. Am. A 3, 1673–1683 (1986). [CrossRef] [PubMed]

]:

r=j=1Jajbj,
(5)

where aj is the coefficient of b j. b j can be calculated using principle component analysis of reflectance data. By combining Eq. (5) and (1), the response u can then be represented as [4–7

4. V. Cheung, S. Westland, C. Li, J. Hardeberg, and D. Connah, “Characterization of trichromatic color cameras by using a new multispectral imaging technique,” J. Opt. Soc. Am. A 22, 1231–1240 (2005). [CrossRef]

]:

u=Mr=j=1JajMbj.
(6)

Thus coefficient aj can be estimated by pseudo-inverse of the L×J matrix [Mb j]. The reflectance rô can then be obtained by substituting the estimated aj into Eq. (5). In this study, J=10 basis functions are used, as they are generally adequate for spectral reflectance construction [12

12. L. Maloney, “Evaluation of linear models of surface spectral reflectance with small numbers of parameters,” J. Opt. Soc. Am. A 3, 1673–1683 (1986). [CrossRef] [PubMed]

, 5

5. M. Shi and G. Healey, “Using reflectance models for color scanner calibration,” J. Opt. Soc. Am. A 19, 645–656 (2002). [CrossRef]

].

4. The proposed method

4.1 Statistical properties of different observations

Suppose a quantity is measured using K (K=3 in this study) different techniques, and xk is the observation of the kth technique. To obtain an improved estimation of that quantity, it is feasible to combine these observations as the following:

x=k=1Kwkxk,
(7)

where wk is the weighting of the observation of the kth technique. The estimate and standard deviation of x can be expressed according to Eq. (8) and (9), respectively.

E(x)=k=1KwkE(xk),
(8)
σ2(x)=k=1Kl=1Kwkwlσ(xk,xl),
(9)

where E(xk) is the averaged observation of the kth technique, and σ(xk, xl) is the covariance between observations of the kth and lth techniques.

4.2 Combination of different spectral characterization techniques

Based on the statistical properties discussed above, a new method for reflectance estimation by combining different techniques is proposed in the following.

Let r̂k be the estimated reflectance of the kth technique, the spectral root mean square (rms) error can then be calculated as

δrk=((rr̂k)T(rr̂k)N)12,
(10)

and the combined rms error can be expressed as

δr=k=1Kwkδrk.
(11)

Similarly, the CIE 1994 color difference [13

13. R. McDonald and K. J. Smith, “CIE94 - a new colour difference formaula,” J. Soc. Dyers Colour. 111, 376–379 (1995). [CrossRef]

] error is represented as

δek=ΔE94*(r,r̂k),
(12)

and the combined color difference error then becomes

δe=k=1Kwkδek.
(13)

It is noted that the spectral error δr is linear to reflectance r, while the colorimetric error δe is nonlinear to r, due to the third-root-square transform between CIEXYZ space and CIELAB space. To account for both spectral and colorimetric accuracy in reflectance reconstruction, these two error terms can be merged into a single error term δ.

Consequently, the estimate of δ can then be expressed as

E(δ)=ck=1KwkE(δrk)+(1c)k=1KwkE(δek)
=cErTw+(1c)EeTw,
(14)

where 0≤c≤1 is used to adjust the proportions of spectral and colorimetric errors, E r and E e denote the K×1 vectors of the average spectral and colorimetric errors, respectively, and w denotes the K×1 weighting vector. Accordingly, the standard deviation of δ can be expressed as

σ2(δ)=ck=1Kl=1Kwkwlσ(δrk,δrl)+(1c)k=1Kl=1Kwkwlσ(δek,δel)
=cwTrw+(1c)wTew,
(15)

where r and ∑e denote the K×K covariance matrix of the spectral and colorimetric error, respectively. Considering that the spectral and colorimetric errors are always of different magnitudes, E r, E e, E r, and e are normalized in the calculation of weighting w.

The appropriate weighting w can be obtained by solving the following objective function:

minimizeσ2(δ),
(16)

or, alternatively,

minimizeσ2(δ)+E(δ).
(17)

Furthermore, the following two constraints should be imposed:

k=1Kwk=1,
(18)
0wk1.
(19)

The first constraint ensures that the weightings of the K techniques sum to 1, while the second constraint forces positive contribution of each technique.

The objective functions (16) and (17), together with the constraints (18) and (19), can be solved using quadric programming. When the weighting is obtained, the reflectance of the proposed method can be calculated as

r̂=k=1Kwkr̂k.
(20)

It is noted that the weightings obtained from the objective functions are suboptimal, due to the nonlinear transform between reflectance and color difference error. Nevertheless, it will be shown in the experiment section that this strategy can produce considerable color accuracy improvement, when compared with the three techniques.

5. Experimental results and discussion

In this study, a QImaging monochrome digital camera model Retiga-EXi, as well as a liquid crystal tunable filter (LCTF) made by Cambridge Research and Instrument Co., were used to acquire multispectral images. The LCTF contains a series of electronically-tunable narrowband filters, with the nominal bandwidth of 10nm Full-Width at Half-Maximum (FWHM). The color target used was GretagMacBeth ColorChecker DC (CDC), and its reflectance data were measured using a GretagMacBeth spectrophotometer 7000A in the visible spectrum of 400 to 700 nm, with sampling interval of 10 nm. Totally L=16 multispectral images of CDC were taken using the filters with center wavelengths at 400 nm, 420 nm, 440 nm, …, 700nm, under an approximate D65 lighting condition. The spatial non-uniformity of the lighting field was corrected by using a white paper [4

4. V. Cheung, S. Westland, C. Li, J. Hardeberg, and D. Connah, “Characterization of trichromatic color cameras by using a new multispectral imaging technique,” J. Opt. Soc. Am. A 22, 1231–1240 (2005). [CrossRef]

]. In total, 198 color patches on CDC were used for reflectance estimation, excluding the duplicated most dark ones and the glossy ones. These color patches were divided into one training set and one testing set, with each set containing 99 colors. The training set contains the color patches with odd sequence numbers, while the testing set contains the color patches with even sequence numbers.

Fig. 1. Distribution of colorimetric errors ΔE* 94 under D65 and spectral errors obtained using the objective function (16) with respect to different c values. The green square-symbol curve represents colorimetric errors, and the blue circle-symbol curve represents spectral rms errors.

Table 1. Spectral and colorimetric error statistics of 6 methods when c=0.2: wiener estimation, pseudo-inverse method, finite-dimensional modeling method, simple averaging method, and the proposed methods.

table-icon
View This Table

For comparison, the spectral and colorimetric error statistics including mean, standard deviation, and maximum of 6 different methods (wiener estimation, pseudo-inverse, finite-dimensional modeling, simple averaging, and the proposed methods with two different objective functions) are given in Table 1. In the averaging method, w 1=w 2=w 3=1/3. Nonparametric statistical test indicates that the color difference errors of the proposed methods and the averaging method are smaller than those of the other three methods at a significant level p=0.05. The performance of the averaging method is close to that of the proposed method 1 using objective function (16). The advantage of the proposed method 1 over the averaging method lies that the former can balance the colorimetric accuracy and spectral accuracy by adjusting the c value. The reconstructed spectral reflectance curves with minimum and maximum color difference errors are shown in Fig.2. The proposed method 2 using objective function (17) does not yield better performance, indicating that it may be not feasible to minimize both average color error and standard deviation. Actually, minimizing standard deviation will also decrease the average color error to a certain extent.

Fig. 2. Reconstructed spectral reflectances of the proposed method using objective function (16). (a) the best case, with ΔE* 94=0.11 under D65, (b) the worst case, with ΔE* 94=3.68 under D65

6. Conclusion

A new reflectance reconstruction method for multispectral imaging has been proposed, by combining three different spectral characterization techniques. The weightings of the estimated reflectances of these techniques are calculated based on the minimization of spectral and colorimetric errors. In terms of color difference error, the performance of the proposed method is better than those of the three techniques, while is close to that of the simple averaging method. The proposed method is applicable in industrial applications such as textile color measurement and color quality control.

Acknowledgments

This work was supported by the NSF of China under Grant No. 60602027.

References and links

1.

J. Y. Hardeberg, “Acquisition and reproduction of color images: colorimetric and multispectral approaches,” Ph.D. dissertation (Ecole Nationale Superieure des Telecommunications, 1999).

2.

H. L. Shen and J. H. Xin, “Spectral characterization of a color scanner by adaptive estimation,” J. Opt. Soc. Am. A 21, 1125–1130 (2004). [CrossRef]

3.

H. Haneishi, T. Hasegawa, A. Hosoi, Y. Yokoyama, N. Tsumura, and Y. Miyake, “System design for accurately estimating the spectral reflectance of art paintings,” Appl. Opt. 39, 6621–6632 (2000). [CrossRef]

4.

V. Cheung, S. Westland, C. Li, J. Hardeberg, and D. Connah, “Characterization of trichromatic color cameras by using a new multispectral imaging technique,” J. Opt. Soc. Am. A 22, 1231–1240 (2005). [CrossRef]

5.

M. Shi and G. Healey, “Using reflectance models for color scanner calibration,” J. Opt. Soc. Am. A 19, 645–656 (2002). [CrossRef]

6.

F. H. Imai and R. S. Berns, “Spectral estimation using trichromaitc digital cameras,” in Proceedings of the International Symposium on Multispectral Imaging and Color Reproduction for Digital Archives (Society of Multispectral Imaging of Japan, Chiba, Japan, 1999).

7.

M. Soriano, W. Oblefias, and C. Saloma, “Fluorescence spectrum estimation using multiple color images and minimum negativity constraint,” Opt. Express , 16, 1458–1464 (2002).

8.

V. Cardei and B. Funt, “Committee-based color constancy,” in Proc. IS&T/SID Seventh Color Imaging Conf: Color Science, Systems, and Applications (Society for Imaging Science and Technology, Virginia, 1999), pp.311–313.

9.

H. Stokman and T. Gevers, “Selection and fusion of color models for image feature detection,” IEEE Trans. Pattern Anal. Mach. Intell. 29, 371–381 (2007). [CrossRef] [PubMed]

10.

K. Barnard and B. Funt, “Camera characterization for color research,” Color Res. Appl. 27, 152–163 (2002). [CrossRef]

11.

W. K. Pratt, Digital Image Processing, 2nd ed. (Wiley, New York, 1991).

12.

L. Maloney, “Evaluation of linear models of surface spectral reflectance with small numbers of parameters,” J. Opt. Soc. Am. A 3, 1673–1683 (1986). [CrossRef] [PubMed]

13.

R. McDonald and K. J. Smith, “CIE94 - a new colour difference formaula,” J. Soc. Dyers Colour. 111, 376–379 (1995). [CrossRef]

OCIS Codes
(110.4190) Imaging systems : Multiple imaging
(330.1710) Vision, color, and visual optics : Color, measurement
(330.1730) Vision, color, and visual optics : Colorimetry

ToC Category:
Imaging Systems

History
Original Manuscript: February 5, 2007
Revised Manuscript: March 13, 2007
Manuscript Accepted: April 9, 2007
Published: April 20, 2007

Virtual Issues
Vol. 2, Iss. 5 Virtual Journal for Biomedical Optics

Citation
Hui-Liang Shen, John H. Xin, and Si-Jie Shao, "Improved reflectance reconstruction for multispectral imaging by combining different techniques," Opt. Express 15, 5531-5536 (2007)
http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-15-9-5531


Sort:  Author  |  Year  |  Journal  |  Reset  

References

  1. J. Y. Hardeberg, "Acquisition and reproduction of color images: colorimetric and multispectral approaches," Ph.D. dissertation (Ecole Nationale Superieure des Telecommunications, 1999).
  2. H. L. Shen and J. H. Xin, "Spectral characterization of a color scanner by adaptive estimation," J. Opt. Soc. Am. A 21, 1125-1130 (2004). [CrossRef]
  3. H. Haneishi, T. Hasegawa, A. Hosoi, Y. Yokoyama, N. Tsumura, and Y. Miyake, "System design for accurately estimating the spectral reflectance of art paintings," Appl. Opt. 39, 6621-6632 (2000). [CrossRef]
  4. V. Cheung, S. Westland, C. Li, J. Hardeberg, and D. Connah, "Characterization of trichromatic color cameras by using a new multispectral imaging technique," J. Opt. Soc. Am. A 22, 1231-1240 (2005). [CrossRef]
  5. M. Shi and G. Healey, "Using reflectance models for color scanner calibration," J. Opt. Soc. Am. A 19, 645-656 (2002). [CrossRef]
  6. F. H. Imai and R. S. Berns, "Spectral estimation using trichromaitc digital cameras," in Proceedings of the International Symposium on Multispectral Imaging and Color Reproduction for Digital Archives (Society of Multispectral Imaging of Japan, Chiba, Japan, 1999).
  7. M. Soriano, W. Oblefias, and C. Saloma, "Fluorescence spectrum estimation using multiple color images and minimum negativity constraint," Opt. Express 16, 1458-1464 (2002).
  8. V. Cardei and B. Funt, "Committee-based color constancy," in Proc. IS&T/SID Seventh Color Imaging Conf.: Color Science, Systems, and Applications (Society for Imaging Science and Technology, Virginia, 1999), pp. 311-313.
  9. H. Stokman and T. Gevers, "Selection and fusion of color models for image feature detection," IEEE Trans. Pattern Anal. Mach. Intell. 29, 371-381 (2007). [CrossRef] [PubMed]
  10. K. Barnard and B. Funt, "Camera characterization for color research," Color Res. Appl. 27, 152-163 (2002). [CrossRef]
  11. W. K. Pratt, Digital Image Processing, 2nd ed. (Wiley, New York, 1991).
  12. L. Maloney, "Evaluation of linear models of surface spectral reflectance with small numbers of parameters," J. Opt. Soc. Am. A 3, 1673-1683 (1986). [CrossRef] [PubMed]
  13. R. McDonald and K. J. Smith, "CIE94 - a new colour difference formaula," J. Soc. Dyers Colour. 111, 376-379 (1995). [CrossRef]

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

Figures

Fig. 1. Fig. 2.
 

« Previous Article  |  Next Article »

OSA is a member of CrossRef.

CrossCheck Deposited