OSA's Digital Library

Applied Optics

Applied Optics

APPLICATIONS-CENTERED RESEARCH IN OPTICS

  • Editor: Joseph N. Mait
  • Vol. 51, Iss. 4 — Feb. 1, 2012
  • pp: A80–A90

Digital camera simulation

Joyce E. Farrell, Peter B. Catrysse, and Brian A. Wandell  »View Author Affiliations


Applied Optics, Vol. 51, Issue 4, pp. A80-A90 (2012)
http://dx.doi.org/10.1364/AO.51.000A80


View Full Text Article

Enhanced HTML    Acrobat PDF (707 KB)





Browse Journals / Lookup Meetings

Browse by Journal and Year


   


Lookup Conference Papers

Close Browse Journals / Lookup Meetings

Article Tools

Share
Citations

Abstract

We describe a simulation of the complete image processing pipeline of a digital camera, beginning with a radiometric description of the scene captured by the camera and ending with a radiometric description of the image rendered on a display. We show that there is a good correspondence between measured and simulated sensor performance. Through the use of simulation, we can quantify the effects of individual digital camera components on system performance and image quality. This computational approach can be helpful for both camera design and image quality assessment.

© 2012 Optical Society of America

OCIS Codes
(100.0100) Image processing : Image processing

History
Original Manuscript: October 5, 2011
Revised Manuscript: December 22, 2011
Manuscript Accepted: December 23, 2011
Published: February 1, 2012

Citation
Joyce E. Farrell, Peter B. Catrysse, and Brian A. Wandell, "Digital camera simulation," Appl. Opt. 51, A80-A90 (2012)
http://www.opticsinfobase.org/ao/abstract.cfm?URI=ao-51-4-A80


Sort:  Author  |  Year  |  Journal  |  Reset  

References

  1. C. Kolb, D. Mitchell, and P. Hanrahan, “A realistic camera model for computer graphics,” in SIGGRAPH ’95 Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (ACM, 1995), pp. 317–324.
  2. M. Potmesil and I. Chakravarty, “A lens and aperture camera model for synthetic image generation,” in SIGGRAPH ’83 Proceedings of the 10th Annual Conference on Computer Graphics and Interactive Techniques (ACM, 1983), Vol. 15, No. 3, pp. 297–305.
  3. B. Barsky, D. R. Horn, S. A. Klein, J. A. Pang, and M. Yu, “Camera models and optical systems used in computer graphics: Part I, object based techniques,” in Proceedings of the 2003 International Conference on Computational Science and its Applications (ICCCSA ’03), Lecture Notes in Computer Science (Springer-Verlag2003), pp. 246–255.
  4. R. Gow, D. Renshaw, and K. Findlater, “A comprehensive tool for modeling CMOS image-sensor-noise performance,” IEEE Trans. Electron Devices 54, 1321–1329 (2007). [CrossRef]
  5. P. E. Haralabidis and C. Pilinis, “Linear color camera model for a skylight colorimeter with emphasis on the imaging pipeline noise performance,” J. Electron. Imaging 14, 043005 (2005). [CrossRef]
  6. R. Constantini and S. Susstrunk, “Virtual sensor design,” Proc. SPIE 5301, 408–419 (2004).
  7. H. B. Wach and J. E. R. Dowski, “Noise modeling for design and simulation of computational imaging systems,” Proc. SPIE 5438, 159–170 (2004).
  8. B. F. A. El Gamal, H. Min, and X. Liu, “Modeling and estimation of FPN components in CMOS image sensors,” Proc. SPIE 3301, 168–177 (1998).
  9. B. F. H. Tian and A. El Gamal, “Analysis of temporal noise in CMOS APS,” IEEE J. Solid-State Circuits 36, 92–101 (2001).
  10. B. A. Wandell, “The synthesis and analysis of color images,” IEEE Trans. Pattern Anal. Machine Intell. 9, 2–13 (1987). [CrossRef]
  11. L. T. Maloney, “Evaluation of linear models of surface spectral reflectance with small numbers of parameters,” J. Opt. Soc. Am. A 3, 1673–1683 (1986). [CrossRef]
  12. D. H. Marimont and B. A. Wandell, “Linear models of surface and illuminant spectra,” J. Opt. Soc. Am. A 9, 1905–1913 (1992). [CrossRef]
  13. S. Tominaga and B. A. Wandell, “Component estimation of surface spectral reflectance,” J. Opt. Soc. Am. A 7, 312–317 (1990).
  14. B. K. P. Horn and R. W. Sjoberg, “Calculating the reflectance map,” Appl. Opt. 18, 1770–1779 (1979). [CrossRef]
  15. D. B. Judd, D. L. MacAdam, and G. W. Wyszecki, “Spectral distribution of typical daylight as a function of correlated color temperature,” J. Opt. Soc. Am. 54, 1031–1036 (1964). [CrossRef]
  16. J. P. S. Parkkinen, J. Hallikainen, and T. Jaaskelainen, “Characteristic spectra of Munsell colors,” J. Opt. Soc. Am. A 6, 318–322 (1989).
  17. J. Cohen, “Dependency of the spectral reflectance curves of the Munsell color chips,” Psychonomic Sci. 1, 369–370 (1964).
  18. B. Gunturk, J. Glotzbach, Y. Altunbasak, R. Schafer, and R. M. Mersereau, “Demosaicking: color filter array interpolation,” IEEE Signal Process. Mag. 22, 44–54 (2005). [CrossRef]
  19. J. P. Kerekes and J. E. Baum, “Spectral imaging system analytical model for subpixel target detection,” IEEE Trans. Geosci. Remote Sens. 40, 1088–1101 (2002).
  20. M. Bernhardt, P. Clare, C. Cowell, and M. Smith, “A hyperspectral model for target detection,” Proc. SPIE 6565, 65650F (2007).
  21. J. R. Schott, Remote Sensing: The Image Chain Approach, 2nd ed. (Oxford University, 2007).
  22. R. D. Fiete, Modeling the Imaging Chain of Digital Cameras Tutorial Texts in Optical Engineering (SPIE Press, 2010), Vol. TT92.
  23. X. Wang, J. Zhang, Z. Feng, and H. Chang, “Equation-based triangle orientation discrimination sensor performance model based on sampling effects,” Appl. Opt. 44, 498–505 (2005). [CrossRef]
  24. J. Farrell, M. Okincha, M. Parmar, and B. Wandell, “Using visible SNR (vSNR) to compare the image quality of pixel binning and digital resizing,” Proc. SPIE 7537, 75370C (2010).
  25. J. Xu, R. Bowen, J. Wang, and J. Farrell, “Visibility of uncorrelated image noise,” Proc. SPIE 7537, 753703 (2010).
  26. J. Farrell, F. Xiao, P. B. Catrysse, and B. Wandell, “A simulation tool for evaluating digital camera image quality,” Proc. SPIE 5294, 124–131 (2004).
  27. P. L. Vora, J. E. Farrell, J. D. Tietz, and D. H. Brainard, “Image capture: simulation of sensor responses from hyperspectral images,” IEEE Trans. Image Process. 10, 307–316 (2001). [CrossRef]
  28. P. Longere and D. H. Brainard, “Simulation of digital camera images from hyperspectral input,” in Vision Models and Applications to Image and Video Processing, C. v. d. B. Lambrecht, ed. (Kluwer, 2001), pp. 123–150.
  29. P. B. Catrysse and B. A. Wandell, “Optical efficiency of image sensor pixels,” J. Opt. Soc. Am. A 19, 1610–1620 (2002).
  30. J. Chen, K. Venkataraman, D. Bakin, B. Rodricks, R. Gravelle, P. Rao, and Y. Ni, “Digital camera imaging system simulation,” IEEE Trans. Electron Devices 56, 2496–2505 (2009). [CrossRef]
  31. J. Farrell, M. Okincha, and M. Parmar, “Sensor calibration and simulation,” Proc. SPIE 6817, 68170R (2008).
  32. J. Farrell, F. Xiao, and S. Kavusi, “Resolution and light sensitivity tradeoff with pixel size,” Proc. SPIE 6069, 60690N (2006).
  33. M. Parmar and B. A. Wandell, “Interleaved imaging: an imaging system design inspired by rod-cone vision,” Proc. SPIE 7250, 725008 (2009).
  34. F. Xiao, A. Silverstein, and J. Farrell, “Camera motion and effective spatial resolution,” in Proceedings of the International Congress of Imaging Science (Society for Imaging Science and Technology, 2006), pp. 33–36.
  35. F. Xiao, J. E. Farrell, P. B. Catrysse, and B. Wandell, “Mobile imaging: the big challenge of the small pixel,” Proc. SPIE 7250, 72500K (2009).
  36. D. H. Brainard and C. Broussard, “Rendertoolbox Wiki,” http://rendertoolbox.org .
  37. B. Barsky and T. Kosloff, “Algorithms for rendering depth of field effects in computer graphics,” in ICCOMP ’08 Proceedings of the 12th WSEAS International Conference on Computers (World Scientific and Engineering Academy and Society, 2008), 999–1010.
  38. D. Lanman, R. Raskar, and G. Taubin, “Modeling and synthesis of aperture effects in camera,” in Proceedings of the International Symposium on Computational Aesthetics in Graphics, Visualization, and Imaging (CAe, 2008), pp. 81–88.
  39. R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a hand-held plenoptic camera,” Stanford Tech. Report CTSR 2005-02 (2005).
  40. K. Martinez, J. Cupitt, and D. Saunders, “High resolution colorimetric imaging of paintings,” Proc. SPIE 1901, 25–36 (1993).
  41. P. L. Vora, M. L. Harville, J. E. Farrell, J. D. Tietz, and D. H. Brainard, “Image capture: synthesis of sensor responses from multispectral images,” Proc. SPIE 3018, 2–11 (1997).
  42. P. D. Burns and R. S. Berns, “Analysis of multispectral image capture,” in Proceedings of the 4th IS&T/SID Color Imaging Conference (IS&T, 1996), pp. 19–22.
  43. S. M. C. Nascimento, F. P. Ferreira, and D. H. Foster, “Statistics of spatial cone-excitation ratios in natural scenes,” J. Opt. Soc. Am. A 19, 1484–1490 (2002).
  44. M. Parmar, F. Imai, S. H. Park, and J. Farrell, “A database of high dynamic range visible and near-infrared multispectral images,” Proc. SPIE 6817, 68170N (2008).
  45. P. B. Catrysse is preparing a manuscript to be called “Imaging optics.”
  46. R. G. Driggers, Encyclopedia of Optical Engineering (CRC, 2003), Vol. 1.
  47. P. B. Catrysse and B. A. Wandell, “Roadmap for CMOS image sensors: Moore meets Planck and Sommerfeld,” Proc. SPIE 5678, 1–13 (2005).
  48. W. J. Smith, Modern Optical Engineering: the Design of Optical Systems, 4th ed., Optical and Electro-Optical Engineering Series (McGraw-Hill, 2007).
  49. J. Goodman, “The frequency response of a defocused optical system,” Proc. R. Soc. Lond. Ser. A 231, 91–103 (1955).
  50. J. W. Goodman, Introduction to Fourier Optics, 2nd ed. (McGraw-Hill, 1996).
  51. P. Maeda, P. B. Catrysse, and B. A. Wandell, “Integrating lens design with digital camera simulation,” Proc. SPIE 5678, 48–58 (2005).
  52. P. B. Catrysse, X. Liu, and A. El Gamal, “Quantum efficiency reduction due to pixel vignetting in CMOS image sensors,” Proc. SPIE 3965, 420–430 (2000).
  53. H. Tian and A. El Gamal, “Analysis of 1/f noise in CMOS APS,” Proc. SPIE 3965, 168–176 (2000).
  54. J. R. Janesick, Scientific Charge-Coupled Devices (SPIE, 2001), Vol. PM83.
  55. J. Farrell, G. Ng, X. Ding, K. Larson, and B. Wandell, “A display simulation toolbox for image quality evaluation,” J. Disp. Technol. 4, 262–270 (2008).
  56. It is possible to use lens specification data from lens design software programs such as ZEMAX and CODE V. When these data are available, a ray-trace method can be used to calculate PSFs that are both wavelength dependent and position dependent (not shift invariant). In our simulations, we analyze the central portion of the sensor image, where departures from diffraction-limited models are less evident.
  57. J.-Y. Bouguet, “Camera calibration toolbox for Matlab,” http://www.vision.caltech.edu/bouguetj/calib_doc/ .
  58. M. Stokes, M. Anderson, S. Chandrasekar, and R. Motta, “Standard default color space for the internet—sRGB,” http://www.w3.org/Graphics/Color/sRGB.html .
  59. C. Mornet, J. Vaillant, T. Decroux, D. Herault, and I. Schanen, “Evaluation of color error and noise on simulated images,” Proc. SPIE 7537, 75370Y (2010).
  60. A. Adam, E. Talvala, S. H. Park, D. E. Jacobs, B. Ajdin, N. Gelfand, N. Dolson, J. Vaquero, J. Baek, M. Tico, H. P. Lensch, W. Matusik, K. Pulli, M. Horowitz, and M. Levoy, “The Frankencamera: an experimental platform for computational photography,” in SIGGRAPH ’10 ACM SIGGRAPH 2010 Papers (ACM, 2010), pp. 1–12.
  61. P. B. Catrysse and B. A. Wandell, “Integrated color pixels in 0.18-μm complementary metal oxide semiconductor technology,” J. Opt. Soc. Am. A 20, 2293–2306 (2003).
  62. G. Langfelder, F. Zaraga, and A. Longoni, “Tunable spectral responses in a color-sensitive CMOS pixel for imaging applications,” IEEE Trans. Electron Devices 56, 2563–2569 (2009). [CrossRef]
  63. S. Lansel and B. Wandell, “Local linear learned image processing pipeline,” in Imaging Systems Applications, OSA Technical Digest (CD) (Optical Society of America, 2011), paper IMC3.
  64. F. Yasuma, T. Mitsunaga, D. Iso, and S. K. Nayar, “Generalized assorted pixel camera: postcapture control of resolution, dynamic range, and spectrum,” IEEE Trans. Image Process. 19, 2241–2253 (2010). [CrossRef]
  65. K. K. Ghosh, L. D. Burns, E. D. Cocker, A. Nimmerjahn, Y. Ziv, A. El Gamal, and M. J. Schnitzer, “Miniaturized integration of a fluorescence microscope,” Nat. Methods 8, 871–878 (2011). [CrossRef]
  66. G. Leseur, N. Meunier, G. Georgiadis, L. Huang, B. Wandell, P. B. Catrysse, and J. M. DiCarlo, “High-speed document sensing and misprint detection in digital presses,” Proc. SPIE 7536 (2010).
  67. J. M. DiCarlo, E. Montgomery, and S. W. Trovinger, “Emissive chart for imager calibration,” in Twelfth Color Imaging Conference: Color Science and Engineering Systems (The Society for Imaging Science and Technology, 2004), pp. 295–301.

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.


« Previous Article  |  Next Article »

OSA is a member of CrossRef.

CrossCheck Deposited