OSA's Digital Library

Optics Express

Optics Express

  • Editor: Michael Duncan
  • Vol. 10, Iss. 1 — Jan. 14, 2002
  • pp: 60–65
« Show journal navigation

Foveated imaging demonstration

D. V. Wick, T. Martinez, S. R. Restaino, and B. R. Stone  »View Author Affiliations


Optics Express, Vol. 10, Issue 1, pp. 60-65 (2002)
http://dx.doi.org/10.1364/OE.10.000060


View Full Text Article

Acrobat PDF (2129 KB)





Browse Journals / Lookup Meetings

Browse by Journal and Year


   


Lookup Conference Papers

Close Browse Journals / Lookup Meetings

Article Tools

Share
Citations

Abstract

A wide field-of-view (FOV), theoretically diffraction-limited imaging system is demonstrated using a single positive lens (a singlet), a reflective liquid crystal spatial light modulator (SLM), a turning mirror and a CCD camera. The SLM is used to correct the off-axis aberrations that would otherwise limit the useful FOV of our system. Foveated imaging refers to the variation in spatial resolution across the image caused by using the SLM in this manner.

© Optical Society of America

1. Introduction

There are typically two methods for increasing the field-of-view (FOV) of conventional imaging systems while still maintaining high image quality out to the edge of the field [1

1. See for example, R.R. Shannon, The art and science of optical design (Cambridge University Press, New York, 1997), Ch. 7.

]. The first approach is to stop down the entrance pupil and thus increase the f/# of the system. A smaller entrance pupil reduces both the resolving power of the system and the irradiance on the image plane. The second method is to add optical elements to the system in order to balance off-axis aberrations at larger field angles. Adding elements increases the complexity, size, and weight of the system. If either resolving power or flux density is critical, adding multiple elements is often the only way to maintain high image quality across the entire FOV. For systems where size, weight, and/or expense are also critical, a compromise between the useable FOV and the system complexity must be met.

Currently, there exists a growing need for small, lightweight imaging sensors with wide FOV and high data transmission rates. There are a variety of military and civilian applications including surveillance activities, target acquisition and tracking, and the remote operation of unmanned vehicles that could benefit by using such systems. In addition, there has been increasing interest in developing bio-mimetic sensors that mimic human sensors. Previously, we presented a concept for increasing the theoretically diffraction-limited FOV of a simple, monochromatic imaging system [2

2. T. Martinez, D.V. Wick, and S.R. Restaino, “Foveated, wide field-of-view imaging system using a liquid crystal spatial light modulator,” Opt. Express 8, 555–560 (2001), http://www.opticsexpress.org/oearchive/source/32932.htm. [CrossRef] [PubMed]

]. Our system mimicked the human vision system by creating a variable resolution image. In this paper, we present preliminary laboratory results that validate the concept.

2. Foveated imaging background

Foveated imaging simply refers to a system that creates images with spatially varying resolution. The term originates from the operation of the human eye, where a limited area within a few degrees of the point of gaze is highly resolved and resolution falls off rapidly with increasing field angle. The imaging system that we previously proposed [2

2. T. Martinez, D.V. Wick, and S.R. Restaino, “Foveated, wide field-of-view imaging system using a liquid crystal spatial light modulator,” Opt. Express 8, 555–560 (2001), http://www.opticsexpress.org/oearchive/source/32932.htm. [CrossRef] [PubMed]

] creates an image with a limited region of interest that is well corrected and appears in focus, while peripheral areas are aberrated and appear blurred. The region of interest can be changed dynamically on a millisecond time scale, such that any area within the FOV of the system can be highly resolved. The reduction in resolution outside the region of interest depends on the particular system design.

The key to our proposed design was using a liquid crystal spatial light modulator (SLM) at or near a pupil plane. The SLM allows us to correct the off-axis aberrations that would have otherwise limited the useful FOV of our simple imaging system. A pixilated, liquid-crystal SLM used in a monochromatic application is the transmissive analogue of a segmented deformable mirror [3

3. D.S. Acton, “Correction of static optical errors in a segmented adaptive optical system,” Appl. Opt. 34, 7965–7968 (1995). [CrossRef] [PubMed]

]; it imposes a user-controlled, spatially varying optical path across the wavefront. This change in optical path length, called the optical path difference (OPD), is adjusted to compensate wavefront aberrations that would otherwise degrade the image. With a deformable mirror, the OPD is adjusted by deflecting the mirror using actuators or electrodes. Thus, the physical path length the light traverses is either increased or decreased depending on whether the mirror surface is pulled or pushed (i.e. OPD = nz Δz, where nz is the index of refraction in the direction of propagation, typically 1.0003 when the mirror is used in air, and Δz is the change in physical path length). With a liquid crystal SLM the OPD is adjusted by changing the index of refraction in each pixel (i.e. OPD = Δnz z), but the physical path length does not change. The index of refraction in the direction of propagation is altered by applying a small voltage to each of the individual pixels, typically less than 40 volts, in order to reorient the liquid crystal molecules [4

4. See for example, S-T Wu, “Nematic Liquid Crystals,” Spatial Light Modulator Technology - Materials, Devices, and Applications, U. Efron, ed. (Marcel Dekker, Inc., New York, 1995), Ch. 1.

].

The ability of the SLM to correct aberrations and increase the FOV of an imaging system depends on the maximum OPD (i.e. dynamic range), the total number of pixels of the SLM, and the spectral bandwidth of the system. If the wavefront aberration is less than the dynamic range of the SLM, the spectral bandwidth is simply limited by dispersion in the liquid crystal, and passive imaging over a finite spectral bandwidth is possible. If, on the other hand, the phase errors of the wavefront are greater than the dynamic range of the SLM, correction can still be done modulo 2π [3

3. D.S. Acton, “Correction of static optical errors in a segmented adaptive optical system,” Appl. Opt. 34, 7965–7968 (1995). [CrossRef] [PubMed]

,5

5. G. D. Love, “Wave-front correction and production of Zernike modes with a liquid-crystal spatial light modulator,” Appl. Opt. 36, 1517–1524 (1997). [CrossRef] [PubMed]

]. In this case, however, the optical bandwidth of the system may be severely limited [6

6. P.F. McManamon, E.A. Watson, T. Dorshner, and L.J. Barnes, “Applications look at the use of liquid crystal writable gratings for steering passive radiation,” Opt. Eng. 32, 2657–2664 (1993). [CrossRef]

]. We are currently investigating the trade space between image quality and spectral bandwidth.

By systematically applying the appropriate voltage to each pixel, the proper transversely varying OPD can be established to correctly compensate the aberrated wavefront. However, because aberrations vary as a function of field angle, only a single angle can be perfectly corrected at any given time. Centered about that angle, there will exist a range of field angles that will be well corrected. That range will depend on the aberration present at that particular field angle, the correction capability of the SLM, and the particular imaging system that is being used. Thus, theoretically diffraction-limited performance can be achieved in a fast imaging system over a limited spatial region within a limited spectral bandwidth. Because the aberrations at each field angle can be determined in advance, either with ray trace calculations or direct measurements, the set of voltages required to correct the wavefront at a given field angle can be stored as a look-up table. Dynamic control is accomplished by simply changing the voltages to correct for a new field angle of interest.

Foveated imaging can reduce data transmission bandwidth requirements for transmitting digital images [7

7. N. Tsumura, C. Endo, H. Haneishi, and Y. Miyake, “Image compression and decompression based on gazing area,” Proc. SPIE 2657, 361–367 (1996). [CrossRef]

,8

8. W.S. Geisler and J.S. Perry, “A real-time foveated multi-resolution system for low-bandwidth video communication,” Proc. SPIE 3299, 294–305 (1998). [CrossRef]

]. Because only the area of interest contains high-resolution data, computer software or hardware such as ASICs (application specific integrated circuits) could be used to bin pixels outside the area of interest as appropriate for the system, reducing the total amount of data that must be transmitted. Even though resolution is lower in peripheral areas, there is still useful information within the entire FOV. Movement or changes outside the region of interest are still detectable, in the same way the human system detects something “out of the corner of the eye”. Motion tracking systems could be integrated with this system for automated tracking applications. Surveillance applications could utilize a cleverly designed display system that matches the resolution properties of the eye and integrates eye-tracking techniques to determine the area of interest to make the variable resolution completely unnoticeable to the observer [8

8. W.S. Geisler and J.S. Perry, “A real-time foveated multi-resolution system for low-bandwidth video communication,” Proc. SPIE 3299, 294–305 (1998). [CrossRef]

].

2. Laboratory demonstration

The laboratory demonstration, Figures 1 and 2, consists of a single plano-convex lens with a reflective SLM located in a pupil plane. Light from the object plane passes through the lens, is corrected/reflected by the SLM, passes back through the lens, and is reflected by a turning mirror onto the CCD camera. In this case, the CCD housing limited how close we could position the CCD to the lens, see Figure 2, and therefore we were only able to image a limited FOV (from approximately +10° to +25° in x, and +/-13° in y). We believe that this limitation can be overcome with a customized system, however, our off-the-shelf design sufficed to demonstrate the concept.

Although a transmissive SLM is preferable in order to simplify the optical setup [2

2. T. Martinez, D.V. Wick, and S.R. Restaino, “Foveated, wide field-of-view imaging system using a liquid crystal spatial light modulator,” Opt. Express 8, 555–560 (2001), http://www.opticsexpress.org/oearchive/source/32932.htm. [CrossRef] [PubMed]

], there currently are no commercially available transmissive devices that have a sufficient number of pixels to adequately correct aberrations at larger field angles. For this reason, we chose the Boulder Nonlinear Systems 512 × 512 reflective SLM to demonstrate our technique. It measured 3.6 × 3.6 mm, with 7 μm square pixels and a 77% fill factor. A large area (35 mm × 23 mm) Atmel Grenoble CCD camera was used to image the scene.

Using the setup in Figure 2, we were able to correct aberrations at field angles up to 25°. In order to determine how resolution varied across the image, a checkerboard pattern was positioned in the object plane and imaged onto the CCD. The most difficult part of the experiment was computing the correct voltage scheme for the SLM. The optimum phase retardance required for each pixel is dependent on the optical path, which is a function of the incident angle. Computer analysis was used to determine the optimum correction schemes for 10° and 25°.

Figure 3 shows how the different correction schemes correct aberrations and thus improve the image at different field angles. In (a), no voltage is applied to the SLM, and, although the image is drastically better moving left to right (i.e. closer to the optical axis), even the image at the far right edge (which corresponds to a 10° field angle in x) is fuzzy. When voltages for correction at 10° in x and 0° in y are applied in (b), the right edge of the image sharpens, although the image is still washed out at larger field angles. In (c), where the voltage scheme for correction at 25° in x and 0° in y has been applied, the left edge of the image becomes visible. Because the fill factor of the SLM is only 77%, there is a ghosting of uncorrected light that degrades the image, although our present optical set-up contributes to the nonuniformity in these figures, and we are redesigning our system.

Fig. 1. Optical layout for foveated imaging system with up to a 25° field-of-view
Fig. 2. Foveated imaging experiment
Fig. 3. Images of checkerboard transparency. For each image, the left edge corresponds to a 25° field angle in x and the right edge to a 10 ° field angle in x (see Fig. 1). The voltages schemes applied to the SLM are (a) none, (b) correction at 10° in x (0° in y), and (c) correction at 25° in x (0° in y)

The ghosting is very evident in Figure 4 below. Here the checkerboard transparency is replaced by a 25 μm pinhole positioned in the object plane at a 25° field angle. In (a), with no voltages applied to the SLM, the image of the pinhole is completely smeared out due to the aberrations in the system. However, when we apply the correction for 25°, the image of the pinhole is dramatically improved, as seen in (b). The remnants of the uncorrected image are due to the 77% fill factor of the SLM. With the camera gain reduced, the ghosting is not as visible, and the image is measured to be approximately two times diffraction limited.

Fig. 4. Images of a 25 μm pinhole at a 25° field angle (a) uncorrected, (b) corrected, and (c) corrected with camera gain reduced by a factor of four to avoid saturation.

5. Conclusion

We have presented experimental confirmation showing that a liquid crystal spatial light modulator placed in a pupil plane can increase the useable field-of-view of an extremely simple imaging system. We have corrected the image of a point source to two times the diffraction limit. We have also shown that the region of interest can be shifted to anywhere within the field-of-view of the system. The creation of variable resolution or foveated images can be used to reduce bandwidth requirements for data transmission.

Acknowledgments

The authors would gratefully like to acknowledge support and enlightening discussions with Paul McManamon, Ed Watson, and Scott Harris. This project was supported by DARPA. Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy under Contract DE-AC04-94AL85000.

References and links

1.

See for example, R.R. Shannon, The art and science of optical design (Cambridge University Press, New York, 1997), Ch. 7.

2.

T. Martinez, D.V. Wick, and S.R. Restaino, “Foveated, wide field-of-view imaging system using a liquid crystal spatial light modulator,” Opt. Express 8, 555–560 (2001), http://www.opticsexpress.org/oearchive/source/32932.htm. [CrossRef] [PubMed]

3.

D.S. Acton, “Correction of static optical errors in a segmented adaptive optical system,” Appl. Opt. 34, 7965–7968 (1995). [CrossRef] [PubMed]

4.

See for example, S-T Wu, “Nematic Liquid Crystals,” Spatial Light Modulator Technology - Materials, Devices, and Applications, U. Efron, ed. (Marcel Dekker, Inc., New York, 1995), Ch. 1.

5.

G. D. Love, “Wave-front correction and production of Zernike modes with a liquid-crystal spatial light modulator,” Appl. Opt. 36, 1517–1524 (1997). [CrossRef] [PubMed]

6.

P.F. McManamon, E.A. Watson, T. Dorshner, and L.J. Barnes, “Applications look at the use of liquid crystal writable gratings for steering passive radiation,” Opt. Eng. 32, 2657–2664 (1993). [CrossRef]

7.

N. Tsumura, C. Endo, H. Haneishi, and Y. Miyake, “Image compression and decompression based on gazing area,” Proc. SPIE 2657, 361–367 (1996). [CrossRef]

8.

W.S. Geisler and J.S. Perry, “A real-time foveated multi-resolution system for low-bandwidth video communication,” Proc. SPIE 3299, 294–305 (1998). [CrossRef]

OCIS Codes
(010.1080) Atmospheric and oceanic optics : Active or adaptive optics
(090.1000) Holography : Aberration compensation
(110.0110) Imaging systems : Imaging systems
(230.3720) Optical devices : Liquid-crystal devices
(230.6120) Optical devices : Spatial light modulators

ToC Category:
Research Papers

History
Original Manuscript: December 20, 2001
Published: January 14, 2002

Citation
David Wick, Ty Martinez, Sergio Restaino, and B. Stone, "Foveated imaging demonstration," Opt. Express 10, 60-65 (2002)
http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-10-1-60


Sort:  Journal  |  Reset  

References

  1. See for example, R.R. Shannon, The art and science of optical design (Cambridge University Press, New York, 1997), Ch. 7.
  2. T. Martinez, D.V. Wick, and S.R. Restaino, "Foveated, wide field-of-view imaging system using a liquid crystal spatial light modulator," Opt. Express 8, 555-560 (2001), http://www.opticsexpress.org/oearchive/source/32932.htm. [CrossRef] [PubMed]
  3. D.S. Acton, "Correction of static optical errors in a segmented adaptive optical system," Appl. Opt. 34, 7965-7968 (1995). [CrossRef] [PubMed]
  4. See for example, S-TWu, "Nematic Liquid Crystals," Spatial Light Modulator Technology - Materials, Devices, and Applications, U. Efron, ed. (Marcel Dekker, Inc., New York, 1995), Ch. 1.
  5. G. D. Love, "Wave-front correction and production of Zernike modes with a liquid-crystal spatial light modulator," Appl. Opt. 36, 1517-1524 (1997). [CrossRef] [PubMed]
  6. P.F. McManamon, E.A. Watson, T. Dorshner, and L.J. Barnes, "Applications look at the use of liquid crystal writable gratings for steering passive radiation," Opt. Eng. 32, 2657-2664 (1993). [CrossRef]
  7. N. Tsumura, C. Endo, H. Haneishi, and Y. Miyake, "Image compression and decompression based on gazing area," Proc. SPIE 2657, 361-367 (1996). [CrossRef]
  8. W.S. Geisler and J.S. Perry, "A real-time foveated multi-resolution system for low-bandwidth video communication," Proc. SPIE 3299, 294-305 (1998). [CrossRef]

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

Figures

Fig. 1. Fig. 2. Fig. 3.
 
Fig. 4.
 

« Previous Article  |  Next Article »

OSA is a member of CrossRef.

CrossCheck Deposited