OSA's Digital Library

Optics Express

Optics Express

  • Editor: C. Martijn de Sterke
  • Vol. 17, Iss. 13 — Jun. 22, 2009
  • pp: 10659–10674
« Show journal navigation

Multiscale lens design

David J. Brady and Nathan Hagen  »View Author Affiliations


Optics Express, Vol. 17, Issue 13, pp. 10659-10674 (2009)
http://dx.doi.org/10.1364/OE.17.010659


View Full Text Article

Acrobat PDF (859 KB)





Browse Journals / Lookup Meetings

Browse by Journal and Year


   


Lookup Conference Papers

Close Browse Journals / Lookup Meetings

Article Tools

Share
Citations

Abstract

While lenses of aperture less than 1000λ frequently form images with pixel counts approaching the space-bandwidth limit, only heroic designs approach the theoretical information capacity at larger scales. We propose to use the field processing capabilities of small-scale secondary lens arrays to correct aberrations due to larger scale objective lenses, with an ultimate goal of achieving diffraction-limited imaging for apertures greater than 10,000λ. We present an example optical design using an 8 mm entrance pupil capable of resolving 20 megapixels.

© 2009 Optical Society of America

1. Introduction

The number of degrees of freedom in an image collected through a finite aperture was analyzed in numerous studies in the 1960’s. The number of degrees of freedom is roughly equivalent to the number of independent samples (pixels) necessary to fully characterize an image. Toraldo di Francia was particularly clear in demonstrating that this number is proportional to the Shannon number, e.g. the space-bandwidth product [1

1. G. T. di Francia, “Degrees of freedom of an image,” J. Opt. Soc. Am. 59, 799–804 (1969). [CrossRef]

]—often approximated as the lens area divided by λ2. Given the long history of this result, one is surprised to note that few modern optical systems approach Shannon-limited performance. The goal of this paper is to demonstrate moderate to large scale optical systems approaching the Shannon limit may be achieved by relaxing traditional constraints on lens system design and field uniformity.

Briefly, optical systems have not achieved Shannon limited performance because (1) focal planes with commensurate pixel counts have not been available, (2) practical lens systems with apertures greater than a few millimeters are incapable of processing Shannon-limited images and (3) digital systems capable of analyzing, transmitting and databasing large images have been unavailable. The first and third challenges have been resolved over the past decade through evolutionary improvements in focal plane and computational technology. Large scale manufacturing for mobile platforms has driven the cost per pixel of CMOS focal planes below 1 microdollar, meaning that gigapixel or even terapixel detector arrays may be reasonably affordable. Similarly, numerous gigapixel image management and analysis platforms have recently emerged [2

2. J. Kopf, M. Uyttendaele, O. Deussen, and M. F. Cohen, “Capturing and viewing gigapixel images,” ACM Trans. Graphics 26, 93 (2007). [CrossRef]

].

This paper proposes multiscale lens design as a solution to the second challenge. Multiscale design separates a lens system into objective and field processing optics. The objective is a single aperture system on a scale matched to the target angular resolution. The processing optics consist of smaller aperture components that can be fabricated with complex surface figures. The multiscale approach is motivated by scaling analysis suggesting that smaller lenses outperform larger lenses. The performance of a lens is quantified by the ratio of space-bandwidth product actually achieved to the diffraction-limited Shannon number. We represent this ratio by γ. As discussed in a pioneering study by Lohmann, performance against this metric saturates as a function of spatial scale, meaning that γ tends to fall in inverse proportion to aperture area [3

3. A. W. Lohmann, “Scaling laws for lens systems,” Appl. Opt. 28, 4996–4998 (1989). [CrossRef] [PubMed]

]. Of course, one may make heroic efforts to maintain γ by increasing lens complexity as aperture area increases. This approach was adopted to great effect in the development of lithographic lens systems [4

4. T. Matsuyama, Y. Ohmura, and D. M. Williamson, “The lithographic lens: its history and evolution,” in Optical Microlithography XIX, D. G. Flagello, ed., vol. 6154 of Proc. SPIE (2006).

], but is ultimately unsustainable in imaging applications.

As a corollary to the idea that smaller lenses are better suited to large γ image formation, one may also suppose that smaller lenses are better at wavefront correction. This is the case for two reasons. First, wavefront correction and image formation both yield geometric solutions with less wavelength-scale error over smaller apertures. Second, manufacturing of complex lens surfaces is much easier in smaller scale systems. Sophisticated small scale manufacturing techniques have been particularly advanced by recent efforts to develop wafer-level cameras for mobile devices [5

5. R. Völkel, M. Eisner, and K. J. Weible, “Miniaturized imaging systems,” Microelectron. Eng. 67–68, 461–472 (2003).

, 6

6. Y. Dagan, “Wafer-level optics enables low cost camera phones,” in Integrated Optics: Devices, Materials, and Technologies XIII, J.-E. Broquin and C. M. Greiner, eds., vol. 7218 of Proc. SPIE (2009).

].

Fig. 1. The basic layout and example raw measurements from the three imager types: (a) conventional, (b) 3×3 multiaperture, and (c) 5×5 two-level multiscale.

The aperture size of the collector subsystem is determined by angular resolution and light collection area specifications. The collector typically includes multiple surfaces and must process the field sufficiently to enable image restoration by the processor optics and digital post-processing subsystems. The aperture size of the processor subsystem is jointly determined by the quality of the wavefront delivered from the collector and by lens fabrication and detector array interface constraints. The optimal processor aperture size (typically 1–10 mm at visible wavelengths) is determined by geometric complexity and manufacturing constraints. While the system of Fig. 1 shows just two aperture sizes, multiscale design may in general include a hierarchy of aperture sizes stepping the field down from the collector aperture to small scale processing optics (Fig. 2).

A conspicuous feature of multiscale design is that the image is fractured into subimages which must be computationally combined in post-processing. Computational fusion has been demonstrated in numerous studies of multi-aperture cameras [7

7. J. Tanida, T. Kumagai, K. Yamada, S. Miyatake, K. Ishida, T. Morimoto, N. Kondou, D. Miyazaki, and Y. Ichioka, “Thin observation module by bound optics (TOMBO): concept and experimental verification,” Appl. Opt. 40, 1806–1813 (2001). [CrossRef]

, 8

8. M. Shankar, R. Willett, N. Pitsianis, T. Schulz, R. Gibbons, R. T. Kolste, J. Carriere, C. Chen, D. Prather, and D. Brady, “Thin infrared imaging systems through multichannel sampling,” Appl. Opt. 47, B1–B10 (2008). [CrossRef] [PubMed]

, 9

9. T. Mirani, D. Rajan, M. P. Christensen, S. C. Douglas, and S. L. Wood, “Computational imaging systems: joint design and end-to-end optimality,” Appl. Opt. 47, B86–B103 (2008). [CrossRef] [PubMed]

, 10

10. K. Choi and T. J. Schulz, “Signal-processing approaches for image-resolution restoration for TOMBO imagery,” Appl. Opt. 47, B104–B116 (2008). [CrossRef] [PubMed]

, 11

11. A. V. Kanaev, D. A. Scribner, J. R. Ackerman, and E. F. Fleet, “Analysis and application of multiframe superresolution processing for conventional imaging systems and lenslet arrays,” Appl. Opt. 46, 4320–4328 (2007). [CrossRef] [PubMed]

] with primary goal of reducing camera thickness by using digital superresolution to decrease effective pixel size. To emphasize the computational nature of the proposed imager, Fig. 1 shows an example image from a standard camera, from a 3×3 multi-aperture camera, and from a multiscale design using a 5×5 lenslet array. The multiscale imager splits the full field of view into subfields which partially overlap with their adjacent regions.

Fig. 2. An example three-level multiscale imager design.

The multiscale approach exploits recent advances in optical manufacturing that allow the accurate production of small integrated lens systems and in generalized sampling and coding strategies that jointly optimize physical filtering, sampling, and digital processing [12

12. A. D. Portnoy, N. P. Pitsianis, X. Sun, and D. J. Brady, “Multichannel sampling schemes for optical imaging systems,” Appl. Opt. 47, B76–B85 (2008). [CrossRef] [PubMed]

]. Co-design of physical and digital processing in multiscale systems design can be used to improve the following system metrics:

• Field of view. A multi-aperture processing element in combination with a large scale collector greatly reduces the need for balancing geometric aberration and field of view.

• Image resolution. For imagers which are not operating at the diffraction limit, the addition of a multi-aperture processor can greatly improve image quality.

• Camera volume. Field correction in processing optics may enable faster collection optics, thus reducing camera volume.

• Manufacturing and system cost. Complex surface figures and compound elements are much easier to make at smaller scales. Some fabrication technologies, such as molding and wafer-level integration, are uniquely applicable to small scale optics.

• Depth of field and 3D imaging. Processor optics may be designed to focus at diverse ranges with overlapping fields to enable tomographic object reconstruction, combining multiscale design with multidimensional image capture as explored in TOMBO [13

13. R. Horisaki, S. Irie, Y. Ogura, and J. Tanida, “Three-dimensional information acquisition using a compound imaging system,” Opt. Rev. 14, 347–350 (2007). [CrossRef]

], plenoptic [14

14. E. H. Adelson and J. Y. Wang, “Single lens stereo with a plenoptic camera,” IEEE Trans. Pattern Anal. Mach. Intel. 14, 99–106 (1992). [CrossRef]

, 15

15. M. Levoy, R. Ng, A. Adams, M. Footer, and M. Horowitz, “Light field microscopy,” ACM Trans. Graphics 25, 924–934 (2006). [CrossRef]

] and integral imaging systems[16

16. H.-J. Lee, D.-H. Shin, H. Yoo, J.-J. Lee, and E.-S. Kim, “Computational integral imaging reconstruction scheme of far 3D objects by additional use of an imaging lens,” Opt. Comm. 281, 2026–2032 (2007). [CrossRef]

].

• Detector mosaicking. Multiscale design utilizes multiple discrete focal plane arrays to synthesize a single image without stitching and field uniformity issues encountered in conventional design.

The design example given in Sec. 3 below attempts to illustrate improvement in the first two metrics.

Considerable recent work focuses on plenoptic or lightfield cameras [15

15. M. Levoy, R. Ng, A. Adams, M. Footer, and M. Horowitz, “Light field microscopy,” ACM Trans. Graphics 25, 924–934 (2006). [CrossRef]

, 14

14. E. H. Adelson and J. Y. Wang, “Single lens stereo with a plenoptic camera,” IEEE Trans. Pattern Anal. Mach. Intel. 14, 99–106 (1992). [CrossRef]

], which measure the radiance field by placing a large array of lenslets at the conventional image plane. The detector array is displaced away from this image plane by the focal length of the lenslets, such that each pixel on the FPA encodes not just the field angle of a ray entering the camera but also its pupil coordinate. The basic goal of the plenoptic camera is to measure the radiance at a spatial resolution corresponding to the lenslet spacing, and provides only an inefficient means of correcting aberrations. Similarly, integral imaging systems use arrays of microlenses to form multiple images of a scene at slightly different perspectives to perform 3D imaging. Recent work by Lee et al. [16

16. H.-J. Lee, D.-H. Shin, H. Yoo, J.-J. Lee, and E.-S. Kim, “Computational integral imaging reconstruction scheme of far 3D objects by additional use of an imaging lens,” Opt. Comm. 281, 2026–2032 (2007). [CrossRef]

] has included a macro-optic imaging lens placed before the micro-lens array in order to increase the field-of-view of the integral imaging system.

Integral field spectrometers [19

19. R. Bacon, P. Y. Copin, G. Monnet, B. W. Miller, J. R. Allington-Smith, M. Bureau, C. M. Carollo, R. L. Davies, E. Emsellem, H. Kuntschner, R. F. Peletier, E. K. Verolme, and P. T. de Zeeuw, “The SAURON project—I. The panoramic integral-field spectrograph,” Monthly Notices of the Royal Astronomical Society 326, 23–35 (2001). [CrossRef]

] use either an array of lenslets or of “image slicers” to integrate the light within subregions of the image, while each integrated signal is then dispersed by a prism or grating in a geometry specially designed to prevent crosstalk. An integral field spectrometer treats each lenslet field as a single spatial pixel, and the lenslet is used to concentrate light for spectral analysis. As with detector array microlenses, no imaging or aberration processing is implemented in the subapertures.

2. Scale and merit functions

The Shannon number for a lens may be equivalently evaluated at various planes, such as the entrance pupil, the exit pupil, or the image plane. The Shannon number of the aperture at the entrance pupil, for example, is the product of the area of the pupil and the area of the spatial bandpass of the field. The pupil area is π(A/2)2 for pupil diameter A. Letting FOV represent the full angular field of view of the system, the maximum spatial frequency in the coherent field illuminating the aperture is 1λ. The effective maximum spatial frequency for incoherent imaging is twice this value, so that the Shannon number for a circular aperture imaging an incoherent object is

S=π(A2)2π[2λsin(FOV2)]2=π2A2λ2sin2(FOV2).
(1)

Precise evaluation of the Shannon number is challenging for specific lens designs. For simplicity, we choose to evaluate lenses using “the number of resolvable spots,” which we define as the ratio of image area to the impulse response spotsize. Since the spotsize is generally shift variant, the number of resolvable spots must be evaluated by local integration of this ratio over the field. For a diffraction limited lens, the spotsize diameter is approximately shift-invariant

Table 1. Shannon number (in megapixels) as a function of FOV and aperture.

table-icon
View This Table

and equal to σ=2.44λ f# for small field angles, where f # is the f-number of the lens. The diameter of the image is roughly 2F tan(FOV/2) and the number of resolvable spots is therefore

Nmax=0.67A2λ2tan2(FOV2)
(2)

Comparing Eqns. (1) and (2), we see that N max is approximately one order of magnitude less than S. The information efficiency of a lens may be equivalently defined as the ratio of the number of resolvable spots or the ratio of the Shannon number achieved by the lens to the corresponding diffraction limited figure. For a lens achieving N resolvable spots, substituting from Eqn. (2) yields

γ=Nλ20.67A2tan2(FOV2)
(3)

Given diverse definitions of spotsize, the value of γ under this definition may differ somewhat from the Shannon number based value. Of course, we expect γ≤1. In practice, γ decreases monotonically in A because N does not grow in proportion to A 2.

To understand why achieved values of γ fall as a function of aperture size, consider the ray tracing diagram for a simple lens shown in Fig. 3. Ray tracing is, of course, commonly used to analyze geometric aberrations. In this case, the on-axis ray bundle comes to reasonable focus, but the off-axis rays are severely blurred due to aberrations. For present purposes, the most interesting feature of the diagram is that it is scale invariant, meaning that the drawing is independent of the spatial units used. If we take a given design and multiply all length parameters by M (i.e. scale the lens by the factor M), we can use the scale parameter to explore the balance between the diffraction-limited (small M) and aberration-limited (large M) regimes of image quality.

For a given lens system, geometric ray tracing can be used to estimate an aberration spotsize ξ. If the aberration spotsize is smaller than the diffraction limited radius, then the lens achieves diffraction-limited performance. Otherwise, geometric aberration dominates. The important point is that the scale invariance of the ray tracing diagram indicates that the size of the geometric aberration scales linearly in M. The diffraction-limited spotsize, in contrast, is scale-invariant.

The composite resolvable element blur radius varies with scale as

δ2+ξ2=32λ2f#2+M2ξ12,,

where ξ1 represents the mean aberration spotsize radius for the default scale, M=1. Defining D 1 to be the image diameter on the default scale, the number of resolvable elements varies with M as

NM2D124[32λ2f#2+M2ξ12],
(4)

At the two extremes of scale, this expression takes on the asymptotic forms

Msmall:N~M2D126λ2f#2,Mlarge:N~D124ξ12

At small scales, diffraction-limited performance yields a pixel count proportional to the Shannon number. At larger scales, geometric aberration makes the pixel count constant with respect to lens size for a given design. The information efficiency as a function of scale is found by substituting Eqn. (4) into Eqn. (3)

γ=11+M2(ξ11.22λf#)2.
(5)
Fig. 3. Ray tracing for a simple lens, and the corresponding spot diagrams for the image evaluated at three field angles. The RMS spotsizes (in units of microns) are given below each diagram.

Figure 4 plots γ as a function of M for three different values for ξ1. For ξ1=0, geometric aberration is eliminated and the system achieves γ=1 at all scales. For ξ1>0, diffraction limited performance is achieved for M<λ f #1, but beyond this limit γ decreases in inverse proportion to M 2 and the number of pixels in the image becomes scale invariant.

To achieve some advantage for increasing aperture size, conventional designs must prevent Mξ1/(λ f #) from increasing linearly in M. Combinations of the following strategies are chosen to decrease ξ1/(λ f #) as M increases:

1. One may reduce the field of view (and thus reduce the image size MD 1). Geometric aberration generally increases with field angle, so that reducing FOV allows the existing degrees of freedom in the design to better compensate for the aberrations at remaining field angles.

2. One may increase lens complexity with scale, adding more optical surfaces to help reduce the growing aberrations. This approach is essential in lithographic systems, where the goal is to simultaneously maximize image field size and minimize f #. The extraordinary growth in lithographic lens mass and complexity to achieve this objective is well documented [4

4. T. Matsuyama, Y. Ohmura, and D. M. Williamson, “The lithographic lens: its history and evolution,” in Optical Microlithography XIX, D. G. Flagello, ed., vol. 6154 of Proc. SPIE (2006).

].

3. One may increase f # as a function of M. This approach reduces geometric aberration while also increasing the image space diffractive blur size. As illustrated in Fig. 5, Lohmann’s empirical rule [3

3. A. W. Lohmann, “Scaling laws for lens systems,” Appl. Opt. 28, 4996–4998 (1989). [CrossRef] [PubMed]

] that f #M 1/3 describes this f-number scaling in many systems, with the notable exception of telescopes, which adopt strategy (1) and lithographic lenses which adopt strategy (2).

Fig. 4. Information efficiency γ vs. scale M for ξ1/(1.22λ f #)=0,0.25,0.5. While the information efficiency of an unaberrated system is scale-invariant, the aberrated systems’ efficiency declines sharply with M above the default scale (M=1).

Multiscale design is a fourth strategy for reducing the growth in Mξ1/(λ f #) as M grows. Its basic advantage derives from the potential to overcome the information capacity saturation effect illustrated in Fig. 4. In a multi-aperture camera, the number of resolvable spots may be expected to increase linearly with the number of apertures. For fixed subaperture size, the scaling parameter is proportional to the square root of the number of apertures and the system information capacity scales in proportion to M 2. Of course, this presupposes that the information collected by each subaperture is independent. This assumption may be valid in “close imaging” applications, such as document scanning and microscopy, but is not valid when subaperture fields of view substantially overlap. The delivery of nonredundant views of distant objects to a multi-aperture array is the purpose of the collector element in a multiscale design.

Specification of the processor element aperture size is the first challenge of multiscale design. If the only role of the processor element is to relay the collector image onto a detector array, then one should choose the minimum possible lenslet aperture size. This is, in fact, the strategy selected for current focal plane microlenses. In multiscale design, however, the lenslets must correct collector optical aberrations in addition to relaying the image. Just as a large aperture introduces greater aberration, wavefront correction capacity grows with aperture size. The wavefront correction capacity may be quantified as the maximum phase delay one can achieve between different points on the aperture. In general, the optimal processor aperture is the largest aperture over which one can achieve near diffraction-limited imaging given lens manufacturing and integration constraints. Current designs suggest subaperture sizes of 100–1000λ may be optimal.

Fig. 5. F-number scaling for (1) photographic cameras, (2) the giant Magellan telescope [20], (3) the Keck telescope (using the HIRES instrument) [21], (4) 1988 VLSI lithography lens, (5) 2004 lithography lens, (6) aerial photography lens, (7) 10×microscope objective, (8) 60×microscope objective. Lenses lying well above the Lohmann line indicate that other design tradeoffs have been made, either in system complexity or reduced field of view.

3. Design principles and examples

The function of the objective lens is to collect light and present an coarse image to the processor subsystem. The processor is placed behind the coarse image plane and simultaneously performs the functions of correcting aberration and of relaying the image onto a mosaic of detector arrays. Using a lenslet array for performing these functions has several advantages: (1) they can operate at their ideal scale to minimize aberrations generated during the image relay; (2) they are much more compact than full-aperture relay optics would be; and (3) dividing the field of view into smaller subregions allows us to relax some design constraints.

The optical design of a multiscale imager starts with the collector optics used together with an idealized (aberration-free and zero-thickness) lens where the processor lens array will later be placed. This allows all of the appropriate distances and element powers to be defined, giving a first-order system design. Given a conventional collector objective lens consisting of rotationally symmetric surfaces, we can express the aberration function in terms of the fourth-order wavefront aberration coefficients by the polynomial expression

W(H,ρ,ϕ)=W040ρ4+W131Hρ3cosϕ+W222H2ρ2cos2ϕ+W220H2cos2ϕ+W311H3ρcosϕ,
(6)

where H is the normalized field angle, ρ the normalized pupil radius, φ the azimuth angle of the pupil coordinate, and Wijk the wavefront aberration coefficient expressed in units of length.

The goal of lens design is to minimize W(H,ρ,ϕ) at targeted field points. Multiscale design deviates from conventional design in that the goal is to correct local rather than global wavefront error. Since the lenslets divide the field into subregions, each lenslet can be used to correct the local aberration centered around its appropriate field angle. In pursuit of this objective, we expand Eqn. (6) in terms of the central field angle Hn for the nth lenslet:

W(HHn,ρ,ϕ)=W040ρ4+W131(HHn)ρcosϕ+W222(HHn)2ρ2cos2ϕ
+W220(HHn)2ρ2+W311(HHn)3ρcosϕ
=[W040ρ4sphericalaberration]
+[W131Hnρ3cosϕconstantcoma+W131Hρ3cosϕlinearcoma]
+[W222Hn2ρ2cos2ϕconstantastigmatism2W222HHnρ2cos2ϕlinearastigmatism+W222H2ρ2cos2ϕquadraticastigmatism]
+[W220Hn2ρ2defocus2W220HHnρ2lineardefocus+W220H2ρ2quadraticdefocus(i.e.FC)]
+[W311Hn3ρcosϕfielddisplacement+3W311HHn2ρcosϕtilt(i.e.magnificationerror)3W311H2Hnρcosϕquadraticdistortion+W311H3ρcosϕcubicdistortion]
(7)

Expressing the aberration function localized about a given lenslet’s central field angle thus produces a number of aberration terms which are not of Seidel form. The important result of this expansion, however, is that the aberrations with high-order field dependence are greatly reduced, having much of their wavefront error shifted into lower-order terms, which are easier to correct if freeform surfaces are allowed. An example of this is shown in Fig. 6 for the case of field curvature.

One item to note when designing corrective optics placed near an image is that care must be taken to design the collector, and to select the positions of the lenslets, such that any caustics are kept away from the lenslet surfaces. This is a consequence of the well-known property that an optical surface placed at a caustic has no ability to correct the bundle of rays near the caustic. The primary consequences are that spherical aberration, and to some extent coma, must be minimized, since these aberrations possess extended caustic surfaces.

As an example we consider a simple f/8 Wollaston landscape lens as the collector. All of the systems were modeled with Zemax optical design software [22] using the visible d wavelength (λ=587 nm), and the freeform lenslet surface parameters are taken to be x-y polynomials of up to sixth order. For the conventional designs, the merit function was chosen to minimize the spotsize at field angles 0°, 20°, and 30°. For the multiscale designs, in order to suppress spherical aberration, and to a lesser extent coma, the merit function weights the field angles with weights 100, 1, and 0.1 respectively. For the design of the lenslets themselves, the merit function is naturally more complicated. In a freeform surface design, one may easily obtain very small spotsizes at the designed field angles while small deviations from the design angles show large spotsizes. (This is sometimes called “overdesigning” the lens.) To prevent this behavior from degrading the lenslet designs, a Cartesian grid of 3×3 field angles is chosen within the angles corresponding to the given lenslet. In practice, the curved multiscale design (Fig. 9(d)) used a subfield of ±1.75° for each lenslet, so that the field angles used to define the merit function were a grid of ±1.25° centered on the lenslet chief ray (see Fig. 8). Additionally, the optimization iterations were truncated early in order to prevent overdesign at those 9 positions.

Fig. 6. A wavefront having 10λ of field curvature (FC) aberration is shown at left. For a processor consisting of a 3×3 lenslet array, the aberration expressed locally across each lenslet is shown on the right hand side: the off-axis lenslets see a large defocus (4.9λ) and linear variation in defocus across the field (4.6λ), but only a small conventional field curvature (0.7λ).

In order to compare the multiscale approach with existing conventional design methods, we analyze the average RMS spotsize as a function of field angle for both designs, using either a planar or a curved focal surface. The conventional design trades off defocus with field curvature, blurring the axial field in order to improve the image at higher field angles. Introducing a curved focal surface (here assumed to be a sphere) greatly improves the image spotsizes (see Fig. 10) relative to the flat focal plane, but the system is still unable to achieve diffraction-limited performance primarily due to the presence of spherical aberration.

In the two multiscale systems, the design of the collector lens has been modified by shifting the stop and bending the lens in order to minimize spherical aberration and to reduce coma to a tolerable level. This has the effect of increasing field curvature and astigmatism aberration, but these are easy to correct using the multiscale approach. The planar multiscale design consists of an array of 11×11 lenslets, of diameters 2, 2, 3, 4, 4, and 6 mm (going from the axial field towards higher field angles). The field of view extends to ±32°, above which the lenslets have difficulty correcting the large aberrations. The lenslets are placed 10 mm behind the nominal image plane and re-image the scene with a magnification of m=-0.9. (This demagnification increases with field angle, reaching m=-0.4 at the highest field angles, due to the inward curving focal surface.) Since this design maps the final image onto a plane, it is compatible for use with either a monolithic detector array or with multiple small arrays fixed behind each lenslet. In both cases the various sub-images will still require post-processing in order to properly register and fuse them, while correcting for varying magnification and distortion as well as vignetting effects.

Fig. 7. (a) An f/8 objective lens, used with (b) a spherical focal surface, and (c) its corresponding planar and (d) curved multiscale designs. The entrance pupil diameter for each design is 8 mm. In (c) and (d), only those lenslets mapping to positive field angles in the meridional plane are shown for clarity, and each lenslet is labeled with the field angle of its corresponding chief ray. In (c) rays from three field angles are shown for each lenslet, whereas in (d) only the chief ray for each lenslet is drawn for clarity. The locally linear multiscale design is depicted with a series of small detector arrays placed near the focal sphere in (d).
Fig. 8. The “subfield” of a given lenslet, where the central point is the lenslet chief ray, and the 3×3 Cartesian grid of distributed points indicate the field angles on which the merit function is defined.
Fig. 9. A closeup of three field angles (32.5°±1.25°) incident on the 10th lenslet of the curved multiscale design. The tangential, medial, and sagittal focal surfaces are shown together with the ideal location of a local subimage (where one could place a small detector array). Only rays in the meridional plane are shown here.

The curved multiscale design is a more complex but also more powerful approach. Fixing a single small detector array behind each lenslet and allowing the lenslet-detector pair to be placed at any desired position and angle within the focal volume provides extraordinary design flexibility, and an appearance much like an inside-out fly’s eye. One way of interpreting this system layout is the following. If we have a set of small detector arrays, we can place each of them along the medial focal surface as a kind of piecewise linear approximation to the ideal curved detector, as in Fig. 7(d). (A closeup is shown in Fig. 9 for the 32.5° field angle lenslet.) However, this piecewise detector setup would inevitably result in gaps at some field angles and would not be able to correct for other aberrations. If we place ideal lenses behind these subimages, re-imaging with demagnification would allow for recovery of the gaps between the detected subimages but would not produce aberration correction. Using freeform lenslets adds the ability to correct aberrations. In the resulting layout, the lenslets and small detector arrays are in general perpendicular to the corresponding chief rays, but at higher field angles the local field tilt becomes significant (as shown in Fig. 9), and the corresponding detector plane must be tilted in the opposite sense to satisfy the Scheimpflug condition.

The RMS geometric spotsize as a function of field angle for each of the designs is shown in Fig. 10. These spotsizes are calculated by tracing 471 rays from each sampled field angle into the pupil with an 8-ring hexapolar sampling pattern. The RMS radial distance of the ray intercepts at the image plane from their centroid give the spotsize radius. From Fig. 10, we see that although having a curved focal surface greatly improves the conventional planar design, it cannot achieve diffraction-limited performance with such a simple objective lens. The curved multiscale design, however, is able to maintain a spotsize near the diffraction limit until it begins to slowly increase at higher field angles (>25°).

Fig. 10. The RMS geometric spotsize of each design as a function of field angle, sampled every 0.25°. The five designs compared are: the conventional lens with (“conv”) planar or (“conv. sphere”) spherical focal surface; and the multiscale designs using (“multi-local”) only a series of small detector arrays placed in 3D, (“multi-planar”) a planar arrangement processor array, and (“multi-sphere”) a curved focal surface arrangement. “Multi-local” refers to the technique of tiling a series of small detector arrays along the focal surface.

The number N of resolvable spots for each design is estimated by dividing up the field in 0.25° increments, and a Zemax macro was used to locate the image height corresponding to each angular position. From each of these we form a corresponding annulus on the image plane, whose area divided by the corresponding average spotsize gives a rough estimate of the number of resolvable points. For the planar designs, the area of the diffraction-limited Airy disk is approximately given by π(1.22λ f #)2/cos4 θ [23

23. M. V. R. K. Murty, “On the theoretical limit of resolution,” J. Opt. Soc. Am. 47, 667–668 (1957). [CrossRef]

] for image space field angle θ, whereas for the curved designs cos3 θ appears in the denominator instead, since these designs lack the extra image plane obliquity factor. Fig. 11 shows the result of these calculations at each field angle. Summing the resolvable spots in each annulus, with the combined result summing the geometric and diffraction spotsizes in quadrature, gives the total number of resolvable elements for each system:

megapixels Design type GeometricDiff-limitCombined
conventional, planar0.544.40.5
conventional, spherical8.840.38.6
multiscale, locally linear8.739.98.5
multiscale, planar11.442.39.6
multiscale, curved24.039.719.9

For an 8 mm aperture over a ±36° FOV, the theoretical limit on the number of resolvable elements (using Eqn 2) is 44 Mpix, so that the curved multiscale design shown here corresponds to γ=0.45. Note that the diffraction-limited performances of the designs vary due to differences in image size and geometry. The planar multiscale design shows a more erratic performance in spotsize vs. field angle, largely due to the fact that the collector lens chosen is unsuited to the design. A collector lens which is more nearly telecentric in image space can achieve better performance. While the curved multiscale design is able to outperform the conventional planar design by a factor of 40, and the conventional design using a spherical focal surface by a factor of 2, it is a much more complicated optical arrangement. Using a more complex objective lens can of course greatly improve the performance of the conventional design, but the aim of the multiscale approach is to show an alternative means of avoiding the complexity limit in design.

Fig. 11. The estimated number N of resolvable spots for each design per unit field angle.

The Zemax design files giving prescriptions for the lens designs discussed here have been posted at the url http://www.disp.duke.edu/projects/multiscale. Interested readers are encouraged to download the files and use them freely.

4. Conclusion

Multiscale lens design presents a different way of thinking about how optical design can be used for image formation. Current methods use optical elements which operate only on the full optical field, resulting in some of the familiar trade-offs in field-of-view and f-number. Multiscale design chooses to process the field-dependent aberrations locally rather than globally, thereby easing design constraints. Moreover, it allows much of the image formation process to be done by lenses operating at their ideal scale, on the order of millimeters in diameter, rather than suboptimal scales. Not only can this allow for wider FOV imagers or improved resolution in wide field systems, but can also be used to reduce system volume, reduce cost, ease detector mosaicking, and allow for 3D imaging.

The resulting instrument design can be considered as a fusing of conventional and multiaperture strategies, the one used as a front-end and the other as a back-end. This allows one to take the advantages of each type — the light collection efficiency and resolving power of the conventional lens and the image processing capability of the multiaperture camera — without their deficits. The multiscale combination of the two provides a design methodology which can achieve system information capacities that scale linearly with aperture area.

The use of nonuniform lenslet arrays, especially ones using aspheric or freeform surfaces, brings up the question of manufacturability. The past two decades have seen huge advances in the state of the art for micro- and meso-optical lenses, thus the limitation appears to be less one of manufacturing than of the various optical design issues arising from integrating the micro-, meso-, and macro-optical elements into a single system. For example, if the required surface sag of the lenslets is too much, we can choose a higher index glass, we can split the lens and effectively double the available sag by use of doublet lenslets, or we can choose a multi-stage processor. Thus, for large surface sag and high curvature shapes, optical design issues such as surface reflection, polarization dependence, and scatter are likely to cause more of a problem than manufacturability.

There are a host of questions that arise by proposing this technique which have been only briefly mentioned, and we are currently exploring these questions. These include robust design techniques for determining optical lenslet shapes and maximizing lenslet placement tolerance, chromatic effects on the multiscale approach, specifying the constraints on collector design, and how to design lenslet baffles to prevent crosstalk between subimages. We are currently developing an experimental prototype to illustrate the technique and to help refine our ideas.

Acknowledgments

The authors thank Dr. Scott McCain of Applied Quantum Technologies for fruitful discussions on the multiscale design approach. This research was supported in part by a DARPA MTO seedling funded under AFOSR contract FA9550-06-1-0230.

References and links

1.

G. T. di Francia, “Degrees of freedom of an image,” J. Opt. Soc. Am. 59, 799–804 (1969). [CrossRef]

2.

J. Kopf, M. Uyttendaele, O. Deussen, and M. F. Cohen, “Capturing and viewing gigapixel images,” ACM Trans. Graphics 26, 93 (2007). [CrossRef]

3.

A. W. Lohmann, “Scaling laws for lens systems,” Appl. Opt. 28, 4996–4998 (1989). [CrossRef] [PubMed]

4.

T. Matsuyama, Y. Ohmura, and D. M. Williamson, “The lithographic lens: its history and evolution,” in Optical Microlithography XIX, D. G. Flagello, ed., vol. 6154 of Proc. SPIE (2006).

5.

R. Völkel, M. Eisner, and K. J. Weible, “Miniaturized imaging systems,” Microelectron. Eng. 67–68, 461–472 (2003).

6.

Y. Dagan, “Wafer-level optics enables low cost camera phones,” in Integrated Optics: Devices, Materials, and Technologies XIII, J.-E. Broquin and C. M. Greiner, eds., vol. 7218 of Proc. SPIE (2009).

7.

J. Tanida, T. Kumagai, K. Yamada, S. Miyatake, K. Ishida, T. Morimoto, N. Kondou, D. Miyazaki, and Y. Ichioka, “Thin observation module by bound optics (TOMBO): concept and experimental verification,” Appl. Opt. 40, 1806–1813 (2001). [CrossRef]

8.

M. Shankar, R. Willett, N. Pitsianis, T. Schulz, R. Gibbons, R. T. Kolste, J. Carriere, C. Chen, D. Prather, and D. Brady, “Thin infrared imaging systems through multichannel sampling,” Appl. Opt. 47, B1–B10 (2008). [CrossRef] [PubMed]

9.

T. Mirani, D. Rajan, M. P. Christensen, S. C. Douglas, and S. L. Wood, “Computational imaging systems: joint design and end-to-end optimality,” Appl. Opt. 47, B86–B103 (2008). [CrossRef] [PubMed]

10.

K. Choi and T. J. Schulz, “Signal-processing approaches for image-resolution restoration for TOMBO imagery,” Appl. Opt. 47, B104–B116 (2008). [CrossRef] [PubMed]

11.

A. V. Kanaev, D. A. Scribner, J. R. Ackerman, and E. F. Fleet, “Analysis and application of multiframe superresolution processing for conventional imaging systems and lenslet arrays,” Appl. Opt. 46, 4320–4328 (2007). [CrossRef] [PubMed]

12.

A. D. Portnoy, N. P. Pitsianis, X. Sun, and D. J. Brady, “Multichannel sampling schemes for optical imaging systems,” Appl. Opt. 47, B76–B85 (2008). [CrossRef] [PubMed]

13.

R. Horisaki, S. Irie, Y. Ogura, and J. Tanida, “Three-dimensional information acquisition using a compound imaging system,” Opt. Rev. 14, 347–350 (2007). [CrossRef]

14.

E. H. Adelson and J. Y. Wang, “Single lens stereo with a plenoptic camera,” IEEE Trans. Pattern Anal. Mach. Intel. 14, 99–106 (1992). [CrossRef]

15.

M. Levoy, R. Ng, A. Adams, M. Footer, and M. Horowitz, “Light field microscopy,” ACM Trans. Graphics 25, 924–934 (2006). [CrossRef]

16.

H.-J. Lee, D.-H. Shin, H. Yoo, J.-J. Lee, and E.-S. Kim, “Computational integral imaging reconstruction scheme of far 3D objects by additional use of an imaging lens,” Opt. Comm. 281, 2026–2032 (2007). [CrossRef]

17.

J. Duparré, P. Schreiber, A. Matthes, E. Pshenay-Severin, A. Bräuer, A. Tünnermann, R. Völkel, M. Eisner, and T. Scharf, “Microoptical telescope compound eye,” Opt. Express 13, 889–903 (2005). [CrossRef] [PubMed]

18.

J. A. Cox and B. S. Fritz, “Variable focal length micro lens array field curvature corrector,” (2003). US Patent 6556349.

19.

R. Bacon, P. Y. Copin, G. Monnet, B. W. Miller, J. R. Allington-Smith, M. Bureau, C. M. Carollo, R. L. Davies, E. Emsellem, H. Kuntschner, R. F. Peletier, E. K. Verolme, and P. T. de Zeeuw, “The SAURON project—I. The panoramic integral-field spectrograph,” Monthly Notices of the Royal Astronomical Society 326, 23–35 (2001). [CrossRef]

20.

URL http://www.gmto.org/codrfolder/GMT-ID-01467-Chapter 6 Optics.pdf/.

21.

URL http://www2.keck.hawaii.edu/inst/hires/.

22.

URL http://www.zemax.com.

23.

M. V. R. K. Murty, “On the theoretical limit of resolution,” J. Opt. Soc. Am. 47, 667–668 (1957). [CrossRef]

OCIS Codes
(100.0100) Image processing : Image processing
(110.0110) Imaging systems : Imaging systems
(220.1000) Optical design and fabrication : Aberration compensation
(220.3620) Optical design and fabrication : Lens system design
(110.1758) Imaging systems : Computational imaging

ToC Category:
Optical Design and Fabrication

History
Original Manuscript: March 20, 2009
Revised Manuscript: June 1, 2009
Manuscript Accepted: June 5, 2009
Published: June 10, 2009

Citation
David J. Brady and Nathan Hagen, "Multiscale lens design," Opt. Express 17, 10659-10674 (2009)
http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-17-13-10659


Sort:  Author  |  Year  |  Journal  |  Reset  

References

  1. G. T. di Francia, "Degrees of freedom of an image," J. Opt. Soc. Am. 59, 799-804 (1969). [CrossRef]
  2. J. Kopf, M. Uyttendaele, O. Deussen, and M. F. Cohen, "Capturing and viewing gigapixel images," ACM Trans. Graphics 26, 93 (2007). [CrossRef]
  3. A. W. Lohmann, "Scaling laws for lens systems," Appl. Opt. 28, 4996-4998 (1989). [CrossRef] [PubMed]
  4. T. Matsuyama, Y. Ohmura, and D. M. Williamson, "The lithographic lens: its history and evolution," in Optical Microlithography XIX, D. G. Flagello, ed., vol. 6154 of Proc. SPIE (2006).
  5. R. Völkel, M. Eisner, and K. J. Weible, "Miniaturized imaging systems," Microelectron. Eng. 67-68, 461-472 (2003).
  6. Y. Dagan, "Wafer-level optics enables low cost camera phones," in Integrated Optics: Devices, Materials, and Technologies XIII, J.-E. Broquin and C. M. Greiner, eds., vol. 7218 of Proc. SPIE (2009).
  7. J. Tanida, T. Kumagai, K. Yamada, S. Miyatake, K. Ishida, T. Morimoto, N. Kondou, D. Miyazaki, and Y. Ichioka, "Thin observation module by bound optics (TOMBO): concept and experimental verification," Appl. Opt. 40, 1806-1813 (2001). [CrossRef]
  8. M. Shankar, R. Willett, N. Pitsianis, T. Schulz, R. Gibbons, R. T. Kolste, J. Carriere, C. Chen, D. Prather, and D. Brady, "Thin infrared imaging systems through multichannel sampling," Appl. Opt. 47, B1-B10 (2008). [CrossRef] [PubMed]
  9. T. Mirani, D. Rajan, M. P. Christensen, S. C. Douglas, and S. L. Wood, "Computational imaging systems: joint design and end-to-end optimality," Appl. Opt. 47, B86-B103 (2008). [CrossRef] [PubMed]
  10. K. Choi and T. J. Schulz, "Signal-processing approaches for image-resolution restoration for TOMBO imagery," Appl. Opt. 47, B104-B116 (2008). [CrossRef] [PubMed]
  11. A. V. Kanaev, D. A. Scribner, J. R. Ackerman, and E. F. Fleet, "Analysis and application of multiframe superresolution processing for conventional imaging systems and lenslet arrays," Appl. Opt. 46, 4320-4328 (2007). [CrossRef] [PubMed]
  12. A. D. Portnoy, N. P. Pitsianis, X. Sun, and D. J. Brady, "Multichannel sampling schemes for optical imaging systems," Appl. Opt. 47, B76-B85 (2008). [CrossRef] [PubMed]
  13. R. Horisaki, S. Irie, Y. Ogura, and J. Tanida, "Three-dimensional information acquisition using a compound imaging system," Opt. Rev. 14, 347-350 (2007). [CrossRef]
  14. E. H. Adelson and J. Y. Wang, "Single lens stereo with a plenoptic camera," IEEE Trans. Pattern Anal. Mach. Intel. 14, 99-106 (1992). [CrossRef]
  15. M. Levoy, R. Ng, A. Adams, M. Footer, and M. Horowitz, "Light field microscopy," ACM Trans. Graphics 25, 924-934 (2006). [CrossRef]
  16. H.-J. Lee, D.-H. Shin, H. Yoo, J.-J. Lee, and E.-S. Kim, "Computational integral imaging reconstruction scheme of far 3D objects by additional use of an imaging lens," Opt. Comm. 281, 2026-2032 (2007). [CrossRef]
  17. J. Duparré, P. Schreiber, A. Matthes, E. Pshenay-Severin, A. Bräuer, A. Tünnermann, R. Völkel, M. Eisner, and T. Scharf, "Microoptical telescope compound eye," Opt. Express 13, 889-903 (2005). [CrossRef] [PubMed]
  18. J. A. Cox and B. S. Fritz, "Variable focal length micro lens array field curvature corrector," (2003). US Patent 6556349.
  19. R. Bacon, P. Y. Copin, G. Monnet, B. W. Miller, J. R. Allington-Smith, M. Bureau, C. M. Carollo, R. L. Davies, E. Emsellem, H. Kuntschner, R. F. Peletier, E. K. Verolme, and P. T. de Zeeuw, "The SAURON project—I. The panoramic integral-field spectrograph," Monthly Notices of the Royal Astronomical Society 326, 23-35 (2001). [CrossRef]
  20. http://www.gmto.org/codrfolder/GMT-ID-01467-Chapter 6 Optics.pdf/.
  21. http://www2.keck.hawaii.edu/inst/hires/.
  22. http://www.zemax.com.
  23. M. V. R. K. Murty, "On the theoretical limit of resolution," J. Opt. Soc. Am. 47, 667-668 (1957). [CrossRef]

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.


« Previous Article  |  Next Article »

OSA is a member of CrossRef.

CrossCheck Deposited