OSA's Digital Library

Optics Express

Optics Express

  • Editor: Andrew M. Weiner
  • Vol. 22, Iss. 11 — Jun. 2, 2014
  • pp: 13896–13903
« Show journal navigation

High-resolution optical see-through multi-focal-plane head-mounted display using freeform optics

Xinda Hu and Hong Hua  »View Author Affiliations


Optics Express, Vol. 22, Issue 11, pp. 13896-13903 (2014)
http://dx.doi.org/10.1364/OE.22.013896


View Full Text Article

Acrobat PDF (3921 KB)





Browse Journals / Lookup Meetings

Browse by Journal and Year


   


Lookup Conference Papers

Close Browse Journals / Lookup Meetings

Article Tools

Share
Citations

Abstract

Conventional stereoscopic displays force an unnatural decoupling of the accommodation and convergence cues, which may contribute to various visual artifacts and have adverse effects on depth perception accuracy. In this paper, we present the design and implementation of a high-resolution optical see-through multi-focal-plane head-mounted display enabled by state-of-the-art freeform optics. The prototype system is capable of rendering nearly-correct focus cues for a large volume of 3D space, extending into a depth range from 0 to 3 diopters. The freeform optics, consisting of a freeform prism eyepiece and a freeform lens, demonstrates an angular resolution of 1.8 arcminutes across a 40-degree diagonal field of view in the virtual display path while providing a 0.5 arcminutes angular resolution to the see-through view.

© 2014 Optical Society of America

1. Introduction

The multi-focal-plane (MFP) display is one of the technologies proposed to minimize and eliminate the accommodation and convergence conflict — a fundamental problem in conventional stereoscopic 3D displays [1

1. S. Suyama, S. Ohtsuka, H. Takada, K. Uehira, and S. Sakai, “Apparent 3-D image perceived from luminance-modulated two 2-D images displayed at different depths,” Vision Res. 44(8), 785–793 (2004). [CrossRef] [PubMed]

5

5. X. Hu and Hua, “Distinguished student paper: a depth-fused multi-focal-plane display prototype enabling focus cues in stereoscopic displays,” in SID Symposium Digest of Technical Papers (2011), Vol. 42, pp. 691–694. [CrossRef]

]. By presenting multiple image planes at different accommodation distances to the viewer and aligning them appropriately for each eye, three-dimensional scenes correctly rendered by MFP displays can correctly stimulate the eyes to accommodate at the same distances to which the eyes converge. The accommodation-convergence response when viewing MFP display is very similar to the one in viewing natural environments where the convergence cue and accommodation cue are matched. Therefore MFP displays have demonstrated great potentials in reducing perceptual distortions, improving visual performance and reducing visual discomfort and fatigue.

Many valuable works have advanced the MFP system designs as well as depth-rendering algorithms to improve image quality and to accurately cue eye accommodation within discrete focal planes [1

1. S. Suyama, S. Ohtsuka, H. Takada, K. Uehira, and S. Sakai, “Apparent 3-D image perceived from luminance-modulated two 2-D images displayed at different depths,” Vision Res. 44(8), 785–793 (2004). [CrossRef] [PubMed]

9

9. X. Hu and H. Hua, “Design and assessment of a depth-fused multi-focal-plane display prototype,” J. Display Technol. 10(4), 308–316 (2014). [CrossRef]

]. It is still technically challenging, however, to design a compact, high frame rate, and high performance MFP display with adequate number of focal planes. The challenge is further aggregated for the use of this technology in see-through head-mounted displays where viewers such as in medical and military applications can benefit most from accurate depth perception and improved visual performance.

Recently, we presented the implementation of a depth-fused six-focal-plane prototype system — the first system capable of displaying 6-focal-plane depth-fused 3D scene extending from 3 diopters to infinity at a flicker-free speed [5

5. X. Hu and Hua, “Distinguished student paper: a depth-fused multi-focal-plane display prototype enabling focus cues in stereoscopic displays,” in SID Symposium Digest of Technical Papers (2011), Vol. 42, pp. 691–694. [CrossRef]

, 9

9. X. Hu and H. Hua, “Design and assessment of a depth-fused multi-focal-plane display prototype,” J. Display Technol. 10(4), 308–316 (2014). [CrossRef]

]. However, built from off-the-shelf lenses, the preliminary prototype suffered from unsatisfactory image resolution and contrast, and most importantly it lacked the optical see-through capability, which significantly limited its usefulness in general 3D display application as well as fundamental depth-perception vision research.

2. Design approach

2.1 System functions and layout

The overall optical system layout of a monocular MFP display is shown in Fig. 1
Fig. 1 (a) top-view optical layout of the right-eye module of the MFP-HMD prototype. (b) detailed layout of the Image Generation Subsystem (IGS).
. Each monocular setup consists of two parts: the composite optical see-though eyepiece and the image generation subsystem (IGS). Figure 1(a) shows the optical layout of the right-eye setup looking from the top, where the freeform eyepiece folds the virtual display optical path to the right and the rotational symmetric IGS, illustrated in Fig. 1(b), stands vertically to the temple side of the human head. Rays from the center and edge field angles are drawn with different colors to show conjugated pupil and image locations more clearly.

The composite eyepiece based on freeform optics is the core component enabling the successful integration of multi-focal-plane display and OST-HMD system. It consists of a wedge-shaped freeform eyepiece, a freeform see-through compensator lens and a cylindrical lens. The freeform eyepiece, formed by three freeform surfaces S1, S2 and S3, serves as a single-element HMD eyepiece for magnifying the intermediate image created by the IGS and presenting the virtual content to the viewer. The light from the IGS propagates into the freeform eyepiece through S3, after total-internally reflected on S1 and partially reflected by the beamsplitter coating on S2, exits the eyepiece structure and enters the viewer’s eye. Unlike conventional HMD eyepiece with rotational symmetric optical surfaces, which requires components to be collinear with the optical axis and uses a separate beamsplitter to fold the system, the freeform prism eyepiece folds the light path within a single plastic element so that not only the system weight, size and complexity are greatly reduced, the system at the same time achieves a large field of view, high image quality and very desirable long eye-relief. The optical see-through capability of the system is enabled by cementing the freeform eyepiece with a compensator lens, formed by two freeform surfaces S2 and S4, to correct the undesirable aberrations and distortions introduced by the prism eyepiece. In this prototype, a cylindrical lens was further incorporated into the optical see-through path design, which helps to correct both aberration-induced blur and distortion of the real-world view to a negligible level.

The second part of the system, the IGS, with detailed optical layout in Fig. 1(b), achieves the core function of generating the multi-focal-plane contents. It consists of a high-speed digital micromirror device (DMD) microdisplay by Texas Instruments, a deformable membrane mirror device (DMMD) and relay lens groups. The IGS is essentially a zoomed relay system based on an active optical element (the deformable mirror) whose optical power can be electronically controlled at a high speed as fast as 1 kHz. The DMD microdisplay, illuminated by an RGB LED light source, is first magnified by two field lenses and then relayed by a double-pass double-telecentric lens group, forming an intermediate image between the IGS and the eyepiece The DMMD is placed at the stop of the relay lens group and its surface shape can change from a plane to a concave parabola when voltage is applied. The change of optical power on the DMMD rapidly changes the axial location of the relayed intermediate image and in turn creates a stack of addressable virtual focal planes with different distances to the viewer in the viewing space. The depth volume of the 3D scene appearing to the viewer and the spacing of the focal planes can be controlled by adjusting the voltage control on the DMMD. By synchronizing the voltage applied on the deformable mirror, the contents being rendered on microdisplay and the RGB LED illumination at a high speed, multiple focal planes are time-multiplexed and different virtual contents can be generated at their correct depths, thus forming a continuous true-3D space even with the monocular setup [5

5. X. Hu and Hua, “Distinguished student paper: a depth-fused multi-focal-plane display prototype enabling focus cues in stereoscopic displays,” in SID Symposium Digest of Technical Papers (2011), Vol. 42, pp. 691–694. [CrossRef]

].

2.2 System optical design

In the virtual display path, the design goal of a MFP display is to present to the viewer a large volume of high-quality 3D contents, rendered with correct accommodation cues. This requires the display to have a wide field of view as well as a large addressable depth range of the focal planes.

In this prototype, we’ve selected the virtual display field of view to be 40° diagonally, or 33° × 25° in the horizontal and vertical directions, respectively. The source of the virtual image is the 0.7-inch DMD microdisplay with a pixel resolution of 1024 × 768 (XGA) and a pixel size of 14µm. Therefore in the visual space, the virtual display equivalently affords an angular resolution of 1.8 arcminutes per pixel. As described in Section 2.1, a group of lenses relays the DMD microdisplay image to the freeform eyepiece. Since the double-pass double-telecentric relay group provides no magnification, to help reduce the optical power of the freeform eyepiece, i.e. its design difficulty, two field lenses were designed to pre-magnify the image as well as to correct aberrations (mainly lateral chromatic aberration).

On the other hand, in this prototype the focal-plane depth can be addressed from 33cm to infinity in front of the viewer, in a total range of 3 diopters. This depth range is optically magnified from the 10mm-diameter DMMD, which only has an adjustable optical power range of approximately 1.2 diopters. From the first-order design analysis in [9

9. X. Hu and H. Hua, “Design and assessment of a depth-fused multi-focal-plane display prototype,” J. Display Technol. 10(4), 308–316 (2014). [CrossRef]

], we have identified the trade-off between the addressable depth range and the exit pupil diameter (EPD). To magnify the focal-plane depth range to 3 diopters from DMMD’s 1.2 diopters, the focal lengths of the freeform eyepiece and the half relay-lens group (the two doublet lenses and a singlet lens close to DMMD) must maintain a ratio of 0.632 (feyepiece/fhalfrelay). This in return limits the maximum achievable non-vignetted EPD of the display to 6.3mm, since the DMMD is conjugated to the exit pupil in this pupil-forming display.

Taking into account many system design factors, such as the placement of IGS relative to the head, the desired long 25mm eye-clearance to accommodate most viewers wearing eyeglasses and the overall optical design difficulties, we chose the focal length of the freeform eyepiece (formed by S1-3) to be 27.5mm and a F-number of 4.3. The design form of this freeform prism eyepiece is well-known and its basic design methods can be found in many literatures such as Refs. [10

10. D. Cheng, Y. Wang, H. Hua, and M. M. Talha, “Design of an optical see-through head-mounted display with a low f-number and large field of view using a freeform prism,” Appl. Opt. 48(14), 2655–2668 (2009). [CrossRef] [PubMed]

] and [11

11. Q. Wang, D. Cheng, Y. Wang, H. Hua, and G. Jin, “Design, tolerance, and fabrication of an optical see-through head-mounted display with free-form surface elements,” Appl. Opt. 52(7), C88–C99 (2013). [CrossRef] [PubMed]

]. In this design, the three freeform surfaces are described by plane-symmetric 10th-order XY-polynomials and they are decentered and tilted to form the prism structure on PMMA plastic.

Besides typical requirements of wide FOV, large pupil and high image contrast, the optical design of the freeform eyepiece and the overall design of our MFP-HMD system face several unique challenges not in typical HMD designs where freeform eyepiece directly magnifies a microdisplay. The first challenge results from the coupling between the freeform optics with the rotational symmetric IGS. The advantages of freeform eyepiece came from breaking the system symmetry and folding the optical path with freeform surfaces. However this also introduces a substantial amount of undesired aberrations not presented in rotational-symmetric systems. Additionally, due the pupil-forming nature of the overall optical system and the fact that the IGS is mainly composed of off-the-shelf lenses, the freeform prism needs to be strategically optimized to match the pupil and to correct aberrations of the IGS, pushing the limits of the prism eyepiece structure. Secondly, the depth of focal-plane can be addressed within a large range of 3 diopters by shifting the intermediate image plane with the zoomed IGS. Therefore the through-focus performance of the optical system must be optimized to ensure high image quality on focal planes of any depth. Last but not the least, in order to properly apply depth-fusion technique [6

6. S. Liu and H. Hua, “A systematic method for designing depth-fused multi-focal plane three-dimensional displays,” Opt. Express 18(11), 11562–11573 (2010). [CrossRef] [PubMed]

9

9. X. Hu and H. Hua, “Design and assessment of a depth-fused multi-focal-plane display prototype,” J. Display Technol. 10(4), 308–316 (2014). [CrossRef]

] in MFP-displays which requires that the fusing pixels on different focal planes maintain constant angular magnification across the entire depth range of interest, the optical system needs to be optimized to ensure high telecentricity and low distortion variation across the 3 diopters of depth range.

In the optical see-through path, the freeform compensator (formed by S2 and S4, described with 10th-order XY-Polynomials) and an off-the-shelf cylindrical lens were designed together to achieve a see-through field of view of 50° × 45° in the horizontal and vertical directions, respectively. The see-through field of view is much larger than the virtual display to minimize the obstruction and “tunnel-feeling” to the real-world view. The central 40° of FOV, where virtual 3D contents can augment the real-world view, was optimized to have MTF values of about 50% at human normal visual acuity of 1 arcminute and minimal distortion so that the viewer’s vision of the real-world is not degraded. Although the see-through path does not have a confined eye box, the compensator optics was optimized for a large 10mm pupil to ensure high visual quality and viewing comfort with eye rotation.

2.3 Optical design results

The optical design results of the virtual display path and optical see-through path are presented separately in this section. Figure 2(a)
Fig. 2 (a) Full-field MTF plots of the virtual display; (b) distortion grid of the virtual display; (c) Through-focus performance across 3 diopters of accommodation range.
shows the full-field display of the polychromatic modulation transfer function (MTF) of the virtual display path evaluated with 3mm centered pupil for spatial frequencies of 20 lp/mm and 35 lp/mm, where 35 lp/mm is the Nyquist frequency of the DMD microdisplay. The sizes of the circles are proportional to the MTF values. Across the entire 40° of FOV the image contrast is very high and uniform, with an average MTF of 0.58 at 20 lp/mm and 0.36 at 35 lp/mm. Figure 2(b) shows the distortion grid of the virtual display. The maximum distortion of the virtual display is kept very low, less than 5% at the edge field of 40°. With very strict telecentricity of less than 0.15° at the intermediate image space, the maximum pixel shift across the focal planes is less than 0.25 pixels, making fusing multi-focal-plane images an ease. As shown in Fig. 2(c), the optical performance through the 3-diopter focus range was also very well balanced. Calculated from the average RMS spot sizes, it varies from + 3% to −8% across the 3 diopter range with respect to the nominal performance on dioptric mid-plane of 1.5 diopters.

Figure 3(a)
Fig. 3 (a) Full-field MTF plots of the see-through path; (b) distortion grid of the see-through view showing minimal distortion at the corners of the temporal side.
shows the MTF full-field plot of the see-through optical path with a 3mm centered pupil for the spatial frequency of 0.5 cycle/arcminute and 1.0 cycle/arcminute, which corresponds to an angular resolution of 1 arcminute and 0.5 arcminute in the visual space, respectively. The performance across the 40° is almost all diffraction-limited by the eye pupil. The see-though view has very high contrast, with an average MTF value of 0.49 at human normal visual acuity of 1 arcminute and 0.16 at 0.5 arc minute. Figure 3(b) shows the distortion grid of the see-through view. The distortion caused by the composite eyepiece is negligible with less than 2% pincushion at the edge fields of the temporal side.

3. System prototype

Figure 4(a)
Fig. 4 (a) the freeform composite eyepiece assembly; (b) 3D model of the as-built binocular bench prototype with part of the mechanical mountings moved to show the optical lenses.
shows the freeform composite eyepiece assembly in the mounting hardware. The freeform optics was prototyped by diamond-turning on PMMA plastic. The 3D geometry of the as-built binocular bench prototype is shown in Fig. 4(b). The mechanical mountings for the image generation subsystem of the right eye module are hidden to show the lens assembly more clearly.

Figure 6(a)
Fig. 6 (a) Camera captured see-through view of 40-degree resolution target; (b) Standard eye chart placed at 20ft seeing through the composite freeform eyepiece.
shows an image of the see-through view captured with a camera for a resolution target placed at approximately 75cm away, covering the FOV of approximately 40 degrees diagonally. The image of the resolution target demonstrated superb quality and negligible distortion. Figure 6(b) shows an image of the see-through view for viewing a standard Snellen eye chart placed at 20 feet, where Group 8 (20/20 vision) corresponds to the angular resolution of 1 arcminute and Group 11 (20/10 vision) corresponds to the angular resolution of 0.5 arcminute. The camera was configured to have an angular resolution of 0.23 arcminutes per pixel. From the image, all letters from Group 8 to Group 11 are readable and have very good contrast. These results clearly demonstrated that little degradation and negligible distortion on the real-world view is introduced by the see-through optics.

4. Conclusion

Acknowledgments

This work is funded by National Science Foundation Grant Awards 1115489 and 0915035.

References and links

1.

S. Suyama, S. Ohtsuka, H. Takada, K. Uehira, and S. Sakai, “Apparent 3-D image perceived from luminance-modulated two 2-D images displayed at different depths,” Vision Res. 44(8), 785–793 (2004). [CrossRef] [PubMed]

2.

K. Akeley, S. J. Watt, A. R. Girshick, and M. S. Banks, “A stereo display prototype with multiple focal distances,” ACM Trans. Graph. 23(3), 804–813 (2004). [CrossRef]

3.

G. D. Love, D. M. Hoffman, P. J. W. Hands, J. Gao, A. K. Kirby, and M. S. Banks, “High-speed switchable lens enables the development of a volumetric stereoscopic display,” Opt. Express 17(18), 15716–15725 (2009). [CrossRef] [PubMed]

4.

S. Liu, H. Hua, and D. Cheng, “A novel prototype for an optical see-through head-mounted display with addressable focus cues,” IEEE Trans. Vis. Comput. Graph. 16(3), 381–393 (2010). [CrossRef] [PubMed]

5.

X. Hu and Hua, “Distinguished student paper: a depth-fused multi-focal-plane display prototype enabling focus cues in stereoscopic displays,” in SID Symposium Digest of Technical Papers (2011), Vol. 42, pp. 691–694. [CrossRef]

6.

S. Liu and H. Hua, “A systematic method for designing depth-fused multi-focal plane three-dimensional displays,” Opt. Express 18(11), 11562–11573 (2010). [CrossRef] [PubMed]

7.

S. Ravikumar, K. Akeley, and M. S. Banks, “Creating effective focus cues in multi-plane 3D displays,” Opt. Express 19(21), 20940–20952 (2011). [CrossRef] [PubMed]

8.

K. J. MacKenzie, D. M. Hoffman, and S. J. Watt, “Accommodation to multiple-focal-plane displays: Implications for improving stereoscopic displays and for accommodation control,” J. Vis. 10(8), 22 (2010). [CrossRef] [PubMed]

9.

X. Hu and H. Hua, “Design and assessment of a depth-fused multi-focal-plane display prototype,” J. Display Technol. 10(4), 308–316 (2014). [CrossRef]

10.

D. Cheng, Y. Wang, H. Hua, and M. M. Talha, “Design of an optical see-through head-mounted display with a low f-number and large field of view using a freeform prism,” Appl. Opt. 48(14), 2655–2668 (2009). [CrossRef] [PubMed]

11.

Q. Wang, D. Cheng, Y. Wang, H. Hua, and G. Jin, “Design, tolerance, and fabrication of an optical see-through head-mounted display with free-form surface elements,” Appl. Opt. 52(7), C88–C99 (2013). [CrossRef] [PubMed]

OCIS Codes
(120.2040) Instrumentation, measurement, and metrology : Displays
(120.2820) Instrumentation, measurement, and metrology : Heads-up displays
(120.4570) Instrumentation, measurement, and metrology : Optical design of instruments
(220.1250) Optical design and fabrication : Aspherics
(220.3620) Optical design and fabrication : Lens system design
(330.7338) Vision, color, and visual optics : Visually coupled optical systems

ToC Category:
Geometric Optics

History
Original Manuscript: March 31, 2014
Revised Manuscript: May 20, 2014
Manuscript Accepted: May 21, 2014
Published: May 30, 2014

Citation
Xinda Hu and Hong Hua, "High-resolution optical see-through multi-focal-plane head-mounted display using freeform optics," Opt. Express 22, 13896-13903 (2014)
http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-22-11-13896


Sort:  Author  |  Year  |  Journal  |  Reset  

References

  1. S. Suyama, S. Ohtsuka, H. Takada, K. Uehira, S. Sakai, “Apparent 3-D image perceived from luminance-modulated two 2-D images displayed at different depths,” Vision Res. 44(8), 785–793 (2004). [CrossRef] [PubMed]
  2. K. Akeley, S. J. Watt, A. R. Girshick, M. S. Banks, “A stereo display prototype with multiple focal distances,” ACM Trans. Graph. 23(3), 804–813 (2004). [CrossRef]
  3. G. D. Love, D. M. Hoffman, P. J. W. Hands, J. Gao, A. K. Kirby, M. S. Banks, “High-speed switchable lens enables the development of a volumetric stereoscopic display,” Opt. Express 17(18), 15716–15725 (2009). [CrossRef] [PubMed]
  4. S. Liu, H. Hua, D. Cheng, “A novel prototype for an optical see-through head-mounted display with addressable focus cues,” IEEE Trans. Vis. Comput. Graph. 16(3), 381–393 (2010). [CrossRef] [PubMed]
  5. X. Hu, Hua, “Distinguished student paper: a depth-fused multi-focal-plane display prototype enabling focus cues in stereoscopic displays,” in SID Symposium Digest of Technical Papers (2011), Vol. 42, pp. 691–694. [CrossRef]
  6. S. Liu, H. Hua, “A systematic method for designing depth-fused multi-focal plane three-dimensional displays,” Opt. Express 18(11), 11562–11573 (2010). [CrossRef] [PubMed]
  7. S. Ravikumar, K. Akeley, M. S. Banks, “Creating effective focus cues in multi-plane 3D displays,” Opt. Express 19(21), 20940–20952 (2011). [CrossRef] [PubMed]
  8. K. J. MacKenzie, D. M. Hoffman, S. J. Watt, “Accommodation to multiple-focal-plane displays: Implications for improving stereoscopic displays and for accommodation control,” J. Vis. 10(8), 22 (2010). [CrossRef] [PubMed]
  9. X. Hu, H. Hua, “Design and assessment of a depth-fused multi-focal-plane display prototype,” J. Display Technol. 10(4), 308–316 (2014). [CrossRef]
  10. D. Cheng, Y. Wang, H. Hua, M. M. Talha, “Design of an optical see-through head-mounted display with a low f-number and large field of view using a freeform prism,” Appl. Opt. 48(14), 2655–2668 (2009). [CrossRef] [PubMed]
  11. Q. Wang, D. Cheng, Y. Wang, H. Hua, G. Jin, “Design, tolerance, and fabrication of an optical see-through head-mounted display with free-form surface elements,” Appl. Opt. 52(7), C88–C99 (2013). [CrossRef] [PubMed]

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.


« Previous Article  |  Next Article »

OSA is a member of CrossRef.

CrossCheck Deposited