Integral volumetric imaging with dual layer fly-eye lenses

doi:10.1364/OE.20.001963

« Show journal navigation

Integral volumetric imaging with dual layer fly-eye lenses

Hideki Kakeya, Shimpei Sawada, Yukio Ueda, and Tomoya Kurokawa »View Author Affiliations

Optics Express, Vol. 20, Issue 3, pp. 1963-1968 (2012)
http://dx.doi.org/10.1364/OE.20.001963

View Full Text Article

Acrobat PDF (1338 KB)

Journal Search
Article Lookup

Article Tools

Citations

Alert me when this article is cited
Export Citation/Save

Full Text
Article Info
References (12)
Cited By
Figures (7)
Multimedia (2)
Metrics

Abstract

This paper proposes an integral volumetric imaging system that uses a coarse fly-eye lens and a fine fly-eye lens to show smooth and deep 3D image. Conventional integral volumetric imaging displays using a coarse fly-eye lens have suffered from low image quality due to distinct seam of lenses and moiré pattern caused by layered panel structure. To solve these problems the proposed system uses a fine fly-eye lens whose elemental lens has a long focal distance. By placing a fine fly-eye lens near the layered real image, the seam and the moiré are removed while the degradation of the presented 3D image is kept small.

1. Introduction

Integral imaging [1

G. Lippmann, “La photograhie integrale,” C. R. Acad. Sci. 146, 446–451 (1908).

] is a 3D display system that reproduces light-ray space. When the pixel pitch of the display behind the fly-eye lens (convex lens array) is fine enough, it can express wide range of depth. In reality, however, the resolution of electronic display is limited, which makes it hard for electronic integral imaging to express deep light-ray space.

One way to solve this problem is to combine volumetric solution with integral imaging. The first proposal was given by Lee et al. [2

B. Lee, S. Jung, S.-W. Min, and J.-H. Park, “Three-dimensional display by use of integral photography with dynamically variable image planes,” Opt. Lett. 26(19), 1481–1482 (2001). [CrossRef] [PubMed]

]. This technology, however, is not a practical solution because it includes moving parts. To do away with moving parts, Kim et al. proposed integral imaging systems with multilayer screens [3

Y. Kim, J.-H. Park, H. Choi, J. Kim, S.-W. Cho, and B. Lee, “Depth-enhanced three-dimensional integral imaging by use of multilayered display devices,” Appl. Opt. 45(18), 4334–4343 (2006). [CrossRef] [PubMed]

Y. Kim, H. Choi, J. Kim, S.-W. Cho, Y. Kim, G. Park, and B. Lee, “Depth-enhanced integral imaging display system with electrically variable image planes using polymer-dispersed liquid-crystal layers,” Appl. Opt. 46(18), 3766–3773 (2007). [CrossRef] [PubMed]

]. In these systems multilayer translucent display panel or screens are used in place of a single display panel behind the fly-eye lens sheet. These systems, however, have a couple of problems. Firstly the images from adjacent elemental lenses are not smoothly connected because the distortion of image due to refraction is not taken into account. Secondly these systems cannot show objects with large depth because connectivity of the images from different depths is not considered. Therefore these systems, though multilayered, do not meet up with the requirement of volumetric displays for general use.

To solve the problem of connectivity between adjacent elemental images and adjacent layered panels, Kakeya proposed coarse integral volumetric imaging (CIVI), where large aperture lens is added to decrease aberration [5

H. Kakeya, “Coarse integral imaging and its applications,” Proc. SPIE 6803, 680317 (2008). [CrossRef]

] and depth-fused 3D (DFD) [6

S. Suyama, H. Takada, K. Uehira, S. Sakai, and S. Ohtsuka, “A Novel Direct-Vision 3-D Display using Luminance-Modulated Two 2-D Images Displayed at Different Depths,” SID Int. Symp. Digest Tech. Papers 31(1), 1208–1211 (2000). [CrossRef]

S. Suyama, H. Takada, and S. Ohtsuka, “A Direct-Vision 3-D Display Using a New Depth-fusing Perceptual Phenomenon in 2-D Displays with Different Depths,” IEICE Trans. Electron. E85-C(11), 1911–1915 (2002).

] for curved image planes is applied to realize smooth image connection [8

H. Kakeya, “Improving image quality of coarse integral volumetric display,” Proc. SPIE 7237, 723726 (2009). [CrossRef]

H. Kakeya, “Realization of undistorted volumetric multiview image with multilayered integral imaging,” Opt. Express 19(21), 20395–20404 (2011). [CrossRef] [PubMed]

]. Also brightness reduction due to multilayer structure can be overcome by layering monochrome and color panels to emulate color volumetric image [8

H. Kakeya, “Improving image quality of coarse integral volumetric display,” Proc. SPIE 7237, 723726 (2009). [CrossRef]

H. Kakeya, “Realization of undistorted volumetric multiview image with multilayered integral imaging,” Opt. Express 19(21), 20395–20404 (2011). [CrossRef] [PubMed]

] and removing polarizing filters between monochrome panels [10

Y. Ueda and H. Kakeya, “Multilayered Integral Imaging with Improved Image Clarity,” SID Int. Symp. Digest Tech. Papers 42(1), 1116–1119 (2011). [CrossRef]

]. Aberration is further decreased by compensating not only field curvature but also barrel distortion and color aberration using texture mapping technique [11

H. Kodaira, A. Nakao, Y. Mano, and H. Kakeya, “Distortion correction of coarse integral volumetric imaging with large lens,” Proc. of 3DSA (2010), pp. 205–208.

,12

S. Sawada and H. Kakeya, “Coarse integral volumetric imaging with flat screen and wide viewing angle,” Proc. SPIE 7863, 78631L (2011). [CrossRef]

CIVI system, however, has two problems related to the image quality. The first problem is the distinct seam of the lenses because of the coarse fly-eye lens. Though the seam can be obscured when a large aperture Fresnel lens is placed far ahead the fly-eye lens, it leads to discrete motion parallax, which makes the view unnatural when the viewer moves his or her head. The second problem is the moiré pattern that appears due to the layering structure of LCD (liquid crystal display) panels with limited pixel aperture.

In this paper the authors propose an integral volumetric imaging (IVI) system with dual layer fly-eye lenses to attain moiré-free 3D image with smooth motion parallax. This paper is organized as follows. In Section 2 the conventional integral imaging with fine fly-eye lens and the coarse integral volumetric imaging are explained and compared. In Section 3 integral volumetric imaging system with dual layer fly-eye lenses is introduced and the mechanism and the merit of the proposed system are elucidated. In Section 4 the experimental system based on the proposed method is described. In Section 5 we conclude this paper.

2. Integral imaging with fine and coarse fly-eye lenses

Electronic integral imaging is composed of an electronic display panel and a fly-eye lens whose elemental lens covers multiple pixels of the display panel. Integral imaging with fine fly-eye lens is often called a 3D display based on light-ray reproduction. When the interval between the display panel and the fly-eye lens is the same as the focal distance of the elemental lens, parallel light is emitted from each elemental lens. Each pixel value on the display panel expresses the color of the light ray that goes through the pixel and the center of the elemental lens. When the pixel pitch of the display panel is fine enough, integral imaging can express many directional light rays. In reality, however, the pixel pitch of the electronic display is not fine enough to reproduce dense light-ray space. When the pixel pitch of the display is coarse, the resolution of the image becomes extremely low when the presented image is far from the surface of fly-eye lens as shown in Fig. 1 .

Fig. 1 Comparison between conventional integral imaging and coarse integral imaging.

One way to express deep space with low resolution display panels is to layer the panels to show volumetric elemental images. When each display panel is not thin enough, however, the resolutions and the viewing angles of the observed images from the front layer panel and the back layer panel become quite different from each other. Standard LCD panels are about 2 mm thick, which is close to the size of elemental lenses constituting the fly-eye lens used for the conventional integral imaging.

To solve this problem, the size of the elemental lens is enlarged in CIVI. When the light is collimated by the elemental lens so that each elemental lens may represent one pixel, the resolution of the image presented by the coarse fly-eye lens becomes low. To increase the resolution of the presented image, the lens should be set so that multiple pixels may be observed through each elemental lens. In this setting, real images or virtual images are formed with the lenses. CIVI presents undistorted 3D image by showing elemental images distorted in the reverse direction on the display panel. Figure 2(a) shows an example of CIVI display system setup. Here a large aperture lens is added to generate a real image in front of the viewer.

Fig. 2 Optical setup of CIVI.

Though the optical distortion can be corrected on the software basis, CIVI system has two hardware problems related to the image quality. The first problem is the distinct seam of lenses because of the coarse fly-eye lens. The seam can be obscured when a large aperture Fresnel lens is placed far from the fly-eye lens so that only one of the elemental images may be observed by each eye (Fig. 2(b)), which in turn leads to discontinuous motion parallax. The second problem the moiré pattern that appears due to the layering structure of LCD panels with limited pixel aperture.

3. Integral volumetric imaging with dual fly-eye lens

To overcome the damage of image quality due to coarse fly-eye lens and multilayer structure, we propose use of another fly-eye lens. CIVI displays shown in Fig. 2 generate layered real images in the air. Around the layered real images presented by the lenses, we insert a fine fly-eye lens sheet whose elemental lenses have a long focal distance as shown in Fig. 3 .

Fig. 3 Proposed system design.

When another fly-eye lens sheet is placed, the images close to it remains almost the same while those far from it are strongly blurred. Therefore the seam and moiré can be made indistinct without damaging the image quality when the interval between the fine fly-eye lens newly added and the farthest real image layer is shorter enough than the distance between the fine fly-eye lens and the coarse fly-eye lens.

The elemental lens pitch (diameter of elemental lens) of fine fly-eye lens should be small enough so that the presented image can maintain high resolution. Here it is important to select proper focal length of the fine fly-eye lens. When the focal length is too short, the images close to the fine fly-eye lens are also blurred, which restricts the depth of image to be presented. When the focal length is too long, the fine fly-eye lens cannot blur the seam of coarse fly-eye lens and the moiré generated by multilayer panels. The two factors should be balanced to present high quality image to the viewer.

The relationship between the blur and the focal length of fine fly-eye lens f is given as shown in Fig. 4 . Here let us assume that the acceptable limit of blur is equal to the diameter of fine fly-eye lens d. When the real image is behind the inserted fine fly-eye lens, the light scattered from a pixel on the real image is collimated when the distance between the real image and the fly-eye lens is equal to f. In this case it is obvious that the blur is equal to d, for the width of blur is given by extending the collimated light to the real image plane as shown in Fig. 4(a).

Fig. 4 Blur of image by fine fly-eye lens.

When the real image is in front of the inserted fine fly-eye lens, the light proceeding toward a pixel on the real image plane is refracted and forms an image closer to the fine fly-eye lens. Let us define the distance between the inserted fine fly-eye lens and the original image as a and the distance between the inserted lens and the image newly generated by it as b. Then the blur of pixel on the original image plane is equal to d when b = a/2. By substituting a/2 for b in the basic lens equation

\frac{1}{b} - \frac{1}{a} = \frac{1}{f},

(1)

we can derive a = f. Therefore the blur of image is equal to d when the distance between the original image and the fine fly-eye lens is equal to f again as shown in Fig. 4(b).

From the discussion here it is clear that the blur of image can be kept small when we use a fly-eye lens that consists of elemental lenses with small diameter and large focal length.

Now let us examine how the seam of coarse fly-eye lens is obscured by inserting the fine fly-eye lens. Here we define the distance between the coarse fly-eye lens and the fine fly-eye lens as L and the distance between the fine fly-eye lens and the real image of seam generated by the fine lens as l. Then we can derive

l = \frac{f L}{L - f}

(2)

from the basic lens equation

\frac{1}{L} + \frac{1}{l} = \frac{1}{f} .

(3)

By extending the light ray as shown in Fig. 5 , the blur of seam D is given by

D = \frac{l + L}{l} d = \frac{d L}{f} .

(4)

From the above discussion it is clear that f should be small and d and L should be large to blur the seam of lens strongly. Small f and large d increase the blur of presented image also, while large L can blur the seam without damaging the quality of image.

Fig. 5 Blur of lens seam by fine fly-eye lens.

4. Prototype system

Based on the discussion in the previous section, we made a prototype system of integral volumetric imaging (IVI) with dual layer fly-eye lenses. The system is composed of two layers of fly-eye lenses and three LCD panels. As for the panels, we use one color panel and two monochrome panels, each of which has 18.1 inch display area with the resolution of 1280 × 1024 pixels. These panels with 2 mm thickness are connected together to be layered in front of the backlight and placed 90 mm behind the coarse fly-eye lens sheet, which is composed of 76 ( = 9.5 × 8) hexagon elemental lenses with 15 mm edges and 90 mm focal distance. A large aperture lens whose focal length is 500 mm is placed just in front of the coarse fly-eye lens sheet. With this configuration real image of the display is generated 500 mm in front of the lens because the light emitted from the panel is collimated by the elemental lens of coarse fly-eye lens and the parallel light is to be converged by the large aperture lens whose focal length is 500 mm. Therefore the fine fly-eye lens is placed 500 mm in front of the large aperture lens. The diameter of elemental lenses constituting the fine fly-eye lens used here is 0.5 mm, and the focal length of elemental lenses is 10mm. The frame size of the fine fly-eye lens is 100 mm × 100 mm.

Figure 6 shows a picture of the prototype IVI display hardware. Figure 7 (Media 1 and Media 2) shows the pictures of 3D images generated by the conventional system without fine fly-eye lens (CIVI) and the proposed IVI system. As shown in the figure, the proposed system can successfully remove moiré pattern and seam of lenses and can show clear 3D image with proper binocular and motion parallax. The presented image is partially blurred because the depth of three image planes is wider than 2f (20 mm). The extent of blur in the image plane farther than f from the fine fly-eye lens can be given by Eq. (4), which gives the blur of far objects in general.

Fig. 6 Prototype system.

Fig. 7 (Media 1, Media 2) Observed images presented by conventional CIVI and proposed IVI systems.

5. Conclusion

This paper proposes an integral volumetric imaging system that uses a coarse fly-eye lens and a fine fly-eye lens to show smooth and deep 3D image. The fine fly-eye lens layer can blur and remove the seam of lenses and the moiré pattern, which have been the cause of low image quality in the conventional integral volumetric imaging systems. Also the blur of presented 3D image can be kept small by placing a fine fly-eye lens whose elemental lens has a long focal distance near the layered real image. System downsizing is the main problem remained to be solved in the future.

Acknowledgments

This research is supported by the Grant-in-Aid for Scientific Research, MEXT, Japan, Grant number: 22680008.

References and links

1.	G. Lippmann, “La photograhie integrale,” C. R. Acad. Sci. 146, 446–451 (1908).
2.	B. Lee, S. Jung, S.-W. Min, and J.-H. Park, “Three-dimensional display by use of integral photography with dynamically variable image planes,” Opt. Lett. 26(19), 1481–1482 (2001). [CrossRef] [PubMed]
3.	Y. Kim, J.-H. Park, H. Choi, J. Kim, S.-W. Cho, and B. Lee, “Depth-enhanced three-dimensional integral imaging by use of multilayered display devices,” Appl. Opt. 45(18), 4334–4343 (2006). [CrossRef] [PubMed]
4.	Y. Kim, H. Choi, J. Kim, S.-W. Cho, Y. Kim, G. Park, and B. Lee, “Depth-enhanced integral imaging display system with electrically variable image planes using polymer-dispersed liquid-crystal layers,” Appl. Opt. 46(18), 3766–3773 (2007). [CrossRef] [PubMed]
5.	H. Kakeya, “Coarse integral imaging and its applications,” Proc. SPIE 6803, 680317 (2008). [CrossRef]
6.	S. Suyama, H. Takada, K. Uehira, S. Sakai, and S. Ohtsuka, “A Novel Direct-Vision 3-D Display using Luminance-Modulated Two 2-D Images Displayed at Different Depths,” SID Int. Symp. Digest Tech. Papers 31(1), 1208–1211 (2000). [CrossRef]
7.	S. Suyama, H. Takada, and S. Ohtsuka, “A Direct-Vision 3-D Display Using a New Depth-fusing Perceptual Phenomenon in 2-D Displays with Different Depths,” IEICE Trans. Electron. E85-C(11), 1911–1915 (2002).
8.	H. Kakeya, “Improving image quality of coarse integral volumetric display,” Proc. SPIE 7237, 723726 (2009). [CrossRef]
9.	H. Kakeya, “Realization of undistorted volumetric multiview image with multilayered integral imaging,” Opt. Express 19(21), 20395–20404 (2011). [CrossRef] [PubMed]
10.	Y. Ueda and H. Kakeya, “Multilayered Integral Imaging with Improved Image Clarity,” SID Int. Symp. Digest Tech. Papers 42(1), 1116–1119 (2011). [CrossRef]
11.	H. Kodaira, A. Nakao, Y. Mano, and H. Kakeya, “Distortion correction of coarse integral volumetric imaging with large lens,” Proc. of 3DSA (2010), pp. 205–208.
12.	S. Sawada and H. Kakeya, “Coarse integral volumetric imaging with flat screen and wide viewing angle,” Proc. SPIE 7863, 78631L (2011). [CrossRef]

OCIS Codes
(100.6890) Image processing : Three-dimensional image processing
(120.2040) Instrumentation, measurement, and metrology : Displays
(220.2740) Optical design and fabrication : Geometric optical design

ToC Category:
Image Processing

History
Original Manuscript: November 23, 2011
Revised Manuscript: January 5, 2012
Manuscript Accepted: January 10, 2012
Published: January 13, 2012

Virtual Issues
Vol. 7, Iss. 3 Virtual Journal for Biomedical Optics

Citation
Hideki Kakeya, Shimpei Sawada, Yukio Ueda, and Tomoya Kurokawa, "Integral volumetric imaging with dual layer fly-eye lenses," Opt. Express 20, 1963-1968 (2012)
http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-20-3-1963

Sort: Author | Year | Journal | Reset

References

G. Lippmann, “La photograhie integrale,” C. R. Acad. Sci.146, 446–451 (1908).
B. Lee, S. Jung, S.-W. Min, and J.-H. Park, “Three-dimensional display by use of integral photography with dynamically variable image planes,” Opt. Lett.26(19), 1481–1482 (2001). [CrossRef] [PubMed]
Y. Kim, J.-H. Park, H. Choi, J. Kim, S.-W. Cho, and B. Lee, “Depth-enhanced three-dimensional integral imaging by use of multilayered display devices,” Appl. Opt.45(18), 4334–4343 (2006). [CrossRef] [PubMed]
Y. Kim, H. Choi, J. Kim, S.-W. Cho, Y. Kim, G. Park, and B. Lee, “Depth-enhanced integral imaging display system with electrically variable image planes using polymer-dispersed liquid-crystal layers,” Appl. Opt.46(18), 3766–3773 (2007). [CrossRef] [PubMed]
H. Kakeya, “Coarse integral imaging and its applications,” Proc. SPIE6803, 680317 (2008). [CrossRef]
S. Suyama, H. Takada, K. Uehira, S. Sakai, and S. Ohtsuka, “A Novel Direct-Vision 3-D Display using Luminance-Modulated Two 2-D Images Displayed at Different Depths,” SID Int. Symp. Digest Tech. Papers31(1), 1208–1211 (2000). [CrossRef]
S. Suyama, H. Takada, and S. Ohtsuka, “A Direct-Vision 3-D Display Using a New Depth-fusing Perceptual Phenomenon in 2-D Displays with Different Depths,” IEICE Trans. Electron.E85-C(11), 1911–1915 (2002).
H. Kakeya, “Improving image quality of coarse integral volumetric display,” Proc. SPIE7237, 723726 (2009). [CrossRef]
H. Kakeya, “Realization of undistorted volumetric multiview image with multilayered integral imaging,” Opt. Express19(21), 20395–20404 (2011). [CrossRef] [PubMed]
Y. Ueda and H. Kakeya, “Multilayered Integral Imaging with Improved Image Clarity,” SID Int. Symp. Digest Tech. Papers42(1), 1116–1119 (2011). [CrossRef]
H. Kodaira, A. Nakao, Y. Mano, and H. Kakeya, “Distortion correction of coarse integral volumetric imaging with large lens,” Proc. of 3DSA (2010), pp. 205–208.
S. Sawada and H. Kakeya, “Coarse integral volumetric imaging with flat screen and wide viewing angle,” Proc. SPIE7863, 78631L (2011). [CrossRef]

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

Click here to see a list of articles that cite this paper