## Image quality enhancement in 3D computational integral imaging by use of interpolation methods

Optics Express, Vol. 15, Issue 19, pp. 12039-12049 (2007)

http://dx.doi.org/10.1364/OE.15.012039

Acrobat PDF (362 KB)

### Abstract

In this paper, we propose a computational integral imaging reconstruction (CIIR) method by use of image interpolation algorithms to improve the visual quality of 3D reconstructed images. We investigate the characteristics of the conventional CIIR method along the distance between lenslet and objects. What we observe is that the visual quality of reconstructed images is periodically degraded. The experimentally observed period is half size of the elemental image. To remedy this problem, we focus on the interpolation methods in computational integral imaging. Several interpolation methods are applied to the conventional CIIR method and their performances are analyzed. To objectively evaluate the proposed CIIR method, we introduce an experimental framework for the computational pickup process and the CIIR process using a Gaussian function. We also carry out experiments on real objects to subjectively evaluate the proposed method. Experimental results indicate that our method outperforms the conventional CIIR method. In addition, our method reduces the grid noise that the conventional CIIR method suffers from.

© 2007 Optical Society of America

## 1. Introduction

16. B. Javidi, R. Ponce-Diaz, and S.-H. Hong,
“Three-dimensional recognition of occluded objects using volumetric
reconstruction,” Opt. Lett. **31**, 1106–1108
(2006). [CrossRef] [PubMed]

9. D.-H. Shin, B. Lee, and E.-S. Kim,
“Multi-direction-curved integral imaging with large depth by
additional use of a large-aperture lens,” Appl.
Opt. **45**, 7375–7381
(2006). [CrossRef] [PubMed]

10. H. Arimoto and B. Javidi,
“Integral three-dimensional imaging with digital
reconstruction,” Opt. Lett. **26**, 157–159 (2001) [CrossRef]

12. S.-H. Hong, J.-S. Jang, and B. Javidi,
“Three-dimensional volumetric object reconstruction using
computational integral imaging,” Opt. Express **12**, 483–491
(2004). [CrossRef] [PubMed]

10. H. Arimoto and B. Javidi,
“Integral three-dimensional imaging with digital
reconstruction,” Opt. Lett. **26**, 157–159 (2001) [CrossRef]

12. S.-H. Hong, J.-S. Jang, and B. Javidi,
“Three-dimensional volumetric object reconstruction using
computational integral imaging,” Opt. Express **12**, 483–491
(2004). [CrossRef] [PubMed]

15. S. -H. Hong and B. Javidi,
“Three-dimensional visualization of partially occluded objects
using integral imaging,” J. Display Technol. **1**, 354- (2005). [CrossRef]

12. S.-H. Hong, J.-S. Jang, and B. Javidi,
“Three-dimensional volumetric object reconstruction using
computational integral imaging,” Opt. Express **12**, 483–491
(2004). [CrossRef] [PubMed]

13. S. -H. Hong and B. Javidi,
“Improved resolution 3D object reconstruction using computational
integral imaging with time multiplexing,” Opt.
Express **12**, 4579–4588
(2004) [CrossRef] [PubMed]

15. S. -H. Hong and B. Javidi,
“Three-dimensional visualization of partially occluded objects
using integral imaging,” J. Display Technol. **1**, 354- (2005). [CrossRef]

16. B. Javidi, R. Ponce-Diaz, and S.-H. Hong,
“Three-dimensional recognition of occluded objects using volumetric
reconstruction,” Opt. Lett. **31**, 1106–1108
(2006). [CrossRef] [PubMed]

13. S. -H. Hong and B. Javidi,
“Improved resolution 3D object reconstruction using computational
integral imaging with time multiplexing,” Opt.
Express **12**, 4579–4588
(2004) [CrossRef] [PubMed]

14. J.-S. Park, D.-C. Hwang, D.-H. Shin, and E.-S. Kim,
“Resolution-enhanced computational integral imaging reconstruction
using intermediate-view reconstruction technique,” Opt.
Eng. **45**, 117004 (2006). [CrossRef]

13. S. -H. Hong and B. Javidi,
“Improved resolution 3D object reconstruction using computational
integral imaging with time multiplexing,” Opt.
Express **12**, 4579–4588
(2004) [CrossRef] [PubMed]

21. R.G Keys,
“Cubic convolution interpolation for digital image
processing,” IEEE Trans. Acoust. Speech Signal
Process. **29**, 1153–1160
(1981). [CrossRef]

## 2. Proposed CIIR Methods

### 2.1 The conventional CIIR method

**12**, 483–491
(2004). [CrossRef] [PubMed]

*z*/

*g*, where

*z*is the distance between the reconstructed output plane and the virtual pinhole array and

*g*is the distance between the elemental images and the virtual pinhole array. Each magnified elemental image is overlapped each other and an reconstructed image is finally produced at the reconstructed output plane

*z*. Iterative computation of the above process by varying the

*z*value provides a series of images along

*z*axis. This 3D image is a so called CIIR image.

*z*/

*g*=2. Each pixel is simply magnified into 2×2 pixels so the intensity values of these four pixels are equal. This method is considered to be the zero-order interpolation algorithm in 2D image processing. Therefore magnification process of the conventional CIIR is identical with the zero-order interpolation algorithm.

**12**, 483–491
(2004). [CrossRef] [PubMed]

**12**, 4579–4588
(2004) [CrossRef] [PubMed]

### 2.2 Proposed CIIR method by use of an image interpolation algorithm

19. T. Blu, P. Thevenaz, and M. Unser,
“Linear interpolation revitalized,” IEEE
Trans. Image Proc. **13**, pp.710–719
(2004). [CrossRef]

21. R.G Keys,
“Cubic convolution interpolation for digital image
processing,” IEEE Trans. Acoust. Speech Signal
Process. **29**, 1153–1160
(1981). [CrossRef]

*f*(

*x*) be the sampled version of a continuous function

_{k}*f*(

*x*). The Shannon theorem states that the Sinc interpolation perfectly reconstructs the continuous function

*f*(

*x*) from its samples

*f*(

*x*) if the sampling frequency is larger than twice the maximum frequency of the function

_{k}*f*(

*x*). The relationship between

*f*(

*x*) and its samples

*f*(

*x*) is represented in the form

_{k}*β*(

*x*) is the interpolation kernel and the sinc interpolation kernel is called as the sinc function defined by sin(

*πx*)/

*πx*. Unfortunately, the practical implementation of Eq. (1) is impossible because of the infinite support of the sinc function. Thus finite-support kernels have been studied. Especially, a short support is desirable in image processing to obtain the computational efficiency. We now introduce three interpolation kernels mentioned in the previous section. The kernel of the zero-order interpolation used in the conventional CIIR method is defined as

*β*

_{0}(

*x*),

*β*

_{1}(

*x*), and

*β*

_{3}(

*x*) are one, two, and four, respectively. It can be easily seen that the complexity of interpolation increases as the support of interpolation kernels increases. We apply the linear interpolation and the CCI to the conventional CIIR method as an image magnification technique.

*M*=

*z*/

*g*. Thus an elemental image having

*K*×

*K*pixels becomes larger as much as

*MK*×

*MK*pixels after the process of image interpolation. Each magnified elemental image is overlapped each other to reconstruct 3D images at the reconstruction output plane. To completely reconstruct a 3D plane image, this same process is repeatedly performed to all of the elemental images through each corresponding pinhole. As shown in Fig. 3, the proposed method use resolution-improved elemental images compared with conventional one because of using an image interpolation algorithm thus it provides improvement of reconstructed 3D images.

## 3. Experiments and Results

### 3.1 Experiments using Gaussian function

*G*(

*x*) as the continuous function

*f*(

*x*), whose pixels are

*N*, be located at the distance

*z*. The Gaussian function used in this paper is defined by

*z*=0 mm. The interval between pinholes is 1.08 mm and the gap

*g*between the elemental images and the pinhole array is 3 mm. Then 1D elemental images are computed by a computational pickup [9

9. D.-H. Shin, B. Lee, and E.-S. Kim,
“Multi-direction-curved integral imaging with large depth by
additional use of a large-aperture lens,” Appl.
Opt. **45**, 7375–7381
(2006). [CrossRef] [PubMed]

*z*/

*g*by using an interpolation algorithm and are superimposed on the reconstruction plane at

*z*. Here we consider that 1D elemental images are interpolated at the distance

*z*equal to a multiple of

*g*. Finally, the reconstructed function

*R*(

*x*) at

_{k}*z*is obtained after superimposition of all elemental images. To objectively evaluate the quality of a reconstructed function (image), we calculate the mean square error (MSE) defined as

*z*value, a series of

*R*(

*x*) is calculated and compared with their original version in terms of MSE.

_{k}### 3.2 Analysis of Gaussian function test

*z*for three kinds of interpolation algorithm are shown in Fig. 5. Also the results were shown for different pixel number p of each elemental image. Here the distance was normalized to

*z*/

*g*. It is shown that conventional CIIR method, which is identical with the zero-order interpolation algorithm, has large variation of MSE value because of appearing serious intensity irregularities. The position of large variation appears periodically. The period is half size

*p*/2 of the elemental image. On the other hand, CCI algorithm gives us better results regardless of the distance

*z*. Figure 6 shows two examples of the reconstructed images at

*z*/

*g*=14 and 15 in case that

*p*=30. Figure 5(a) indicates that the performances of the zero-order interpolation method, the linear interpolation method, and the CCI method are similar to each other for

*z*/

*g*=14. This is illustrated in Fig. 6(a). The reconstructed 1D-images of the three methods looks similar. On the other hand, the performances of the three interpolation methods are totally different for

*z*/

*g*=15. This is illustrated in Fig. 6(b). The reconstructed 1D image of the zero-order interpolation method suffers from the blocking artifact much higher than the others. Thus the previous method using the zero-order interpolation method suffers from the artifact at some locations whereas the two proposed methods do not produce the artifact. CCI algorithm is well known as having good performance of resolution enhancement. This fact also agrees with our proposed CIIR method. From results of Fig. 6, the proposed CIIR method can provide decreasing of intensity variation in the reconstructed images and improve the quality of image.

### 3.3 Experiments using 3D objects

*z*=18 mm and

*z*=45 mm, respectively.

9. D.-H. Shin, B. Lee, and E.-S. Kim,
“Multi-direction-curved integral imaging with large depth by
additional use of a large-aperture lens,” Appl.
Opt. **45**, 7375–7381
(2006). [CrossRef] [PubMed]

*z*=45 mm where ‘car’ pattern was originally located. Reconstructed images for three kinds of interpolation algorithms are shown, respectively. Here we can see the nature of the CIIR method. Figure 8 shows that the image reconstructed at the output plane (where the ‘car’ pattern is originally located) is clearly focused, whereas ‘tree’ pattern is out of focus. The ‘car’ image is clearly shown in the three reconstructed images. The partial ‘tire’ images were enlarged for visual test. In case of using zero-order interpolation algorithm, the reconstructed image was composed of large square-type pixel, which is the reason why the intensity variation exists as shown in Fig. 5. On the other hand, in case of using linear interpolation and CCI algorithm, the reconstructed images were smoother so that intensity variation might be reduced.

*z*/

*g*=15 which is the same with the pixel number of elemental image.

### 3.4 Experiments using real 3D objects

*z*=30 mm and

*z*=45 mm, respectively. The lenslet array with 34×25 lenslets is located at

*z*=0 mm. Each lenslet size

*d*is 1.08 mm and single elemental image is composed of 60×60 pixels. The elemental images obtained through an optical pickup are shown in Fig. 10(b).

*z*=30 mm and 45 mm for both conventional method and proposed method, respectively. It must be noted here that there is the improvement of visual quality between two images reconstructed. In the result of the conventional method of Fig. 11(a), we can see the intensity irregularities with a grid structure cased by square-shaped mapping of elemental images. This is the reason why visual quality of the 3D reconstructed image is degraded in the conventional method. However, the proposed method can provide better results as shown in Fig. 11(b). This is due to the superposition of resolution-improved elemental images using CCI algorithm for each magnified elemental image.

## 4. Conclusions

## References and links

1. | G. Lippmann,
“La photographic intergrale,”
Comptes-Rendus, Acad. Sci. |

2. | F. Okano, H. Hoshino, J. Arai, and I. Yuyama,
“Real-time pickup method for a three-dimensional image based on
integral photography,” Appl. Opt. |

3. | B. Lee, S. Jung, and J.-H. Park,
“Viewing-angle-enhanced integral imaging by lens
switching,” Opt. Lett. |

4. | J.-S. Jang and B. Javidi,
“Formation of orthoscopic three-dimensional real images in direct
pickup one-stepintegral imaging,” Opt. Eng. |

5. | A. Stern and B. Javidi,
“Three-dimensional image sensing and reconstruction with
time-division multiplexed computational integral imaging,”
Appl. Opt. |

6. | D.-H. Shin, M. Cho, and E.-S. Kim,
“Computational implementation of asymmetric integral imaging by use
of two crossed lenticular sheets,” ETRI Journal |

7. | M. Martínez-Corral, B. Javidi, R. Martínez-Cuenca, and G. Saavedra,
“Integral imaging with improved depth of field by use of amplitude
modulated microlens array,” Appl. Opt. |

8. | J.-H. Park, J. Kim, Y. Kim, and B. Lee,
“Resolution-enhanced three-dimension/two-dimension convertible
display based on integral imaging,” Opt. Express |

9. | D.-H. Shin, B. Lee, and E.-S. Kim,
“Multi-direction-curved integral imaging with large depth by
additional use of a large-aperture lens,” Appl.
Opt. |

10. | H. Arimoto and B. Javidi,
“Integral three-dimensional imaging with digital
reconstruction,” Opt. Lett. |

11. | Y. Frauel and B. Javidi,
“Digital three-dimensional image correlation by use of
computer-reconstructed integral imaging,” Appl.
Opt. |

12. | S.-H. Hong, J.-S. Jang, and B. Javidi,
“Three-dimensional volumetric object reconstruction using
computational integral imaging,” Opt. Express |

13. | S. -H. Hong and B. Javidi,
“Improved resolution 3D object reconstruction using computational
integral imaging with time multiplexing,” Opt.
Express |

14. | J.-S. Park, D.-C. Hwang, D.-H. Shin, and E.-S. Kim,
“Resolution-enhanced computational integral imaging reconstruction
using intermediate-view reconstruction technique,” Opt.
Eng. |

15. | S. -H. Hong and B. Javidi,
“Three-dimensional visualization of partially occluded objects
using integral imaging,” J. Display Technol. |

16. | B. Javidi, R. Ponce-Diaz, and S.-H. Hong,
“Three-dimensional recognition of occluded objects using volumetric
reconstruction,” Opt. Lett. |

17. | W. K. Pratt, Digital Image Processing, (New York: Wiley, 1991). |

18. | E. Meijering,
“A Chronology of interpolation: From ancient astronomy to modern
signal and image processing,” Proc. IEEE |

19. | T. Blu, P. Thevenaz, and M. Unser,
“Linear interpolation revitalized,” IEEE
Trans. Image Proc. |

20. | H. Yoo,
“Closed-form least-squares technique for adaptive linear image
interpolation,” Elect. Lett. |

21. | R.G Keys,
“Cubic convolution interpolation for digital image
processing,” IEEE Trans. Acoust. Speech Signal
Process. |

**OCIS Codes**

(100.6890) Image processing : Three-dimensional image processing

(110.2990) Imaging systems : Image formation theory

**ToC Category:**

Image Processing

**History**

Original Manuscript: June 11, 2007

Revised Manuscript: August 27, 2007

Manuscript Accepted: August 28, 2007

Published: September 6, 2007

**Virtual Issues**

Vol. 2, Iss. 10 *Virtual Journal for Biomedical Optics*

**Citation**

Dong-Hak Shin and Hoon Yoo, "Image quality enhancement in 3D computational integral imaging by use of interpolation methods," Opt. Express **15**, 12039-12049 (2007)

http://www.opticsinfobase.org/vjbo/abstract.cfm?URI=oe-15-19-12039

Sort: Year | Journal | Reset

### References

- G. Lippmann, "La photographic intergrale," Comptes-Rendus, Acad. Sci. 146, 446-451 (1908).
- F. Okano, H. Hoshino, J. Arai, and I. Yuyama, "Real-time pickup method for a three-dimensional image based on integral photography," Appl. Opt. 36, 1598-1603 (1997). [CrossRef] [PubMed]
- B. Lee, S. Jung, and J.-H. Park, "Viewing-angle-enhanced integral imaging by lens switching," Opt. Lett. 27, 818-820 (2002). [CrossRef]
- J.-S. Jang and B. Javidi, "Formation of orthoscopic three-dimensional real images in direct pickup one-stepintegral imaging," Opt. Eng. 42, 1869-1870 (2003). [CrossRef]
- A. Stern and B. Javidi, "Three-dimensional image sensing and reconstruction with time-division multiplexed computational integral imaging," Appl. Opt. 42, 7036-7042 (2003). [CrossRef] [PubMed]
- D.-H. Shin, M. Cho and E.-S. Kim, "Computational implementation of asymmetric integral imaging by use of two crossed lenticular sheets," ETRI Journal 27, 289-293 (2005). [CrossRef]
- M. Martínez-Corral, B. Javidi, R. Martínez-Cuenca, and G. Saavedra, "Integral imaging with improved depth of field by use of amplitude modulated microlens array," Appl. Opt. 43, 5806-5813 (2004). [CrossRef] [PubMed]
- J.-H. Park, J. Kim, Y. Kim, and B. Lee, "Resolution-enhanced three-dimension/two-dimension convertible display based on integral imaging," Opt. Express 13, 1875-1884 (2005). [CrossRef] [PubMed]
- D.-H. Shin, B. Lee and E.-S. Kim, "Multi-direction-curved integral imaging with large depth by additional use of a large-aperture lens," Appl. Opt. 45, 7375-7381 (2006). [CrossRef] [PubMed]
- H. Arimoto and B. Javidi, "Integral three-dimensional imaging with digital reconstruction," Opt. Lett. 26, 157-159 (2001) [CrossRef]
- Y. Frauel and B. Javidi, "Digital three-dimensional image correlation by use of computer-reconstructed integral imaging," Appl. Opt. 41, 5488-5496 (2002). [CrossRef] [PubMed]
- S.-H. Hong, J.-S. Jang, and B. Javidi, "Three-dimensional volumetric object reconstruction using computational integral imaging," Opt. Express 12, 483-491 (2004). [CrossRef] [PubMed]
- S. -H. Hong and B. Javidi, "Improved resolution 3D object reconstruction using computational integral imaging with time multiplexing," Opt. Express 12, 4579-4588 (2004) [CrossRef] [PubMed]
- J.-S. Park, D.-C. Hwang, D.-H. Shin, and E.-S. Kim, "Resolution-enhanced computational integral imaging reconstruction using intermediate-view reconstruction technique," Opt. Eng. 45, 117004 (2006). [CrossRef]
- S. -H. Hong and B. Javidi, "Three-dimensional visualization of partially occluded objects using integral imaging," J. Display Technol. 1, 354 (2005). [CrossRef]
- B. Javidi, R. Ponce-Diaz, and S.-H. Hong, "Three-dimensional recognition of occluded objects using volumetric reconstruction," Opt. Lett. 31, 1106-1108 (2006). [CrossRef] [PubMed]
- W. K. Pratt, Digital Image Processing, (New York: Wiley, 1991).
- E. Meijering, "A Chronology of interpolation: From ancient astronomy to modern signal and image processing," Proc. IEEE 90, 319-342 (2002). [CrossRef]
- T. Blu, P. Thevenaz, and M. Unser, "Linear interpolation revitalized," IEEE Trans. Image Proc. 13, pp.710-719 (2004). [CrossRef]
- H. Yoo, "Closed-form least-squares technique for adaptive linear image interpolation," Elect. Lett. 43, pp. 210-212 (2007). [CrossRef]
- Keys, R.G , "Cubic convolution interpolation for digital image processing," IEEE Trans. Acoust. Speech Signal Process. 29, 1153-1160 (1981). [CrossRef]

## Cited By |
Alert me when this paper is cited |

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

« Previous Article | Next Article »

OSA is a member of CrossRef.