## Influence of lenslet number on performance of image restoration algorithms for the TOMBO imaging system |

Optics Express, Vol. 22, Issue 7, pp. 8298-8308 (2014)

http://dx.doi.org/10.1364/OE.22.008298

Acrobat PDF (1386 KB)

### Abstract

In this paper the influence of the number of lenslets on the performance of image restoration algorithms for the thin observation module by bound optics (TOMBO) imaging system was investigated, and the lenslet number was optimized to achieve thin system and high imaging performance. Subimages with different numbers of lenslets were generated following the TOMBO observation model, and image restoration algorithms were applied to evaluate the imaging performance of the TOMBO system. The optimal lenslet number was determined via theoretical performance optimization and verified via experimental comparisons of angular resolutions of two TOMBO systems and a conventional single-lens system.

© 2014 Optical Society of America

## 1. Introduction

1. J. Tanida, T. Kumagai, K. Yamada, S. Miyatake, K. Ishida, T. Morimoto, N. Kondou, D. Miyazaki, and Y. Ichioka, “Thin observation module by bound optics (TOMBO): concept and experimental verification,” Appl. Opt. **40**(11), 1806–1813 (2001). [CrossRef] [PubMed]

3. D. Mendlovic, “Toward a super imaging system,” Appl. Opt. **52**(4), 561–566 (2013). [CrossRef] [PubMed]

4. K. Choi and T. J. Schulz, “Signal-processing approaches for image-resolution restoration for TOMBO imagery,” Appl. Opt. **47**(10), B104–B116 (2008). [CrossRef] [PubMed]

*n*of the conventional single-lens system with equal

*f*number, with

*n*being the number of subimages of the scene in one direction across the image sensor. The reduction in the focal length of the lenslet (keeping the

*f*number the same) would keep the focal resolution unaffected with the compromise of reduction in angular resolution [5

5. M. Shankar, R. Willett, N. Pitsianis, T. Schulz, R. Gibbons, R. Te Kolste, J. Carriere, C. Chen, D. Prather, and D. Brady, “Thin infrared imaging systems through multichannel sampling,” Appl. Opt. **47**(10), B1–B10 (2008). [CrossRef] [PubMed]

6. A. V. Kanaev, D. A. Scribner, J. R. Ackerman, and E. F. Fleet, “Analysis and application of multiframe superresolution processing for conventional imaging systems and lenslet arrays,” Appl. Opt. **46**(20), 4320–4328 (2007). [CrossRef] [PubMed]

7. Y. Kitamura, R. Shogenji, K. Yamada, S. Miyatake, M. Miyamoto, T. Morimoto, Y. Masaki, N. Kondou, D. Miyazaki, J. Tanida, and Y. Ichioka, “Reconstruction of a high-resolution image on a compound-eye image-capturing system,” Appl. Opt. **43**(8), 1719–1727 (2004). [CrossRef] [PubMed]

12. S. Mendelowitz, I. Klapp, and D. Mendlovic, “Design of an image restoration algorithm for the TOMBO imaging system,” J. Opt. Soc. Am. A **30**(6), 1193–1204 (2013). [CrossRef] [PubMed]

13. Z. Lin and H. Y. Shum, “Fundamental limits of reconstruction-based superresolution algorithms under local translation,” IEEE Trans. Pattern Anal. Mach. Intell. **26**(1), 83–97 (2004). [CrossRef] [PubMed]

2. J. W. Duparré and F. C. Wippermann, “Micro-optical artificial compound eyes,” Bioinspir. Biomim. **1**(1), R1–R16 (2006). [CrossRef] [PubMed]

13. Z. Lin and H. Y. Shum, “Fundamental limits of reconstruction-based superresolution algorithms under local translation,” IEEE Trans. Pattern Anal. Mach. Intell. **26**(1), 83–97 (2004). [CrossRef] [PubMed]

15. D. Robinson and P. Milanfar, “Statistical performance analysis of super-resolution,” IEEE Trans. Image Process. **15**(6), 1413–1428 (2006). [CrossRef] [PubMed]

8. A. Stern and B. Javidi, “Three-dimensional image sensing and reconstruction with time-division multiplexed computational integral imaging,” Appl. Opt. **42**(35), 7036–7042 (2003). [CrossRef] [PubMed]

16. S. Farsiu, M. D. Robinson, M. Elad, and P. Milanfar, “Fast and robust multiframe super resolution,” IEEE Trans. Image Process. **13**(10), 1327–1344 (2004). [CrossRef] [PubMed]

17. S. Villena, M. Vega, S. D. Babaccan, R. Molina, and A. K. Katsaggelos, “Bayesian combination of sparse and non-sparse priors in image super resolution,” Digit. Signal Process. **23**(2), 530–541 (2013). [CrossRef]

18. D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. **60**(2), 91–110 (2004). [CrossRef]

19. M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM **24**(6), 381–395 (1981). [CrossRef]

## 2. Performance simulation

*n*×

*n*subimaging units. Each captured subimage can be modeled as [12

12. S. Mendelowitz, I. Klapp, and D. Mendlovic, “Design of an image restoration algorithm for the TOMBO imaging system,” J. Opt. Soc. Am. A **30**(6), 1193–1204 (2013). [CrossRef] [PubMed]

*L*represents the blurred, noisy, and low-resolution output image captured by subimaging system in the

_{i,j}*i*row of the

*j*column of TOMBO (

*i*,

*j*= 1, 2, …,

*n*);

*H*is the input high-resolution image;

*b*is a two-dimensional PSF representing the channel blur for each imaging unit;

_{i,j}*t*(

_{i,j}*r*) is a global translation shift operator which has

_{i,j}*r*= [Δ

_{i,j}*x*, Δ

_{i}*y*] translation with respect to the input image;

_{j}*↓D*is the downsampling operator; and

*v*is the noise, such as fixed-detector thermal noise, signal-dependent shot noise, and background noise [20

_{i,j}20. M. W. Haney, “Performance scaling in flat imagers,” Appl. Opt. **45**(13), 2901–2910 (2006). [CrossRef] [PubMed]

5. M. Shankar, R. Willett, N. Pitsianis, T. Schulz, R. Gibbons, R. Te Kolste, J. Carriere, C. Chen, D. Prather, and D. Brady, “Thin infrared imaging systems through multichannel sampling,” Appl. Opt. **47**(10), B1–B10 (2008). [CrossRef] [PubMed]

^{2}= 1 is used as channel blur

*b*, variance σ

_{i,j}^{2}of two-dimensional zero mean white Gaussian noise is set to 2, and downsampling factor

*d*equals the lenslet number

*n*.

*M*

^{2}when the translation between adjacent subimages is 1/

*d*(

*M*is an integer magnification factor of superresolution algorithms) [13

13. Z. Lin and H. Y. Shum, “Fundamental limits of reconstruction-based superresolution algorithms under local translation,” IEEE Trans. Pattern Anal. Mach. Intell. **26**(1), 83–97 (2004). [CrossRef] [PubMed]

*r*= [(

_{i,j}*i*– 1), (

*j*– 1)] (i.e., Δ

*x*=

_{i}*i*– 1 and Δ

*y*=

_{j}*j*– 1), and magnification factor

*M*equals the lenslet number

*n*in our simulation. For instance, translation between two low-resolution pixels is 1/3 in horizontal and vertical as shown in Fig. 2 when downsampling factor

*d*is 3. Take the system in [5

5. M. Shankar, R. Willett, N. Pitsianis, T. Schulz, R. Gibbons, R. Te Kolste, J. Carriere, C. Chen, D. Prather, and D. Brady, “Thin infrared imaging systems through multichannel sampling,” Appl. Opt. **47**(10), B1–B10 (2008). [CrossRef] [PubMed]

*M*equals the lenslet number

*n*, and the number of iterations is 20. Furthermore, for consideration of practical applications, restored images using the same algorithm with estimated translation parameters by SIFT-RANSAC are shown in Fig. 5. Matched pixel pairs between any two subimages are generated by SIFT, and then RANSAC is used to remove incorrect matches.

7. Y. Kitamura, R. Shogenji, K. Yamada, S. Miyatake, M. Miyamoto, T. Morimoto, Y. Masaki, N. Kondou, D. Miyazaki, J. Tanida, and Y. Ichioka, “Reconstruction of a high-resolution image on a compound-eye image-capturing system,” Appl. Opt. **43**(8), 1719–1727 (2004). [CrossRef] [PubMed]

*n*, higher registration accuracy is required as the lenslet number increases. Under this condition, larger registration errors lead to restoration performance deterioration.

## 3. Experiment

*f*and the diameter of each lens are 20 mm and 2.6 mm, respectively. For the 5 × 5 lenslet TOMBO system, lenses are aligned with a pitch of 2 mm. The focal length

_{l}*f*and the diameter of each lens are 16 mm and 2 mm, respectively. For the single-lens system, we utilize Nikon Nikkor lens whose focal lens

_{l}*f*is 85 mm and

_{s}*f*number ranges from 1.8 to 16.

*f*number ( = 8) are measured by a collimator. In the collimator, an illuminated USAF 1951 resolution target is positioned at the front focal plane of the objective lens as shown in Fig. 12. With this configuration, all light beams passing a point in the resolution target plane form a collimated light bundle behind the objective lens. The focal length

*f*and clear aperture of the collimator are 1000 mm and 100 mm, respectively. Figure 13 presents a photo of the experimental setup for a TOMBO system.

_{c}*b*is the minimal interval of white bars that can be distinguished,

*f*is the focal length of the collimator, and the value 206,265 is the arcseconds for one radian. As shown in Figs. 15(a) and 15(c),

_{c}*b*in yellow squares corresponds to the resolution target’s group 1 element 3, which indicates 2.52 line pairs/mm. So the angular resolution for the 4 × 4 lenslet TOMBO system and the single-lens system is 81.85″, whereas the 5 × 5 lenslet TOMBO system has a poorer angular resolution performance. These results confirm that 4 × 4 is the optimal lenslet number for TOMBO systems, as the simulations showed.

## 4. Conclusion and future work

## Acknowledgments

## References and links

1. | J. Tanida, T. Kumagai, K. Yamada, S. Miyatake, K. Ishida, T. Morimoto, N. Kondou, D. Miyazaki, and Y. Ichioka, “Thin observation module by bound optics (TOMBO): concept and experimental verification,” Appl. Opt. |

2. | J. W. Duparré and F. C. Wippermann, “Micro-optical artificial compound eyes,” Bioinspir. Biomim. |

3. | D. Mendlovic, “Toward a super imaging system,” Appl. Opt. |

4. | K. Choi and T. J. Schulz, “Signal-processing approaches for image-resolution restoration for TOMBO imagery,” Appl. Opt. |

5. | M. Shankar, R. Willett, N. Pitsianis, T. Schulz, R. Gibbons, R. Te Kolste, J. Carriere, C. Chen, D. Prather, and D. Brady, “Thin infrared imaging systems through multichannel sampling,” Appl. Opt. |

6. | A. V. Kanaev, D. A. Scribner, J. R. Ackerman, and E. F. Fleet, “Analysis and application of multiframe superresolution processing for conventional imaging systems and lenslet arrays,” Appl. Opt. |

7. | Y. Kitamura, R. Shogenji, K. Yamada, S. Miyatake, M. Miyamoto, T. Morimoto, Y. Masaki, N. Kondou, D. Miyazaki, J. Tanida, and Y. Ichioka, “Reconstruction of a high-resolution image on a compound-eye image-capturing system,” Appl. Opt. |

8. | A. Stern and B. Javidi, “Three-dimensional image sensing and reconstruction with time-division multiplexed computational integral imaging,” Appl. Opt. |

9. | R. Horisaki, S. Irie, Y. Ogura, and J. Tanida, “Three-dimensional information acqusition using a compound imaging system,” Opt. Rev. |

10. | A. V. Kanaev, J. R. Ackerman, E. F. Fleet, and D. A. Scribner, “TOMBO sensor with scene-independent superresolution processing,” Opt. Lett. |

11. | A. A. El-Sallam and F. Boussaid, “Spectral-based blind image restoration method for thin TOMBO imagers,” Sensors (Basel Switzerland) |

12. | S. Mendelowitz, I. Klapp, and D. Mendlovic, “Design of an image restoration algorithm for the TOMBO imaging system,” J. Opt. Soc. Am. A |

13. | Z. Lin and H. Y. Shum, “Fundamental limits of reconstruction-based superresolution algorithms under local translation,” IEEE Trans. Pattern Anal. Mach. Intell. |

14. | S. Baker and T. Kanade, “Limits on super-resolution and how to break them,” IEEE Trans. Pattern Anal. Mach. Intell. |

15. | D. Robinson and P. Milanfar, “Statistical performance analysis of super-resolution,” IEEE Trans. Image Process. |

16. | S. Farsiu, M. D. Robinson, M. Elad, and P. Milanfar, “Fast and robust multiframe super resolution,” IEEE Trans. Image Process. |

17. | S. Villena, M. Vega, S. D. Babaccan, R. Molina, and A. K. Katsaggelos, “Bayesian combination of sparse and non-sparse priors in image super resolution,” Digit. Signal Process. |

18. | D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. |

19. | M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM |

20. | M. W. Haney, “Performance scaling in flat imagers,” Appl. Opt. |

**OCIS Codes**

(110.1758) Imaging systems : Computational imaging

(110.3010) Imaging systems : Image reconstruction techniques

**ToC Category:**

Imaging Systems

**History**

Original Manuscript: January 21, 2014

Revised Manuscript: March 7, 2014

Manuscript Accepted: March 17, 2014

Published: April 1, 2014

**Citation**

Yuan Gao, Lizhi Dong, Ping Yang, Guomao Tang, and Bing Xu, "Influence of lenslet number on performance of image restoration algorithms for the TOMBO imaging system," Opt. Express **22**, 8298-8308 (2014)

http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-22-7-8298

Sort: Year | Journal | Reset

### References

- J. Tanida, T. Kumagai, K. Yamada, S. Miyatake, K. Ishida, T. Morimoto, N. Kondou, D. Miyazaki, Y. Ichioka, “Thin observation module by bound optics (TOMBO): concept and experimental verification,” Appl. Opt. 40(11), 1806–1813 (2001). [CrossRef] [PubMed]
- J. W. Duparré, F. C. Wippermann, “Micro-optical artificial compound eyes,” Bioinspir. Biomim. 1(1), R1–R16 (2006). [CrossRef] [PubMed]
- D. Mendlovic, “Toward a super imaging system,” Appl. Opt. 52(4), 561–566 (2013). [CrossRef] [PubMed]
- K. Choi, T. J. Schulz, “Signal-processing approaches for image-resolution restoration for TOMBO imagery,” Appl. Opt. 47(10), B104–B116 (2008). [CrossRef] [PubMed]
- M. Shankar, R. Willett, N. Pitsianis, T. Schulz, R. Gibbons, R. Te Kolste, J. Carriere, C. Chen, D. Prather, D. Brady, “Thin infrared imaging systems through multichannel sampling,” Appl. Opt. 47(10), B1–B10 (2008). [CrossRef] [PubMed]
- A. V. Kanaev, D. A. Scribner, J. R. Ackerman, E. F. Fleet, “Analysis and application of multiframe superresolution processing for conventional imaging systems and lenslet arrays,” Appl. Opt. 46(20), 4320–4328 (2007). [CrossRef] [PubMed]
- Y. Kitamura, R. Shogenji, K. Yamada, S. Miyatake, M. Miyamoto, T. Morimoto, Y. Masaki, N. Kondou, D. Miyazaki, J. Tanida, Y. Ichioka, “Reconstruction of a high-resolution image on a compound-eye image-capturing system,” Appl. Opt. 43(8), 1719–1727 (2004). [CrossRef] [PubMed]
- A. Stern, B. Javidi, “Three-dimensional image sensing and reconstruction with time-division multiplexed computational integral imaging,” Appl. Opt. 42(35), 7036–7042 (2003). [CrossRef] [PubMed]
- R. Horisaki, S. Irie, Y. Ogura, J. Tanida, “Three-dimensional information acqusition using a compound imaging system,” Opt. Rev. 14(5), 347–350 (2007). [CrossRef]
- A. V. Kanaev, J. R. Ackerman, E. F. Fleet, D. A. Scribner, “TOMBO sensor with scene-independent superresolution processing,” Opt. Lett. 32(19), 2855–2857 (2007). [CrossRef] [PubMed]
- A. A. El-Sallam, F. Boussaid, “Spectral-based blind image restoration method for thin TOMBO imagers,” Sensors (Basel Switzerland) 8(9), 6108–6124 (2008). [CrossRef]
- S. Mendelowitz, I. Klapp, D. Mendlovic, “Design of an image restoration algorithm for the TOMBO imaging system,” J. Opt. Soc. Am. A 30(6), 1193–1204 (2013). [CrossRef] [PubMed]
- Z. Lin, H. Y. Shum, “Fundamental limits of reconstruction-based superresolution algorithms under local translation,” IEEE Trans. Pattern Anal. Mach. Intell. 26(1), 83–97 (2004). [CrossRef] [PubMed]
- S. Baker, T. Kanade, “Limits on super-resolution and how to break them,” IEEE Trans. Pattern Anal. Mach. Intell. 24(9), 1167–1183 (2002). [CrossRef]
- D. Robinson, P. Milanfar, “Statistical performance analysis of super-resolution,” IEEE Trans. Image Process. 15(6), 1413–1428 (2006). [CrossRef] [PubMed]
- S. Farsiu, M. D. Robinson, M. Elad, P. Milanfar, “Fast and robust multiframe super resolution,” IEEE Trans. Image Process. 13(10), 1327–1344 (2004). [CrossRef] [PubMed]
- S. Villena, M. Vega, S. D. Babaccan, R. Molina, A. K. Katsaggelos, “Bayesian combination of sparse and non-sparse priors in image super resolution,” Digit. Signal Process. 23(2), 530–541 (2013). [CrossRef]
- D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60(2), 91–110 (2004). [CrossRef]
- M. A. Fischler, R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24(6), 381–395 (1981). [CrossRef]
- M. W. Haney, “Performance scaling in flat imagers,” Appl. Opt. 45(13), 2901–2910 (2006). [CrossRef] [PubMed]

## Cited By |
Alert me when this paper is cited |

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

### Figures

Fig. 1 |
Fig. 2 |
Fig. 3 |

Fig. 4 |
Fig. 5 |
Fig. 6 |

Fig. 7 |
Fig. 8 |
Fig. 9 |

Fig. 10 |
Fig. 11 |
Fig. 12 |

Fig. 13 |
Fig. 14 |
Fig. 15 |

« Previous Article | Next Article »

OSA is a member of CrossRef.