## Iterative reconstruction of projection images from a microlens-based optical detector |

Optics Express, Vol. 19, Issue 13, pp. 11932-11943 (2011)

http://dx.doi.org/10.1364/OE.19.011932

Acrobat PDF (2611 KB)

### Abstract

A microlens-based optical detector was developed to perform small animal optical imaging. In this paper we present an iterative reconstruction algorithm yielding improved image quality and spatial resolution as compared to conventional inverse mapping. The reconstruction method utilizes the compressive sensing concept to cope with the undersampling nature of the problem. Each iteration in the algorithm contains two separate steps to ensure both the convergence of the least-square solution and the minimization of the *l*_{1}-norm of the sparsifying transform. The results estimated from measurements, employing a Derenzo-like pattern and a Siemens star phantom, illustrate significant improvements in contrast and spatial resolution in comparison to results calculated by inverse mapping.

© 2011 OSA

## 1. Introduction

1. N. Henriquez, P. van Overveld, I. Que, J. Buijs, R. Bachelier, E. Kaijzel, C. Löwik, P. Clezardin, and G. van der Pluijm, “Advances in optical imaging and novel model systems for cancer metastasis research,” Clin. Exp. Metastasis **24**, 699–705 (2007). [CrossRef] [PubMed]

2. D. Rowland, J. Lewis, and M. Welch, “Molecular imaging: the application of small animal positron emission tomography,” J. Cell. Biochem. **87**, 110–115 (2002). [CrossRef]

3. G. Kelloff, K. Krohn, S. Larson, R. Weissleder, D. Mankoff, J. Hoffman, J. Link, K. Guyton, W. Eckelman, H. Scher, J. O’Shaughnessy, B. D. Cheson, C. C. Sigman, J. L. Tatum, G. Q. Mills, D. C. Sullivan, and J. Woodcock, “The progress and promise of molecular imaging probes in oncologic drug development,” Clin. Cancer Res. **11**, 7967–7985 (2005). [CrossRef] [PubMed]

4. R. Weissleder and V. Ntziachristos, “Shedding light onto live molecular targets,” Nat. Med. **9**, 123–128 (2003). [CrossRef] [PubMed]

5. J. Peter, D. Unholtz, R. Schulz, J. Doll, and W. Semmler, “Development and initial results of a tomographic dual-modality positron/optical small animal imager,” IEEE Trans. Nucl. Sci. **54**, 1553–1560 (2007). [CrossRef]

15. S. Hong and B. Javidi, “Improved resolution 3D object reconstruction using computational integral imaging with time multiplexing,” Opt. Express **12**, 4579–4588 (2004). [CrossRef] [PubMed]

17. J. Jang and B. Javidi, “Improved viewing resolution of three-dimensional integral imaging by use of nonstationary micro-optics,” Opt. Lett. **27**, 324–326 (2002). [CrossRef]

18. D. Shin and H. Yoo, “Image quality enhancement in 3D computational integral imaging by use of interpolation methods,” Opt. Express **15**, 12039–12049 (2007). [CrossRef] [PubMed]

19. D. Hwang, J. Park, S. Kim, D. Shin, and E. Kim, “Magnification of 3D reconstructed images in integral imaging using an intermediate-view reconstruction technique,” Appl. Opt. **45**, 4631–4637 (2006). [CrossRef] [PubMed]

20. K. Lee, D. Hwang, S. Kim, and E. Kim, “Blur-metric-based resolution enhancement of computationally reconstructed integral images,” Appl. Opt. **47**, 2859–2869 (2008). [CrossRef] [PubMed]

21. G. Saavedra, R. Martinez-Cuenca, M. Martinez-Corral, H. Navarro, M. Daneshpanah, and B. Javidi, “Digital slicing of 3D scenes by Fourier filtering of integral images,” Opt. Express **16**, 17154–17160 (2008). [CrossRef] [PubMed]

22. J. Qi and R. Leahy, “Iterative reconstruction techniques in emission computed tomography,” Phys. Med. Biol. **51**, R541–R578 (2006). [CrossRef] [PubMed]

24. G. Chen, J. Tang, and S. Leng, “Prior image constrained compressed sensing (PICCS): a method to accurately reconstruct dynamic CT images from highly undersampled projection data sets,” Med. Phys. **35**, 660–663 (2008). [CrossRef] [PubMed]

## 2. Methods

### 2.1. System Description

5. J. Peter, D. Unholtz, R. Schulz, J. Doll, and W. Semmler, “Development and initial results of a tomographic dual-modality positron/optical small animal imager,” IEEE Trans. Nucl. Sci. **54**, 1553–1560 (2007). [CrossRef]

*H*of the system beyond which all objects are acceptably sharp can be calculated with [25] where

*N*is the f-number which is calculated by the proportion of focal length

*f*to the borehole diameter of the septum mask

*d*as: and

*c*is the circle of confusion which is in this case limited by the pixel size on the photon sensor, 0.048 mm. Therefore, the hyperfocal distance of this system as calculated from Eq. 1 is 18.3 mm, which means that all objects placed farther than 18.3 mm to the MLA are adequately focused. Applying the ray tracing theory with the thin lens approximation, this MLA can then be simplified to a pinhole array for all the imaging objects behind this plane.

### 2.2. Inverse Mapping Method

6. D. Unholtz, W. Semmler, O. Dössel, and J. Peter, “Image formation with a microlens-based optical detector: a three-dimensional mapping approach,” Appl. Opt. **48**, D273–D279 (2009). [CrossRef] [PubMed]

14. S. Hong, J. Jang, and B. Javidi, “Three-dimensional volumetric object reconstruction using computational integral imaging,” Opt. Express **12**, 483–491 (2004). [CrossRef] [PubMed]

15. S. Hong and B. Javidi, “Improved resolution 3D object reconstruction using computational integral imaging with time multiplexing,” Opt. Express **12**, 4579–4588 (2004). [CrossRef] [PubMed]

*z*towards the MLA plane, and the photon sensor is aligned at the focal plane with a length

*f*to the lens units. Assuming that the MLA is modeled as a pinhole array, the inverse mapping method is implemented in this work with the following three steps:

### 2.3. Iterative Reconstruction Method

**X**is the image to be estimated and a vector

**Y**refers to the measured elemental image on the photon sensor. From the geometry shown in Fig. 3(b), we can construct a system matrix

**A**, which maps any pixel at the reconstructed image plane to the sensor plane. Since the sizes of the raw detector data and the reconstructed image in our case are 512 × 1000, the dimension of the matrix would be 512000 × 512000. However, since only few pixels in the reconstructed image correspond to one specific detector pixel, this matrix is very sparse. Therefore, we store the matrix in a sparse manner with an index vector and a value vector for each detector pixel. Therefore, the reconstruction problem is to find

**X**, which conforms to

**AX = Y**.

**X**can be mapped to various pixel positions in

**Y**, which causes redundancy in the measured image, the solution of this inverse problem is not unique. Assuming a simple 1-dimensional example with 5 microlenses shown in Fig. 4, a relatively large light source will generate the same elemental images as a very small point source. In the other words, the acquisition is considered to be undersampled if a high-resolution image is supposed to be reconstructed. The compressive sensing (CS) theory can deal with such data incompleteness [26

26. E. Candès, J. Romberg, and T. Tao, “Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information,” IEEE Trans. Inf. Theory **52**, 489–509 (2006). [CrossRef]

27. D. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory **52**, 1289–1306 (2006). [CrossRef]

**X**is more likely to be smooth, which represents the upper large light source in Fig. 4, rather than the lower sharp point one. With this assumption, the resultant image is compressive and can be sparsified with a gradient operation. This estimation of

**X**with given

**A**and

**Y**can be set in the framework of CS image reconstruction by solving the following

*l*

_{1}-minimization problem: where Φ is a sparsifying transform which is specifically formulated as a discrete gradient transform in two orthogonal directions for this problem as:

*X*(

*i*,

*j*) is the value of

**X**at pixel (

*i*,

*j*).

*AX*=

*Y*). Instead, this minimization is numerically implemented in an iterative process with two separate steps for each iteration:

**AX = Y**. The second one is a step of steepest descent method which minimize the

*l*

_{1}-norm of |Φ

**X|**. The parameter

*α*defines the step length for

*l*

_{1}-minimization at each iteration process. The reconstruction is terminated when the convergence criterion ||

**X**

^{(}

^{k}^{+1)}−

**X**

*||*

^{k}^{2}<

*ε*is reached.

### 2.4. Experimental Setup

## 3. Results

### 3.1. Experiment with Derenzo-like Geometry

^{12}= 4096). Fig. 6(b) shows the result calculated with the inverse mapping method, and Fig. 6(c) is the result estimated from the iterative algorithm. The parameter

*α*was selected to 0.25, and the reconstruction stopped at 37 iterations. Both reconstruction results were normalized to the same total sum of pixel intensities, and shown with the scale, 0–1, for better comparison. Fig. 6(d) shows the profiles for both reconstruction results. When

*α*was set to 0 (identical to SIRT reconstruction), the reconstruction process did converge after 338 iterations and the result is shown in Fig. 7.

*α*for the iterative method was chosen as 0.00075, and the reconstruction process did converge after 15 iterations.

### 3.2. Experiment with Siemens Star

*α*was set to 0.4, 0.3 and 0.25 for the measurements with the distances of 25 mm, 30 mm and 35 mm, respectively. The spatial resolutions are then calculated in line pairs per millimeter according to the minimum detectable contrast as listed in table 1.

## 4. Discussion and Conclusion

*M*used in the inverse mapping approach is about 10–20. With such magnification the enlarged mappings of the elemental images at the selected reconstruction plane are greatly overlapped with their neighbors. This overlapping and superposition can cause blurring of the reconstructed images and degradation of the resolution of the system. The iterative algorithm solves the problem in the other way round. It searches the optimized estimation that has the least difference between the computer calculated elemental images and the measured sensor data. Since the inverse problem is not unique, the proposed algorithm tries to look for the estimation that is locally smooth according to the theory of compressive sensing. The sparsifying transform Φ in the algorithm is designed in such a way to ensure the local smoothness and, in the meanwhile, to reduce the square-like artifact from the natural characteristic of microlens-based imaging [15

15. S. Hong and B. Javidi, “Improved resolution 3D object reconstruction using computational integral imaging with time multiplexing,” Opt. Express **12**, 4579–4588 (2004). [CrossRef] [PubMed]

28. M. Martínez-Corral, B. Javidi, R. Martínez-Cuenca, and G. Saavedra, “Multifacet structure of observed reconstructed integral images,” J. Opt. Soc. Am. A **22**, 597–603 (2005). [CrossRef]

*in vivo*imaging applications. It has been included here to illustrate the potential of the proposed algorithm in general.

*α*is according to the overall illuminance level of the acquisition. If

*α*is too large, the estimated image becomes uniform, losing the information from the raw detected data. On the other hand, if the parameter is too small (set to 0, as the extreme case), the algorithm is then reduced to SIRT reconstruction. In such case, the reconstruction is converged to a result that is filled with sharp speckles (Fig. 7). For the results shown in this article we selected

*α*as between 1/10000 and 1/8000 of max grey value of the raw detector data based on various tests. This selection keeps a good balance between the SIRT step and the steepest gradient descent of the l1-norm. The method seems to be robust towards the slight changes to the

*α*parameter. Further investigation of the selection of parameters may improve the performance.

29. D. Shin, E. Kim, and B. Lee, “Computational reconstruction of three-dimensional objects in integral imaging using lenslet array,” Jpn. J. Appl. Phys. **44**, 8016–8018 (2005). [CrossRef]

**A′**instead of considering a pinhole array for the simulation of ray tracing. The remaining iteration steps presented in this paper are the same.

*z*is known. This precondition can be easily overcome as our optical detector is supposed to acquire data simultaneously with other imaging modalities, such as PET and MRI. Hence the distance information of a three-dimensional object surface can be extracted from the modality easily.

## References and links

1. | N. Henriquez, P. van Overveld, I. Que, J. Buijs, R. Bachelier, E. Kaijzel, C. Löwik, P. Clezardin, and G. van der Pluijm, “Advances in optical imaging and novel model systems for cancer metastasis research,” Clin. Exp. Metastasis |

2. | D. Rowland, J. Lewis, and M. Welch, “Molecular imaging: the application of small animal positron emission tomography,” J. Cell. Biochem. |

3. | G. Kelloff, K. Krohn, S. Larson, R. Weissleder, D. Mankoff, J. Hoffman, J. Link, K. Guyton, W. Eckelman, H. Scher, J. O’Shaughnessy, B. D. Cheson, C. C. Sigman, J. L. Tatum, G. Q. Mills, D. C. Sullivan, and J. Woodcock, “The progress and promise of molecular imaging probes in oncologic drug development,” Clin. Cancer Res. |

4. | R. Weissleder and V. Ntziachristos, “Shedding light onto live molecular targets,” Nat. Med. |

5. | J. Peter, D. Unholtz, R. Schulz, J. Doll, and W. Semmler, “Development and initial results of a tomographic dual-modality positron/optical small animal imager,” IEEE Trans. Nucl. Sci. |

6. | D. Unholtz, W. Semmler, O. Dössel, and J. Peter, “Image formation with a microlens-based optical detector: a three-dimensional mapping approach,” Appl. Opt. |

7. | R. Martinez-Cuenca, G. Saavedra, M. Martinez-Corral, and B. Javidi, “Progress in 3-D multiperspective display by integral imaging,” Proc. IEEE |

8. | M. Cho, M. Daneshpanah, I. Moon, and B. Javidi, “Three-dimensional optical sensing and visualization using integral imaging,” Proc. IEEE |

9. | A. Stern and B. Javidi, “Three-dimensional image sensing, visualization, and processing using integral imaging,” Proc. IEEE |

10. | N. Davies, M. McCormick, and M. Brewin, “Design and analysis of an image transfer system using microlens arrays (Journal Paper),” Opt. Eng. |

11. | S. Min, B. Javidi, and B. Lee, “Enhanced three-dimensional integral imaging system by use of double display devices,” Appl. Opt. |

12. | J. Arai, F. Okano, H. Hoshino, and I. Yuyama, “Gradient-index lens-array method based on real-time integral photography for three-dimensional images,” Appl. Opt. |

13. | H. Arimoto and B. Javidi, “Integral three-dimensional imaging with digital reconstruction,” Opt. Lett. |

14. | S. Hong, J. Jang, and B. Javidi, “Three-dimensional volumetric object reconstruction using computational integral imaging,” Opt. Express |

15. | S. Hong and B. Javidi, “Improved resolution 3D object reconstruction using computational integral imaging with time multiplexing,” Opt. Express |

16. | J. Park, K. Hong, and B. Lee, “Recent progress in three-dimensional information processing based on integral imaging,” Appl. Opt. |

17. | J. Jang and B. Javidi, “Improved viewing resolution of three-dimensional integral imaging by use of nonstationary micro-optics,” Opt. Lett. |

18. | D. Shin and H. Yoo, “Image quality enhancement in 3D computational integral imaging by use of interpolation methods,” Opt. Express |

19. | D. Hwang, J. Park, S. Kim, D. Shin, and E. Kim, “Magnification of 3D reconstructed images in integral imaging using an intermediate-view reconstruction technique,” Appl. Opt. |

20. | K. Lee, D. Hwang, S. Kim, and E. Kim, “Blur-metric-based resolution enhancement of computationally reconstructed integral images,” Appl. Opt. |

21. | G. Saavedra, R. Martinez-Cuenca, M. Martinez-Corral, H. Navarro, M. Daneshpanah, and B. Javidi, “Digital slicing of 3D scenes by Fourier filtering of integral images,” Opt. Express |

22. | J. Qi and R. Leahy, “Iterative reconstruction techniques in emission computed tomography,” Phys. Med. Biol. |

23. | E. Sidky, C. Kao, and X. Pan, “Accurate image reconstruction from few-views and limited-angle data in divergent-beam CT,” J. X-Ray Sci. Technol. |

24. | G. Chen, J. Tang, and S. Leng, “Prior image constrained compressed sensing (PICCS): a method to accurately reconstruct dynamic CT images from highly undersampled projection data sets,” Med. Phys. |

25. | M. Katz, |

26. | E. Candès, J. Romberg, and T. Tao, “Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information,” IEEE Trans. Inf. Theory |

27. | D. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory |

28. | M. Martínez-Corral, B. Javidi, R. Martínez-Cuenca, and G. Saavedra, “Multifacet structure of observed reconstructed integral images,” J. Opt. Soc. Am. A |

29. | D. Shin, E. Kim, and B. Lee, “Computational reconstruction of three-dimensional objects in integral imaging using lenslet array,” Jpn. J. Appl. Phys. |

**OCIS Codes**

(080.0080) Geometric optics : Geometric optics

(120.3890) Instrumentation, measurement, and metrology : Medical optics instrumentation

(170.3010) Medical optics and biotechnology : Image reconstruction techniques

**ToC Category:**

Medical Optics and Biotechnology

**History**

Original Manuscript: February 14, 2011

Revised Manuscript: April 13, 2011

Manuscript Accepted: April 14, 2011

Published: June 6, 2011

**Virtual Issues**

Vol. 6, Iss. 7 *Virtual Journal for Biomedical Optics*

**Citation**

Liji Cao and Jörg Peter, "Iterative reconstruction of projection images from a microlens-based optical detector," Opt. Express **19**, 11932-11943 (2011)

http://www.opticsinfobase.org/vjbo/abstract.cfm?URI=oe-19-13-11932

Sort: Year | Journal | Reset

### References

- N. Henriquez, P. van Overveld, I. Que, J. Buijs, R. Bachelier, E. Kaijzel, C. Löwik, P. Clezardin, and G. van der Pluijm, “Advances in optical imaging and novel model systems for cancer metastasis research,” Clin. Exp. Metastasis 24, 699–705 (2007). [CrossRef] [PubMed]
- D. Rowland, J. Lewis, and M. Welch, “Molecular imaging: the application of small animal positron emission tomography,” J. Cell. Biochem. 87, 110–115 (2002). [CrossRef]
- G. Kelloff, K. Krohn, S. Larson, R. Weissleder, D. Mankoff, J. Hoffman, J. Link, K. Guyton, W. Eckelman, H. Scher, J. O’Shaughnessy, B. D. Cheson, C. C. Sigman, J. L. Tatum, G. Q. Mills, D. C. Sullivan, and J. Woodcock, “The progress and promise of molecular imaging probes in oncologic drug development,” Clin. Cancer Res. 11, 7967–7985 (2005). [CrossRef] [PubMed]
- R. Weissleder and V. Ntziachristos, “Shedding light onto live molecular targets,” Nat. Med. 9, 123–128 (2003). [CrossRef] [PubMed]
- J. Peter, D. Unholtz, R. Schulz, J. Doll, and W. Semmler, “Development and initial results of a tomographic dual-modality positron/optical small animal imager,” IEEE Trans. Nucl. Sci. 54, 1553–1560 (2007). [CrossRef]
- D. Unholtz, W. Semmler, O. Dössel, and J. Peter, “Image formation with a microlens-based optical detector: a three-dimensional mapping approach,” Appl. Opt. 48, D273–D279 (2009). [CrossRef] [PubMed]
- R. Martinez-Cuenca, G. Saavedra, M. Martinez-Corral, and B. Javidi, “Progress in 3-D multiperspective display by integral imaging,” Proc. IEEE 97, 1067–1077 (2009). [CrossRef]
- M. Cho, M. Daneshpanah, I. Moon, and B. Javidi, “Three-dimensional optical sensing and visualization using integral imaging,” Proc. IEEE 99, 556–575 (2011). [CrossRef]
- A. Stern and B. Javidi, “Three-dimensional image sensing, visualization, and processing using integral imaging,” Proc. IEEE 94, 591 –607 (2006). [CrossRef]
- N. Davies, M. McCormick, and M. Brewin, “Design and analysis of an image transfer system using microlens arrays (Journal Paper),” Opt. Eng. 33, 3624–3633 (1994). [CrossRef]
- S. Min, B. Javidi, and B. Lee, “Enhanced three-dimensional integral imaging system by use of double display devices,” Appl. Opt. 42, 4186–4195 (2003). [CrossRef] [PubMed]
- J. Arai, F. Okano, H. Hoshino, and I. Yuyama, “Gradient-index lens-array method based on real-time integral photography for three-dimensional images,” Appl. Opt. 37, 2034–2045 (1998). [CrossRef]
- H. Arimoto and B. Javidi, “Integral three-dimensional imaging with digital reconstruction,” Opt. Lett. 26, 157–159 (2001). [CrossRef]
- S. Hong, J. Jang, and B. Javidi, “Three-dimensional volumetric object reconstruction using computational integral imaging,” Opt. Express 12, 483–491 (2004). [CrossRef] [PubMed]
- S. Hong and B. Javidi, “Improved resolution 3D object reconstruction using computational integral imaging with time multiplexing,” Opt. Express 12, 4579–4588 (2004). [CrossRef] [PubMed]
- J. Park, K. Hong, and B. Lee, “Recent progress in three-dimensional information processing based on integral imaging,” Appl. Opt. 48, 77–94 (2009). [CrossRef]
- J. Jang and B. Javidi, “Improved viewing resolution of three-dimensional integral imaging by use of nonstationary micro-optics,” Opt. Lett. 27, 324–326 (2002). [CrossRef]
- D. Shin and H. Yoo, “Image quality enhancement in 3D computational integral imaging by use of interpolation methods,” Opt. Express 15, 12039–12049 (2007). [CrossRef] [PubMed]
- D. Hwang, J. Park, S. Kim, D. Shin, and E. Kim, “Magnification of 3D reconstructed images in integral imaging using an intermediate-view reconstruction technique,” Appl. Opt. 45, 4631–4637 (2006). [CrossRef] [PubMed]
- K. Lee, D. Hwang, S. Kim, and E. Kim, “Blur-metric-based resolution enhancement of computationally reconstructed integral images,” Appl. Opt. 47, 2859–2869 (2008). [CrossRef] [PubMed]
- G. Saavedra, R. Martinez-Cuenca, M. Martinez-Corral, H. Navarro, M. Daneshpanah, and B. Javidi, “Digital slicing of 3D scenes by Fourier filtering of integral images,” Opt. Express 16, 17154–17160 (2008). [CrossRef] [PubMed]
- J. Qi and R. Leahy, “Iterative reconstruction techniques in emission computed tomography,” Phys. Med. Biol. 51, R541–R578 (2006). [CrossRef] [PubMed]
- E. Sidky, C. Kao, and X. Pan, “Accurate image reconstruction from few-views and limited-angle data in divergent-beam CT,” J. X-Ray Sci. Technol. 14, 119–139 (2006).
- G. Chen, J. Tang, and S. Leng, “Prior image constrained compressed sensing (PICCS): a method to accurately reconstruct dynamic CT images from highly undersampled projection data sets,” Med. Phys. 35, 660–663 (2008). [CrossRef] [PubMed]
- M. Katz, Introduction to Geometrical Optics (World Scientific Pub. Co. Inc., 2002).
- E. Candès, J. Romberg, and T. Tao, “Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information,” IEEE Trans. Inf. Theory 52, 489–509 (2006). [CrossRef]
- D. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory 52, 1289–1306 (2006). [CrossRef]
- M. Martínez-Corral, B. Javidi, R. Martínez-Cuenca, and G. Saavedra, “Multifacet structure of observed reconstructed integral images,” J. Opt. Soc. Am. A 22, 597–603 (2005). [CrossRef]
- D. Shin, E. Kim, and B. Lee, “Computational reconstruction of three-dimensional objects in integral imaging using lenslet array,” Jpn. J. Appl. Phys. 44, 8016–8018 (2005). [CrossRef]

## Cited By |
Alert me when this paper is cited |

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

« Previous Article | Next Article »

OSA is a member of CrossRef.