## Single exposure super-resolution compressive imaging by double phase encoding |

Optics Express, Vol. 18, Issue 14, pp. 15094-15103 (2010)

http://dx.doi.org/10.1364/OE.18.015094

Acrobat PDF (4318 KB)

### Abstract

Super-resolution is an important goal of many image acquisition systems. Here we demonstrate the possibility of achieving super-resolution with a single exposure by combining the well known optical scheme of double random phase encoding which has been traditionally used for encryption with results from the relatively new and emerging field of compressive sensing. It is shown that the proposed model can be applied for recovering images from a general image degrading model caused by both diffraction and geometrical limited resolution.

© 2010 OSA

## 1. Introduction

1. S. Park, M. Park, and M. Gang, “Super-resolution image reconstruction: a technical overview,” IEEE Signal Process. Mag. **20**(3), 21–36 (2003). [CrossRef]

7. A. Borkowski, Z. Zalevsky, and B. Javidi, “Geometrical superresolved imaging using nonperiodic spatial masking,” J. Opt. Soc. Am. A **26**(3), 589–601 (2009). [CrossRef]

5. A. Stern, Y. Porat, A. Ben-Dor, and N. S. Kopeika, “Enhanced-resolution image restoration from a sequence of low-frequency vibrated images by use of convex projections,” Appl. Opt. **40**(26), 4706–4715 (2001). [CrossRef]

7. A. Borkowski, Z. Zalevsky, and B. Javidi, “Geometrical superresolved imaging using nonperiodic spatial masking,” J. Opt. Soc. Am. A **26**(3), 589–601 (2009). [CrossRef]

5. A. Stern, Y. Porat, A. Ben-Dor, and N. S. Kopeika, “Enhanced-resolution image restoration from a sequence of low-frequency vibrated images by use of convex projections,” Appl. Opt. **40**(26), 4706–4715 (2001). [CrossRef]

7. A. Borkowski, Z. Zalevsky, and B. Javidi, “Geometrical superresolved imaging using nonperiodic spatial masking,” J. Opt. Soc. Am. A **26**(3), 589–601 (2009). [CrossRef]

3. S. Farsiu, M. D. Robinson, M. Elad, and P. Milanfar, “Fast and robust multiframe super resolution,” IEEE Trans. Image Process. **13**(10), 1327–1344 (2004). [CrossRef] [PubMed]

1. S. Park, M. Park, and M. Gang, “Super-resolution image reconstruction: a technical overview,” IEEE Signal Process. Mag. **20**(3), 21–36 (2003). [CrossRef]

4. S. Prasad and X. Luo, “Support-assisted optical superresolution of low-resolution image sequences: the one-dimensional problem,” Opt. Express **17**(25), 23213–23233 (2009). [CrossRef]

*single*exposure without sacrificing the field of view, or requiring any other measurement dimensions. The key to achieve SR without additional data is by utilizing the fact that the information in all human intelligible images is very redundant. Therefore, instead of acquiring extra data we properly encode the data within one image and apply reconstruction algorithms that reconstruct the desired high resolution image.

8. P. Réfrégier and B. Javidi, “Optical image encryption based on input plane and Fourier plane random encoding,” Opt. Lett. **20**(7), 767–769 (1995). [CrossRef] [PubMed]

9. B. Javidi, G. Zhang, and J. Li, “Encrypted optical memory using double-random phase encoding,” Appl. Opt. **36**(5), 1054–1058 (1997). [CrossRef] [PubMed]

*both*diffraction and geometrical SR. It uses a single shot acquisition process and does not sacrifice any measurement dimensions. Unlike some conventional SR systems, the approach proposed here does not require any movements.

## 2. Double random phase encoding (DRPE)

8. P. Réfrégier and B. Javidi, “Optical image encryption based on input plane and Fourier plane random encoding,” Opt. Lett. **20**(7), 767–769 (1995). [CrossRef] [PubMed]

9. B. Javidi, G. Zhang, and J. Li, “Encrypted optical memory using double-random phase encoding,” Appl. Opt. **36**(5), 1054–1058 (1997). [CrossRef] [PubMed]

11. E. Tajahuerce, O. Matoba, S. C. Verrall, and B. Javidi, “Optoelectronic information encryption with phase-shifting interferometry,” Appl. Opt. **39**(14), 2313–2320 (2000). [CrossRef]

13. O. Matoba and B. Javidi, “Encrypted optical memory system using three-dimensional keys in the Fresnel domain,” Opt. Lett. **24**(11), 762–764 (1999). [CrossRef]

14. G. Unnikrishnan, J. Joseph, and K. Singh, “Optical encryption by double-random phase encoding in the fractional Fourier domain,” Opt. Lett. **25**(12), 887–889 (2000). [CrossRef]

15. P. C. Mogensen and J. Glückstad, “Phase-only optical encryption,” Opt. Lett. **25**(8), 566–568 (2000). [CrossRef]

19. X. Tan, O. Matoba, Y. Okada-Shudo, M. Ide, T. Shimura, and K. Kuroda, “Secure optical memory system with polarization encryption,” Appl. Opt. **40**(14), 2310–2315 (2001). [CrossRef]

*universal*CS scheme, as demonstrated in the next section.

## 3. Double random phase encoder as a universal compressive sensing encoder

### 3.1 Brief introduction to Compressive sensing

20. D. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory **52**(4), 1289–1306 (2006). [CrossRef]

**t**which is described by an

*N*dimensional real valued vector (in case that the object represents an image of

*N*pixels,

**t**is a one dimensional vector obtained by rearranging the image in a lexicographic order) being projected (imaged) to

**u**which is an

*M*dimensional vector. One can also think of

*M*as the number of detector pixels. In CS we are interested in the case of

*M*<

*N*, i.e., the signal is undersampled according to the Shannon-Nyquist theorem. The sensing process is given by:where

**Φ**is an

*M*by

*N*matrix.

21. E. Candes and M. Wakin, “An introduction to compressive sampling,” IEEE Signal Process. Mag. **25**(2), 21–30 (2008). [CrossRef]

**t**can be sparsely represented in some arbitrary orthonormal basis

**Ψ**(e.g., wavelet, or DCT). Thus,

**α**is the

*K*-sparse representation of image

**t**projected on

**Ψ**

^{T}, meaning, that

**α**has only

*K*non-zero terms.

**Φ**and

**Ψ**respectively,

*N*is the length of the column vector. The mutual coherence is bounded by

21. E. Candes and M. Wakin, “An introduction to compressive sampling,” IEEE Signal Process. Mag. **25**(2), 21–30 (2008). [CrossRef]

**Φ**, can be actually recovered by

*l*

_{1}-norm minimization. The estimated coefficients vector

21. E. Candes and M. Wakin, “An introduction to compressive sampling,” IEEE Signal Process. Mag. **25**(2), 21–30 (2008). [CrossRef]

_{1}-norm. One way of guaranteeing recovery via the ℓ

_{1}-norm minimization of a

*K*-sparse signal

**t**is by taking

*M*measurements satisfying [20

20. D. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory **52**(4), 1289–1306 (2006). [CrossRef]

**25**(2), 21–30 (2008). [CrossRef]

*C*is some small positive constant. The role of the mutual coherence becomes clear. The larger it is, the more samples you need. One can also think of the mutual coherence as a measure of how much the projection

**Φ**spreads the information among many entries. Thus, if every coefficient from the sparsely represented signal is spread on many projections, we may have a better chance of reconstructing the signal from less available samples. A Gaussian random sensing basis is often chosen since it is universal CS operator, meaning that it fits to signals sparse in any domain [21

**25**(2), 21–30 (2008). [CrossRef]

**Ψ**.

*M*out of

*N*measurements uniformly at random [20

20. D. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory **52**(4), 1289–1306 (2006). [CrossRef]

**25**(2), 21–30 (2008). [CrossRef]

*M*measurements, which is more relevant to our physically constrained blurring and sampling scheme.

### 3.2 Double phase encoding as a universal sensing operator

**25**(2), 21–30 (2008). [CrossRef]

**16

16. B. M. Hennelly, T. J. Naughton, J. McDonald, J. T. Sheridan, G. Unnikrishnan, D. P. Kelly, and B. Javidi, “Spread-space spread-spectrum technique for secure multiplexing,” Opt. Lett. **32**(9), 1060–1062 (2007). [CrossRef] [PubMed]

*compressive signals, in order to guarantee the reconstruction of the signal, that is, the amount of samples we need to reconstruct a signal*sensed in DRPE process.

**F**is the

*N*

^{2}x

*N*

^{2}discrete Fourier transform matrix. The input

**t**and output

**u**are an

*N*x1 lexicographical arrangement of the input and output fields, respectively. All the matrices are of size

^{2}*N*

^{2}x

*N*

^{2}. Thus, we may write the

**FP**in Eq. (6) representing random scrambling in the frequency domain matrix as:

*p*is drawn from a uniform distribution on {0,1}. Please recall that all the entries of

_{i}*b(u,v)*and

*p(x,y)*are drawn independently from a uniform distribution between [0,1]; therefore, since

**FP**holds an inter-column statistical independence. In this sense the

**FP**operator behaves equivalently to the operation of the random Gaussian sensing scheme [22]. Now, according to most CS implementations, at this stage, we should randomly sub-sample

**FPt**, in order to guarantee the measurements independence [

**21

**25**(2), 21–30 (2008). [CrossRef]

**F*H**operator in Eq. (6). Performing

**F*H**has the de-correlating effect on the result of the

**FPt**operator (similar to the effect

**FP**had on

**t**). This can also be seen as guaranteeing inter-row statistical independence. Now, a statistical independence between the measurements is guaranteed. After the second scrambling, the signal may undergo a deterministic blurring or sub-sampling, and enjoy the powerful results of CS theory such as described in section 3.

## 4. Super resolution with double random phase encoding

### 4.1 DRPE image degradation model

*x*and

*y*directions. The random phase masks have a pixel size of

*L*samples. The

*h*

_{s}operator accounts for the entire blurring caused by the optical system, including blur due to sensor's geometrical limits, small NA (diffraction limited imaging), motion blur and defocusing blur.

*h*. Therefore, without the first multiplication, the high spatial frequencies are lost due the sensor or optics limitations and there is no practical way they could be fully recovered. In the next section, we demonstrate that this way of encoding provides an effective spatial bandwidth extension both for the case that

_{s}*h*is dominated by the pixelated sensor or determined by diffraction.

_{s}### 4.2 Geometrical sub-sampling

*L*and

_{x}*L*in the

_{y}*x*and

*y*directions, respectively, i.e. each image pixel has the size of

*N/L*where

*L*=

*L*

_{x}

*L*

_{y}. The image obtained with the DRPE system is given by:whererepresents the averaging (integration) over the sensor pixel area.

*L*= 4 this operator is written in a matrix notation (for a 1-D signal) as:where

**S**denotes the sub-sampling matrix, and

**D**performs the averaging (low-pass) operation. In order to reconstruct the object

**t**from the measured

**u**according Eq. (9) we choose to solve the problem:where TV stands for the total variation operator defined asThe measured signal is denoted by

**u**, and

**Ψ**, is the sparsifying operator. The TV functional used here is well-known in signal and image processing for its tendency to suppress spurious high-frequency features.

*∆*by 4

*∆,*i.e. 2 and 4 times larger in the vertical and horizontal directions, respectively. Accordingly, the number of pixels in the captured image was

*N/L*x

_{x}*N/L*, with

_{y}*L*= 2 and

_{x}*L*= 4. Figure 4(b) is obtained by averaging and sub-sampling the output data from the DRPE system (Fig. 1(a)) 4 times less in the horizontal direction and 2 times in the vertical direction. As a result, the CCD captures 512x256 pixels, with pixels size of 2

_{y}*∆*x4

*∆.*Fig. 4(c) shows the reconstructed image after solving Eq. (12) for the output of the DPRE-CS system. Figure 4 (d) - (f) show a zoom in on the finest resolution details, comparing the low-resolution image obtained with conventional imaging system to the super-resolution image obtained with DRPE-CS. It is evident that the DRPE-CS method resolves almost perfectly the finest details that are obviously lost with conventional imaging systems. It can be seen in Fig. 4 (e) that details which correspond to 1/4 line pairs per pixel are irresolvable, while in Fig. 4(f) the finest detail resolution corresponding to 1/2 line pairs per pixel is evident. Thus a resolution gain of at least 2 is demonstrated.

23. J. Romberg, “Compressive sensing by random convolution,” SIAM J. Imaging Sci. **2**(4), 1098–1128 (2009). [CrossRef]

23. J. Romberg, “Compressive sensing by random convolution,” SIAM J. Imaging Sci. **2**(4), 1098–1128 (2009). [CrossRef]

23. J. Romberg, “Compressive sensing by random convolution,” SIAM J. Imaging Sci. **2**(4), 1098–1128 (2009). [CrossRef]

**22]. Thus, we also offer the connection between the two approaches for structured CS. Despite the similarity between the model in Eq. (9) and that of Eq. (14), the DPRE-CS offers some practical advantages, which are described in the next subsection.

### 4.3 Lens blurring

*f*as the nominal cutoff frequency of the lens aperture in order to capture the image without blurring. In such a case we have to solve the following:where

_{nom}**A**is the matrix-vector representation describing the aperture function, in the spatial frequency domain. Figure 5 presents simulation results for a lens with a cutoff spatial radial frequency 6 times smaller than the one required for no blurring,, i.e., with a spatial frequency cutoff

*f*represents the diffraction cutoff frequency matched to the rest of the system. In our simulations,

_{nom}*f*is the cutoff frequency set by the object's pixel size. We also added measurement noise to the captured image such that the SNR was 37dB. The reconstruction of the fine resolution details using DRPE-CS strategy is evident from the zooming in of Fig. 5 despite of the substantial spatial frequency sub-sampling. We can notice in Fig. 5 (d) that due to blurring, targets with spatial frequency larger than 1/12 line pairs per pixels became irresolvable using conventional imaging. On the other hand, when acquiring and reconstructing data using the DRPE-CS, details corresponding to 1/2 line pairs per pixels (see Fig. 5 (e)) are clearly resolvable. Hence SR by approximately a factor of 6 is demonstrated.

_{nom}### 4.4 General degrading model

**2**(4), 1098–1128 (2009). [CrossRef]

*h*immediately on the input signal

_{diff}**t**all the high frequencies are filtered out before entering the sensing system. This information would be lost forever and cannot be uniquely reconstructed.

## 5. Conclusions

## Acknowledgements

## References and links

1. | S. Park, M. Park, and M. Gang, “Super-resolution image reconstruction: a technical overview,” IEEE Signal Process. Mag. |

2. | Z. Zalevsky and D. Mendlovic, |

3. | S. Farsiu, M. D. Robinson, M. Elad, and P. Milanfar, “Fast and robust multiframe super resolution,” IEEE Trans. Image Process. |

4. | S. Prasad and X. Luo, “Support-assisted optical superresolution of low-resolution image sequences: the one-dimensional problem,” Opt. Express |

5. | A. Stern, Y. Porat, A. Ben-Dor, and N. S. Kopeika, “Enhanced-resolution image restoration from a sequence of low-frequency vibrated images by use of convex projections,” Appl. Opt. |

6. | J. García, Z. Zalevsky, and D. Fixler, “Synthetic aperture superresolution by speckle pattern projection,” Opt. Express |

7. | A. Borkowski, Z. Zalevsky, and B. Javidi, “Geometrical superresolved imaging using nonperiodic spatial masking,” J. Opt. Soc. Am. A |

8. | P. Réfrégier and B. Javidi, “Optical image encryption based on input plane and Fourier plane random encoding,” Opt. Lett. |

9. | B. Javidi, G. Zhang, and J. Li, “Encrypted optical memory using double-random phase encoding,” Appl. Opt. |

10. | O. Matoba, T. Nomura, E. Perez-Cabre, M. S. Millan, and B. Javidi, “Optical techniques for information security,” Proc. IEEEl |

11. | E. Tajahuerce, O. Matoba, S. C. Verrall, and B. Javidi, “Optoelectronic information encryption with phase-shifting interferometry,” Appl. Opt. |

12. | E. Tajahuerce, J. Lancis, P. Andres, V. Climent, and B. Javidi, “Optoelectronic Information Encryption with Incoherent Light,” in |

13. | O. Matoba and B. Javidi, “Encrypted optical memory system using three-dimensional keys in the Fresnel domain,” Opt. Lett. |

14. | G. Unnikrishnan, J. Joseph, and K. Singh, “Optical encryption by double-random phase encoding in the fractional Fourier domain,” Opt. Lett. |

15. | P. C. Mogensen and J. Glückstad, “Phase-only optical encryption,” Opt. Lett. |

16. | B. M. Hennelly, T. J. Naughton, J. McDonald, J. T. Sheridan, G. Unnikrishnan, D. P. Kelly, and B. Javidi, “Spread-space spread-spectrum technique for secure multiplexing,” Opt. Lett. |

17. | O. Matoba and B. Javidi, “Encrypted optical storage with angular multiplexing,” Appl. Opt. |

18. | E. Tajahuerce and B. Javidi, “Encrypting three-dimensional information with digital holography,” Appl. Opt. |

19. | X. Tan, O. Matoba, Y. Okada-Shudo, M. Ide, T. Shimura, and K. Kuroda, “Secure optical memory system with polarization encryption,” Appl. Opt. |

20. | D. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory |

21. | E. Candes and M. Wakin, “An introduction to compressive sampling,” IEEE Signal Process. Mag. |

22. | T. Do, T. Tran, and L. Gan, “Fast compressive sampling with structurally random matrices,” in Proc. ICASSP, 3369–3372, (2008). |

23. | J. Romberg, “Compressive sensing by random convolution,” SIAM J. Imaging Sci. |

24. | Y. Rivenson, A. Stern, and B. Javidi, “Compressive Fresnel Holography,” to appear in IEEE/OSA J. on Display Technology, (2010). |

25. | A. Stern and B. Javidi, “Random projections imaging with extended space-bandwidth product,” IEEE/OSA Journal on Display Technology, |

**OCIS Codes**

(100.2000) Image processing : Digital image processing

(100.6640) Image processing : Superresolution

**ToC Category:**

Image Processing

**History**

Original Manuscript: April 7, 2010

Revised Manuscript: June 18, 2010

Manuscript Accepted: June 23, 2010

Published: June 30, 2010

**Citation**

Yair Rivenson, Adrian Stern, and Bahram Javidi, "Single exposure super-resolution compressive imaging by double phase encoding," Opt. Express **18**, 15094-15103 (2010)

http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-18-14-15094

Sort: Year | Journal | Reset

### References

- S. Park, M. Park, and M. Gang, “Super-resolution image reconstruction: a technical overview,” IEEE Signal Process. Mag. 20(3), 21–36 (2003). [CrossRef]
- Z. Zalevsky and D. Mendlovic, Optical Super Resolution (Springer-Verlag, 2003).
- S. Farsiu, M. D. Robinson, M. Elad, and P. Milanfar, “Fast and robust multiframe super resolution,” IEEE Trans. Image Process. 13(10), 1327–1344 (2004). [CrossRef] [PubMed]
- S. Prasad and X. Luo, “Support-assisted optical superresolution of low-resolution image sequences: the one-dimensional problem,” Opt. Express 17(25), 23213–23233 (2009). [CrossRef]
- A. Stern, Y. Porat, A. Ben-Dor, and N. S. Kopeika, “Enhanced-resolution image restoration from a sequence of low-frequency vibrated images by use of convex projections,” Appl. Opt. 40(26), 4706–4715 (2001). [CrossRef]
- J. García, Z. Zalevsky, and D. Fixler, “Synthetic aperture superresolution by speckle pattern projection,” Opt. Express 13(16), 6073–6078 (2005). [CrossRef] [PubMed]
- A. Borkowski, Z. Zalevsky, and B. Javidi, “Geometrical superresolved imaging using nonperiodic spatial masking,” J. Opt. Soc. Am. A 26(3), 589–601 (2009). [CrossRef]
- P. Réfrégier and B. Javidi, “Optical image encryption based on input plane and Fourier plane random encoding,” Opt. Lett. 20(7), 767–769 (1995). [CrossRef] [PubMed]
- B. Javidi, G. Zhang, and J. Li, “Encrypted optical memory using double-random phase encoding,” Appl. Opt. 36(5), 1054–1058 (1997). [CrossRef] [PubMed]
- O. Matoba, T. Nomura, E. Perez-Cabre, M. S. Millan, and B. Javidi, “Optical techniques for information security,” Proc. IEEEl 97(6), 1128–1148 (2009). [CrossRef]
- E. Tajahuerce, O. Matoba, S. C. Verrall, and B. Javidi, “Optoelectronic information encryption with phase-shifting interferometry,” Appl. Opt. 39(14), 2313–2320 (2000). [CrossRef]
- E. Tajahuerce, J. Lancis, P. Andres, V. Climent, and B. Javidi, “Optoelectronic Information Encryption with Incoherent Light,” in Optical and Digital Techniques for Information Security, B. Javidi, ed. (Springer-Verlag, 2004).
- O. Matoba and B. Javidi, “Encrypted optical memory system using three-dimensional keys in the Fresnel domain,” Opt. Lett. 24(11), 762–764 (1999). [CrossRef]
- G. Unnikrishnan, J. Joseph, and K. Singh, “Optical encryption by double-random phase encoding in the fractional Fourier domain,” Opt. Lett. 25(12), 887–889 (2000). [CrossRef]
- P. C. Mogensen and J. Glückstad, “Phase-only optical encryption,” Opt. Lett. 25(8), 566–568 (2000). [CrossRef]
- B. M. Hennelly, T. J. Naughton, J. McDonald, J. T. Sheridan, G. Unnikrishnan, D. P. Kelly, and B. Javidi, “Spread-space spread-spectrum technique for secure multiplexing,” Opt. Lett. 32(9), 1060–1062 (2007). [CrossRef] [PubMed]
- O. Matoba and B. Javidi, “Encrypted optical storage with angular multiplexing,” Appl. Opt. 38(35), 7288–7293 (1999). [CrossRef]
- E. Tajahuerce and B. Javidi, “Encrypting three-dimensional information with digital holography,” Appl. Opt. 39(35), 6595–6601 (2000). [CrossRef]
- X. Tan, O. Matoba, Y. Okada-Shudo, M. Ide, T. Shimura, and K. Kuroda, “Secure optical memory system with polarization encryption,” Appl. Opt. 40(14), 2310–2315 (2001). [CrossRef]
- D. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006). [CrossRef]
- E. Candes and M. Wakin, “An introduction to compressive sampling,” IEEE Signal Process. Mag. 25(2), 21–30 (2008). [CrossRef]
- T. Do, T. Tran, and L. Gan, “Fast compressive sampling with structurally random matrices,” in Proc. ICASSP, 3369–3372, (2008).
- J. Romberg, “Compressive sensing by random convolution,” SIAM J. Imaging Sci. 2(4), 1098–1128 (2009). [CrossRef]
- Y. Rivenson, A. Stern, and B. Javidi, “Compressive Fresnel Holography,” to appear in IEEE/OSA J. on Display Technology, (2010).
- A. Stern and B. Javidi, “Random projections imaging with extended space-bandwidth product,” IEEE/OSA Journal on Display Technology, 3(3), 315–320 (2007).

## Cited By |
Alert me when this paper is cited |

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

« Previous Article | Next Article »

OSA is a member of CrossRef.