## A depth estimation algorithm with a single image

Optics Express, Vol. 15, Issue 8, pp. 5024-5029 (2007)

http://dx.doi.org/10.1364/OE.15.005024

Acrobat PDF (138 KB)

### Abstract

This paper explains the use of a sharpening filter to calculate the depth of an object from a blurred image of it. It presents a technique which is independent of edge orientation. The technique is based on the assumption that a defocused image of an object is the convolution of a sharp image of the same object with a two-dimensional Gaussian function whose spread parameter (SP) is related to the object depth. A sharp image of an object is obtained from a defocused image of the same object by applying sharpening filters. The defocused and sharp images of the object are used to calculate the SP which is then related to the object depth. The paper gives experimental results which show the feasibility of employing sharpening filters for depth computation.

© 2007 Optical Society of America

## 1. Introduction

2. P. Grossmann, “Depth from focus,” Pattern Recogn. Lett. **5**, 63–69 (1987). [CrossRef]

## 2. Problem formulation

*x*and

*y*are image coordinates, ξ and η are two spatial variables,

*s*(

*x,y*) and

*i*(

*x,y*) are the sharp and defocused images of the source object respectively,

*d*(

*x,y*) is the distance from the object to the plane of best focus (PBF) and

*h*(

*x,y,d*) is the PSF. If the distance from the object to the PBF is constant, then the PSF

*h*(

*x,y,d*) can be written as

*h*(

*x,y*) and the defocusing process is defined as a convolution integral:

*i*(

*x,y*),

*I*(

*u,v*)}, {

*h*(

*x,y*),

*H*(

*u,v*)} and {

*s*(

*x,y*),

*S*(

*u,v*)} are Fourier pairs. Most of the focus based techniques assume that the distance function

*d*(

*x,y*) is slowly varying, so that it is almost constant over local regions. The defocus is then modeled by the convolution integral over these regions.

### 2.1 Form of point-spread function

*circle of confusion*” or “

*blur circle*” with diameter 2

*R*, provided that the aperture of the lens is also circular. According to geometrical optics, the intensity distribution within the blur circle is assumed to be approximately uniform i.e., the PSF is a circular “pillbox”. In reality, however, diffraction effects and characteristics of the system play a major role in forming the intensity distribution within the blur circle. After examining the net distribution of several wavelengths and considering the effects of lens aberrations the net PSF is best described by a 2D Gaussian function [3

3. A. P. Pentland, “A new sense for depth of field,” IEEE Trans. Pattern Anal. Mach. Intell. **9**, 523–531 (1987). [CrossRef] [PubMed]

*R*of the blur circle.

*k*depends on the system and can be determined through calibration. The optical transfer function

*H*(

*u,v*) for geometrical optics can be obtained by taking the Fourier transform (FT) of

*h*(

*x,y*):

*u,v*) by employing Eq. (8). However, a more accurate value can be obtained by averaging σover some domain in the frequency space:

*P*is a region in the (

*u,v*) space containing points where (

*I*(

*u,v*)/

*S*(

*u,v*)>0 and

*A*is the area of

*P*[5].

### 2.2 Relating depth to camera parameters and defocus

*F*can be derived to establish the relationship between the radius

*R*of the blur circle and the distance

*D*from a point in a scene to the lens [3

_{OL}3. A. P. Pentland, “A new sense for depth of field,” IEEE Trans. Pattern Anal. Mach. Intell. **9**, 523–531 (1987). [CrossRef] [PubMed]

12. V. Aslantas and D. T. Pham, “Depth from automatic defocusing” Opt. Express **15**, 1011–1023 (2007). [CrossRef] [PubMed]

*D*is the distance between the lens and the sensor plane,

_{LS}*f*is the

*f*-number of a given lens. When the object is in front of the PBF, Eq. (10) becomes:

## 3. Obtaining sharp images

### 3.1 Optical method for obtaining sharp images

*L*. Therefore, when

*L*is very small, σis also very small. Then the PSF may be approximated by an impulse function. Consequently, the captured image closely resembles the focused image. However, setting the aperture diameter to a very small value causes some serious practical problems. First, as the aperture diameter decreases, the diffraction effects increase. Therefore, the observed image is distorted. Second, a small aperture gathers only a small amount of light. Consequently, the period of exposure of the sensor has to be lengthened and the light intensity must be increased to take advantage of the sensor′s full dynamic range. This increases the time required. Also, the scene must be stationary not only while each of the two images (one obtained with a large aperture and the other with a small aperture) is captured but also in the interval between the acquisitions of those images.

### 3.2 The Laplacian sharpening filter

*c*is a constant and

*i*is a function of

*x, y*and

*t*(time).

*i*(

*x,y*,0) is the sharp image

*s*(

*x,y*) at

*t*= 0. The blurred image

*i*(

*x,y,t*) is obtained at some

*t*=τ>

*0*. Then,

*i*(

*x,y,t*) is approximated at

*t*=τ by the following Taylor polynomial:

*i*(

*x,y*,0),

*i*(

*x,y*) for

*i*(

*x,y,τ*) and

*c*∇

^{2}/

*i*for

*∂i*/

*∂t*, a mathematical expression can be derived for

*s*(

*x, y*) as:

*i*a positive multiple of its Laplacian. If higher-order approximations based on the Taylor series expansion are used, better results can be achieved. However, this will increase the computational cost. The aim of this paper is to find a relation between blur and depth rather than restoring the exact sharp image and the above first-order approximation is sufficient to derive that relation.

*c*. Therefore,

*c*can be estimated by fitting a Gaussian to the PSF [14]. By convolving both sides of Eq. (15) with the PSF

*h*(

*x,y*) and substituting σ for

*cτ*, the following formula is obtained:

*h*(

*x,y*) can be searched iteratively to minimize the difference between the left and right hand sides of Eq. (17) over a region P, namely:

*h*(

*x,y*) is the unique indicator of the depth of a scene. Thus, when the

*h*(

*x,y*) that minimizes the above expression is obtained, the depth can be computed using the SP of that

*h*(

*x,y*). By taking the Laplacian of the blurred edge and subtracting the result from the blurred edge (

*c*= 1), the sharpened edge is obtained. However, it also produces overshoot or “ringing” on either side of the edge. This problem can be solved by “clipping” the extreme low and high grey level values.

## 4. Data collection

*f*-number of the lens used were set to 20mm and 2.8 respectively. The distance of the step edge from the camera varied for different images but the camera parameters were the same for all images. The camera was set such that the PBF was at infinity (thus the object would always be between the PBF and the lens). For each distance, fifteen images were employed for averaging to minimize the effects of noise.

## 5. Results

## 6. Conclusion

## References and links

1. | M. Hebert, “Active and passive range sensing for robotics,” in |

2. | P. Grossmann, “Depth from focus,” Pattern Recogn. Lett. |

3. | A. P. Pentland, “A new sense for depth of field,” IEEE Trans. Pattern Anal. Mach. Intell. |

4. | M. Subbarao and N. Gurumoorthy, “Depth recovery from blurred edges,” in |

5. | M. Subbarao, “Efficient depth recovery through inverse optics,” |

6. | C. Cardillo and M. A. Sid-Ahmed, “3-D position sensing using passive monocular vision system,” IEEE Trans. Pattern Anal. Mach. Intell. |

7. | R. V. Dantu, N. J. Dimopoulos, R. V. Patel, and A. J. Al-Khalili, “Depth perception using blurring and its application in VLSI wafer probing,” Mach. Vision Appl. |

8. | S. H. Lai, C. W. Fu, and S. Chang, “A generalised depth estimation algorithm with a single image,” IEEE Trans. Pattern Anal. Mach. Intell. |

9. | D. T. Pham and V. Aslantas, “Depth from defocusing using a neural network,” J. Pattern Recogn. |

10. | V. Aslantas, “Estimation of depth from defocusing using a neural network,” in |

11. | B. K. P. Horn, |

12. | V. Aslantas and D. T. Pham, “Depth from automatic defocusing” Opt. Express |

13. | R. C. Gonzalez and R. E. Woods, |

14. | A. Rosenfeld and C. Kak, |

**OCIS Codes**

(100.2000) Image processing : Digital image processing

(150.5670) Machine vision : Range finding

(150.6910) Machine vision : Three-dimensional sensing

**ToC Category:**

Image Processing

**History**

Original Manuscript: February 20, 2007

Revised Manuscript: April 6, 2007

Manuscript Accepted: April 9, 2007

Published: April 10, 2007

**Citation**

V. Aslantas, "A depth estimation algorithm with a single image," Opt. Express **15**, 5024-5029 (2007)

http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-15-8-5024

Sort: Year | Journal | Reset

### References

- M. Hebert, "Active and passive range sensing for robotics," in Proceedings of IEEE Conference on Robotics and Automation, (Institute of Electrical and Electronics Engineers, San Francisco, CA, 2000), pp. 102-110.
- P. Grossmann, "Depth from focus," Pattern Recogn. Lett. 5, 63-69 (1987). [CrossRef]
- A. P. Pentland, "A new sense for depth of field," IEEE Trans. Pattern Anal. Mach. Intell. 9, 523-531 (1987). [CrossRef] [PubMed]
- M. Subbarao and N. Gurumoorthy, "Depth recovery from blurred edges," in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, (Institute of Electrical and Electronics Engineers, Ann Arbor, MI, 1988), pp. 498-503.
- M. Subbarao, "Efficient depth recovery through inverse optics," Machine Vision Inspection and Measurement, H. Freeman, ed., (Academic, Boston, 1989).
- C. Cardillo and M. A. Sid-Ahmed, "3-D position sensing using passive monocular vision system," IEEE Trans. Pattern Anal. Mach. Intell. 13, 809-813 (1991). [CrossRef]
- R. V. Dantu, N. J. Dimopoulos, R. V. Patel, and A. J. Al-Khalili, "Depth perception using blurring and its application in VLSI wafer probing," Mach. Vision Appl. 5, 35-45 (1992). [CrossRef]
- S. H. Lai, C. W. Fu, and S. Chang, "A generalised depth estimation algorithm with a single image," IEEE Trans. Pattern Anal. Mach. Intell. 14, 405-411 (1992). [CrossRef]
- D. T. Pham and V. Aslantas, "Depth from defocusing using a neural network," J. Pattern Recogn. 32, 715-727 (1999). [CrossRef]
- V. Aslantas, "Estimation of depth from defocusing using a neural network," in International Conference on Signal Processing, (Canakkale, Turkey, 2003), pp. 305-309.
- B. K. P. Horn, Robot vision (McGraw-Hill, New York, 1986).
- V. Aslantas and D. T. Pham, "Depth from automatic defocusing" Opt. Express 15, 1011-1023 (2007). [CrossRef] [PubMed]
- R. C. Gonzalez, and R. E. Woods, Digital image processing (Addison-Wesley, Reading, MA 1992).
- A. Rosenfeld and C. Kak, Digital picture processing, Second Edition, (Academic Press, New York 1982).

## Cited By |
Alert me when this paper is cited |

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

« Previous Article | Next Article »

OSA is a member of CrossRef.