## Joint iris boundary detection and fit: a real-time method for accurate pupil tracking |

Biomedical Optics Express, Vol. 5, Issue 8, pp. 2458-2470 (2014)

http://dx.doi.org/10.1364/BOE.5.002458

Acrobat PDF (1120 KB)

### Abstract

A range of applications in visual science rely on accurate tracking of the human pupil’s movement and contraction in response to light. While the literature for independent contour detection and fitting of the iris-pupil boundary is vast, a joint approach, in which it is assumed that the pupil has a given geometric shape has been largely overlooked. We present here a global method for simultaneously finding and fitting of an elliptic or circular contour against a dark interior, which produces consistently accurate results even under non-ideal recording conditions, such as reflections near and over the boundary, droopy eye lids, or the sudden formation of tears. The specific form of the proposed optimization problem allows us to write down closed analytic formulae for the gradient and the Hessian of the objective function. Moreover, both the objective function and its derivatives can be cast into vectorized form, making the proposed algorithm significantly faster than its closest relative in the literature. We compare methods in multiple ways, both analytically and numerically, using real iris images as well as idealizations of the iris for which the ground truth boundary is precisely known. The method proposed here is illustrated under challenging recording conditions and it is shown to be robust.

© 2014 Optical Society of America

## 1. Introduction

1. E. S. Maini, “Robust ellipse-specific fitting for real-time machine vision,” in *Brain, Vision, and Artificial Intelligence*, M. Gregorio, V. Maio, M. Frucci, and C. Musio, eds. (Springer Berlin Heidelberg, 2005) vol. 3704, pp. 318–327. [CrossRef]

3. K. Kanatani, “Ellipse fitting with hyperaccuracy,” IEICE Trans. Inf. Syst. **E89-D**, 2653–2660 (2006). [CrossRef]

4. K. Kanatani, “Statistical bias of conic fitting and renormalization,” IEEE Trans. Pattern Analysis Mach. Intell. **16**, 320–326 (1994). [CrossRef]

*find and fit*methods. In such methods, it is often assumed that the data points representing the geometric curve are uniformly sampled and this is hardly the case when an arc or section are missing. When part of a pupil image is uncertain, e.g. momentarily covered by eyelids or eyelashes, this uncertainty will propagate into the fit [5

5. J. Porrill, “Fitting ellipses and predicting confidence envelopes using a bias corrected kalman filter,” Image Vis. Comput. **8**, 37–41 (1990). [CrossRef]

6. J. Daugman, “High confidence visual recognition of persons by a test of statistical independence,” IEEE Trans. Pattern Analysis Mach. Intell. **15**(11), 1148–1161 (1993). [CrossRef]

7. S. A. C. Schuckers, N. A. Schmid, A. Abhyankar, V. Dorairaj, C. K. Boyce, and L. A. Hornak, “On techniques for angle compensation in nonideal iris recognition,” IEEE transactions on systems, man, cybernetics. Part B, Cybern. a publication IEEE Syst. Man, Cybern. Soc. , **37**, 1176–1190 (2007). [CrossRef] [PubMed]

*global methods*—which enforce a priori knowledge of the geometric curve describing the boundary—have been mostly overlooked. The integro-differential operator method leverages the fact that the pupil is generally darker than the iris. By taking the absolute value of the optimization function, the method can be extended to detect brighter than iris, in cases of abnormal lens opacity (cataract) or the occasional red-eye effect caused by coaxial illumination. However, for non-ideal, noisy images this method is very sensitive to artifacts, particularly reflections seen inside the pupil or at its border. In specific contexts this method can be successfully implemented [8

8. W. Sankowski, K. Grabowski, M. Napieralska, M. Zubert, and A. Napieralski, “Reliable algorithm for iris segmentation in eye image,” Image Vis. Comput. , **28**, 231–237, 2010. [CrossRef]

*find and fit*methods, eroding its attractiveness for simple shapes.

9. H. Yuen, J. Princen, J. Illingworth, and J. Kittler, “Comparative study of hough transform methods for circle finding,” Image Vis. Comput. **8**, 71–77 (1990). [CrossRef]

14. Y. Chen, M. Adjouadi, C. Han, J. Wang, A. Barreto, N. Rishe, and J. Andrian, “A highly accurate and computationally efficient approach for unconstrained iris segmentation,” Image Vis. Comput. **28**, 261–269 (2010). [CrossRef]

15. A. Bell, A. C. James, M. Kolic, R. W. Essex, and T. Maddess, “Dichoptic multifocal pupillography reveals afferent visual field defects in early type 2 diabetes,” Invest. Ophthalmol. Vis. Sci. **51**, 602–608 (2010). [CrossRef]

16. C. F. Carle, T. Maddess, and A. C. James, “Contraction anisocoria: Segregation, summation, and saturation in the pupillary pathway,” Invest. Ophthalmol. Vis. Sci. **52**, 2365–2371 (2011). [CrossRef] [PubMed]

10. H. Proenca, “Iris recognition: On the segmentation of degraded images acquired in the visible wavelength,” IEEE Trans. Pattern Analysis Mach. Intell. **32**(8), 1502–1516 (2010). [CrossRef]

## 2. Optimization methods

### 2.1. Integro-differential operators

6. J. Daugman, “High confidence visual recognition of persons by a test of statistical independence,” IEEE Trans. Pattern Analysis Mach. Intell. **15**(11), 1148–1161 (1993). [CrossRef]

*c*(

*r*,

*x*

_{0},

*y*

_{0}) that maximizes the absolute value of the functional Where the path

*c*is a circle of integration slightly larger than a concentric circle

*c′*. The optimisation search is over a three-parameter space for the centre (x,y) and average radius r of these two concentric circles. When those three parameters happen to correspond closely to those of an actual pupil in an eye image, the blurred partial derivative of Eq 1 has a large maximum. This rationale can be easily appreciated by representing the image gray levels

*I*(

*x*,

*y*) by a simple, rotationally invariant, step (Heaviside) function,

*H*(

*r*). Assuming the center of the image is correct, the image functional in Eq. (1) simplifies to Note that which, for

*r*>>

*a*i.e,

*a*small, is just a

*δ*(

*r*) so the image functional in Eq. (1) has a gaussian profile

*G*(

_{σ}*r*−

*r′*), with extremum at

*r*=

*r′*, the optimal radius. Similar analysis can be performed considering a mismatch between the coordinates for the origin on both integrals and the center of the image. Again the objective function simplifies to a gaussian profile centered at the correct origin.

### 2.2. Proposed method

*x*,

*y*) belonging to a quadric with generalized radius

*r*and center (

_{sq}*a*,

*b*) satisfy the equation With the change of variables, where the contour is defined at

*z*= 0 and the parameter

*δ*serves as a scale (or range 1/

*δ*), enforcing a gaussian decay profile on

*w*, so values of

*r*away from unity (away from the contour) give a contribution to

_{sq}*w*exponentially smaller. We search for a set of parameters {

*a*,

*b*,

*c*,

*d*,

*e*} which minimizes the functional where

*I*(

*x*,

*y*) is a gray-level image. This optimal set of parameters corresponds to the largest elliptical curve with a dark interior found in

*I*(

*x*,

*y*). This can be readily seen, as the functional in Eq. (6) is the accumulation of the product of the exponential profile and the image: whenever this product moves away from the correct position it contributes more (as

*w*(

*z*) gets smaller) to the sum parcels, increasing the value of the objective function. Figure 1 shows an image of a reasonably circular pupil and the initial and final circle: the solution of the minimization problem proposed in this paper is the circle displayed using a dashed line, the circle in a continuous line is the, far off, initialization of the algorithm. Note that the solution is close to what would have been obtained by visual inspection, and showcases the potential of the proposed method, particulary in face of presence of strong reflections. The objective function in Eq. (6) has two very useful properties. Firstly, analytic derivatives are easily obtained, with A similar expression for the second-order derivative of

*Z*is straightforward. Secondly, both the objective function and its derivatives can be fully vectorized. The objective function

*Z*itself is a simple scalar product

*w*

_{(xx,yy)}(:) *

*I*

_{(x,y)}(:), where

*aa*is a matrix obtained by concatenating

*n*copies of the column vector

*a*and

*V*(:) stands for the linear expansion of

_{aa}*V*.

## 3. Optimization landscape

### 3.1. Idealized pupil, analytic

*Z*becomes where

*α*= (1/2 +

*kδ*

^{2})

^{1/2}.

*kδ*.

### 3.2. Idealized pupil, numeric

*c*= (160, 160) and radius

*r*= 80. On the left we have the objective function contours for the integro-differential method Eq. (1), while on the right we display the optimization landscape for the objective function of this paper Eq. (6). Note that although both pictures display well-defined global maxima, with a smooth profile near the true optimum, for the integro-differential method there are two routes towards the two lower corners that can either slow down the gradient ascent or derail the search for the maximum. Also note that there is an upper limit on the initial radius that would lead to local maxima. None of these features are present on the right side picture for the method presented in this paper, the landscape is a smooth gradient throughout the whole range of initial conditions shown. This is the first indication that our method should be robust regarding variations of the initial conditions, a very important feature for tracking, as we recycle the previous frame parameters as initial values for the next frame, and this can be in considerable error after a blink or a saccade.

### 3.3. Real pupil

## 4. Stability analysis

`offset`, from

`20px`to

`120px`. We also vary the angle of the line connecting the correct center and the initial center, the

`offset angle`, by 45 degrees to a full turn. The initial radius is varied from the correct radius plus minus a gap of

`20px`, every

`5px`. We start with the pupil simulation and then repeat the stability analysis for the real pupil image of Fig 1.

### 4.1. Ideal pupil

18. J. A. Nelder and R. Mead, “A simplex method for function minimization,” Comp. J. **7**, 308–313 (1965). [CrossRef]

19. J. C. Lagarias, J. A. Reeds, M. H. Wright, and P. E. Wright, “Convergence properties of the nelder-mead simplex method in low dimensions,” SIAM J. Optim. **9**, 112–147 (1998). [CrossRef]

### 4.2. Real pupil

## 5. Tracking

### 5.1. Hardware

16. C. F. Carle, T. Maddess, and A. C. James, “Contraction anisocoria: Segregation, summation, and saturation in the pupillary pathway,” Invest. Ophthalmol. Vis. Sci. **52**, 2365–2371 (2011). [CrossRef] [PubMed]

### 5.2. Offline method

*find and fit*method, using the approach described in [20

20. G. Taubin, “Estimation of planar curves, surfaces, and nonplanar space curves defined by implicit equations with applications to edge and range image segmentation,” IEEE Trans. Pattern Analysis Mach. Intell. **13**, 1115–1138 (1991). [CrossRef]

### 5.3. Results

## 6. Discussion

18. J. A. Nelder and R. Mead, “A simplex method for function minimization,” Comp. J. **7**, 308–313 (1965). [CrossRef]

19. J. C. Lagarias, J. A. Reeds, M. H. Wright, and P. E. Wright, “Convergence properties of the nelder-mead simplex method in low dimensions,” SIAM J. Optim. **9**, 112–147 (1998). [CrossRef]

21. C. Broyden, “The convergence of a class of double-rank minimization algorithms 1. general considerations,” IMA J. Appl. Math. **6**, 76–90 (1970). [CrossRef]

24. D. F. Shanno, “Conditioning of quasi-newton methods for function minimization,” Math. Comput. **24**, 647–656 (1970). [CrossRef]

*Matlab*

^{®}. We assume by

*real time*a value close to 1/24s per frame, thus real time performance is likely to be achieved by using a non-interpreted programming language. The

*Matlab*

^{®}implementation of our algorithm will be made openly available.

## 7. Conclusion

## References and links

1. | E. S. Maini, “Robust ellipse-specific fitting for real-time machine vision,” in |

2. | A. Fitzgibbon, M. Pilu, and R. B. Fisher, “Direct least square fitting of ellipses,” IEEE Trans. Pattern Analysis Mach. Intell. |

3. | K. Kanatani, “Ellipse fitting with hyperaccuracy,” IEICE Trans. Inf. Syst. |

4. | K. Kanatani, “Statistical bias of conic fitting and renormalization,” IEEE Trans. Pattern Analysis Mach. Intell. |

5. | J. Porrill, “Fitting ellipses and predicting confidence envelopes using a bias corrected kalman filter,” Image Vis. Comput. |

6. | J. Daugman, “High confidence visual recognition of persons by a test of statistical independence,” IEEE Trans. Pattern Analysis Mach. Intell. |

7. | S. A. C. Schuckers, N. A. Schmid, A. Abhyankar, V. Dorairaj, C. K. Boyce, and L. A. Hornak, “On techniques for angle compensation in nonideal iris recognition,” IEEE transactions on systems, man, cybernetics. Part B, Cybern. a publication IEEE Syst. Man, Cybern. Soc. , |

8. | W. Sankowski, K. Grabowski, M. Napieralska, M. Zubert, and A. Napieralski, “Reliable algorithm for iris segmentation in eye image,” Image Vis. Comput. , |

9. | H. Yuen, J. Princen, J. Illingworth, and J. Kittler, “Comparative study of hough transform methods for circle finding,” Image Vis. Comput. |

10. | H. Proenca, “Iris recognition: On the segmentation of degraded images acquired in the visible wavelength,” IEEE Trans. Pattern Analysis Mach. Intell. |

11. | Z. He, T. Tan, Z. Sun, and X. Qiu, “Toward accurate and fast iris segmentation for iris biometrics,” IEEE Trans. Pattern Analysis Mach. Intell. |

12. | Z. He, T. Tan, and Z. Sun, “Iris localization via pulling and pushing,” in 18th International Conference on Pattern Recognition, 2006. ICPR 2006, 4, 366–369, 2006. |

13. | T. Camus and R. Wildes, “Reliable and fast eye finding in close-up images,” in 16th International Conference on Pattern Recognition, 2002. Proceedings, 1, 389–394 vol. 1, 2002. |

14. | Y. Chen, M. Adjouadi, C. Han, J. Wang, A. Barreto, N. Rishe, and J. Andrian, “A highly accurate and computationally efficient approach for unconstrained iris segmentation,” Image Vis. Comput. |

15. | A. Bell, A. C. James, M. Kolic, R. W. Essex, and T. Maddess, “Dichoptic multifocal pupillography reveals afferent visual field defects in early type 2 diabetes,” Invest. Ophthalmol. Vis. Sci. |

16. | C. F. Carle, T. Maddess, and A. C. James, “Contraction anisocoria: Segregation, summation, and saturation in the pupillary pathway,” Invest. Ophthalmol. Vis. Sci. |

17. | J. Miles., www.milesresearch.com. Image use permission kindly granted by owner. |

18. | J. A. Nelder and R. Mead, “A simplex method for function minimization,” Comp. J. |

19. | J. C. Lagarias, J. A. Reeds, M. H. Wright, and P. E. Wright, “Convergence properties of the nelder-mead simplex method in low dimensions,” SIAM J. Optim. |

20. | G. Taubin, “Estimation of planar curves, surfaces, and nonplanar space curves defined by implicit equations with applications to edge and range image segmentation,” IEEE Trans. Pattern Analysis Mach. Intell. |

21. | C. Broyden, “The convergence of a class of double-rank minimization algorithms 1. general considerations,” IMA J. Appl. Math. |

22. | R. Fletcher, “A new approach to variable metric algorithms,” The Comp. J. |

23. | D. Goldfarb, “A family of variable-metric methods derived by variational means,” Math. Comput. |

24. | D. F. Shanno, “Conditioning of quasi-newton methods for function minimization,” Math. Comput. |

**OCIS Codes**

(100.0100) Image processing : Image processing

(150.0150) Machine vision : Machine vision

(170.4470) Medical optics and biotechnology : Ophthalmology

(330.2210) Vision, color, and visual optics : Vision - eye movements

(150.1135) Machine vision : Algorithms

(100.4999) Image processing : Pattern recognition, target tracking

**ToC Category:**

Image Processing

**History**

Original Manuscript: April 17, 2014

Revised Manuscript: June 20, 2014

Manuscript Accepted: June 22, 2014

Published: July 2, 2014

**Citation**

Marconi Barbosa and Andrew C. James, "Joint iris boundary detection and fit: a real-time method for accurate pupil tracking," Biomed. Opt. Express **5**, 2458-2470 (2014)

http://www.opticsinfobase.org/boe/abstract.cfm?URI=boe-5-8-2458

Sort: Year | Journal | Reset

### References

- E. S. Maini, “Robust ellipse-specific fitting for real-time machine vision,” in Brain, Vision, and Artificial Intelligence, M. Gregorio, V. Maio, M. Frucci, and C. Musio, eds. (Springer Berlin Heidelberg, 2005) vol. 3704, pp. 318–327. [CrossRef]
- A. Fitzgibbon, M. Pilu, and R. B. Fisher, “Direct least square fitting of ellipses,” IEEE Trans. Pattern Analysis Mach. Intell.21, 476–480 (1999). [CrossRef]
- K. Kanatani, “Ellipse fitting with hyperaccuracy,” IEICE Trans. Inf. Syst.E89-D, 2653–2660 (2006). [CrossRef]
- K. Kanatani, “Statistical bias of conic fitting and renormalization,” IEEE Trans. Pattern Analysis Mach. Intell.16, 320–326 (1994). [CrossRef]
- J. Porrill, “Fitting ellipses and predicting confidence envelopes using a bias corrected kalman filter,” Image Vis. Comput.8, 37–41 (1990). [CrossRef]
- J. Daugman, “High confidence visual recognition of persons by a test of statistical independence,” IEEE Trans. Pattern Analysis Mach. Intell.15(11), 1148–1161 (1993). [CrossRef]
- S. A. C. Schuckers, N. A. Schmid, A. Abhyankar, V. Dorairaj, C. K. Boyce, and L. A. Hornak, “On techniques for angle compensation in nonideal iris recognition,” IEEE transactions on systems, man, cybernetics. Part B, Cybern. a publication IEEE Syst. Man, Cybern. Soc., 37, 1176–1190 (2007). [CrossRef] [PubMed]
- W. Sankowski, K. Grabowski, M. Napieralska, M. Zubert, and A. Napieralski, “Reliable algorithm for iris segmentation in eye image,” Image Vis. Comput., 28, 231–237, 2010. [CrossRef]
- H. Yuen, J. Princen, J. Illingworth, and J. Kittler, “Comparative study of hough transform methods for circle finding,” Image Vis. Comput.8, 71–77 (1990). [CrossRef]
- H. Proenca, “Iris recognition: On the segmentation of degraded images acquired in the visible wavelength,” IEEE Trans. Pattern Analysis Mach. Intell.32(8), 1502–1516 (2010). [CrossRef]
- Z. He, T. Tan, Z. Sun, and X. Qiu, “Toward accurate and fast iris segmentation for iris biometrics,” IEEE Trans. Pattern Analysis Mach. Intell.31(9), 1670–1684 (2009). [CrossRef]
- Z. He, T. Tan, and Z. Sun, “Iris localization via pulling and pushing,” in 18th International Conference on Pattern Recognition, 2006. ICPR 2006, 4, 366–369, 2006.
- T. Camus and R. Wildes, “Reliable and fast eye finding in close-up images,” in 16th International Conference on Pattern Recognition, 2002. Proceedings, 1, 389–394 vol. 1, 2002.
- Y. Chen, M. Adjouadi, C. Han, J. Wang, A. Barreto, N. Rishe, and J. Andrian, “A highly accurate and computationally efficient approach for unconstrained iris segmentation,” Image Vis. Comput.28, 261–269 (2010). [CrossRef]
- A. Bell, A. C. James, M. Kolic, R. W. Essex, and T. Maddess, “Dichoptic multifocal pupillography reveals afferent visual field defects in early type 2 diabetes,” Invest. Ophthalmol. Vis. Sci.51, 602–608 (2010). [CrossRef]
- C. F. Carle, T. Maddess, and A. C. James, “Contraction anisocoria: Segregation, summation, and saturation in the pupillary pathway,” Invest. Ophthalmol. Vis. Sci.52, 2365–2371 (2011). [CrossRef] [PubMed]
- J. Miles., www.milesresearch.com . Image use permission kindly granted by owner.
- J. A. Nelder and R. Mead, “A simplex method for function minimization,” Comp. J.7, 308–313 (1965). [CrossRef]
- J. C. Lagarias, J. A. Reeds, M. H. Wright, and P. E. Wright, “Convergence properties of the nelder-mead simplex method in low dimensions,” SIAM J. Optim.9, 112–147 (1998). [CrossRef]
- G. Taubin, “Estimation of planar curves, surfaces, and nonplanar space curves defined by implicit equations with applications to edge and range image segmentation,” IEEE Trans. Pattern Analysis Mach. Intell.13, 1115–1138 (1991). [CrossRef]
- C. Broyden, “The convergence of a class of double-rank minimization algorithms 1. general considerations,” IMA J. Appl. Math.6, 76–90 (1970). [CrossRef]
- R. Fletcher, “A new approach to variable metric algorithms,” The Comp. J.13, 317–322 (1970). [CrossRef]
- D. Goldfarb, “A family of variable-metric methods derived by variational means,” Math. Comput.24, 23–26 (1970). [CrossRef]
- D. F. Shanno, “Conditioning of quasi-newton methods for function minimization,” Math. Comput.24, 647–656 (1970). [CrossRef]

## Cited By |
Alert me when this paper is cited |

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

OSA is a member of CrossRef.