OSA's Digital Library

Optics Express

Optics Express

  • Editor: Michael Duncan
  • Vol. 14, Iss. 10 — May. 15, 2006
  • pp: 4328–4350

Video-based eyetracking methods and algorithms in head-mounted displays

Hong Hua, Prasanna Krishnaswamy, and Jannick P. Rolland  »View Author Affiliations

Optics Express, Vol. 14, Issue 10, pp. 4328-4350 (2006)

View Full Text Article

Enhanced HTML    Acrobat PDF (2507 KB)

Browse Journals / Lookup Meetings

Browse by Journal and Year


Lookup Conference Papers

Close Browse Journals / Lookup Meetings

Article Tools



Head pose is utilized to approximate a user’s line-of-sight for real-time image rendering and interaction in most of the 3D visualization applications using head-mounted displays (HMD). The eye often reaches an object of interest before the completion of most head movements. It is highly desirable to integrate eye-tracking capability into HMDs in various applications. While the added complexity of an eyetracked-HMD (ET-HMD) imposes challenges on designing a compact, portable, and robust system, the integration offers opportunities to improve eye tracking accuracy and robustness. In this paper, based on the modeling of an eye imaging and tracking system, we examine the challenges and identify parametric requirements for video-based pupil-glint tracking methods in an ET-HMD design, and predict how these parameters may affect the tracking accuracy, resolution, and robustness. We further present novel methods and associated algorithms that effectively improve eye-tracking accuracy and extend the tracking range.

© 2006 Optical Society of America

OCIS Codes
(120.2040) Instrumentation, measurement, and metrology : Displays
(120.2820) Instrumentation, measurement, and metrology : Heads-up displays
(120.4640) Instrumentation, measurement, and metrology : Optical instruments
(330.2210) Vision, color, and visual optics : Vision - eye movements

ToC Category:
Instrumentation, Measurement, and Metrology

Original Manuscript: February 14, 2006
Revised Manuscript: May 2, 2006
Manuscript Accepted: May 8, 2006
Published: May 15, 2006

Virtual Issues
Vol. 1, Iss. 6 Virtual Journal for Biomedical Optics

Hong Hua, Prasanna Krishnaswamy, and Jannick P. Rolland, "Video-based eyetracking methods and algorithms in head-mounted displays," Opt. Express 14, 4328-4350 (2006)

Sort:  Author  |  Year  |  Journal  |  Reset  


  1. M. Bajura, H. Fuchs, and R. Ohbuchi, "Merging virtual objects with the real world: Seeing ultrasound imagery within the patient," in Proceedings of ACM SIGGRAPH (ACM, Chicago, IL, 1992), pp. 203-210.
  2. T. Caudell and D. Mizell, "Augmented reality: An application of heads-up display technology to manual manufacturing processes," in Proceedings of Hawaii International Conferences on Systems Sciences (Hawaii, HI, 1992), pp. 659-69.
  3. J. P. Rolland and H. Fuchs, "Optical versus video see-through head-mounted displays in medical visualization," Presence: Teleoperators and Virtual Environments 9, 287-309 (2000). [CrossRef]
  4. H. L. Pryor, T. A. Furness, and E. Viirre, "The Virtual Retinal Display: A New Display Technology Using Scanned Laser Light," in Proceedings of Human Factors and Ergonomics Society, 42nd Annual Meeting (1998) pp. 1570-1574.
  5. H. Hua, A. Girardot, C. Gao, and J. P. Rolland, "Engineering of head-mounted projective displays," Appl. Opt. 39, 3814-3824 (2000). [CrossRef]
  6. J. P. Rolland, L. Davis, and Y. Baillot, "A survey of tracking technology for virtual environments," in Fundamentals of Wearable Computers and Augmented Reality. M. Barfield and T. Caudell, eds. (Mahwah, NJ, 2000).
  7. J. Rolland, Y. Ha, and C. Fidopiastis, "Albertian errors in head-mounted displays: I. Choice of eye-point location for a near- or far-field task visualization," J. Opt. Soc. Am. A 21, 901-912 (2004). [CrossRef]
  8. K. Iwamoto, K. Komoriya, K. Tanie, "Eye Movement Tracking Type Image Display System for Wide View Image Presentation with High-Resolution: -Evaluation of High-resolution Image Presentation," in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (Institute of Electrical and Electronics Engineers, EPFL, Switzerland, 2002), pp.1190-95.
  9. J. P. Rolland, A. Yoshida, L. Davis, and J.H. Reif, "High resolution inset head-mounted display," Appl. Opt. 37, 4183-4193 (1998). [CrossRef]
  10. L. R. Young and D. Sheena, "Methods and Designs: Survey of eye movement recording methods," Behav. Res. Methods Instrum. 7, 397-429 (1975). [CrossRef]
  11. R. J.K. Jacob and K.S. Karn, "Eye Tracking in Human-Computer Interaction and Usability Research: Ready to Deliver the Promises (Section Commentary)," in The Mind's Eye: Cognitive and Applied Aspects of Eye Movement Research, J. Hyona, R. Radach, and H. Deubel, eds. (Elsevier Science, Amsterdam, 2003), pp. 573-605.
  12. A. T. Duchowski, "Incorporating the viewer's Point-Of-Regard (POR) in gaze-contingent virtual environments," in Proc.SPIE 3295, 332-43 (1998). [CrossRef]
  13. M. Hayhoe, D. Ballard, J. Triesch, and H. Shinoda, "Vision in natural and virtual environments," Proceedings, in Proceedings of ACM 2002 Symposium on Eye Tracking Research and Applications (ACM, New Orleans, USA 2002), pp. 7-13.
  14. L. Vaissie, J. Rolland, "Head mounted display with eyetracking capability," US Patent 6,433,760 B1 (2002).
  15. H. Hua, "Integration of eye tracking capability into optical see-through head mounted displays," in Helmet-Mounted Displays, Proc. SPIE 4297, 496-503 (2001). [CrossRef]
  16. C. W. Hess, R. Muri, O. Meienberg, "Recording of horizontal saccadic eye movements: methodological comparison between electro-oculography and infrared reflection oculography," Neuro-Ophthalmology 6, 264-272 (1986). [CrossRef]
  17. D. A. Robinson, "A method of measuring eye movements using a scleral search coil in a magnetic field," IEEE Trans. Biomed. Electron. BME 10, 137-145 (1963).
  18. Applied Science Laboratories: Technology and Systems for Eye Tracking, http://www.a-s-l.com.
  19. M. Eizenman, R. C. Frecker, P. E. Hallett, "Precise non-contacting measurement of eye movements using the corneal reflex," Vision Res. 24, 167-74 (1984). [CrossRef] [PubMed]
  20. T. N. Cornsweet, H. D. Crane, "Accurate two-dimensional eye tracker using first and fourth Purkinje images," J. Opt. Soc. Am. 63, 921-8 (1973). [CrossRef] [PubMed]
  21. J. Merchant, R. Morrissette, J. L. Porterfield, "Remote measurement of eye direction allowing subject motion over one cubic foot of space," IEEE Trans. Biomed. Engineering BME 21, 309-17 (1974). [CrossRef]
  22. Y. Ebisawa, "Improved video-based eye-gaze detection method," IEEE Trans. Instrum. Meas. 47, 948-55 (1998). [CrossRef]
  23. C. H. Morimoto, D. Koons, A. Amir, and M. Flickner, "Pupil detection and tracking using multiple light sources," Image Vision Comput. 18, 331-5 (2000). [CrossRef]
  24. D. Yoo, J. Kim, B. Lee, M., "Non-contact eye gaze tracking system by mapping of corneal reflections," in Proceedings of Fifth IEEE International Conference on Automatic Face and Gesture Recognition (IEEE, Piscataway, NJ, USA, 2002), pp. 0101-06.
  25. S. W. Shih, and Jin Liu, "A novel approach to 3-D gaze tracking using stereo cameras," IEEE Trans. on Systems, Man, and Cybernetics-Part B: Cybernetics 34, 234-245 (2004).
  26. K. Ryoung Park, "Gaze detection by wide and narrow view stereo camera," Lecture Notes in Computer Science (CIARP’2004) 3287, 140-147 (2004).
  27. M. L. Thomas, W. P. Siegmund, S. E. Antos, and R. M. Robinson, "Fiber optic development for use on the fiber optic helmet-mounted display," in Helmet-Mounted Displays, J. T. Carollo, ed., Proc. SPIE 116, 90-101 (1989).
  28. G. Beach, CharlesJ  Cohen, Jeff Braun, G. Moody, "Eye tracker system for use with head mounted displays," in Proceedings of the IEEE International Conference on Systems, Man and Cybernetics (IEEE, Piscataway, NJ, USA, 1998) 5, pp. 4348-52.
  29. C. Curatu, H. Hua, and J. Rolland, "Projection-based head mounted display with eye tracking capabilities," in Novel Optical Systems Design and Optimization VIII, Jose M. Sasian, R. John Koshel, and Richard C. Juergens Eds., Proc. SPIE. 5875, 128-140 (2005).
  30. C. W. Pansing, H. Hua, and J. Rolland, "Optimization of illumination schemes in a head-mounted display integrated with eye tracking capabilities," in Novel Optical Systems Design and Optimization VIII, Jose M. Sasian, R. John Koshel, and Richard C. Juergens Eds., Proc. SPIE. 5875, 128-140 (2005).
  31. J. Schwiegerling, Field Guide to Visual and Opthalmic Optics (SPIE Press, Bellingham, WA, 2004). [CrossRef]
  32. L. G. Farkas, Anthropometry of the Head and Face, Second Ed. (Raven Press, New York, 1994).
  33. J. D. Foley and A. van Dam, Fundamentals of Interactive Computer Graphics, Second Ed. (Addison-Wesley, Reading, Mass., 1996).
  34. Z. Zhang, "A flexible new technique of camera calibration," IEEE Trans. Pattern Anal. Mach. Intell. 22, 1330-1334 (2000). [CrossRef]
  35. Karlene Nguyen, Cindy Wagner, David Koons, and Myron Flickner, "Differences in the infrared bright pupil response of human eyes," in Proceedings of ACM 2002 Symposium on Eye Tracking Research and Applications (ACM, New Orleans, USA 2002). pp. 133-138.
  36. P. Krishnaswamy, "Design and assessment of improved feature-based eye tracking methods for head-mounted displays," University of Arizona M.S. thesis report, 2005.

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

« Previous Article  |  Next Article »

OSA is a member of CrossRef.

CrossCheck Deposited