OSA's Digital Library

Journal of Display Technology

Journal of Display Technology

| A JOINT IEEE/OSA PUBLICATION

  • Vol. 4, Iss. 4 — Dec. 1, 2008
  • pp: 491–501

Intra-Operative Visualizations: Perceptual Fidelity and Human Factors

Danail Stoyanov, George P. Mylonas, Mirna Lerotic, Adrian J. Chung, and Guang-Zhong Yang

Journal of Display Technology, Vol. 4, Issue 4, pp. 491-501 (2008)


View Full Text Article

Acrobat PDF (1544 KB)





Browse Journals / Lookup Meetings

Browse by Journal and Year


   


Lookup Conference Papers

Close Browse Journals / Lookup Meetings

Article Tools

Share
Citations
  • Export Citation/Save Click for help

Abstract

With increasing capability and complexity of surgical interventions, intra-operative visualization is becoming an important part of a surgical environment. This paper reviews some of our recent progress in the intelligent use of pre- and intra-operative data for enhanced surgical navigation and motion compensated visualization. High fidelity augmented reality (AR) with enhanced 3D depth perception is proposed to provide effective surgical guidance. To cater for large scale tissue deformation, real-time depth recovery based on stereo disparity and eye gaze tracking is introduced. This allows the development of motion compensated visualization for improved visual perception and for facilitating motion adaptive AR displays. The discussion of the paper is focused on how to ensure perceptual fidelity of AR and the need for real-time tissue deformation recovery and modeling, as well as the importance of incorporating human perceptual factors in surgical displays.

© 2008 IEEE

Citation
Danail Stoyanov, George P. Mylonas, Mirna Lerotic, Adrian J. Chung, and Guang-Zhong Yang, "Intra-Operative Visualizations: Perceptual Fidelity and Human Factors," J. Display Technol. 4, 491-501 (2008)
http://www.opticsinfobase.org/jdt/abstract.cfm?URI=jdt-4-4-491


Sort:  Year  |  Journal  |  Reset

References

  1. R. H. Taylor, D. Stoianovici, "Medical robotics in computer-integrated surgery," IEEE Trans. Robot. Autom. 19, 765-781 (2003).
  2. R. J. Damiano, "Robotics in cardiac surgery: The emperor's new clothes," J., Thoracic & Cardiovasc. Surgery 134, 559-561 (2007).
  3. H. Delingette, X. Pennec, L. Soler, J. Marescaux, N. Ayache, "Computational models for image-guided robot-assisted and simulated medical interventions," Proc. IEEE 94, 1678-1688 (2006).
  4. T. M. Peters, "Image-guidance for surgical procedures," Phys. in Med. Biol. 51, R505-R540 (2006).
  5. S. Vogt, A. Khamene, F. Sauer, "Reality augmentation for medical procedures: System architecture, single camera marker tracking, and system evaluation," Int. J. Comput. Vision 70, 179-190 (2006).
  6. T. Sielhorst, C. Bichlmeier, S. M. Heining, N. Navab, "Depth perception—A major issue in medical AR: Evaluation study by twenty surgeons," Int. Conf. on Medical Image Computing and Computer Assisted Intervention (2006) pp. 364-372.
  7. M. Lerotic, A. J. Chung, G. Mylonas, G. Z. Yang, "pq -space based non-photorealistic rendering for augmented reality," Int. Conf. on Medical Image Computing and Computer Assisted Intervention (2007) pp. 102-109.
  8. T. Okatani, K. Deguchi, "Shape reconstruction from an endoscope image by shape from shading technique for a point light source at the projection centre," Computer Vision and Image Understanding 66, 119-131 (1997).
  9. F. Deligianni, A. J. Chung, G. Z. Yang, "Patient-specific bronchoscope simulation with pq-space-based 2D/3D registration," Computer Aided Surgery 9, 215-226 (2004).
  10. B. K. P. Horn, M. J. Brooks, Shape from Shading (The MIT Press, 1989).
  11. H. U. Rashid, P. Burger, "Differential algorithm for the determination of shape from shading using a point light source," Image and Vision Computing 10, 119-127 (1992).
  12. R. Hartley, A. Zisserman, Multiple View Geometry in Computer Vision (Cambridge Univ. Press, 2000).
  13. J. Heikkila, O. Silven, "A four-step camera calibration procedure with implicit image correction," IEEE Comput. Soc. Conf. on Computer Vision and Pattern Recogn. (1997) pp. 1106-1112.
  14. Z. Zhang, "A flexible new technique for camera calibration," IEEE Trans. Pattern Anal. Mach. Intell. 22, 1330-1334 (2000).
  15. D. Stoyanov, A. Darzi, G. Z. Yang, "Laparoscope self-calibration for robotic assisted minimally invasive surgery," Int. Conf. on Medical Image Computing and Computer Assisted Intervention (2005) pp. 114-121.
  16. S. Baker, T. Sim, T. Kanade, "When is the shape of a scene unique given its light-field: A fundamental theorem of 3D vision?," IEEE Trans. Pattern Anal. Mach. Intell. 25, 100-109 (2003).
  17. D. Stoyanov, A. Darzi, G. Z. Yang, "Dense 3d depth recovery for soft tissue deformation during robotically assisted laparoscopic surgery," Int. Conf. on Medical Image Computing and Computer Assisted Intervention (2004) pp. 41-48.
  18. W. W. Lau, N. A. Ramey, J. Corso, N. V. Thakor, G. D. Hager, "Stereo-based endoscopic tracking of cardiac surface deformation," Int. Conf. on Medical Image Computing and Computer Assisted Intervention (2004) pp. 494-501.
  19. D. Stoyanov, A. Darzi, G. Z. Yang, "A practical approach towards accurate dense 3D depth recovery for robotic laparoscopic surgery," Computer Aided Surgery 10, 199-208 (2005).
  20. K. Mori, D. Deguchi, J. Sugiyama, Y. Suenagaa, J. Toriwakia, C. R. M. Jr, H. Takabatake, H. Natori, "Tracking of a bronchoscope using epipolar geometry analysis and intensity-based image registration of real and virtual endoscopic images," Medical Image Analysis 6, 321-336 (2002).
  21. D. Burschka, M. Li, R. Taylor, G. D. Hager, "Scale-invariant registration of monocular endoscopic images to ct-scans for sinus surgery," International Conference on Medical Image Computing and Computer Assisted Intervention (2004) pp. 413-421.
  22. M. Gröger, T. Ortmaier, W. Sepp, G. Hirzinger, "Tracking local motion on the beating heart," SPIE Med. Imag. Conf. (2002) pp. 233-241.
  23. D. Stoyanov, G. P. Mylonas, F. Deligianni, A. Darzi, G. Z. Yang, "Soft-tissue motion tracking and structure estimation for robotic assisted MIS procedures," Int. Conf. on Medical Image Computing and Computer Assisted Intervention (2005) pp. 139-146.
  24. B. Lucas, T. Kanade, "An iterative image registration technique with an application to stereo vision," Int. Joint Conf. Artif. Intell. (1981) pp. 674-679.
  25. M. Hu, G. P. Penney, P. J. Edwards, M. Figl, D. J. Hawkes, "3D reconstruction of internal organ surfaces for minimal invasive surgery," Int. Conf. on Med. Image Computing and Computer Assisted Intervention (2007) pp. 68-77.
  26. M. Gröger, T. Ortmaier, W. Sepp, G. Hirzinger, "Reconstruction of image structure in presence of specular reflections," DAGM-Symp. on Pattern Recogn. (2001) pp. 53-60.
  27. D. Stoyanov, G. Z. Yang, "Removing specular reflection components for robotic assisted laparoscopic surgery," IEEE Int. Conf. on Image Process. (2005) pp. 350-359.
  28. D. Stoyanov, G. Z. Yang, "Stabilization of image motion for robotic assisted beating heart surgery," Int. Conf. on Medical Image Computing and Computer Assisted Intervention (2007) pp. 417-424.
  29. G. P. Mylonas, D. Stoyanov, A. Darzi, G. Z. Yang, "Assessment of perceptual quality for gaze-contingent motion stabilization in robotic assisted minimally invasive surgery," Int. Conf. on Medical Image Computing and Computer Assisted Intervention (2007) pp. 660-667.
  30. H. Togami, "Effects on visual search performance of individual differences in fixation time and number of fixations," Ergonomics 27, 789-799 (1984).
  31. A. J. Chung, F. Deligianni, P. Shah, A. Wells, M. Elhelw, G. Z. Yang, "Patient specific bronchoscopy visualisation through BRDF estimation and disocclusion correction," IEEE Trans. Med. Imag. 25, 503-513 (2006).
  32. M. ElHelw, M. Nicolaou, A. Chung, G. Z. Yang, M. S. Atkins, "A gaze-based framework for investigating the perception of visual realism in simulated scenes," ACM Trans. Appl. Perception 5, (2008).
  33. X. P. Hu, L. Dempere-Marco, G. Z. Yang, "Hot spot detection based on feature space representation of visual search," IEEE Trans. Med. Imag. 22, 1152-1162 (2003).
  34. L. Dempere-Marco, X. P. Hu, S. L. S. Macdonald, S. M. Ellis, D. M. Hansell, G. Z. Yang, "The use of visual search for knowledge gathering in image decision support," IEEE Trans. Med. Imag. 21, 741-754 (2002).
  35. G. Z. Yang, L. Dempre-Marco, X. P. Hu, A. Rowe, "Visual search: Psychophysical models and practical applications," Image and Vision Computing 20, 291-305 (2002).
  36. A. James, D. Viera, B. P. L. Lo, A. Darzi, G. Z. Yang, "Eye-gaze driven surgical workflow segmentation," Int. Conf. on Medical Image Computing and Computer Assisted Intervention (2007) pp. 110-117.
  37. G. P. Mylonas, A. Darzi, G. Z. Yang, "Gaze-contingent control for minimally invasive robotic surgery," Computer Aided Surgery 11, 256-266 (2006).

Cited By

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

« Previous Article

OSA is a member of CrossRef.

CrossCheck Deposited