OSA's Digital Library

Biomedical Optics Express

Biomedical Optics Express

  • Editor: Joseph A. Izatt
  • Vol. 3, Iss. 10 — Oct. 1, 2012
  • pp: 2567–2578

Multispectral image alignment using a three channel endoscope in vivo during minimally invasive surgery

Neil T. Clancy, Danail Stoyanov, David R. C. James, Aimee Di Marco, Vincent Sauvage, James Clark, Guang-Zhong Yang, and Daniel S. Elson  »View Author Affiliations

Biomedical Optics Express, Vol. 3, Issue 10, pp. 2567-2578 (2012)

View Full Text Article

Enhanced HTML    Acrobat PDF (2035 KB)

Browse Journals / Lookup Meetings

Browse by Journal and Year


Lookup Conference Papers

Close Browse Journals / Lookup Meetings

Article Tools



Sequential multispectral imaging is an acquisition technique that involves collecting images of a target at different wavelengths, to compile a spectrum for each pixel. In surgical applications it suffers from low illumination levels and motion artefacts. A three-channel rigid endoscope system has been developed that allows simultaneous recording of stereoscopic and multispectral images. Salient features on the tissue surface may be tracked during the acquisition in the stereo cameras and, using multiple camera triangulation techniques, this information used to align the multispectral images automatically even though the tissue or camera is moving. This paper describes a detailed validation of the set-up in a controlled experiment before presenting the first in vivo use of the device in a porcine minimally invasive surgical procedure. Multispectral images of the large bowel were acquired and used to extract the relative concentration of haemoglobin in the tissue despite motion due to breathing during the acquisition. Using the stereoscopic information it was also possible to overlay the multispectral information on the reconstructed 3D surface. This experiment demonstrates the ability of this system for measuring blood perfusion changes in the tissue during surgery and its potential use as a platform for other sequential imaging modalities.

© 2012 OSA

OCIS Codes
(170.2150) Medical optics and biotechnology : Endoscopic imaging
(170.3010) Medical optics and biotechnology : Image reconstruction techniques
(170.6510) Medical optics and biotechnology : Spectroscopy, tissue diagnostics

ToC Category:
Endoscopes, Catheters and Micro-Optics

Original Manuscript: June 18, 2012
Revised Manuscript: September 7, 2012
Manuscript Accepted: September 11, 2012
Published: September 14, 2012

Virtual Issues
BIOMED 2012 (2012) Biomedical Optics Express

Neil T. Clancy, Danail Stoyanov, David R. C. James, Aimee Di Marco, Vincent Sauvage, James Clark, Guang-Zhong Yang, and Daniel S. Elson, "Multispectral image alignment using a three channel endoscope in vivo during minimally invasive surgery," Biomed. Opt. Express 3, 2567-2578 (2012)

Sort:  Author  |  Year  |  Journal  |  Reset  


  1. M. A. Ilias, E. Häggblad, C. Anderson, and E. G. Salerud, “Visible, hyperspectral imaging evaluating the cutaneous response to ultraviolet radiation,” Proc. SPIE6441, 644103 (2007). [CrossRef]
  2. B. S. Sorg, B. J. Moeller, O. Donovan, Y. Cao, and M. W. Dewhirst, “Hyperspectral imaging of hemoglobin saturation in tumor microvasculature and tumor hypoxia development,” J. Biomed. Opt.10(4), 044004 (2005). [CrossRef] [PubMed]
  3. S. P. Nighswander-Rempel, R. Anthony Shaw, J. R. Mansfield, M. Hewko, V. V. Kupriyanov, and H. H. Mantsch, “Regional variations in myocardial tissue oxygenation mapped by near-infrared spectroscopic imaging,” J. Mol. Cell. Cardiol.34(9), 1195–1203 (2002). [CrossRef] [PubMed]
  4. M. G. Sowa, J. R. Payette, M. D. Hewko, and H. H. Mantsch, “Visible-near infrared multispectral imaging of the rat dorsal skin flap,” J. Biomed. Opt.4(4), 474–481 (1999). [CrossRef]
  5. R. T. Bryan, L. J. Billingham, and D. M. A. Wallace, “Narrow-band imaging flexible cystoscopy in the detection of recurrent urothelial cancer of the bladder,” BJU Int.101(6), 702–706 (2008). [CrossRef] [PubMed]
  6. J. H. Takano, T. Yakushiji, I. Kamiyama, T. Nomura, A. Katakura, N. Takano, and T. Shibahara, “Detecting early oral cancer: narrowband imaging system observation of the oral mucosa microvasculature,” Int. J. Oral Maxillofac. Surg.39(3), 208–213 (2010). [CrossRef] [PubMed]
  7. H. C. Wolfsen, J. E. Crook, M. Krishna, S. R. Achem, K. R. Devault, E. P. Bouras, D. S. Loeb, M. E. Stark, T. A. Woodward, L. L. Hemminger, F. K. Cayer, and M. B. Wallace, “Prospective, controlled tandem endoscopy study of narrow band imaging for dysplasia detection in Barrett’s Esophagus,” Gastroenterology135(1), 24–31 (2008). [CrossRef] [PubMed]
  8. V. X. D. Yang, P. J. Muller, P. Herman, and B. C. Wilson, “A multispectral fluorescence imaging system: design and initial clinical tests in intra-operative Photofrin-photodynamic therapy of brain tumors,” Lasers Surg. Med.32(3), 224–232 (2003). [CrossRef] [PubMed]
  9. L. Zhou and W. S. El-Deiry, “Multispectral fluorescence imaging,” J. Nucl. Med.50(10), 1563–1566 (2009). [CrossRef] [PubMed]
  10. J. Qi, C. Barrière, T. C. Wood, and D. S. Elson, “Polarized multispectral imaging in a rigid endoscope based on elastic light scattering spectroscopy,” Biomed. Opt. Express3(9), 2087–2099 (2012). [CrossRef]
  11. P. K. Frykman, E. H. Lindsley, M. Gaon, and D. L. Farkas, “Spectral imaging for precise surgical intervention in Hirschsprung’s disease,” J Biophotonics1(2), 97–103 (2008). [CrossRef] [PubMed]
  12. N. T. Clancy, V. Sauvage, S. Saso, D. Stoyanov, D. J. Corless, M. Boyd, D. E. Noakes, G.-Z. Yang, S. Ghaem-Maghami, J. R. Smith, and D. S. Elson, “Registration and analysis of multispectral images acquired during uterine transplantation surgery,” in Biomedical Optics and 3D Imaging, Technical Digest (CD) (Optical Society of America, 2012), paper BSu3A.73.
  13. S. L. Best, A. Thapa, M. J. Holzer, N. Jackson, S. A. Mir, J. A. Cadeddu, and K. J. Zuzak, “Minimal arterial in-flow protects renal oxygenation and function during porcine partial nephrectomy: confirmation by hyperspectral imaging,” Urology78(4), 961–966 (2011). [CrossRef] [PubMed]
  14. K. J. Zuzak, S. C. Naik, G. Alexandrakis, D. Hawkins, K. Behbehani, and E. H. Livingston, “Intraoperative bile duct visualization using near-infrared hyperspectral video imaging,” Am. J. Surg.195(4), 491–497 (2008). [CrossRef] [PubMed]
  15. S. P. Nighswander-Rempel, R. A. Shaw, V. V. Kupriyanov, J. Rendell, B. Xiang, and H. H. Mantsch, “Mapping tissue oxygenation in the beating heart with near-infrared spectroscopic imaging,” Vib. Spectrosc.32(1), 85–94 (2003). [CrossRef]
  16. L. Gao, R. T. Kester, N. Hagen, and T. S. Tkaczyk, “Snapshot Image Mapping Spectrometer (IMS) with high sampling density for hyperspectral microscopy,” Opt. Express18(14), 14330–14344 (2010). [CrossRef] [PubMed]
  17. A. Gorman, D. W. Fletcher-Holmes, and A. R. Harvey, “Generalization of the Lyot filter and its application to snapshot spectral imaging,” Opt. Express18(6), 5602–5608 (2010). [CrossRef] [PubMed]
  18. R. Leitner, M. De Biasio, T. Arnold, C. Viet Dinh, M. Loog, and R. P. W. Duin, “Multi-spectral video endoscopy system for the detection of cancerous tissue,” Pattern Recognit. Lett. (to be published).
  19. G. N. Stamatas, M. Southall, and N. Kollias, “In vivo monitoring of cutaneous edema using spectral imaging in the visible and near infrared,” J. Invest. Dermatol.126(8), 1753–1760 (2006). [CrossRef] [PubMed]
  20. D. Stoyanov, A. Rayshubskiy, and E. Hillman, “Robust registration of multispectral images of the cortical surface in neurosurgery,” in Proceedings of 9th IEEE International Symposium on Biomedical Imaging (Institute of Electrical and Electronic Engineers, Barcelona, 2012), pp. 1643–1646.
  21. K. J. Zuzak, S. C. Naik, G. Alexandrakis, D. Hawkins, K. Behbehani, and E. H. Livingston, “Characterization of a near-infrared laparoscopic hyperspectral imaging system for minimally invasive surgery,” Anal. Chem.79(12), 4709–4715 (2007). [CrossRef] [PubMed]
  22. N. T. Clancy, D. Stoyanov, V. Sauvage, D. R. C. James, G.-Z. Yang, and D. S. Elson, “A Triple Endoscope System for Alignment of Multispectral Images of Moving Tissue,” in Biomedical Optics, Technical Digest (CD) (Optical Society of America, 2010), paper BTuD27.
  23. D. Stoyanov, A. Darzi, and G.-Z. Yang, “Dense 3D depth recovery for soft tissue deformation during robotically assisted laparoscopic surgery,” in MICCAI, C. Barillot, D. R. Haynor, and P. Hellier, eds. (Springer-Verlag, Rennes/Saint-Malo, France, 2004), pp. 41–48.
  24. D. Stoyanov, G. P. Mylonas, F. Deligianni, A. Darzi, and G.-Z. Yang, “Soft-tissue motion tracking and structure estimation for robotic assisted MIS procedures,” in MICCAI, J. Duncan, and G. Gerig, eds. (Springer-Verlag, Palm Springs, USA, 2005), pp. 139–146.
  25. D. Stoyanov and G.-Z. Yang, “Stabilization of image motion for robotic assisted beating heart surgery,” in MICCAI'07 Proceedings of the 10th international conference on Medical image computing and computer-assisted intervention, N. Ayache, S. Ourselin, and A. Maeder, eds. (Springer-Verlag, Brisbane, Australia, 2007), Vol. 1, Part I, pp. 417–424.
  26. R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision (Cambridge University Press, 2000).
  27. Z. Zhang, “A flexible new technique for camera calibration,” IEEE Trans. Pattern Anal.22(11), 1330–1334 (2000). [CrossRef]
  28. D. Scharstein and R. Szeliski, “A taxonomy and evaluation of dense two-frame stereo correspondence algorithms,” Int. J. Comput. Vis.47(1/3), 7–42 (2002). [CrossRef]
  29. S. A. Prahl, “Tabulated molar extinction coefficient for hemoglobin in water” (Oregon Medical Laser Center, 2009), http://omlc.ogi.edu/spectra/hemoglobin/summary.html
  30. K. J. Zuzak, M. D. Schaeberle, M. T. Gladwin, R. O. I. Cannon, and I. W. Levin, “Noninvasive determination of spatially resolved and time-resolved tissue perfusion in humans during nitric oxide inhibition and inhalation by use of a visible-reflectance hyperspectral imaging technique,” Circulation104(24), 2905–2910 (2001). [CrossRef] [PubMed]
  31. D. R. C. James, V. Sauvage, N. T. Clancy, J. Clark, A. W. Darzi, G.-Z. Yang, and D. S. Elson, “Towards intra operative diagnosis of mesenteric oxygenation with hyperspectral imaging,” presented at the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) 12th World Congress of Endoscopic Surgery, National Harbour, Maryland, USA. 2010.
  32. V. Sauvage, D. James, K. Koh, T. Wood, and D. S. Elson, “Development of a hyperspectral laparoscope system for intraoperative diagnosis of intestinal ischemia,” Proc. SPIE7555, 7555-8 (2010).
  33. D. Yudovsky, A. Nouvong, and L. Pilon, “Hyperspectral imaging in diabetic foot wound care,” J. Diabetes Sci. Tech.4(5), 1099–1113 (2010). [PubMed]
  34. J. R. Payette, E. Kohlenberg, L. Leonardi, A. Pabbies, P. Kerr, K.-Z. Liu, and M. G. Sowa, “Assessment of skin flaps using optically based methods for measuring blood flow and oxygenation,” Plast. Reconstr. Surg.115(2), 539–546 (2005). [CrossRef] [PubMed]
  35. C. R. Tracy, J. D. Terrell, R. P. Francis, E. F. Wehner, J. Smith, M. Litorja, D. L. Hawkins, M. S. Pearle, J. A. Cadeddu, and K. J. Zuzak, “Characterization of renal ischemia using DLP hyperspectral imaging: a pilot study comparing artery-only occlusion versus artery and vein occlusion,” J. Endourol.24(3), 321–325 (2010). [CrossRef] [PubMed]
  36. J. M. Bland and D. G. Altman, “Statistical methods for assessing agreement between two methods of clinical measurement,” Lancet327 (8476), 307–310 (1986). [CrossRef] [PubMed]
  37. M. B. Bouchard, B. R. Chen, S. A. Burgess, and E. M. C. Hillman, “Ultra-fast multispectral optical imaging of cortical oxygenation, blood flow, and intracellular calcium dynamics,” Opt. Express17(18), 15670–15678 (2009). [CrossRef] [PubMed]
  38. H. Akbari, L. V. Halig, D. M. Schuster, A. Osunkoya, V. Master, P. T. Nieh, G. Z. Chen, and B. Fei, “Hyperspectral imaging and quantitative analysis for prostate cancer detection,” J. Biomed. Opt.17(7), 076005 (2012). [CrossRef] [PubMed]

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

Supplementary Material

» Media 1: AVI (685 KB)     
» Media 2: AVI (684 KB)     
» Media 3: AVI (685 KB)     

« Previous Article  |  Next Article »

OSA is a member of CrossRef.

CrossCheck Deposited