OSA's Digital Library

Journal of the Optical Society of America A

Journal of the Optical Society of America A

| OPTICS, IMAGE SCIENCE, AND VISION

  • Vol. 16, Iss. 3 — Mar. 1, 1999
  • pp: 679–693

Prediction of human observer performance by numerical observers: an experimental study

T. K. Narayan and Gabor T. Herman  »View Author Affiliations


JOSA A, Vol. 16, Issue 3, pp. 679-693 (1999)
http://dx.doi.org/10.1364/JOSAA.16.000679


View Full Text Article

Enhanced HTML    Acrobat PDF (1736 KB)





Browse Journals / Lookup Meetings

Browse by Journal and Year


   


Lookup Conference Papers

Close Browse Journals / Lookup Meetings

Article Tools

Share
Citations

Abstract

Numerical observers are investigated for predicting the outcome of a free-response human observer study involving the detection of simulated pulmonary nodules in images reconstructed from low-dose computed tomography projection data by use of several reconstruction algorithms. A new way of calculating the figure of merit of a numerical observer is proposed wherein the detectability of signals in a particular image depends on the noise properties associated with that image and not the other images in the data set. The resulting variants of numerical observers are found to perform better than their traditional counterparts. In particular, the imagewise variant of the region-of-interest observer is found to predict best the rank ordering of algorithms by human observers for the free-response task.

© 1999 Optical Society of America

OCIS Codes
(100.0100) Image processing : Image processing
(100.2960) Image processing : Image analysis
(100.3010) Image processing : Image reconstruction techniques
(100.5010) Image processing : Pattern recognition
(110.4280) Imaging systems : Noise in imaging systems
(330.4060) Vision, color, and visual optics : Vision modeling
(330.5020) Vision, color, and visual optics : Perception psychology
(330.5510) Vision, color, and visual optics : Psychophysics

History
Original Manuscript: May 11, 1998
Revised Manuscript: October 27, 1998
Manuscript Accepted: October 1, 1998
Published: March 1, 1999

Citation
T. K. Narayan and Gabor T. Herman, "Prediction of human observer performance by numerical observers: an experimental study," J. Opt. Soc. Am. A 16, 679-693 (1999)
http://www.opticsinfobase.org/josaa/abstract.cfm?URI=josaa-16-3-679


Sort:  Author  |  Year  |  Journal  |  Reset  

References

  1. G. T. Herman, Image Reconstruction from Projections: The Fundamentals of Computerized Tomography (Academic, New York, 1980).
  2. G. T. Herman, D. Odhner, “Performance evaluation of an iterative image reconstruction algorithm for positron emission tomography,” IEEE Trans. Med. Imaging 10, 336–346 (1991). [CrossRef] [PubMed]
  3. H. H. Barrett, “Objective assessment of image quality: effects of quantum noise and object variability,” J. Opt. Soc. Am. A 7, 1266–1278 (1990). [CrossRef] [PubMed]
  4. T. A. Gooley, H. H. Barrett, “Evaluation of statistical methods of image reconstruction through ROC analysis,” IEEE Trans. Med. Imaging 11, 276–283 (1992). [CrossRef]
  5. K. M. Hanson, “Method of evaluating image-recovery algorithms based on task performance,” J. Opt. Soc. Am. A 7, 1294–1304 (1990). [CrossRef]
  6. G. T. Herman, K. T. D. Yeung, “Evaluators of image reconstruction algorithms,” Int. J. Imag. Syst. Technol. 1, 187–195 (1989). [CrossRef]
  7. M. F. Insana, T. J. Hall, “Methods for estimating the efficiency of human and computational observers in ultrasonography,” in Information Processing in Medical Imaging, H. H. Barrett, A. F. Gmitro, eds. (Springer-Verlag, Berlin, 1993), Vol. 687, pp. 542–552.
  8. J. A. Swets, R. M. Pickett, Evaluation of Diagnostic Systems: Methods from Signal Detection Theory (Academic, New York, 1982).
  9. D. P. Chakraborty, L. H. L. Winter, “Free-response methodology: alternate analysis and a new observer-performance experiment,” Radiology 174, 873–881 (1990). [PubMed]
  10. D. M. Green, J. A. Swets, Signal Detection Theory and Psychophysics (Peninsula, Los Altos, Calif., 1988).
  11. J. Llacer, E. Veklerov, L. R. Baxter, S. T. Grafton, L. K. Griffeth, R. A. Hawkins, C. K. Hoh, J. C. Mazziotta, E. J. Hoffman, C. E. Metz, “Results of a clinical receiver operating characteristic study comparing filtered backprojection and maximum likelihood estimator images in FDG PET studies,” J. Nucl. Med. 34, 1198–1203 (1993). [PubMed]
  12. C. E. Metz, “Some practical issues of experimental design and data analysis in radiological ROC studies,” Invest. Radiol. 24, 234–243 (1989). [CrossRef] [PubMed]
  13. R. G. Swensson, “Measuring detection and localization performance,” in Information Processing in Medical Imaging, H. H. Barrett, A. F. Gmitro, eds. (Springer-Verlag, Berlin, 1993), Vol. 687, pp. 525–541.
  14. J. A. Swets, “ROC analysis applied to the evaluation of medical imaging techniques,” Invest. Radiol. 4, 109–112 (1979). [CrossRef]
  15. R. D. Fiete, H. H. Barrett, W. E. Smith, K. J. Myers, “Hotelling trace criterion and its correlation with human-observer performance,” J. Opt. Soc. Am. A 4, 945–953 (1987). [CrossRef] [PubMed]
  16. S. S. Furuie, G. T. Herman, T. K. Narayan, P. Kinahan, J. S. Karp, R. M. Lewitt, S. Matej, “A methodology for testing for statistically significant differences between fully 3-D PET reconstruction algorithms,” Phys. Med. Biol. 39, 341–354 (1994). [CrossRef] [PubMed]
  17. S. Matej, G. T. Herman, T. K. Narayan, S. S. Furuie, R. M. Lewitt, P. Kinahan, “Evaluation of task-oriented performance of several fully 3-D PET reconstruction algorithms,” Phys. Med. Biol. 39, 355–367 (1994). [CrossRef] [PubMed]
  18. K. J. Myers, H. H. Barrett, M. C. Borgstrom, D. D. Patton, D. W. Seeley, “Effect of noise correlation on detectability of disk signals in medical imaging,” J. Opt. Soc. Am. A 2, 1752–1759 (1985). [CrossRef] [PubMed]
  19. K. J. Myers, H. H. Barrett, “Addition of a channel mechanism to the ideal-observer model,” J. Opt. Soc. Am. A 4, 2447–2457 (1987). [CrossRef] [PubMed]
  20. W. E. Smith, H. H. Barrett, “Hotelling trace criterion as a figure of merit for the optimization of imaging systems,” J. Opt. Soc. Am. A 3, 717–723 (1986). [CrossRef]
  21. R. F. Wagner, K. J. Myers, D. G. Brown, M. J. Tapiovaara, A. E. Burgess, “Higher-order tasks: human vs. machine performance,” in Medical Imaging II, R. H. Schneider, S. J. Dwyer, eds., Proc. SPIE1090, 183–194 (1989). [CrossRef]
  22. Medical Imaging—The Assessment of Image Quality, (International Commission on Radiation Units and Measurement, Bethesda, Md., 1996).
  23. A. E. Burgess, “Statistically defined backgrounds: performance of a modified nonprewhitening observer model,” J. Opt. Soc. Am. A 11, 1237–1242 (1994). [CrossRef]
  24. M. Kendall, J. D. Gibbons, Rank Correlation Methods (Oxford U. Press, New York, 1990).
  25. C. K. Abbey, H. H. Barrett, D. W. Wilson, “Observer signal-to-noise ratios for the ML-EM algorithm,” in Medical Imaging, 1996: Image Perception, H. L. Kundel, ed., Proc. SPIE2712, 47–58 (1996). [CrossRef]
  26. D. P. Chakraborty, “Maximum likelihood analysis of free-response receiver operating characteristic (FROC) data,” Med. Phys. 16, 561–568 (1989). [CrossRef] [PubMed]
  27. P. C. Bunch, J. F. Hamilton, G. K. Sanderson, A. H. Simmons, “A free-response approach to the measurement and characterization of radiographic-observer performance,” J. Appl. Photogr. Eng. 4, 166–171 (1978).
  28. C. E. Metz, “ROC methodology in radiologic imaging,” Invest. Radiol. 21, 720–733 (1986). [CrossRef] [PubMed]
  29. E. Pernkopf, Atlas of Topographical and Applied Human Anatomy (Saunders, Philadelphia, Pa., 1964), Vol. 2.
  30. J. A. Browne, G. T. Herman, D. Odhner, “SNARK93— a programming system for image reconstruction from projections,” (Department of Radiology, University of Pennsylvania, Philadelphia, Pa., 1993).
  31. G. T. Herman, “Algorithms for computed tomography,” in The Digital Signal Processing Handbook, V. K. Madisetti, D. B. Williams, eds. (CRC Press, Boca Raton, Fla., 1998), Chap. 26, pp. 1–9.
  32. R. M. Lewitt, “Alternatives to voxels for image representation in iterative reconstruction algorithms,” Phys. Med. Biol. 37, 705–716 (1992). [CrossRef] [PubMed]
  33. S. Matej, R. M. Lewitt, “Practical considerations for 3-D image reconstruction using spherically symmetric volume elements,” IEEE Trans. Med. Imaging 15, 68–78 (1996). [CrossRef] [PubMed]
  34. P. P. B. Eggermont, G. T. Herman, A. Lent, “Iterative algorithms for larger partitioned systems with applications to image reconstruction,” Linear Algebr. Appl. 40, 37–67 (1981). [CrossRef]
  35. L. A. Shepp, Y. Vardi, “Maximum likelihood reconstruction in positron emission tomography,” IEEE Trans. Med. Imaging 1, 113–122 (1982). [CrossRef]
  36. J. Browne, A. R. De Pierro, “A row-action alternative to the EM algorithm for maximizing likelihoods in emission tomography,” IEEE Trans. Med. Imaging 15, 687–699 (1996). [CrossRef]
  37. K. T. D. Yeung, G. T. Herman, “Objective measures to evaluate the performance of image reconstruction algorithms,” in Medical Imaging III: Image Processing, S. J. Dwyer, R. Jost, R. H. Schneider, eds., Proc. SPIE1092, 326–335 (1989). [CrossRef]
  38. H. H. Barrett, J. Yao, J. P. Rolland, K. J. Myers, “Model observers for assessment of image quality,” Proc. Natl. Acad. Sci. USA 90, 9758–9765 (1993). [CrossRef] [PubMed]
  39. J. Yao, H. H. Barrett, “Predicting human performance by a channelized Hotelling observer model,” in Mathematical Methods in Medical Imaging, D. C. Wilson, J. N. Wilson, eds., Proc. SPIE1768, 161–168 (1992). [CrossRef]
  40. A. Burgess, X. Li, “Experimental evaluation of observer models for detection of signals in statistically defined backgrounds,” in Information Processing in Medical Imaging, Y. Bizais, C. Barillot, R. Di Paola, eds. (Kluwer Academic, Dordrecht, The Netherlands, 1995), pp. 89–100.
  41. C. K. Abbey, H. H. Barrett, “Linear iterative reconstruction algorithms: study of observer performance,” in Information Processing in Medical Imaging, Y. Bizais, C. Barillot, R. Di Paola, eds. (Kluwer Academic, Dordrecht, The Netherlands, 1995), pp. 65–76.
  42. B. Efron, The Jackknife, the Bootstrap and Other Resampling Plans (Society For Industrial and Applied Mathematics, Philadelphia, Pa., 1982).
  43. R. F. Mould, Introduction to Medical Statistics, 2nd ed. (Hilger, Bristol, UK, 1989).
  44. M. A. King, D. J. de Vries, E. J. Soares, “Comparison of the channelized Hotelling and human observers for lesion detection in hepatic SPECT imaging,” in Medical Imaging 1997: Image Perception, H. L. Kundel, ed., Proc. SPIE3036, 14–20 (1997). [CrossRef]

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.


« Previous Article  |  Next Article »

OSA is a member of CrossRef.

CrossCheck Deposited