OSA's Digital Library

Virtual Journal for Biomedical Optics

Virtual Journal for Biomedical Optics


  • Editor: Gregory W. Faris
  • Vol. 2, Iss. 7 — Jul. 16, 2007

What do we perceive from motion pictures? A computational account

Loong-Fah Cheong and Xu Xiang  »View Author Affiliations

JOSA A, Vol. 24, Issue 6, pp. 1485-1500 (2007)

View Full Text Article

Enhanced HTML    Acrobat PDF (774 KB)

Browse Journals / Lookup Meetings

Browse by Journal and Year


Lookup Conference Papers

Close Browse Journals / Lookup Meetings

Article Tools



Cinema viewed from a location other than a canonical viewing point (CVP) presents distortions to the viewer in both its static and its dynamic aspects. Past works have investigated mainly the static aspect of this problem and attempted to explain why viewers still seem to perceive the scene very well. The dynamic aspect of depth perception, which is known as structure from motion, and its possible distortion, have not been well investigated. We derive the dynamic depth cues perceived by the viewer and use the so-called isodistortion framework to understand its distortion. The result is that viewers seated at a reasonably central position experience a shift in the intrinsic parameters of their visual systems. Despite this shift, the key properties of the perceived depths remain largely the same, being determined in the main by the accuracy to which extrinsic motion parameters can be recovered. For a viewer seated at a noncentral position and watching the movie screen at a slant angle, the view is related to the view at the CVP by a homography, resulting in various aberrations such as noncentral projection.

© 2007 Optical Society of America

OCIS Codes
(330.5510) Vision, color, and visual optics : Psychophysics
(330.7310) Vision, color, and visual optics : Vision

ToC Category:
Vision and Color

Original Manuscript: June 19, 2006
Revised Manuscript: December 20, 2006
Manuscript Accepted: December 22, 2006
Published: May 9, 2007

Virtual Issues
Vol. 2, Iss. 7 Virtual Journal for Biomedical Optics

Loong-Fah Cheong and Xu Xiang, "What do we perceive from motion pictures? A computational account," J. Opt. Soc. Am. A 24, 1485-1500 (2007)

Sort:  Author  |  Year  |  Journal  |  Reset  


  1. J. E. Cutting, "Rigidity of cinema seen from the front row, side aisle," J. Exp. Psychol. Hum. Percept. Perform. 13, 323-334 (1987). [CrossRef] [PubMed]
  2. D. N. Perkins, "Compensating for distortion in viewing pictures obliquely," Percept. Psychophys. 14, 13-18 (1973). [CrossRef]
  3. B. Caprile and V. Torre, "Using vanishing points for camera calibration," Int. J. Comput. Vis. 4, 127-140 (1990). [CrossRef]
  4. M. S. Banks, H. F. Rose, D. Vishwanath, and A. R. Girshick, "Where should you sit to watch a movie?" Proc. SPIE 5666, 316-325 (2005). [CrossRef]
  5. C. Metz, Film Language: A Semiotics of the Cinema (U. Chicago Press, 1991).
  6. L. F. Cheong, C. Fermuller, and Y. Aloimonos, "Effects of errors in the viewing geometry on shape estimation," Comput. Vis. Image Underst. 71, 356-372 (1998). [CrossRef]
  7. L. F. Cheong and T. Xiang, "Characterizing depth distortion under different generic motions," Int. J. Comput. Vis. 44, 199-217 (2001). [CrossRef]
  8. T. Viéville, O. D. Faugeras, and Q. T. Luong, "Motion of points and lines in the uncalibrated case," Int. J. Comput. Vis. 17, 7-41 (1996). [CrossRef]
  9. G. Adiv, "Inherent ambiguities in recovering 3-D motion and structure from a noisy flow field," IEEE Trans. Pattern Anal. Mach. Intell. 11, 477-489 (1989). [CrossRef]
  10. K. Daniilidis and M. E. Spetsakis, "Understanding noise sensitivity in structure from motion," in Vision Navigation, Y.Aloimonos, ed. (Hillsdale, 1993), pp. 61-88.
  11. E. Grossmann and J. Santos-Victor, "Uncertainty analysis of 3-D reconstruction from uncalibrated views," Image Vis. Comput. 18, 685-696 (2000). [CrossRef]
  12. J. Oliensis, "A new structure-from-motion ambiguity," IEEE Trans. Pattern Anal. Mach. Intell. 22, 685-700 (2000). [CrossRef]
  13. R. Szeliski and S. B. Kang, "Shape ambiguities in structure-from-motion," IEEE Trans. Pattern Anal. Mach. Intell. 19, 506-512 (1997). [CrossRef]
  14. G. S. Young and R. Chellappa, "Statistical analysis of inherent ambiguities in recovering 3-D motion from a noisy flow field," IEEE Trans. Pattern Anal. Mach. Intell. 14, 995-1013 (1992). [CrossRef]
  15. H. P. Hudson, Cremona Transformations in Plane and Space (Cambridge, 1927).
  16. V. Cornilleau-Pérès and J. Droulez, "The visual perception of 3D shape from self-motion and object-motion," Vision Res. 34, 2331-2336 (1994). [CrossRef] [PubMed]
  17. J. T. Todd and V. J. Perotti, "The visual perception of surface orientation from optical flow," Percept. Psychophys. 61, 1577-1589 (1999). [CrossRef] [PubMed]
  18. F. Domini, C. Caudek, and S. Richmann, "Distortions of depth-order relations and parallelism in structure from motion," Percept. Psychophys. 60, 1164-1174 (1998). [CrossRef] [PubMed]
  19. O. D. Faugeras, Three-Dimensional Computer Vision (MIT, 1993).
  20. J. J. Gibson, The Perception of the Visual World (Houghton Mifflin, 1950).
  21. R. K. Luneburg, Mathematical Analysis of Binocular Vision (Princeton U. Press, 1947).
  22. J. T. Todd and P. Bressan, "The perception of 3-dimensional affine structure from minimal apparent motion sequences," Percept. Psychophys. 48, 419-430 (1990). [CrossRef] [PubMed]
  23. W. C. Hoffman, "The lie algebra of visual perception," J. Math. Psychol. 3, 65-98 (1966). [CrossRef]
  24. J. J. Koenderink and A. J. van Doorn, "Relief: pictorial and otherwise," Image Vis. Comput. 13, 321-334 (1995). [CrossRef]
  25. J. Droulez and V. Cornilleau-Pérès, "Visual perception of surface curvature, the spin variation and its physiological implications," Biol. Cybern. 62, 211-224 (1990). [CrossRef] [PubMed]
  26. T. S. Meese and M. G. Harris, "Computation of surface slant from optic flow: orthogonal components of speed gradient can be combined," Vision Res. 37, 2369-2379 (1997). [CrossRef] [PubMed]
  27. T. Viéville, J. Droulez, C. H. Peh, and A. Negri, "How do we perceive the eye's intrinsic parameters?" Tech. Rep. RR 4030 (INRIA, 2000).
  28. M. A. Goodale and D. A. Westwood, "An evolving view of duplex vision: separate but interacting cortical pathways for perception and action," Curr. Opin. Neurobiol. 14, 203-211 (2004). [CrossRef] [PubMed]
  29. L. F. Cheong, T. Xiang, V. Cornilleau-Pérès, and L. C. Tai, "Not all motions are equivalent in terms of depth recovery," in Computer Vision and Robotics, J.X.Liu, ed. (Nova, 2005), pp. 99-134.
  30. J. S. Tittle, J. T. Todd, V. J. Perotti, and J. F. Norman, "Systematic distortion of perceived three-dimensional structure from motion and binocular stereopsis," J. Exp. Psychol. Hum. Percept. Perform. 21, 663-678 (1995). [CrossRef] [PubMed]
  31. V. Cornilleau-Pérès and J. Droulez, "Visual perception of surface curvature: Psychophysics of curvature detection induced by motion parallax," Percept. Psychophys. 46, 351-364 (1989). [CrossRef] [PubMed]
  32. J. F. Norman and J. S. Lappin, "The detection of surface curvatures defined by optical motion," Percept. Psychophys. 51, 386-396 (1992). [CrossRef] [PubMed]
  33. B. J. Rogers and M. E. Graham, "Anisotropies in the perception of three-dimensional surfaces," Science 221, 1409-1411 (1983). [CrossRef] [PubMed]
  34. W. J. M. Damme and W. A. van de Grind, "Active vision and the identification of 3D shape," Vision Res. 11, 1581-1587 (1993).
  35. W. J. M. Damme, F. H. Oosterhoff, and W. A. van de Grind, "Discrimination of 3-D shape and 3-D curvature from motion in active vision," Percept. Psychophys. 55, 340-349 (1994). [CrossRef] [PubMed]
  36. H. C. Longuet-Higgins, "A computer algorithm for reconstructing a scene from two projections," Nature (London) 293, 133-135 (1981). [CrossRef]
  37. O. D. Faugeras, "Stratification of 3D vision: projective, affine, and metric representations," J. Opt. Soc. Am. A 12, 465-484 (1995). [CrossRef]
  38. A. Heyden and K. Åström, "Euclidean reconstruction from image sequences with varying and unknown focal length and principal point," In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 1997), pp. 438-443.
  39. R. I. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd ed. (Cambridge U., 2004). [CrossRef]
  40. R. Petrozzo and S. W. Singer, "Cinema projection distortion," in Proceedings of the 141st Society of Motion Picture and Television Engineers Technical Conference and Exhibition (SMPTE, 1999).
  41. P. F. Sturm and S. Ramalingam, "A generic concept for camera calibration," in Proceedings of European Conference on Computer Vision, T.Pajdla and J.Matas, eds. (Springer, 2004), pp. 1-13.
  42. P. F. Sturm, "Multi-view geometry for general camera models," in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2005) pp. I: 206-212.
  43. W. N. Charman, Visual Optics and Instrumentation (Macmillan, 1991).
  44. S. J. Galvin, D. R. Williams, and N. J. Coletta, "The spatial grain of motion perception in human peripheral vision," Vision Res. 36, 2283-2296 (1996). [CrossRef] [PubMed]
  45. P. Artal, A. Derrington, and E. Colombo, "Refraction, aliasing, and the absence of motion reversals in peripheral vision," Vision Res. 35, 939-947 (1995). [CrossRef] [PubMed]
  46. A. Guirao and P. Artal, "Off-axis monochromatic aberrations estimated from double pass measurements in the human eye," Vision Res. 39, 207-217 (1999). [CrossRef] [PubMed]
  47. S. Ullman, The Interpretation of Visual Motion (MIT, 1979).
  48. M. Martha, Producing Videos: A Complete Guide (Australian Film, Television, and Radio School, 1997).
  49. B. Leeuw, Digital Cinematography: Lighting and Photographing Computer-Generated Animation (Morgan Kaufmann, 1997).
  50. D. Raymond, The Strange Case of Alfred Hitchcock (MIT, 1978).
  51. L. F. Cheong and C. H. Peh, "Depth distortion under calibration uncertainty," Comput. Vis. Image Underst. 93, 221-244 (2004). [CrossRef]
  52. L. F. Cheong and X. Xiang, "Error characteristics of sfm with unknown focal length," in Proceedings of Asian Conference on Computer Vision, P.J.Narayanan, S.K.Nayar and H.Y.Shum, eds. (Springer, 2006).
  53. K. N. Ogle, Researches in Binocular Vision (Hafner,1964).
  54. W. C. Gogel, "The analysis of perceived space," in Foundations of Perceptual Theory, S.C.Masin, ed. (Elsevier, 1993), pp. 113-182. [CrossRef]
  55. J. M. Loomis and J. W. Philbeck, "Is the anisotropy of perceived 3-d shape invariant across scale?" Percept. Psychophys. 61, 397-402 (1999). [CrossRef] [PubMed]
  56. M. Wagner, "The metric of visual space," Percept. Psychophys. 38, 483-495 (1985). [CrossRef] [PubMed]
  57. M. J. A. Doumen, A. M. L. Kappers, and J. J. Koenderink, "Visual space under free viewing conditions," Percept. Psychophys. 67, 1177-1189 (2005). [CrossRef]
  58. D. N. Lee, "The optic flow field: The foundation of vision," Philos. Trans. R. Soc. London, Ser. B 290, 169-178 (1980). [CrossRef] [PubMed]
  59. R. C. Nelson and J. Aloimonos, "Obstacle avoidance using flow field divergence," IEEE Trans. Pattern Anal. Mach. Intell. 11, 1102-1106 (1989). [CrossRef]
  60. M. Subbarao, Interpretation of Visual Motion: A Computational Study (Pitman, 1988).
  61. K. A. Stevens and A. Brookes, "Integrating stereopsis with monocular interpretations of planar surfaces," Vision Res. 28, 371-386 (1998). [CrossRef]
  62. J. E. Sparrow and W. M. Stine, "The perceived rigidity of rotating eight-vertex geometric forms; extracting nonrigid structure from rigid motion," Vision Res. 38, 541-556 (1998). [CrossRef] [PubMed]
  63. V. Cornilleau-Pèrés, M. Wexler, J. Droulez, E. Marin, C. Miège, and B. Bourdoncle, "Visual perception of planar orientation: dominance of static depth cues over motion cues," Vision Res. 42, 1403-1412 (2002). [CrossRef] [PubMed]
  64. J. M. Hillis, S. J. Watt, M. S. Landy, and M. S. Banks, "Slant from texture and disparity cues: optimal cue combination," J. Vision 4, 967-992 (2004). [CrossRef]
  65. D. C. Knill and J. Saunders, "Do humans optimally integrate stereo and texture information for judgments of surface slant?" Vision Res. 43, 2539-2558 (2003). [CrossRef] [PubMed]
  66. D. Kersten, P. Mamassian, and A. Yuille, "Object perception as Bayesian inference," Annu. Rev. Psychol. 55, 271-304 (2004). [CrossRef] [PubMed]
  67. J. Trommershauser, S. Gepshtein, L. T. Maloney, M. S. Landy, and M. S. Banks, "Optimal compensation for changes in task-relevant movement variability," J. Neurosci. 25, 7169-7178 (2005). [CrossRef] [PubMed]

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

Next Article »

OSA is a member of CrossRef.

CrossCheck Deposited