A geometric framework for nonlinear visual coding
Optics Express, Vol. 7, Issue 4, pp. 155-165 (2000)
http://dx.doi.org/10.1364/OE.7.000155
Acrobat PDF (232 KB)
Abstract
It is argued that important aspects of early and middle level visual coding may be understood as resulting from basic geometric processing of the visual input. The input is treated as a hypersurface defined by image intensity as a function of two spatial coordinates and time. Analytical results show how the Riemann curvature tensor R of this hypersurface represents speed and direction of motion. Moreover, the results can predict the selectivity of MT neurons for multiple motions and for motion in a direction along the optimal spatial orientation. Finally, a model based on integrated R components predicts global-motion percepts related to the barber-pole illusion.
© Optical Society of America
[Optical Society of America ]
Introduction
H. R. Wilson and J. R. Bergen, “A four mechanisms model for threshold spatial vision,” Vision Research 19, 19–33 (1979). [CrossRef] [PubMed]
C. Zetzsche and E. Barth, “Fundamental limits of linear filters in the visual processing of two-dimensional signals,” Vision Research 30, 1111–1117 (1990). [CrossRef] [PubMed]
R. M. Haralick, L. T. Watson, and T. J. Laffey, “The topographic primal sketch,” International J. of Robotic Research 2, 50–72 (1983). [CrossRef]
P. J. Besl and R. C. Jain, “Segmentation through variable-order surface fitting,” IEEE Trans. Pattern Anal. Mach. Intell. 10, 167–192 (1988). [CrossRef]
J. J. Koenderink and A. J. v. Doorn, “Representation of local geometry in the visual system,” Biol. Cybern. 55, 367–375 (1987). [CrossRef] [PubMed]
D. H. Hubel and T. N. Wiesel, “Receptive fields and functional architecture of monkey striate cortex,” J. Physiol. 195, 215–243 (1968). [PubMed]
J. Y. Lettvin, H. R. Maturana, W. S. McCulloch, and W. H. Pitts, “What the frog’s eye tells the frog’s brain,” Proceedings IRE 47, 1940–1951 (1959). [CrossRef]
G. A. Orban, Neuronal operations in the visual cortex , (Springer, Heidelberg, 1984). [CrossRef]
J. B. Levitt, D. C. Kiper, and J. A. Movshon, “Receptive fields and functional architecture of macaque V2,” J Neurophysiol 71, 2517–42 (1994). [PubMed]
C. Yu and D. M. Levi, “End stopping and length tuning in psychophysical spatial filters,” J. Opt. Soc. Am. A 14, 2346–54 (1997). [CrossRef]
A. Dobbins, S. W. Zucker, and M. S. Cynader, “Endstopping and curvature,” Vision Res 29, 1371–87 (1989). [CrossRef] [PubMed]
H. R. Wilson and W. A. Richards, “Mechanisms of contour curvature discrimination,” J. Opt. Soc. Am. A 6, 106–115 (1989). [CrossRef] [PubMed]
C. Zetzsche and E. Barth, “Fundamental limits of linear filters in the visual processing of two-dimensional signals,” Vision Research 30, 1111–1117 (1990). [CrossRef] [PubMed]
C. Zetzsche and E. Barth, “Fundamental limits of linear filters in the visual processing of two-dimensional signals,” Vision Research 30, 1111–1117 (1990). [CrossRef] [PubMed]
S. P. Liou and R. C. Jain, “Motion detection in spatio-temporal space,” Computer Vision, Graphics, and Image Processing 45, 227–250 (1989). [CrossRef]
C. Zetzsche and E. Barth, “Direct detection of flow discontinuities by 3D-curvature operators,” Pattern Recognition Letters 12, 771–779 (1991). [CrossRef]
Curvature as deviation from flatness
Curvature and redundancy
E. Barth, T. Caelli, and C. Zetzsche, “Image encoding, labelling and reconstruction from differential geometry,” CVGIP:GRAPHICAL MODELS AND IMAGE PROCESSING 55, 428–446 (1993). [CrossRef]
Geometry of movie hypersurfaces
Riemann tensor
E. Barth, C. Zetzsche, and G. Krieger, “Curvature measures in visual information processing,” Open Systems and Information Dynamics 5, 25–39 (1998). [CrossRef]
Riemann-tensor components
E. Barth, Riemann-tensor motion analysis, , (2000), http://www.visionscience.com/vsDemos.html.
Curvature and motion
Motion in terms of R components
T. S. Huang and A. N. Netravali, “Motion and structure from feature correspondence: a review,” Proceedings of the IEEE 82, 252–268 (1994). [CrossRef]
Motion detection
A. B. Watson and A. J. Ahumada Jr.“Model of human visual-motion sensing,” J. Opt. Soc. Am A 2, 322–342 (1985). [CrossRef] [PubMed]
E. Barth, C. Zetzsche, and G. Krieger, “Curvature measures in visual information processing,” Open Systems and Information Dynamics 5, 25–39 (1998). [CrossRef]
C. Zetzsche and E. Barth, “Direct detection of flow discontinuities by 3D-curvature operators,” Pattern Recognition Letters 12, 771–779 (1991). [CrossRef]
S. J. Nowlan and T. J. Sejnowski, “A selection model for motion processing in area MT of primates,” J Neurosci 15, 1195–214 (1995). [PubMed]
| Ambiguous motion | (<-) -> | R=0 |
| Occlusions, discontinuities, | <-> | K≠0 |
| multiple motions | ||
| Defined motion(Eq. 3) | -> | R≠0, K=0 |
| Defined motion(Eq. 3) | <-> | R≠0, K=0, all R components ≠0 |
| Defined motion(Eq. 3) | <-> | R≠0, K=0, R2121 ≠0 |
| Defined motion(Eq. 3) | (<-) -> | Different direction in (6) defined and equal |
Simulations of global motion percepts
S. Wuerger, R. Shapley, and N. Rubin, ““On the visually perceived direction of motion” by Hans Wallach: 60 years later,” Perception 25, 1317–1367 (1996). [CrossRef]
F. L. Kooi, “Local direction of edge motion causes and abolishes the barberpole illusion,” Vision Res 33, 2347–51 (1993). [CrossRef] [PubMed]
E. Barth, C. Zetzsche, and I. Rentschler, “Intrinsic two-dimensional features as textons,” J Opt Soc Am A Opt Image Sci Vis 15, 1723–32 (1998). [CrossRef] [PubMed]
E. Barth, T. Caelli, and C. Zetzsche, “Image encoding, labelling and reconstruction from differential geometry,” CVGIP:GRAPHICAL MODELS AND IMAGE PROCESSING 55, 428–446 (1993). [CrossRef]
Curvature and motion-selective neurons
Orthogonal direction and orientation tunings
T. D. Albright, “Direction and orientation selectivity of neurons in visual area MT of the macaque,” J Neurophysiol 52, 1106–30 (1984). [PubMed]
T. D. Albright, “Direction and orientation selectivity of neurons in visual area MT of the macaque,” J Neurophysiol 52, 1106–30 (1984). [PubMed]
Multiple motions
E. P. Simoncelli and D. J. Heeger, “A model of neuronal responses in visual area MT,” Vision Res 38, 743–61 (1998). [CrossRef] [PubMed]
Computational aspects
Discussion
Conclusion
Acknowledgements
References and links
H. R. Wilson and J. R. Bergen, “A four mechanisms model for threshold spatial vision,” Vision Research 19, 19–33 (1979). [CrossRef] [PubMed] | |
A. B. Watson, “Detection and recognition of simple spatial forms,” in Physical and biological processing of images , O. J. Braddick and A. C. Sleigh, eds. (Springer-Verlag, Berlin, 1983). [CrossRef] | |
A. B. Watson and A. J. Ahumada Jr.“Model of human visual-motion sensing,” J. Opt. Soc. Am A 2, 322–342 (1985). [CrossRef] [PubMed] | |
E. H. Adelson and J. R. Bergen, “The Plenoptic Function and the Elements of Early Vision,” in Computational Models of Visual Processing , M. Landy and J. A. Movshon, eds. (MIT Press, Cambridge, MA, 1991). | |
C. Zetzsche and E. Barth, “Fundamental limits of linear filters in the visual processing of two-dimensional signals,” Vision Research 30, 1111–1117 (1990). [CrossRef] [PubMed] | |
C. Zetzsche, E. Barth, and B. Wegmann, “The importance of intrinsically two-dimensional image features in biological vision and picture coding,” in Digital images and human vision , A. B. Watson, ed. (MIT Press, Cambridge, MA, 1993). | |
R. M. Haralick, L. T. Watson, and T. J. Laffey, “The topographic primal sketch,” International J. of Robotic Research 2, 50–72 (1983). [CrossRef] | |
P. J. Besl and R. C. Jain, “Segmentation through variable-order surface fitting,” IEEE Trans. Pattern Anal. Mach. Intell. 10, 167–192 (1988). [CrossRef] | |
J. Shi and C. Tomasi, “Good features to track,” Proc. of the IEEE Conference on Computer Vision and Pattern Recognition , 593–600 (1994). | |
J. J. Koenderink and A. J. v. Doorn, “Representation of local geometry in the visual system,” Biol. Cybern. 55, 367–375 (1987). [CrossRef] [PubMed] | |
D. H. Hubel and T. N. Wiesel, “Receptive fields and functional architecture of monkey striate cortex,” J. Physiol. 195, 215–243 (1968). [PubMed] | |
J. Y. Lettvin, H. R. Maturana, W. S. McCulloch, and W. H. Pitts, “What the frog’s eye tells the frog’s brain,” Proceedings IRE 47, 1940–1951 (1959). [CrossRef] | |
G. A. Orban, Neuronal operations in the visual cortex , (Springer, Heidelberg, 1984). [CrossRef] | |
E. Peterhans and R. von der Heydt, “Functional organization of area V2 in the alert macaque,” European Journal of Neuroscience 5, 509–24 (1993). [CrossRef] [PubMed] | |
J. B. Levitt, D. C. Kiper, and J. A. Movshon, “Receptive fields and functional architecture of macaque V2,” J Neurophysiol 71, 2517–42 (1994). [PubMed] | |
C. Yu and D. M. Levi, “End stopping and length tuning in psychophysical spatial filters,” J. Opt. Soc. Am. A 14, 2346–54 (1997). [CrossRef] | |
A. Dobbins, S. W. Zucker, and M. S. Cynader, “Endstopping and curvature,” Vision Res 29, 1371–87 (1989). [CrossRef] [PubMed] | |
F. Heitger, L. Rosenthaler, R. von der Heydt, E. Peterhans, and O. Kubler, “Simulation of neural contour mechanisms: from simple to end-stopped cells,” Vision Res 32, 963–81 (1992). [CrossRef] [PubMed] | |
H. R. Wilson and W. A. Richards, “Mechanisms of contour curvature discrimination,” J. Opt. Soc. Am. A 6, 106–115 (1989). [CrossRef] [PubMed] | |
S. P. Liou and R. C. Jain, “Motion detection in spatio-temporal space,” Computer Vision, Graphics, and Image Processing 45, 227–250 (1989). [CrossRef] | |
C. Zetzsche and E. Barth, “Direct detection of flow discontinuities by 3D-curvature operators,” Pattern Recognition Letters 12, 771–779 (1991). [CrossRef] | |
C. Zetzsche, E. Barth, and J. Berkmann, “Spatio-temporal curvature measures for flow field analysis,” Geometric Methods in Computer Vision , B. Vemuri Ed. SPIE 1590, 337–350 (1991). | |
M. P. Do Carmo, Riemannian Geometry , (Birkhäuser, Boston, 1992). | |
S. Weinberg, Gravitation and Cosmology , (Wiley and Sons, New York, 1972). | |
B. Schutz, A first course in general relativity , (Cambridge University Press, Cambridge, 1985). | |
E. Barth, T. Caelli, and C. Zetzsche, “Image encoding, labelling and reconstruction from differential geometry,” CVGIP:GRAPHICAL MODELS AND IMAGE PROCESSING 55, 428–446 (1993). [CrossRef] | |
C. Mota and J. Gomes, “Curvature Operators in Geometric Image Processing,” presented at Brasilian Symposium On Computer Graphics and Image Processing, (Campinas, Brazil, 1999). | |
E. Barth, C. Zetzsche, and G. Krieger, “Curvature measures in visual information processing,” Open Systems and Information Dynamics 5, 25–39 (1998). [CrossRef] | |
E. Barth, Riemann-tensor motion analysis, , (2000), http://www.visionscience.com/vsDemos.html. | |
O. Tretiak and L. Pastor, “Velocity estimation from image sequences with second order differential operators,” presented at Proc. 7th Int. Conf. Pattern Recognition, (Montreal, Canada, 1984). | |
T. S. Huang and A. N. Netravali, “Motion and structure from feature correspondence: a review,” Proceedings of the IEEE 82, 252–268 (1994). [CrossRef] | |
E. Barth, “Spatio-temporal curvature and the visual coding of motion,” in Neural Computation (NC’2000) , vol. 1404–093, H. Bothe and R. Rojas, eds. (ICSC Academic Press, Berlin, 2000). | |
H. Haußecker and H. Spies, “Motion,” in Handbook of Computer Vision and Applications ,B. Jahne, H. Haußecker, and P. Geissler, eds., 1999). | |
S. J. Nowlan and T. J. Sejnowski, “A selection model for motion processing in area MT of primates,” J Neurosci 15, 1195–214 (1995). [PubMed] | |
S. Wuerger, R. Shapley, and N. Rubin, ““On the visually perceived direction of motion” by Hans Wallach: 60 years later,” Perception 25, 1317–1367 (1996). [CrossRef] | |
F. L. Kooi, “Local direction of edge motion causes and abolishes the barberpole illusion,” Vision Res 33, 2347–51 (1993). [CrossRef] [PubMed] | |
E. Barth, C. Zetzsche, and I. Rentschler, “Intrinsic two-dimensional features as textons,” J Opt Soc Am A Opt Image Sci Vis 15, 1723–32 (1998). [CrossRef] [PubMed] | |
T. D. Albright, “Direction and orientation selectivity of neurons in visual area MT of the macaque,” J Neurophysiol 52, 1106–30 (1984). [PubMed] | |
G. H. Recanzone, R. H. Wurtz, and U. Schwarz, “Responses of MT and MST neurons to one and two moving objects in the receptive field,” J Neurophysiol 78, 2904–15 (1997). | |
E. P. Simoncelli and D. J. Heeger, “A model of neuronal responses in visual area MT,” Vision Res 38, 743–61 (1998). [CrossRef] [PubMed] | |
E. Barth and A. B. Watson, “Nonlinear spatio-temporal model based on the geometry of the visual input,” Investigative Ophthalmology and Visual Science 39, S2110 (1998). |
OCIS Codes
(150.4620) Machine vision : Optical flow
(330.4060) Vision, color, and visual optics : Vision modeling
(330.4150) Vision, color, and visual optics : Motion detection
ToC Category:
Research Papers
History
Original Manuscript: June 28, 2000
Published: August 14, 2000
Citation
Erhardt Barth and Andrew Watson, "A geometric framework for nonlinear visual coding," Opt. Express 7, 155-165 (2000)
http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-7-4-155
Sort: Journal | Reset
References
- H. R. Wilson and J. R. Bergen, "A four mechanisms model for threshold spatial vision," Vision Research 19 , 19-33 (1979 ). [CrossRef] [PubMed]
- A. B. Watson, "Detection and recognition of simple spatial forms," in Physical and biological processing of images, O. J. Braddick and A. C. Sleigh, eds. (Springer-Verlag, Berlin, 1983). [CrossRef]
- A. B. Watson and A. J. Ahumada, Jr., "Model of human visual-motion sensing," J. Opt. Soc. Am. A 2, 322-342 (1985). [CrossRef] [PubMed]
- E. H. Adelson and J. R. Bergen, "The Plenoptic Function and the Elements of Early Vision," in Computational Models of Visual Processing, M. Landy and J. A. Movshon, eds. (MIT Press, Cambridge, MA, 1991).
- C. Zetzsche and E. Barth, "Fundamental limits of linear filters in the visual processing of two-dimensional signals," Vision Research 30, 1111-1117 (1990). [CrossRef] [PubMed]
- C. Zetzsche, E. Barth, and B. Wegmann, "The importance of intrinsically two-dimensional image features in biological vision and picture coding," in Digital images and human vision, A. B. Watson, ed. (MIT Press, Cambridge, MA, 1993).
- R. M. Haralick, L. T. Watson, and T. J. Laffey, "The topographic primal sketch," International J. of Robotic Research 2, 50-72 (1983). [CrossRef]
- P. J. Besl and R. C. Jain, "Segmentation through variable-order surface fitting," IEEE Trans. Pattern Anal. Mach. Intell. 10, 167-192 (1988). [CrossRef]
- J. Shi and C. Tomasi, "Good features to track," Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 593-600 (1994).
- J. J. Koenderink and A. J. v. Doorn, "Representation of local geometry in the visual system," Biol. Cybern. 55, 367-375 (1987). [CrossRef] [PubMed]
- D. H. Hubel and T. N. Wiesel, "Receptive fields and functional architecture of monkey striate cortex," J. Physiol. 195, 215-243 (1968). [PubMed]
- J. Y. Lettvin, H. R. Maturana, W. S. McCulloch, and W. H. Pitts, "What the frog's eye tells the frog's brain," Proceedings IRE 47, 1940-1951 (1959). [CrossRef]
- G. A. Orban, Neuronal operations in the visual cortex, (Springer, Heidelberg, 1984). [CrossRef]
- E. Peterhans and R. von der Heydt, "Functional organization of area V2 in the alert macaque," European Journal of Neuroscience 5, 509-24 (1993). [CrossRef] [PubMed]
- J. B. Levitt, D. C. Kiper, and J. A. Movshon, "Receptive fields and functional architecture of macaque V2," J Neurophysiol 71, 2517-42 (1994). [PubMed]
- C. Yu and D. M. Levi, "End stopping and length tuning in psychophysical spatial filters," J. Opt. Soc. Am. A 14, 2346-54 (1997). [CrossRef]
- A. Dobbins, S. W. Zucker, and M. S. Cynader, "Endstopping and curvature," Vision Res 29, 1371-87 (1989). [CrossRef] [PubMed]
- F. Heitger, L. Rosenthaler, R. von der Heydt, E. Peterhans, and O. Kubler, "Simulation of neural contour mechanisms: from simple to end-stopped cells," Vision Res 32, 963-81 (1992). [CrossRef] [PubMed]
- H. R. Wilson and W. A. Richards, "Mechanisms of contour curvature discrimination," J. Opt. Soc. Am. A 6, 106- 115 (1989). [CrossRef] [PubMed]
- S. P. Liou and R. C. Jain, "Motion detection in spatio-temporal space," Computer Vision, Graphics, and Image Processing 45, 227-250 (1989). [CrossRef]
- C. Zetzsche and E. Barth, "Direct detection of flow discontinuities by 3D-curvature operators," Pattern Recognition Letters 12, 771--779 (1991). [CrossRef]
- C. Zetzsche, E. Barth, and J. Berkmann, "Spatio-temporal curvature measures for flow field analysis," Geometric Methods in Computer Vision, B. Vemuri Ed. SPIE 1590, 337--350 (1991).
- M. P. Do Carmo, Riemannian Geometry, (Birkh�user, Boston, 1992).
- S. Weinberg, Gravitation and Cosmology, (Wiley and Sons, New York, 1972).
- B. Schutz, A first course in general relativity, (Cambridge University Press, Cambridge, 1985).
- E. Barth, T. Caelli, and C. Zetzsche, "Image encoding, labelling and reconstruction from differential geometry," CVGIP:GRAPHICAL MODELS AND IMAGE PROCESSING 55, 428--446 (1993). [CrossRef]
- C. Mota and J. Gomes, "Curvature Operators in Geometric Image Processing," presented at Brasilian Symposium On Computer Graphics and Image Processing, (Campinas, Brazil, 1999).
- E. Barth, C. Zetzsche, and G. Krieger, "Curvature measures in visual information processing," Open Systems and Information Dynamics 5, 25-39 (1998). [CrossRef]
- E. Barth, Riemann-tensor motion analysis, (2000), http://www.visionscience.com/vsDemos.html .
- O. Tretiak and L. Pastor, "Velocity estimation from image sequences with second order differential operators," presented at Proc. 7th Int. Conf. Pattern Recognition, (Montreal, Canada, 1984).
- T. S. Huang and A. N. Netravali, "Motion and structure from feature correspondence: a review," Proceedings of the IEEE 82, 252-268 (1994). [CrossRef]
- E. Barth, "Spatio-temporal curvature and the visual coding of motion," in Neural Computation (NC'2000), vol. 1404-093, H. Bothe and R. Rojas, eds. (ICSC Academic Press, Berlin, 2000).
- H. Hau�ecker and H. Spies, "Motion," in Handbook of Computer Vision and Applications, B. Jahne, H. Hau�ecker, and P. Geissler, eds., 1999).
- S. J. Nowlan and T. J. Sejnowski, "A selection model for motion processing in area MT of primates," J Neurosci 15, 1195-214 (1995). [PubMed]
- S. Wuerger, R. Shapley, and N. Rubin, ""On the visually perceived direction of motion" by Hans Wallach: 60 years later," Perception 25, 1317-1367 (1996). [CrossRef]
- F. L. Kooi, "Local direction of edge motion causes and abolishes the barberpole illusion," Vision Res 33, 2347-51 (1993). [CrossRef] [PubMed]
- E. Barth, C. Zetzsche, and I. Rentschler, "Intrinsic two-dimensional features as textons," J. Opt. Soc. Am. A Opt Image Sci Vis 15, 1723-32 (1998). [CrossRef] [PubMed]
- T. D. Albright, "Direction and orientation selectivity of neurons in visual area MT of the macaque," J. Neurophysiol 52, 1106-30 (1984). [PubMed]
- G. H. Recanzone, R. H. Wurtz, and U. Schwarz, "Responses of MT and MST neurons to one and two moving objects in the receptive field," J Neurophysiol 78, 2904-15 (1997).
- E. P. Simoncelli and D. J. Heeger, "A model of neuronal responses in visual area MT," Vision Res 38, 743-61 (1998). [CrossRef] [PubMed]
- E. Barth and A. B. Watson, "Nonlinear spatio-temporal model based on the geometry of the visual input," Investigative Ophthalmology and Visual Science 39, S2110 (1998).
Cited By |
OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.
Multimedia
| Multimedia Files | Recommended Software |
| » Media 1: MOV (179 KB) | |
| » Media 2: MOV (168 KB) | |
| » Media 3: MOV (147 KB) | |
| » Media 4: MOV (144 KB) |





OSA is a member of 