Bidirectionality, forward and backward information flow, is introduced in neural networks to produce two-way associative search for stored stimulus-response associations (A_i,B_i). Two fields of neurons, F_A and F_B, are connected by an n × p synaptic marix M. Passing information through M gives one direction, passing information through its transpose M^T gives the other. Every matrix is bidirectionally stable for bivalent and for continuous neurons. Paired data (A_i,B_i) are encoded in M by summing bipolar correlation matrices. The bidirectional associative memory (BAM) behaves as a two-layer hierarchy of symmetrically connected neurons. When the neurons in F_A and F_B are activated, the network quickly evolves to a stable state of twopattern reverberation, or pseudoadaptive resonance, for every connection topology M. The stable reverberation corresponds to a system energy local minimum. An adaptive BAM allows M to rapidly learn associations without supervision. Stable short-term memory reverberations across F_A and F_B gradually seep pattern information into the long-term memory connections M, allowing input associations (A_i,B_i) to dig their own energy wells in the network state space. The BAM correlation encoding scheme is extended to a general Hebbian learning law. Then every BAM adaptively resonates in the sense that all nodes and edges quickly equilibrate in a system energy local minimum. A sampling adaptive BAM results when many more training samples are presented than there are neurons in F_B and F_B, but presented for brief pulses of learning, not allowing learning to fully or nearly converge. Learning tends to improve with sample size. Sampling adaptive BAMs can learn some simple continuous mappings and can rapidly abstract bivalent associations from several noisy gray-scale samples.

© 1987 Optical Society of America

1. T. Kohonen, "Correlation Matrix Memories," IEEE Trans. Comput. C-21, 353 (1972).
2. T. Kohonen, Self-Organization and Associative Memory (Springer-Verlag, New York, 1984).
3. J. A. Anderson, J. W. Silverstein, S. A. Ritz, and R. S. Jones, "Distinctive Features, Categorical Perception, and Probability Learning: Some Applications of a Neural Model," Psychol. Rev. 84, 413 (1977).
4. G. A. Carpenter and S. Grossberg, "A Massively Parallel Architecture for a Self-Organizing Neural Pattern Recognition Machine," Comput. Vision Graphics Image Process. 37, 54 (1987).
5. S. Grossberg, "Adaptive Pattern Classification and Universal Recoding, II: Feedback, Expectation, Olfaction, and Illusions," Biol. Cybern. 23, 187 (1976).
6. S. Grossberg, "A Theory of Human Memory: Self-Organization and Performance of Sensory-Motor Codes, Maps, and Plans," Prog. Theor. Biol. 5, 000 (1978).
7. S. Grossberg, "How Does a Brain Build a Cognitive Code?," Psychol. Rev. 87, 1 (1980).
8. S. Grossberg, Studies of Mind and Brain: Neural Principles of Learning, Perception, Development, Cognition, and Motor Control (Reidel, Boston, 1982).
9. S. Grossberg, The Adaptive Brain, I and II (North-Holland, Amsterdam, 1987).
10. B. Kosko, "Bidirectional Associative Memories," IEEE Trans. Syst. Man Cybern. SMC-00, 000 (1987).
11. B. Kosko, "Fuzzy Associative Memories," in Fuzzy Expert Systems, A. Kandel, Ed. (Addison-Wesley, Reading, MA, 1987).
12. S. Grossberg, "Contour Enhancement, Short Term Memory, and Constancies in Reverberating Neural Networks," Stud. Appl. Math. 52, 217 (1973).
13. W. S. McCulloch and W. Pitts, "A Logical Calculus of the Ideas Immanent in Nervous Activity," Bull. Math. Biophys. 5, 115 (1943).
14. B. Kosko, "Fuzzy Entropy and Conditioning," Inf. Sci. 40, 165 (1986).
15. J. J. Hopfield, "Neural Networks and Physical Systems with Emergent Collective Computational Abilities," Proc. Natl. Acad. Sci. U.S.A. 79, 2554 (1982).
16. M. A. Cohen and S. Grossberg, "Absolute Stability of Global Pattern Formation and Parallel Memory Storage by Competitive Neural Networks," IEEE Trans. Syst. Man Cybern. SMC-13, 815 (1983).
17. D. B. Parker, "Learning Logic," Invention Report S81-64, File 1, Office of Technology Licensing, Stanford U. (Oct. 1982).
18. D. B. Parker, "Learning Logic," TR-47, Center for Computational Research in Economics and Management Science, MIT (Apr. 1985).
19. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning Internal Representations by Error Propagation," ICS Report 8506, Institute for Cognitive Science, U. California San Diego (Sept. 1985).
20. P. J. Werbos, "Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences," Ph.D. Dissertation in Statistics, Harvard U. (Aug. 1974).
21. B. Kosko and C. Guest, "Optical Bidirectional Associative Memories," Proc. Soc. Photo-Opt. Instrum. Eng. 758, (1987).
22. J. J. Hopfield, "Neurons with Graded Response Have Collective Computational Properties Like Those of Two-State Neurons," Proc. Natl. Acad. Sci. U.S.A. 81, 3088 (1984).
23. S. Grossberg, "Adaptive Pattern Classification and Universal Recoding, I: Parallel Development and Coding of Neural Feature Detectors," Biol. Cybern. 23, 121 (1976).
24. R. Hecht-Nielsen, "CounterPropagation Networks," in Proceedings, First International Conference on Neural Networks (IEEE, New York, 1987).

Biol. Cybern.

• S. Grossberg, "Adaptive Pattern Classification and Universal Recoding, II: Feedback, Expectation, Olfaction, and Illusions," Biol. Cybern. 23, 187 (1976).
• S. Grossberg, "Adaptive Pattern Classification and Universal Recoding, I: Parallel Development and Coding of Neural Feature Detectors," Biol. Cybern. 23, 121 (1976).

Bull. Math. Biophys.

• W. S. McCulloch and W. Pitts, "A Logical Calculus of the Ideas Immanent in Nervous Activity," Bull. Math. Biophys. 5, 115 (1943).

Comput. Vision Graphics Image Process.

• G. A. Carpenter and S. Grossberg, "A Massively Parallel Architecture for a Self-Organizing Neural Pattern Recognition Machine," Comput. Vision Graphics Image Process. 37, 54 (1987).

IEEE Trans. Comput.

• T. Kohonen, "Correlation Matrix Memories," IEEE Trans. Comput. C-21, 353 (1972).

IEEE Trans. Syst. Man Cybern.

• B. Kosko, "Bidirectional Associative Memories," IEEE Trans. Syst. Man Cybern. SMC-00, 000 (1987).
• M. A. Cohen and S. Grossberg, "Absolute Stability of Global Pattern Formation and Parallel Memory Storage by Competitive Neural Networks," IEEE Trans. Syst. Man Cybern. SMC-13, 815 (1983).

Inf. Sci.

• B. Kosko, "Fuzzy Entropy and Conditioning," Inf. Sci. 40, 165 (1986).

Proc. Natl. Acad. Sci. U.S.A.

• J. J. Hopfield, "Neural Networks and Physical Systems with Emergent Collective Computational Abilities," Proc. Natl. Acad. Sci. U.S.A. 79, 2554 (1982).
• J. J. Hopfield, "Neurons with Graded Response Have Collective Computational Properties Like Those of Two-State Neurons," Proc. Natl. Acad. Sci. U.S.A. 81, 3088 (1984).

Proc. Soc. Photo-Opt. Instrum. Eng.

• B. Kosko and C. Guest, "Optical Bidirectional Associative Memories," Proc. Soc. Photo-Opt. Instrum. Eng. 758, (1987).

Prog. Theor. Biol.

• S. Grossberg, "A Theory of Human Memory: Self-Organization and Performance of Sensory-Motor Codes, Maps, and Plans," Prog. Theor. Biol. 5, 000 (1978).

Psychol. Rev.

• S. Grossberg, "How Does a Brain Build a Cognitive Code?," Psychol. Rev. 87, 1 (1980).
• J. A. Anderson, J. W. Silverstein, S. A. Ritz, and R. S. Jones, "Distinctive Features, Categorical Perception, and Probability Learning: Some Applications of a Neural Model," Psychol. Rev. 84, 413 (1977).

Stud. Appl. Math.

• S. Grossberg, "Contour Enhancement, Short Term Memory, and Constancies in Reverberating Neural Networks," Stud. Appl. Math. 52, 217 (1973).

Other

• R. Hecht-Nielsen, "CounterPropagation Networks," in Proceedings, First International Conference on Neural Networks (IEEE, New York, 1987).
• T. Kohonen, Self-Organization and Associative Memory (Springer-Verlag, New York, 1984).
• S. Grossberg, Studies of Mind and Brain: Neural Principles of Learning, Perception, Development, Cognition, and Motor Control (Reidel, Boston, 1982).
• S. Grossberg, The Adaptive Brain, I and II (North-Holland, Amsterdam, 1987).
• B. Kosko, "Fuzzy Associative Memories," in Fuzzy Expert Systems, A. Kandel, Ed. (Addison-Wesley, Reading, MA, 1987).
• D. B. Parker, "Learning Logic," Invention Report S81-64, File 1, Office of Technology Licensing, Stanford U. (Oct. 1982).
• D. B. Parker, "Learning Logic," TR-47, Center for Computational Research in Economics and Management Science, MIT (Apr. 1985).
• D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning Internal Representations by Error Propagation," ICS Report 8506, Institute for Cognitive Science, U. California San Diego (Sept. 1985).
• P. J. Werbos, "Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences," Ph.D. Dissertation in Statistics, Harvard U. (Aug. 1974).

Click here to see a list of articles that cite this paper