OSA's Digital Library

Applied Optics

Applied Optics

APPLICATIONS-CENTERED RESEARCH IN OPTICS

  • Vol. 35, Iss. 8 — Mar. 10, 1996
  • pp: 1328–1343

Convergence of backward-error-propagation learning in photorefractive crystals

Gregory C. Petrisor, Adam A. Goldstein, B. Keith Jenkins, Edward J. Herbulock, and Armand R. Tanguay, Jr  »View Author Affiliations


Applied Optics, Vol. 35, Issue 8, pp. 1328-1343 (1996)
http://dx.doi.org/10.1364/AO.35.001328


View Full Text Article

Enhanced HTML    Acrobat PDF (462 KB)





Browse Journals / Lookup Meetings

Browse by Journal and Year


   


Lookup Conference Papers

Close Browse Journals / Lookup Meetings

Article Tools

Share
Citations

Abstract

We analytically determine that the backward-error-propagation learning algorithm has a well-defined region of convergence in neural learning-parameter space for two classes of photorefractive-based optical neural-network architectures. The first class uses electric-field amplitude encoding of signals and weights in a fully coherent system, whereas the second class uses intensity encoding of signals and weights in an incoherent/coherent system. Under typical assumptions on the grating formation in photorefractive materials used in adaptive optical interconnections, we compute weight updates for both classes of architectures. Using these weight updates, we derive a set of conditions that are sufficient for such a network to operate within the region of convergence. The results are verified empirically by simulations of the XOR sample problem. The computed weight updates for both classes of architectures contain two neural learning parameters: a learning-rate coefficient and a weight-decay coefficient. We show that these learning parameters are directly related to two important design parameters: system gain and exposure energy. The system gain determines the ratio of the learning-rate parameter to decay-rate parameter, and the exposure energy determines the size of the decay-rate parameter. We conclude that convergence is guaranteed (assuming no spurious local minima in the error function) by using a sufficiently high gain and a sufficiently low exposure energy per weight update.

© 1996 Optical Society of America

History
Original Manuscript: June 23, 1995
Revised Manuscript: October 30, 1995
Published: March 10, 1996

Citation
Gregory C. Petrisor, Adam A. Goldstein, B. Keith Jenkins, Edward J. Herbulock, and Armand R. Tanguay, "Convergence of backward-error-propagation learning in photorefractive crystals," Appl. Opt. 35, 1328-1343 (1996)
http://www.opticsinfobase.org/ao/abstract.cfm?URI=ao-35-8-1328


Sort:  Author  |  Year  |  Journal  |  Reset  

References

  1. P. J. van Heerden, “Theory of optical information storage in solids,” Appl. Opt. 2, 393–400 (1963).
  2. F. H. Mok, “Angle-multiplexed storage of 5000 holograms in lithium niobate,” Opt. Lett. 18, 915–917 (1993).
  3. K. Wagner, D. Psaltis, “Multilayer optical learning networks,” Appl. Opt. 26, 5061–5076 (1987).
  4. D. Psaltis, D. Brady, K. Wagner, “Adaptive optical networks using photorefractive crystals,” Appl. Opt. 27,1752–1759 (1988).
  5. P. Yeh, A. E. T. Chiou, J. Hong, “Optical interconnection using photorefractive dynamic holograms,” Appl. Opt. 27, 2093–2095 (1988).
  6. H. Yoshinaga, K. Kitayama, T. Hara, “All-optical error-signal generation for backpropagation learning in optical multilayer neural networks,” Opt. Lett. 14, 262–264 (1989).
  7. E. G. Paek, J. R. Wullert, J. S. Patel, “Holographic implementation of a learning machine based on a multicat-egory perceptron algorithm,” Opt. Lett. 14, 1303–1305 (1989).
  8. D. Psaltis, D. Brady, X.-G. Gu, S. Lin, “Holography inartificial neural networks,” Nature (London) 343, 325–343 (1990).
  9. A. Marrakchi, W. M. Hubbard, S. F. Habiby, J. S. Patel, “Dynamic holographic interconnects with analog weights in photorefractive crystals,” Opt. Eng. 29, 215–224 (1990).
  10. J. H. Hong, S. Campbell, P. Yeh, “Optical pattern classifier with Perceptron learning,” Appl. Opt. 29, 3019–3025 (1990).
  11. C. Peterson, S. Redfield, J. D. Keeler, E. Hartman, “Optoelectronic implementation of multilayer neural net-works in a single photorefractive crystal,” Opt. Eng. 29, 359–368 (1990).
  12. Y. Owechko, B. H. Soffer, “Optical interconnection method for neural networks using self-pumped phase-conjugate mir-rors,” Opt. Lett. 16, 675–677 (1991).
  13. G. J. Dunning, Y. Owechko, B. H. Soffer, “Hybrid optoelectronic neural networks using a mutually pumped phase-conjugate mirror,” Opt. Lett. 16, 928–930 (1991).
  14. Y. Qiao, D. Psaltis, “Learning algorithms for optical multilayer neural networks,” in Proceedings of the Third International Joint Conference on Neural Networks (IEEE, New York, 1991), Vol. 1, pp. 457–462.
  15. Y. Qiao, D. Psaltis, “Local learning algorithm for optical neural networks,” Appl. Opt. 31, 3285–3288 (1992).
  16. B. K. Jenkins, A. R. Tanguay, Photonic implementations of neural networks, in Neural Networks for Signal Processing, Bart Kosko, ed. (Prentice-Hall, Englewood Cliffs, N.J., 1992), pp. 287–379.
  17. Y. Owechko, “Cascaded-grating holography for artificial neural networks,” Appl. Opt. 32, 1380–1398 (1993).
  18. J. Hong, “Applications of photorefractive crystals for optical neural networks,” Opt. Quantum Electron. 25, 551–568 (1993).
  19. K. Y. Hsu, S. H. Lin, P. Yeh, “Conditional convergence of photorefractive perceptron learning,” Opt. Lett. 18, 2135–2137 (1993).
  20. K. Wagner, T. M. Slagle, “Optical competitive learning with VLSI/liquid-crystal winner-take-all modulators,” Appl. Opt. 32, 1408–1435 (1993).
  21. C.J. Cheng, P. C. Yeh, K. Y. Hsu, “Generalized perceptron learning rule and its implications for photorefractive neural networks,” J. Opt. Soc. Am. B 11, 1619–1624 (1994).
  22. T. Galstyan, G. Pauliat, A. Villing, G. Roosen, “Adaptive photorefractive neurons for self-organizing networks,” Opt. Commun. 109, 35–42 (1994).
  23. Y. Qiao, D. Psaltis, C. Gu, J. Hong, P. Yeh, R. R. Neurgaonkar, “Phase-locked sustainment of photorefractive holograms using phase conjugation,” J. Appl. Phys. 70, 4646–4648 (1991).
  24. A. Goldstein, G. C. Petrisor, B. K. Jenkins, “Gain and exposure schedule to compensate for photorefractive neural-network weight decay,” Opt. Lett. 20, 611–613 (1995).
  25. D. C. Plaut, S. J. Nowlan, G. E. Hinton, “Experiments on learning by back propagation,” Tech. Rep. CMU-CS-86-126 (Department of Computer Science, Carnegie-Mellon, Pittsburgh, Pa., 1986), p.8.
  26. P. Asthana, G. P. Nordin, A. R. Tanguay, B. K. Jenkins, “Analysis of weighted fan-out fan-in volume holographic optical interconnections,” Appl. Opt. 32, 1441–1469 (1993).
  27. N. V. Kukhtarev, “Kinetics of hologram recording and erasurein electrooptic crystals,” Sov. Tech. Phys. Lett. 2, 438–448 (1976).
  28. D. Brady, D. Psaltis, “Information capacity of 3-D holo-graphic data storage,” Opt. Quantum Electron. 25, 597–610 (1993).
  29. D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learninginternal representations by error propagation,” in Parallel Dìstrìbuted Processing, D. Rumelhart, J. McClelland, eds. (MIT, Cambridge, Mass., 1986), Vol. 1, Chap. 8, pp. 318–328.
  30. H. Lee, X. G. Gu, D. Psaltis, “Volume holographic interconnections with maximal capacity and minimal crosstalk,” J. Appl. Phys. 65, 2191–2195 (1989).
  31. D. Brady, D. Psaltis, “Control of volume holograms,” J. Opt. Soc. Am. A 9, 1167–1182 (1992).
  32. C. Slinger, “Analysis ofthe N-to-N volume-holographic neural interconnect,” J. Opt. Soc. Am. A 8, 1074–1081 (1991).
  33. B. K. Jenkins, G. C. Petrisor, S. Piazzolla, P. Asthana, A. R. Tanguay, “Photonic architecture for neural nets using incoherent coherent holographic interconnections,” in Optical Computing ‘90, J. Tsujinunchi, Y. Ichioka, S. Ishihara, eds., Proc. Soc. Photo-Opt. Instrum. Eng.1359, 317–318 (1990).
  34. G. P. Nordin, “Analysis of volume diffraction phenomena forphotonic neural network implementations and stratified volume holographic optical elements,” Ph.D. dissertationUniversity of Southern California, Los Angeles, Calif., 1992), pp. 99–130.
  35. G. C. Petrisor, S. Piazzolla, G. P. Nordin, B. K. Jenkins, A. R. Tanguay, “Volume holographic interconnections and copying architectures based on incoherent coherent source arrays,” in Fourth International Conference on Holographs Systems, Components, and Apptications, IEEE Conf. Pub. (Institute of Electrical Engineering, London, September1993), pp. 21–26.
  36. D. Psaltis, X.-G. Gu, D. Brady, “Fractal sampling grids for holographic interconnections,” in Optical Computing ‘88, P. Chavel, J. W. Goodman, G. Roblin, eds., Proc. Soc. Photo-Opt. Instrum. Eng.963, 468–474 (1988).

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.


« Previous Article  |  Next Article »

OSA is a member of CrossRef.

CrossCheck Deposited