## Artificial neural networks using complex numbers and phase encoded weights

Applied Optics, Vol. 49, Issue 10, pp. B71-B82 (2010)

http://dx.doi.org/10.1364/AO.49.000B71

Enhanced HTML Acrobat PDF (654 KB)

### Abstract

The model of a simple perceptron using phase-encoded inputs and complex-valued weights is proposed. The aggregation function, activation function, and learning rule for the proposed neuron are derived and applied to Boolean logic functions and simple computer vision tasks. The complex-valued neuron (CVN) is shown to be superior to traditional perceptrons. An improvement of 135% over the theoretical maximum of 104 linearly separable problems (of three variables) solvable by conventional perceptrons is achieved without additional logic, neuron stages, or higher order terms such as those required in polynomial logic gates. The application of CVN in distortion invariant character recognition and image segmentation is demonstrated. Implementation details are discussed, and the CVN is shown to be very attractive for optical implementation since optical computations are naturally complex. The cost of the CVN is less in all cases than the traditional neuron when implemented optically. Therefore, all the benefits of the CVN can be obtained without additional cost. However, on those implementations dependent on standard serial computers, CVN will be more cost effective only in those applications where its increased power can offset the requirement for additional neurons.

© 2010 Optical Society of America

**OCIS Codes**

(200.4260) Optics in computing : Neural networks

(100.4996) Image processing : Pattern recognition, neural networks

**History**

Original Manuscript: September 8, 2009

Revised Manuscript: December 31, 2009

Manuscript Accepted: January 29, 2010

Published: March 10, 2010

**Citation**

Howard E. Michel and Abdul Ahad S. Awwal, "Artificial neural networks using complex numbers and phase encoded weights," Appl. Opt. **49**, B71-B82 (2010)

http://www.opticsinfobase.org/ao/abstract.cfm?URI=ao-49-10-B71

Sort: Year | Journal | Reset

### References

- J. J. Hopfield, “Pattern recognition computation using action potential timing for stimulus representation,” Nature 376, 33-36 (1995).
- W. Maass and C. M. Bishop, Pulsed Neural Networks (MIT Press, 1998).
- F. Theunissen and J. P. Miller, “Temporal encoding in nervous systems,” J. Comput. Neurosci. 2, 149-162 (1995). [CrossRef]
- S. Thorpe, D. Fize, and C. Marlot, “Speed of processing in the human visual system,” Nature 381, 520-522 (1996).
- J. J. Hopfield and A. V. M. Herz, “Rapid local synchronization of action potentials: toward computation with coupled integrate-and-fire networks,” Proc. Natl. Acad. Sci. U.S.A. 92, 6655-6662 (1995).
- W. S. McCulloch and W. Pitts, “A logical calculus of ideas immanent in nervous activity,” Bull. Mathematical Biophys. 5, 115-133 (1943).
- H. E. Michel, A. A. S. Awwal, and D. Rancour, “Artificial neural networks using complex numbers and phase encoded weights--electronic and optical implementations,” in 2006 International Joint Conference on Neural Networks (2006), pp. 1213-1218.
- H. E. Michel, D. Rancour, and S. Iringentavida, “CMOS Implementation of phase-encoded complex-valued artificial neural networks,” in Proceedings of the International Conference on VLSI, VLSI'04 (2004), pp. 551-557.
- X. Tao and H. E. Michel, “Data clustering via spiking neural networks through spike timing-dependent plasticity,” in Proceedings of the International Conference on Artificial Intelligence, IC-AI'04, Vol. 1 (2004), pp. 168-173.
- N. N. Aizenberg, Yu L. Ivas'kiv, D. A. Pospelov, and G. F. Khudyakov, “Multivalued threshold functions in Boolean complex-threshold functions and their generalization,” Cybern. Syst. Anal. 7, 626-635 (1971).
- I. N. Aizenberg, “A universal logic element over the complex field,” Cybern. Syst. Anal. 27, 467-473 (1991). [CrossRef]
- N. N. Aizenberg and I. N. Aizenberg, “Universal binary and multivalued paradigm: conception, learning, applications,” Lect. Notes Comput. Sci. 1240, 463-472 (1997). [CrossRef]
- I. Aizenberg, D. Paliy, and J. Astola, “Multilayer neural network based on multi-valued neurons and the blur identification problem,” in Proceedings of the 2006 IEEE Joint Conference on Neural Networks (2006), pp. 1200-1207.
- I. Aizenberg and C. Moraga, “Multilayer feedforward neural network based on multi-valued neurons and a backpropagation learning algorithm,” Soft Comput. 11, 169-183(2007).
- I. Aizenberg, “Solving the xor and parity N problems using a single universal binary neuron, Soft Computing 12, 215-222(2008).
- I. Aizenberg and C. Butakoff, “Image processing using cellular neural networks based on multi-valued and universal binary neurons,” J. VLSI Signal Process. 32, 169-188 (2002).
- I. Aizenberg, D. Paliy, J. Zurada, and J. Astola, “Blur identification by multilayer neural network based on multi-valued neurons,” IEEE Trans. Neural Networks 19, 883-898(2008). [CrossRef]
- T. Nitta, “Solving the xor problem and the detection of symmetry using a single complex-valued neuron,” Neural Networks 16, 1101-1105 (2003). [CrossRef]
- T. Nitta, “An extension of the back-propagation algorithm to complex numbers,” Neural Networks 10, 1391-1415 (1997). [CrossRef]
- N. Benvenuto and F. Piazza, “On the complex backpropagation algorithm,” IEEE Trans. Signal Process. 40, 967-969(1992). [CrossRef]
- H. Leung and S. Haykin, “The complex backpropagation algorithm,” IEEE Trans. Signal Process. 39, 2101-2104 (1991). [CrossRef]
- G. M. Georgiou and C. Koutsougeras, “Complex domain backpropagation,” IEEE Trans. Circuits Syst. 2 Analog Digit. Signal Process. 39, 330-334 (1992).
- M. R. Smith and Y. Hui, “A data extrapolation algorithm using a complex domain neural network,” IEEE Trans. Circuits Syst. 2 Analog Digit. Signal Process. 44, 143-147 (1997).
- P. Arena, G. Fortuna, G. Muscato, and M. G. Xibilia, “Multilayer perceptrons to approximate quaternion valued functions,” Neural Networks 10, 335-342 (1997). [CrossRef]
- A. Hirose, “Dynamics of fully complex-valued neural networks,” Electron. Lett. 28, 1492-1494 (1992). [CrossRef]
- D. Casasent and S. Natarajan, “A classifier neural network with complex-valued weights and square-law nonlinearities,” Neural Networks 8, 989-998 (1995). [CrossRef]
- D. M. Weber and D. P. Casasent, “The extended piecewise quadratic neural network,” Neural Networks 11, 837-850(1998). [CrossRef]
- A. Hirose, “Applications of complex-valued neural networks to coherent optical computing using phase-sensitive detection scheme,” Information Sci. Appl. 2, 103-117 (1994).
- J. I. Khan, “Characteristics of multidimensional holographic associative memory in retrieval with dynamic localizable attention,” IEEE Trans.Neural Networks 9, 389-406(1998). [CrossRef]
- J. I. Khan and D. Y. Yun, “A parallel, distributed and associative approach for pattern matching with holographic dynamics,” J. Visual Languages Comput. 8, 303-331(1997). [CrossRef]
- A. A. S. Awwal and G. Power, “Object tracking by an opto-electronic inner product complex neural network,” Opt. Eng. 32, 2782-2787 (1993).
- B. Igelnik, M. Tabib-Azar, and S. LeClair, “A net with complex weights,” IEEE Trans. Neural Networks 12, 236-249(2001). [CrossRef]
- E. J. Bayro-Corrochano, “Geometric neural computing,” IEEE Trans. Neural Networks 12, 968-986 (2001). [CrossRef]
- H. E. Michel and A. A. S. Awwal, “Enhanced Artificial Neural Networks Using Complex-Numbers,” in IJCNN'99 International Joint Conference on Neural Networks, Vol. 1 (1999), pp. 456-461.
- H. E. Michel, “Enhanced artificial neural network using complex numbers,” Ph.D. dissertation (Wright State University, 1999).
- H. E. Michel and A. A. S. Awwal, “How to train a phase only filter,” Proc. SPIE 4046, 24-33 (2000). [CrossRef]
- A. A. S. Awwal and H. E. Michel, “Enhancing the discrimination capability of phase only filter,” Asian J. Phys. 8, 381-383 (2000).
- H. E. Michel and S. Kunjithapatham, “Processing Landsat TM data using complex-valued neural networks,” Proc. SPIE 4730, 43-53 (2002). [CrossRef]
- X. Tao and H. E. Michel, “Processing Landsat TM data using complex-valued NRBF neural network,” in Proceedings of the International Joint Conference on Neural Networks (2005), pp. 3081-8086.
- X. Tao and H. E. Michel, “Novel artificial neural networks for remote-sensing data classification,” Proc. SPIE 5781, 127-138(2005). [CrossRef]
- X. Tao and H. E. Michel, “Classification of multi-spectral satellite image data using improved NRBF neural networks,” Proc. SPIE 5267, 311-320 (2003). [CrossRef]
- V. Bagnoud and J. D. Zuegel, “Independent phase and amplitude control of a laser beam by use of a single-phase-only spatial light modulator,” Opt. Lett. 29, 295-297 (2004). [CrossRef]
- K. Stubkjaer, “Semiconductor optical amplifier-based all optical gates for high-speed optical processing,” IEEE J. Sel. Top. Quantum Electron. 6, 1428-1435 (2000). [CrossRef]
- J. DongX. Zhang, J. Xu, and D. Huang, “40 Gb/s all-optical logic nor and or gates using a semiconductor optical amplifier: experimental demonstration and theoretical analysis,” Opt. Commun. 281, 1710-1715 (2008). [CrossRef]
- J. Kim, J. Kang, T. Kim, and S. Han, “All-optical multiple logic gates with xor, nor, or, and nand functions using parallel SOA-MZI structures: theory and experiment,” J. Lightwave Technol. 24, 3392-3399 (2006). [CrossRef]
- H. Dong, H. Sun, Q. Wang, N. Dutta, and J. Jaques, “80 Gb/s all-optical logic and operation using Mach-Zehnder interferometer with differential scheme,” Opt. Commun. 265, 79-93 (2006). [CrossRef]

## Cited By |
Alert me when this paper is cited |

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

« Previous Article | Next Article »

OSA is a member of CrossRef.