In optical fiber transmission systems using inline amplifiers, the interaction of a signal and amplifier noise through the Kerr effect leads to nonlinear phase noise that can impair the detection of phase-modulated signals. We present analytical expressions for the maximum-likelihood (ML) decision boundaries and symbol-error rate (SER) for phase-shift keying and differential phase-shift keying systems with coherent and differentially coherent detection, respectively. The ML decision boundaries are in the form θ(r) = c₂r² + c₁r + c₀, where θ and r are the phase and the amplitude of the received signal, respectively. Using the expressions for the SER, we show that the impact of phase error from carrier synchronization is small, particularly for transoceanic links. For modulation formats such as 16-quadrature amplitude modulation, we propose various transmitter and receiver phase rotation strategies such that the ML detection is well approximated by using straight-line decision boundaries. The problem of signal constellation design for optimal SER performance is also studied for a system with four signal points.

© 2007 IEEE

You do not have subscription access to this journal. Citation lists with outbound citation links are available to subscribers only. You may subscribe either as an OSA member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Log in to access OSA Member Subscription

You do not have subscription access to this journal. Cited by links are available to subscribers only. You may subscribe either as an OSA member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Log in to access OSA Member Subscription