## Maximum likelihood sequence detection in laser phase noise-impaired coherent optical systems |

Optics Express, Vol. 19, Issue 23, pp. 22600-22606 (2011)

http://dx.doi.org/10.1364/OE.19.022600

Acrobat PDF (1029 KB)

### Abstract

A maximum likelihood sequence detection (MLSD) receiver is used to detect data sequences in single-carrier coherent optical systems in the presence of laser phase noise. It requires no explicit phase estimation and involves only linear operations. It consistently shows improvement in the OSNR penalty (e.g., 1.1 dB at BER = 10^{−4} with memory length *L* =3) and the laser linewidth tolerance (e.g., around 4 times that of DAML at 1dB OSNR penalty at BER = 10^{−4} with memory length *L* =3) over the well-known DAML and *M*th power approaches in laser phase noise (LPN)-impaired coherent optical systems.

© 2011 OSA

## 1. Introduction

1. E. Ip, A. P. Lau, D. J. F. Barros, and J. M. Kahn, “Coherent detection in optical fiber systems,” Opt. Express **16**(2), 753–791 (2008). [CrossRef] [PubMed]

_{2}(

*M*) when advanced modulation formats such as

*M*-ary PSK (MPSK) is used. Nowadays, coherent receivers employing high-speed analog-to-digital converters (ADCs) and high-speed baseband digital signal processing (DSP) have become increasingly attractive, because they recover the carrier phase digitally and thus avoid the use of expensive optical phase lock loops. To allow a free-running local oscillator (LO) laser, it is crucial to recover the carrier phase in the presence of laser phase noise (LPN) due to the mismatch between the transmitter laser and the laser at the LO, which keeps rotating the phase of the received signals and causes a power penalty to the receiver sensitivity.

2. Y. Mori, C. Zhang, K. Igarashi, K. Katoh, and K. Kikuchi, “Unrepeated 200-km transmission of 40-Gbit/s 16-QAM signals using digital coherent receiver,” Opt. Express **17**(3), 1435–1441 (2009). [CrossRef] [PubMed]

3. E. Ip and J. M. Kahn, “Feedforward carrier recovery for coherent optical communications,” J. Lightwave Technol. **25**(9), 2675–2692 (2007). [CrossRef]

*M*th-power [5

5. D.-S. Ly-Gagnon, S. Tsukamoto, K. Katoh, and K. Kikuchi, “Coherent detection of optical quadrature phase-shift keying signals with carrier phase estimation,” J. Lightwave Technol. **24**(1), 12–21 (2006). [CrossRef]

6. S. Zhang, P. Y. Kam, J. Chen, and C. Yu, “Decision-aided maximum likelihood detection in coherent optical phase-shift-keying system,” Opt. Express **17**(2), 703–715 (2009). [CrossRef] [PubMed]

7. S. Zhang, P. Y. Kam, C. Yu, and J. Chen, “Decision-aided carrier phase estimation for coherent optical communications,” J. Lightwave Technol. **28**(11), 1597–1607 (2010). [CrossRef]

6. S. Zhang, P. Y. Kam, J. Chen, and C. Yu, “Decision-aided maximum likelihood detection in coherent optical phase-shift-keying system,” Opt. Express **17**(2), 703–715 (2009). [CrossRef] [PubMed]

*M*th power scheme’s heavy reliance on nonlinear operations, DAML is totally linear and free from phase-unwrapping [7

7. S. Zhang, P. Y. Kam, C. Yu, and J. Chen, “Decision-aided carrier phase estimation for coherent optical communications,” J. Lightwave Technol. **28**(11), 1597–1607 (2010). [CrossRef]

9. P. Y. Kam, “Maximum likelihood carrier phase recovery for linear suppressed-carrier digital data modulations,” IEEE Trans. Commun. **34**(6), 522–527 (1986). [CrossRef]

10. P. Y. Kam, “Maximum-likelihood digital data sequence estimation over the Gaussian channel with unknown carrier phase,” IEEE Trans. Commun. **35**(7), 764–767 (1987). [CrossRef]

11. P. Y. Kam and P. Sinha, “A Viterbi-type algorithm for efficient estimation of M-PSK sequences over the Gaussian channel with unknown carrier phase,” IEEE Trans. Commun. **43**(9), 2429–2433 (1995). [CrossRef]

11. P. Y. Kam and P. Sinha, “A Viterbi-type algorithm for efficient estimation of M-PSK sequences over the Gaussian channel with unknown carrier phase,” IEEE Trans. Commun. **43**(9), 2429–2433 (1995). [CrossRef]

*M*th power and the DAML in the LPN-impaired coherent optical system. This MLSD receiver can be extended to the coded case, where good performance in the presence of LPN can also be obtained.

## 2. System setup

*M*th power and the DAML approaches, the signal is assumed to be differentially pre-coded at the transmitter to avoid ‘runaway’ or error propagation due to cycle slips in the phase tracking at the receiver [5

5. D.-S. Ly-Gagnon, S. Tsukamoto, K. Katoh, and K. Kikuchi, “Coherent detection of optical quadrature phase-shift keying signals with carrier phase estimation,” J. Lightwave Technol. **24**(1), 12–21 (2006). [CrossRef]

7. S. Zhang, P. Y. Kam, C. Yu, and J. Chen, “Decision-aided carrier phase estimation for coherent optical communications,” J. Lightwave Technol. **28**(11), 1597–1607 (2010). [CrossRef]

5. D.-S. Ly-Gagnon, S. Tsukamoto, K. Katoh, and K. Kikuchi, “Coherent detection of optical quadrature phase-shift keying signals with carrier phase estimation,” J. Lightwave Technol. **24**(1), 12–21 (2006). [CrossRef]

12. S. Zhang, X. Li, P. Y. Kam, C. Yu, and J. Chen, “Pilot-assisted decision-aided maximum-likelihood phase estimation in coherent optical phase-modulated systems with nonlinear phase noise,” IEEE Photon. Technol. Lett. **22**(6), 380–382 (2010). [CrossRef]

*E*(

_{r}*t*) beats with the optical field generated by the LO

*E*(

_{LO}*t*) in a standard 2x4 optical hybrid, which has four 3-dB couplers and an additional 90°-phase shifter [13

13. M. Seimetz and C.-M. Weinert, “Options, feasibility, and availability of 2 x4 90° hybrids for coherent optical systems,” J. Lightwave Technol. **24**(3), 1317–1322 (2006). [CrossRef]

*i*(

_{I}*t*)) and quadrature-phase (

*i*(

_{Q}*t*)) signals. These signals are sampled by using high-speed ADCs and then processed by the MLSD receiver for laser phase noise-aware sequence detection. Note that the MLSD receiver is optimum from the standpoint of eliminating laser phase noise, which is the dominant optical impairment in back-to-back coherent optical systems. Given our experimental setup as shown in Fig. 1 and our assumption that all the other impairments such as intersymbol interference due to chromatic dispersion, polarization mode dispersion and attenuation can be handled by using optical means and polarization is fully matched by using a polarization controller, the design is also globally optimum since the DSP unit now only needs to combat the laser phase noise.

## 3. The MLSD receiver

*K*symbols, where

*K*>> 1. Laser phase noise (LPN) is modeled as a Wiener process

*k*th symbol interval [

*kT,*(

*k +*1)

*T*),

*T*is the symbol duration, and {

14. K. Kikuchi, T. Okoshi, M. Nagamatsu, and N. Henmi, “Degradation of bit-error rate in coherent optical communications due to spectral spread of the transmitter and the local oscillator,” J. Lightwave Technol. **2**(6), 1024–1033 (1984). [CrossRef]

*k*th symbol interval can be represented aswhere

*k*th symbol interval, and {

*n*(

*k*)} is the sequence of i.i.d complex Gaussian variables due to the shot noise [6

6. S. Zhang, P. Y. Kam, J. Chen, and C. Yu, “Decision-aided maximum likelihood detection in coherent optical phase-shift-keying system,” Opt. Express **17**(2), 703–715 (2009). [CrossRef] [PubMed]

*E*is the symbol energy. The sequence

_{s}**s**is equivalent to a path through a trellis, whether it is coded or uncoded [11

11. P. Y. Kam and P. Sinha, “A Viterbi-type algorithm for efficient estimation of M-PSK sequences over the Gaussian channel with unknown carrier phase,” IEEE Trans. Commun. **43**(9), 2429–2433 (1995). [CrossRef]

*t*=

*kT*correspond to the possible values assumed by signal phase

**17**(2), 703–715 (2009). [CrossRef] [PubMed]

*L*symbol intervals, where

*t = kT*, when the receiver has to decide on the survivor at each state of the trellis, it chooses the path with the maximum metric

**28**(11), 1597–1607 (2010). [CrossRef]

## 4. Performance evaluation

*M*th power. First, as we can see from Eq. (2), MLSD requires no explicit phase estimation and thus has no associated nonlinear argument operation or phase unwrapping as required in the

*M*th power scheme [5

**24**(1), 12–21 (2006). [CrossRef]

*M*−1) comparisons are performed at each state to choose one survivor form the

*M*paths that enter that state. Therefore, a total of (

*M*−1)

*M*comparisons are performed at any time with

*M*states. Finally, DAML and

*M*th power are both SBS detection approaches, hence the decision of a symbol occurs right after the receipt of the symbol. However, our MLSD is a sequence detection receiver that implements trellis search, where the decision on a data symbol is made only when the tails of all the survivors merge. Hence, the detection delay of the symbols is random. In order to resolve this issue, here we divide a long sequence of

*K*symbols into a train of subsequences each of length

*D*(usually

*D*<<

*K*), where a termination symbol corresponding to a known modulation phase of zero radian is inserted at the end of each subsequence to flush the trellis back to state ‘0’. As a result, decisions on all data symbols prior to that termination symbol can be made at the latest upon the receipt of the termination symbol. Thus, the average detection delay per symbol,

*t*, is not longer than (

_{d}*D*+1)/2.

*K*= 10

^{7}symbols is simulated. Furthermore, a preamble of length equal to the memory length

*L*for the initial computation of the metric

*L*is usually much shorter than the subsequence length

*D*and only used once for transmitting the entire long sequence of

*K*symbols, the overhead is negligible. Note that SNR/bit denotes signal-to-noise ratio per bit, which is obtained by dividing the SNR of the received symbols by the number of bits per symbol, i.e., 2 in this case.

*L*= 3. For the MLSD,

*D*= 100 is used here. Figure 3(a) shows the actual received digitized signal before the high-speed DSP units for signal detection, where the laser phase noise (LPN) keeps rotating the phase of the carrier signal. Figure 3(b) shows the received signal if LPN is absent. From Fig. 3(c) and 3(d), we find that compared to DAML, the majority of the received signals after MLSD are spaced out more obviously. Hence the decision errors after MLSD are mainly due to the signals that are very near to the decision boundaries.

*L*= 1, and Fig. 4(b) for

*L*= 3. As expected, DAML and

*M*th power demonstrate similar BER performances for both cases. On the other hand, MLSD shows consistent improvements, in terms of lower BER and better receiver sensitivity, over these two SBS receivers for both cases throughout the entire SNR region. For example, at SNR/bit = 11dB, when

*L*= 3, the BER obtained is 4x10

^{−5}for MLSD, and 2x10

^{−4}for both DAML and

*M*th power. Furthermore, the improvement of receiver sensitivity is more obvious at lower BER. For example, when

*L*= 3 and BER = 10

^{−4}, MLSD has an SNR/bit improvement of 1.1 dB over both DAML and

*M*th power. For the same memory length, when BER is reduced to 10

^{−5}, the improvement increases further to 1.4dB.

*D*. Here LLW = 10MHz and three different subsequence lengths are investigated. The results are compared with the DAML results only. We observe that the BER performances are very similar for different subsequence lengths. Similar findings are observed with different memory lengths (

*L*= 3 and

*L*= 5). However, in the case of

*L*= 1, a shorter subsequence length achieves lower BER under the same SNR/bit. Note that a memory space is required at the MLSD receiver to hold a copy of the unmerged paths of the survivor sequences before a firm decision is reached, and in the worst case, the mergers only occur through the enforcement of the termination symbol. Hence, the worst case memory required at the receiver will be proportional to

*D*. Thus, a shorter subsequence length means smaller receiver buffer sizes and shorter detection delays. For all cases, apparent improvements over the DAML are observed, e.g., about 0.5 dB improvements of SNR/bit at BER = 10

^{−4}are shown for both

*L*= 3 and

*L*= 5.

^{−4}for MLSD, DAML and

*M*th power as compared to ideal coherent detection. It shows that MLSD is able to achieve a lower OSNR penalty or higher laser linewidth (LLW) tolerance than both DAML and

*M*th Power. For example, for 1dB OSNR penalty, when

*L*= 3, MLSD is able to achieve a LLW tolerance (9MHz, in this case) around 4 times that of DAML (2.3MHz, in this case), where

*M*th Power cannot achieve 1dB penalty for all laser linewidths.

## 5. Conclusions

*M*th power schemes in terms of lower BER at given receiver SNR and higher laser linewidth tolerance for a given OSNR penalty. More importantly, the larger the LPN is, the shorter the required memory length

*L*, and the more obvious the improvement of our MLSD over the DAML and the

*M*th power schemes. This lowers the receiver complexity, leading to less stringent requirements and lower costs for the transmitter and LO lasers.

## Acknowledgments

## References and links

1. | E. Ip, A. P. Lau, D. J. F. Barros, and J. M. Kahn, “Coherent detection in optical fiber systems,” Opt. Express |

2. | Y. Mori, C. Zhang, K. Igarashi, K. Katoh, and K. Kikuchi, “Unrepeated 200-km transmission of 40-Gbit/s 16-QAM signals using digital coherent receiver,” Opt. Express |

3. | E. Ip and J. M. Kahn, “Feedforward carrier recovery for coherent optical communications,” J. Lightwave Technol. |

4. | L. M. Pessoa, H. M. Salgado, and I. Darwazeh, “Joint mitigation of optical impairments and phase estimation in coherent optical systems,” |

5. | D.-S. Ly-Gagnon, S. Tsukamoto, K. Katoh, and K. Kikuchi, “Coherent detection of optical quadrature phase-shift keying signals with carrier phase estimation,” J. Lightwave Technol. |

6. | S. Zhang, P. Y. Kam, J. Chen, and C. Yu, “Decision-aided maximum likelihood detection in coherent optical phase-shift-keying system,” Opt. Express |

7. | S. Zhang, P. Y. Kam, C. Yu, and J. Chen, “Decision-aided carrier phase estimation for coherent optical communications,” J. Lightwave Technol. |

8. | S. Zhang, P. Y. Kam, and C. Yu, “Block length effect of decision-aided maximum likelihood phase estimation in coherent optical communication systems,” |

9. | P. Y. Kam, “Maximum likelihood carrier phase recovery for linear suppressed-carrier digital data modulations,” IEEE Trans. Commun. |

10. | P. Y. Kam, “Maximum-likelihood digital data sequence estimation over the Gaussian channel with unknown carrier phase,” IEEE Trans. Commun. |

11. | P. Y. Kam and P. Sinha, “A Viterbi-type algorithm for efficient estimation of M-PSK sequences over the Gaussian channel with unknown carrier phase,” IEEE Trans. Commun. |

12. | S. Zhang, X. Li, P. Y. Kam, C. Yu, and J. Chen, “Pilot-assisted decision-aided maximum-likelihood phase estimation in coherent optical phase-modulated systems with nonlinear phase noise,” IEEE Photon. Technol. Lett. |

13. | M. Seimetz and C.-M. Weinert, “Options, feasibility, and availability of 2 x4 90° hybrids for coherent optical systems,” J. Lightwave Technol. |

14. | K. Kikuchi, T. Okoshi, M. Nagamatsu, and N. Henmi, “Degradation of bit-error rate in coherent optical communications due to spectral spread of the transmitter and the local oscillator,” J. Lightwave Technol. |

15. | Y. Li, P. Y. Kam, and C. C. Chui, “Adaptive sequence detection for MPSK/MQAM with unknown carrier phase characteristics,” |

**OCIS Codes**

(060.1660) Fiber optics and optical communications : Coherent communications

(060.4510) Fiber optics and optical communications : Optical communications

(060.5060) Fiber optics and optical communications : Phase modulation

**ToC Category:**

Fiber Optics and Optical Communications

**History**

Original Manuscript: July 20, 2011

Revised Manuscript: September 11, 2011

Manuscript Accepted: September 25, 2011

Published: October 25, 2011

**Citation**

Xuguang Shao, Pooi-Yuen Kam, and Changyuan Yu, "Maximum likelihood sequence detection in laser phase noise-impaired coherent optical systems," Opt. Express **19**, 22600-22606 (2011)

http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-19-23-22600

Sort: Year | Journal | Reset

### References

- E. Ip, A. P. Lau, D. J. F. Barros, and J. M. Kahn, “Coherent detection in optical fiber systems,” Opt. Express16(2), 753–791 (2008). [CrossRef] [PubMed]
- Y. Mori, C. Zhang, K. Igarashi, K. Katoh, and K. Kikuchi, “Unrepeated 200-km transmission of 40-Gbit/s 16-QAM signals using digital coherent receiver,” Opt. Express17(3), 1435–1441 (2009). [CrossRef] [PubMed]
- E. Ip and J. M. Kahn, “Feedforward carrier recovery for coherent optical communications,” J. Lightwave Technol.25(9), 2675–2692 (2007). [CrossRef]
- L. M. Pessoa, H. M. Salgado, and I. Darwazeh, “Joint mitigation of optical impairments and phase estimation in coherent optical systems,” in Proceedings of IEEE LEOS Summer Topical Meetings2008, pp. 169–170.
- D.-S. Ly-Gagnon, S. Tsukamoto, K. Katoh, and K. Kikuchi, “Coherent detection of optical quadrature phase-shift keying signals with carrier phase estimation,” J. Lightwave Technol.24(1), 12–21 (2006). [CrossRef]
- S. Zhang, P. Y. Kam, J. Chen, and C. Yu, “Decision-aided maximum likelihood detection in coherent optical phase-shift-keying system,” Opt. Express17(2), 703–715 (2009). [CrossRef] [PubMed]
- S. Zhang, P. Y. Kam, C. Yu, and J. Chen, “Decision-aided carrier phase estimation for coherent optical communications,” J. Lightwave Technol.28(11), 1597–1607 (2010). [CrossRef]
- S. Zhang, P. Y. Kam, and C. Yu, “Block length effect of decision-aided maximum likelihood phase estimation in coherent optical communication systems,” in Proceedings of OSA/CLEO/QELS2009, pp. 1–2, 2–4.
- P. Y. Kam, “Maximum likelihood carrier phase recovery for linear suppressed-carrier digital data modulations,” IEEE Trans. Commun.34(6), 522–527 (1986). [CrossRef]
- P. Y. Kam, “Maximum-likelihood digital data sequence estimation over the Gaussian channel with unknown carrier phase,” IEEE Trans. Commun.35(7), 764–767 (1987). [CrossRef]
- P. Y. Kam and P. Sinha, “A Viterbi-type algorithm for efficient estimation of M-PSK sequences over the Gaussian channel with unknown carrier phase,” IEEE Trans. Commun.43(9), 2429–2433 (1995). [CrossRef]
- S. Zhang, X. Li, P. Y. Kam, C. Yu, and J. Chen, “Pilot-assisted decision-aided maximum-likelihood phase estimation in coherent optical phase-modulated systems with nonlinear phase noise,” IEEE Photon. Technol. Lett.22(6), 380–382 (2010). [CrossRef]
- M. Seimetz and C.-M. Weinert, “Options, feasibility, and availability of 2 x4 90° hybrids for coherent optical systems,” J. Lightwave Technol.24(3), 1317–1322 (2006). [CrossRef]
- K. Kikuchi, T. Okoshi, M. Nagamatsu, and N. Henmi, “Degradation of bit-error rate in coherent optical communications due to spectral spread of the transmitter and the local oscillator,” J. Lightwave Technol.2(6), 1024–1033 (1984). [CrossRef]
- Y. Li, P. Y. Kam, and C. C. Chui, “Adaptive sequence detection for MPSK/MQAM with unknown carrier phase characteristics,” in Proceedings of WCNC2009, pp.1–6

## Cited By |
Alert me when this paper is cited |

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

« Previous Article | Next Article »

OSA is a member of CrossRef.