## Equalization of nonlinear transmission impairments by maximum-likelihood-sequence estimation in digital coherent receivers

Optics Express, Vol. 18, Issue 5, pp. 4776-4782 (2010)

http://dx.doi.org/10.1364/OE.18.004776

Acrobat PDF (159 KB)

### Abstract

We describe a successful introduction of maximum-likelihood-sequence estimation (MLSE) into digital coherent receivers together with finite-impulse response (FIR) filters in order to equalize both linear and nonlinear fiber impairments. The MLSE equalizer based on the Viterbi algorithm is implemented in the offline digital signal processing (DSP) core. We transmit 20-Gbit/s quadrature phase-shift keying (QPSK) signals through a 200-km-long standard single-mode fiber. The bit-error rate performance shows that the MLSE equalizer outperforms the conventional adaptive FIR filter, especially when nonlinear impairments are predominant.

© 2010 OSA

## 1. Introduction

1. A. Faerbert, S. Langenbach, N. Stojanovic, C. Dorschky, T. Kupfer, C. Schulien, J.-P. Elbers, H. Wernz, H. Griesser, and C. Glingener, “Performance of a 10.7Gb/s receiver with digital equalizer using maximum likelihood sequence estimation”, European Conference on Optical Communication (ECOC 2004), Stockholm, Sweden, Post deadline paper PD Th4.1.5 (2004).

8. S. Chandrasekhar and A. H. Gnauck, “Performance of MLSE receiver in a dispersion-managed multispan experiment at 10.7 Gb/s under nonlinear transmission,” IEEE Photon. Technol. Lett. **18**(23), 2448–2450 (2006). [CrossRef]

10. K. Kikuchi, “Phase-diversity homodyne detection of multilevel optical modulation with digital carrier phase estimation”, IEEE J. Sel. Top. Quantum Electron. **12**(4), 563–570 (2006). [CrossRef]

11. S. Tsukamoto, K. Katoh, and K. Kikuchi, “Unrepeated transmission of 20-Gb/s optical quadrature phase-shift-keying signal over 200-km standard single-mode fiber based on digital processing of homodyne-detected signal for group-velocity dispersion compensation,” IEEE Photon. Technol. Lett. **18**(9), 1016–1018 (2006). [CrossRef]

12. S. J. Savory, “Digital filters for coherent optical receivers,” Opt. Express **16**(2), 804–817 (2008). [CrossRef] [PubMed]

13. G. D. Forney Jr., “Maximum-likelihood sequence estimation of digital sequences in the presence of intersymbol interference,” IEEE Trans. Inf. Theory **18**(3), 363–378 (1972). [CrossRef]

14. G. D. Forney Jr., “The Viterbi algorithm,” IEEE Proc. **61**(3), 268–278 (1973). [CrossRef]

## 2. Experimental setup

^{9}-1 pseudorandom pattern after decoding. An optical QPSK signal was created by a LiNbO

_{3}optical IQ modulator (IQM).

*L*

_{f}becomes dominant when

*L*

_{f}is longer than the nonlinearity length

*L*

_{NL}defined by

*γP*, where

*γ*is the nonlinear coefficient of the fiber and

*P*the launched power. When

*P*= 0 dBm and 10 dBm, we calculate

*L*

_{NL}to be 670 km and 67 km, respectively, by using

*γ*= 1.5/W/km. Since

*L*

_{f}= 100 km and 200 km in our case, the SPM effect may appear significantly when the input power is 10 dBm. On the other hand, we can almost neglect the SPM effect when the input power is 0 dBm. The SMF had the dispersion

*D*= 17 ps/nm/km (

*β*22 ps

_{2}= -^{2}/km). We confirmed that the stimulated Brillouin scattering (SBS) never occurred when the input power was increased up to10 dBm.

*P*

_{in}was controlled by a variable optical attenuator (VOA). Then, the signal was pre-amplified by an erbium-doped fiber amplifier (EDFA) and detected by a homodyne phase-diversity receiver. More details on homodyne phase-diversity coherent receivers can be found in [8

8. S. Chandrasekhar and A. H. Gnauck, “Performance of MLSE receiver in a dispersion-managed multispan experiment at 10.7 Gb/s under nonlinear transmission,” IEEE Photon. Technol. Lett. **18**(23), 2448–2450 (2006). [CrossRef]

## 3. MLSE implementation

*M*and the channel-memory length is

*L*, the number of states in the trellis diagram (TD) is

*M*and the number of state transitions is

^{L}*M*In our experiment, we apply the QPSK (

^{L + 1.}*M*= 4) signal with the channel memory

*L*= 2. In such a case, each of 64 state transitions is given as a six-bit word consisting of a symbol together with a predecessor symbol and a post-cursor symbol.

*p*(

*i*) of those sorted symbols, which belong to the

*i*-th state transition. The complex amplitude

*p*(

*i*) thus obtained is stored in the look-up table.

*(*

_{k}*i*) = |z(

*k*)-

*p*(

*i*)

*|*

^{2}, which is the distance between a received complex amplitude z(

*k*) of the

*k*-th symbol and the average complex amplitude

*p*(

*i*) belonging to the

*i*-th state transition. BM can be computed by either histogram or Euclidean-distance method, and we applied the later one because of its simplicity. Each BM is associated with one of the state transitions in TD. The path leading to a certain state accumulates all branch metrics within this path and thus leads a path metric (PM). Therefore, the path metric at the

*k*-th state can be obtained as PM

*= PM*

_{k}

_{k-}_{1}+ BM

*. Each state is developed into a transition by adding a possible new symbol and each transition leads to a new state. This leads to a competition of paths prior to each consecutive state, where only the path with the maximum path metric survives. All other paths are omitted in the next steps. In summary, the operational principles of VA as well as of TD include the following steps:*

_{k}- • Calculation of BM and PM by an add-compare-select module.
- • Maximization of path metrics among all 16-states at the last stage of TD.
- • Obtaining the only the survival path by the trace-back process.

13. G. D. Forney Jr., “Maximum-likelihood sequence estimation of digital sequences in the presence of intersymbol interference,” IEEE Trans. Inf. Theory **18**(3), 363–378 (1972). [CrossRef]

14. G. D. Forney Jr., “The Viterbi algorithm,” IEEE Proc. **61**(3), 268–278 (1973). [CrossRef]

*M*th-power feed-forward scheme, and we restore the symbol

*z*(

*k*). Such symbol is now ready for equalization by an MLSE module consisting of the look up table, the BM computation block and the Viterbi algorithm block. The look up table stores

*p*(

*i*) and sends it to the BM computation block.

## 4. Experimental results

- 1) The case without any equalizer,
- 2) The case using only a fixed FIR filter,
- 3) The case using an adaptive FIR filter along with the fixed FIR filter, and
- 4) The case using MLSE together with the fixed FIR filter.

^{−4}for 100-km and 200-km transmission, respectively.

## 5. Discussion

^{−4}with respect to the theoretical limit. Green curves show the BER performance in the linear case. We can see that GVD can be perfectly compensated for by fixed and adaptive FIR filters, and the sensitivity can be improved by very small amount (less than 1 dB) with the MLSE scheme. Although either FIR filters or MLSE can be the perfect solution to mitigate linear inter-symbol interference (ISI), the FIR-filter-based scheme should be implemented in the digital coherent receiver because of its smaller computational complexity.

## 6. Conclusion

## Acknowledgements

## References

1. | A. Faerbert, S. Langenbach, N. Stojanovic, C. Dorschky, T. Kupfer, C. Schulien, J.-P. Elbers, H. Wernz, H. Griesser, and C. Glingener, “Performance of a 10.7Gb/s receiver with digital equalizer using maximum likelihood sequence estimation”, European Conference on Optical Communication (ECOC 2004), Stockholm, Sweden, Post deadline paper PD Th4.1.5 (2004). |

2. | J. M. Gene, P. J. Winzer, S. Chandrasekhar, and H. Kogelnik, “Joint PMD and chromatic dispersion compensation using an MLSE”, European Conference on Optical Communication (ECOC 2006), Cannes, France, Paper We2.5.2 (2006). |

3. | N. Alić, G. C. Papen, R. E. Saperstein, L. B. Milstein, and Y. Fainman, “Signal statistics and maximum likelihood sequence estimation in intensity modulated fiber optic links containing a single optical preamplifier,” Opt. Express |

4. | F. N. Hauske, B. Lankl, C. Xie, and E.-D. Schmidt, “Iterative electronic equalization utilizing low complexity MLSEs for 40 Gbit/s DQPSK modulation”, Optical Fiber Communication Conference (OFC 2007), Anaheim, CA, USA, Paper OMG2 (2007). |

5. | G. Agrawal, |

6. | O. E. Agazzi, and V. Gopinathan, “The impact of nonlinearity on electronic dispersion compensation of optical channels”, Optical Fiber Communication Conference (OFC 2004), Anaheim, CA, USA, Paper TuG6 (2004). |

7. | O. E. Agazzi, M. R. Hueda, H. S. Carrer, and D. E. Crivelli, “Maximum-likelihood sequence estimation in dispersive optical channels,” J. Lightwave Technol. |

8. | S. Chandrasekhar and A. H. Gnauck, “Performance of MLSE receiver in a dispersion-managed multispan experiment at 10.7 Gb/s under nonlinear transmission,” IEEE Photon. Technol. Lett. |

9. | T. Okoshi, and K. Kikuchi, |

10. | K. Kikuchi, “Phase-diversity homodyne detection of multilevel optical modulation with digital carrier phase estimation”, IEEE J. Sel. Top. Quantum Electron. |

11. | S. Tsukamoto, K. Katoh, and K. Kikuchi, “Unrepeated transmission of 20-Gb/s optical quadrature phase-shift-keying signal over 200-km standard single-mode fiber based on digital processing of homodyne-detected signal for group-velocity dispersion compensation,” IEEE Photon. Technol. Lett. |

12. | S. J. Savory, “Digital filters for coherent optical receivers,” Opt. Express |

13. | G. D. Forney Jr., “Maximum-likelihood sequence estimation of digital sequences in the presence of intersymbol interference,” IEEE Trans. Inf. Theory |

14. | G. D. Forney Jr., “The Viterbi algorithm,” IEEE Proc. |

15. | J. G. Proakis, |

**OCIS Codes**

(060.1660) Fiber optics and optical communications : Coherent communications

(060.2330) Fiber optics and optical communications : Fiber optics communications

(060.2920) Fiber optics and optical communications : Homodyning

**ToC Category:**

Fiber Optics and Optical Communications

**History**

Original Manuscript: November 18, 2009

Revised Manuscript: February 18, 2010

Manuscript Accepted: February 18, 2010

Published: February 23, 2010

**Citation**

Md. Khairuzzaman, Chao Zhang, Koji Igarashi, Kazuhiro Katoh, and Kazuro Kikuchi, "Equalization of nonlinear transmission impairments by maximum-likelihood-sequence estimation in digital coherent receivers," Opt. Express **18**, 4776-4782 (2010)

http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-18-5-4776

Sort: Year | Journal | Reset

### References

- A. Faerbert, S. Langenbach, N. Stojanovic, C. Dorschky, T. Kupfer, C. Schulien, J.-P. Elbers, H. Wernz, H. Griesser, and C. Glingener, “Performance of a 10.7Gb/s receiver with digital equalizer using maximum likelihood sequence estimation,” European Conference on Optical Communication (ECOC 2004), Stockholm, Sweden, Post deadline paper PD Th4.1.5 (2004).
- J. M. Gene, P. J. Winzer, S. Chandrasekhar, and H. Kogelnik, “Joint PMD and chromatic dispersion compensation using an MLSE,” European Conference on Optical Communication (ECOC 2006), Cannes, France, Paper We2.5.2 (2006).
- N. Alić, G. C. Papen, R. E. Saperstein, L. B. Milstein, and Y. Fainman, “Signal statistics and maximum likelihood sequence estimation in intensity modulated fiber optic links containing a single optical preamplifier,” Opt. Express 13(12), 4568–4579 (2005). [CrossRef] [PubMed]
- F. N. Hauske, B. Lankl, C. Xie, and E.-D. Schmidt, “Iterative electronic equalization utilizing low complexity MLSEs for 40 Gbit/s DQPSK modulation,” Optical Fiber Communication Conference (OFC 2007), Anaheim, CA, USA, Paper OMG2 (2007).
- G. Agrawal, Fiber-Optic Communication Systems, (John Wiley & Sons, New York, NY, USA, 2002).
- O. E. Agazzi, and V. Gopinathan, “The impact of nonlinearity on electronic dispersion compensation of optical channels,” Optical Fiber Communication Conference (OFC 2004), Anaheim, CA, USA, Paper TuG6 (2004).
- O. E. Agazzi, M. R. Hueda, H. S. Carrer, and D. E. Crivelli, “Maximum-likelihood sequence estimation in dispersive optical channels,” J. Lightwave Technol. 23(2), 749–763 (2005). [CrossRef]
- S. Chandrasekhar and A. H. Gnauck, “Performance of MLSE receiver in a dispersion-managed multispan experiment at 10.7 Gb/s under nonlinear transmission,” IEEE Photon. Technol. Lett. 18(23), 2448–2450 (2006). [CrossRef]
- T. Okoshi, and K. Kikuchi, Coherent optical fiber communications, (KTK Scientific Publishers, Tokyo, Japan, 1988).
- K. Kikuchi, “Phase-diversity homodyne detection of multilevel optical modulation with digital carrier phase estimation,” IEEE J. Sel. Top. Quantum Electron. 12(4), 563–570 (2006). [CrossRef]
- S. Tsukamoto, K. Katoh, and K. Kikuchi, “Unrepeated transmission of 20-Gb/s optical quadrature phase-shift-keying signal over 200-km standard single-mode fiber based on digital processing of homodyne-detected signal for group-velocity dispersion compensation,” IEEE Photon. Technol. Lett. 18(9), 1016–1018 (2006). [CrossRef]
- S. J. Savory, “Digital filters for coherent optical receivers,” Opt. Express 16(2), 804–817 (2008). [CrossRef] [PubMed]
- G. D. Forney, “Maximum-likelihood sequence estimation of digital sequences in the presence of intersymbol interference,” IEEE Trans. Inf. Theory 18(3), 363–378 (1972). [CrossRef]
- G. D. Forney, “The Viterbi algorithm,” IEEE Proc. 61(3), 268–278 (1973). [CrossRef]
- J. G. Proakis, Digital communications, 4th Edition. (McGraw-Hill, New York, NY, USA, 2001).

## Cited By |
Alert me when this paper is cited |

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

« Previous Article | Next Article »

OSA is a member of CrossRef.