## Anti-noise algorithm of lidar data retrieval by combining the ensemble Kalman filter and the Fernald method |

Optics Express, Vol. 21, Issue 7, pp. 8286-8297 (2013)

http://dx.doi.org/10.1364/OE.21.008286

Acrobat PDF (1136 KB)

### Abstract

The lidar signal-to-noise ratio decreases rapidly with an increase in range, which severely affects the retrieval accuracy and the effective measure range of a lidar based on the Fernald method. To avoid this issue, an alternative approach is proposed to simultaneously retrieve lidar data accurately and obtain a de-noised signal as a by-product by combining the ensemble Kalman filter and the Fernald method. The dynamical model of the new algorithm is generated according to the lidar equation to forecast backscatter coefficients. In this paper, we use the ensemble sizes as 60 and the factor *δ*^{1/2} as 1.2 after being weighed against the accuracy and the time cost based on the performance function we define. The retrieval and de-noising results of both simulated and real signals demonstrate that our method is practical and effective. An extensive application of our method can be useful for the long-term determining of the aerosol optical properties.

© 2013 OSA

## 1. Introduction

2. M. Feiyue, G. Wei, and M. Yingying, “Retrieving the aerosol lidar ratio profile by combining ground- and space-based elastic lidars,” Opt. Lett. **37**(4), 617–619 (2012). [CrossRef] [PubMed]

3. F. G. Fernald, “Analysis of atmospheric lidar observations: some comments,” Appl. Opt. **23**(5), 652–653 (1984). [CrossRef] [PubMed]

*r*is the altitude,

*P*(

*r*) represents the received signal,

*C*is the lidar constant,

*G*(

*r*) is the overlap factor (which can be defined as the ratio of the received energy to the energy hits the telescope mirror [4

4. W. Gong, F. Mao, and J. Li, “OFLID: Simple method of overlap factor calculation with laser intensity distribution for biaxial lidar,” Opt. Commun. **284**(12), 2966–2971 (2011). [CrossRef]

*β*(

*r*) and

*α*(

*r*) are the atmospheric backscattering and extinction coefficient, respectively, the subscript 1 and 2 denote aerosol and molecule, respectively, and

*e*(

*r*) is the noise which can be considered as Gaussian.

3. F. G. Fernald, “Analysis of atmospheric lidar observations: some comments,” Appl. Opt. **23**(5), 652–653 (1984). [CrossRef] [PubMed]

5. R. Collis, “Lidar: a new atmospheric probe,” Q. J. R. Meteorol. Soc. **92**(392), 220–230 (1966). [CrossRef]

8. V. A. Kovalev, “Stable near-end solution of the lidar equation for clear atmospheres,” Appl. Opt. **42**(3), 585–591 (2003). [CrossRef] [PubMed]

3. F. G. Fernald, “Analysis of atmospheric lidar observations: some comments,” Appl. Opt. **23**(5), 652–653 (1984). [CrossRef] [PubMed]

6. J. D. Klett, “Stable analytical inversion solution for processing lidar returns,” Appl. Opt. **20**(2), 211–220 (1981). [CrossRef] [PubMed]

2. M. Feiyue, G. Wei, and M. Yingying, “Retrieving the aerosol lidar ratio profile by combining ground- and space-based elastic lidars,” Opt. Lett. **37**(4), 617–619 (2012). [CrossRef] [PubMed]

6. J. D. Klett, “Stable analytical inversion solution for processing lidar returns,” Appl. Opt. **20**(2), 211–220 (1981). [CrossRef] [PubMed]

9. F. Rocadenbosch, M. N. Reba, M. Sicard, and A. Comerón, “Practical analytical backscatter error bars for elastic one-component lidar inversion algorithm,” Appl. Opt. **49**(17), 3380–3393 (2010). [CrossRef] [PubMed]

10. H. T. Fang and D. S. Huang, “Noise reduction in lidar signal based on discrete wavelet transform,” Opt. Commun. **233**(1-3), 67–76 (2004). [CrossRef]

12. W. Gong, J. Li, F. Mao, and J. Zhang, “Comparison of simultaneous signals obtained from a dual-field-of-view lidar and its application to noise reduction based on empirical mode decomposition,” Chin. Opt. Lett. **9**(5), 050101–050104 (2011). [CrossRef]

*P*(

*r*) is a grossly nonlinear non-stationary signal (due to the strong attenuation caused by the 1/

*r*

^{2}effect), the range of variability can be as large as seven or more orders of magnitude. The moving average and Fourier methods are not well suited for the de-noising of a nonlinear non-stationary lidar signal [11

11. H. T. Fang, D. S. Huang, and Y. H. Wu, “Antinoise approximation of the lidar signal with wavelet neural networks,” Appl. Opt. **44**(6), 1077–1083 (2005). [CrossRef] [PubMed]

12. W. Gong, J. Li, F. Mao, and J. Zhang, “Comparison of simultaneous signals obtained from a dual-field-of-view lidar and its application to noise reduction based on empirical mode decomposition,” Chin. Opt. Lett. **9**(5), 050101–050104 (2011). [CrossRef]

12. W. Gong, J. Li, F. Mao, and J. Zhang, “Comparison of simultaneous signals obtained from a dual-field-of-view lidar and its application to noise reduction based on empirical mode decomposition,” Chin. Opt. Lett. **9**(5), 050101–050104 (2011). [CrossRef]

*P*(

*r*). The magnitude range of the range-corrected signal

*X*(

*r*) [i.e.

*P*(

*r*)

*r*

^{2}] is smaller than that of

*P*(

*r*). However, the noise level of

*X*(

*r*) is equivalent to

*e*(

*r*)

*r*

^{2}, which rapidly increases with increasing range causing de-noising methods based on

*X*(

*r*) to yield undesirable results. Thus, using results based upon the de-noised signal by the Fernald method can be seriously distorted.

*δ*

^{1/2}as 1.2 after weighed against the accuracy and the time cost. The results of both the simulated and real signal demonstrate that our method is practical and effective, even at a far range where the SNR is low. Furthermore, our study suggests that de-noise is unnecessary in the near range because the SNR is very high.

## 2. Principles and methods

### 2.1 Flow of the retrieval and de-noising algorithm

*β*

_{1}(

*i*+

*n*)...

*β*

_{1}(

*i*+ 1),

*β*

_{1}(

*i*) and Eq. (1) to forecast

*X*

_{ratio}(

*i*).

*X*

_{ratio}(

*i*) is defined as the ratio of the range-corrected signal of two adjacent range bins, which can be written aswhere Δ

*r*is the vertical resolution and the altitude of bin

*i*is

*i*∙Δ

*r*. First, we use

*β*

_{1}(

*i*+

*n*)...

*β*

_{1}(

*i*+ 1),

*β*

_{1}(

*i*) to linearly extrapolate and obtain

*β*

_{1}(

*i*-1). For a high vertical resolution lidar, we can consider

*β*

_{1}(

*i*-1) =

*β*

_{1}(

*i*) to simplify the calculation. Second, we apply the

*β*

_{1}(

*i*-1) and the given lidar ratio to derive

*X*

_{ratio}(

*i*). Third, we use the Monte Carlo method with

*X*(

*i*) and

*Std*·

*r*

^{2}(

*i*) to produce a corresponding ensemble range-corrected signal

*j*= 1,...,

*N*, and

*N*is the ensemble size). Finally, we use

*X*

_{ratio}(

*i*) to approximately forecast

*X*(

*i*-1) and

*Std*·

*r*

^{2}(

*i*-1) as the input parameters in order to update

*X*(

*i*-1).

*β*

_{1}(

*i*-1) based on

*X*(

*i*) and the updated

*X*(

*i*-1) by the normal Fernald method, and then iterate the calculation until the whole

*β*

_{1}profile have been retrieved.

### 2.2 De-noising by ensemble Kalman filter

13. R. E. Kalman, “A new approach to linear filtering and prediction problems,” J. Basic Eng. **82**(1), 35–45 (1960). [CrossRef]

14. R. N. Miller, M. Ghil, and F. Gauthiez, “Advanced data assimilation in strongly nonlinear dynamical systems,” J. Atmos. Sci. **51**(8), 1037–1056 (1994). [CrossRef]

15. G. Evensen, “Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics,” J. Geophys. Res. **99**(C5), 143–162 (1994). [CrossRef]

*X*and random model error

_{k}*ε*at time

_{k}*k*, we can express the model equation as

*M*(·) is the model operator for forecasting model states from time

*k*-1 to

*k*, and

*Y*is the observation at time

_{k}*k*,

*H*(·) is the observation operator, and

16. G. Evensen, “The ensemble Kalman filter: Theoretical formulation and practical implementation,” Ocean Dyn. **53**(4), 343–367 (2003). [CrossRef]

*k*can be obtained from model state

*k*-1, where the superscript

*f*and

*a*denote the forecast and analysis scheme, and the superscript (

*i*) denotes the

*i*-th sample of the variables,

*N*samples of

*X*

_{ratio}(

*i*) to forecast

*k*along with the Kalman gain matrix,

*X*and

_{k}*δ*>1, or

17. J. L. Anderson and S. L. Anderson, “A Monte Carlo implementation of the nonlinear filtering problem to produce ensemble assimilations and forecasts,” Mon. Weather Rev. **127**(12), 2741–2758 (1999). [CrossRef]

*δ*

^{1/2}and

*N*will be discussed and selected based on the performance function in Section 3.

### 2.3 Data retrieval by the Fernald method

*S*

_{1}(

*i*) and the backscatter coefficient at a calibration range. Under these assumptions, the total backscatter coefficients at range bin

*i*of the ground-based lidars can be derived based on the two-component Fernald method. The two-component method is the most common inversion method because it can distinguish the contributions between molecules and aerosols. The two-component Fernald method contains backward and forward solutions, which can be respectively given as [3

**23**(5), 652–653 (1984). [CrossRef] [PubMed]

6. J. D. Klett, “Stable analytical inversion solution for processing lidar returns,” Appl. Opt. **20**(2), 211–220 (1981). [CrossRef] [PubMed]

*β*

_{2}(

*i*) is the molecular backscatter coefficient determined from the US standard atmospheric model, and

*S*

_{2}(

*i*) =

*α*

_{2}(

*i*)/

*β*

_{2}(

*i*) = 8π/3 is the molecule lidar ratio. The aerosol lidar ratio

*S*

_{1}(

*i*) changes largely for aerosols with different chemical and physical properties [2

2. M. Feiyue, G. Wei, and M. Yingying, “Retrieving the aerosol lidar ratio profile by combining ground- and space-based elastic lidars,” Opt. Lett. **37**(4), 617–619 (2012). [CrossRef] [PubMed]

18. Y. Sasano, “Tropospheric aerosol extinction coefficient profiles derived from scanning lidar measurements over Tsukuba, Japan, from 1990 to 1993,” Appl. Opt. **35**(24), 4941–4952 (1996). [CrossRef] [PubMed]

*c*can be chosen at about 10 km, and we can assume

*β*(

*c*)/

*β*(

_{2}*c*) = 1.05

*β*(

_{2}*c*) for the real data retrieval.

*N*and

*δ*

^{1/2}) involved. In order to optimize these parameters, we construct a performance function

*F*(

*N*,

*δ*

^{1/2}) based on the final values of

*β*

_{1}(

*i*) retrieved with

*N*and

*δ*

^{1/2}aswhere

*i*corresponds to the maximal range bin we considered and

_{max}*β*

_{1,}

*(*

_{R}*i*) and

*β*

_{1}(

*i*) are the retrieved and reference aerosol backscatter coefficients, respectively. We calculate

*β*

_{1,}

*(*

_{R}*i*) and

*F*(

*N*,

*δ*

^{1/2}) with different

*N*and

*δ*

^{1/2}then search for a relatively stable and low value of

*F*(

*N*,

*δ*

^{1/2}) as well as consider the corresponding

*N*and

*δ*

^{1/2}as the optimal parameters for the retrieval.

## 3. Results and discussion

### 3.1 Testing with simulated signals

*N*and

*δ*

^{1/2}are shown in Fig. 3 , respectively. The values of the performance function with a same

*δ*

^{1/2}but different ensemble sizes decrease with the ensemble size, and tended to be stable after 20. After weighing the de-noising effect and the calculation time, we select 60 as the ensemble size. Furthermore, the performance function with the same ensemble size but different

*δ*

^{1/2}has a minimum when

*δ*

^{1/2}approaches 1.2 which coincides with results from a previous study [19

19. P. Sakov and P. R. Oke, “A deterministic formulation of the ensemble Kalman filter: an alternative to ensemble square root filters,” Tellus, Ser. A, Dyn. Meterol. Oceanogr. **60**(2), 361–371 (2008). [CrossRef]

*δ*

^{1/2}= 1.2 and

*N*= 60 for the retrieval.

### 3.2 Testing with real signals

*r*

^{2}effect and the attenuation of laser beam energy. The aerosol backscatter coefficient retrieved by our method not only has smaller uncertainty above 4 km, but also fits the profile of the normal Fernald method below 4 km well. The results of our method are consistent with that of the 64 minutes averaged signals. That means the effect of our method is approximately equivalent to that of averaging 64 replications in our retrieval, which can halve the standard error of the signal three times.

20. F. Mao, W. Gong, and Z. Zhu, “Simple multiscale algorithm for layer detection with lidar,” Appl. Opt. **50**(36), 6591–6598 (2011). [CrossRef] [PubMed]

**20**(2), 211–220 (1981). [CrossRef] [PubMed]

## 4. Summary

*δ*

^{1/2}as 1.2 as well as the ensemble sizes as 60 in our retrieval. The retrieval results show that our method can obtain accurate backscatter coefficient. The results of both simulated and real signal demonstrate that the effectiveness and efficiency of our method are satisfactory. In particular, the experimental results of real signals show that our method can work at a far range where the signal-to-noise ratio is low. For our lidar data set, the effect of the new method is approximately equivalent to an average of 64 replications, which can halve the standard error three times. Furthermore, this study suggests that a de-noise scheme is unnecessary in the near range because the SNR is very high.

## Acknowledgments

## References and links

1. | V. A. Kovalev and W. E. Eichinger, |

2. | M. Feiyue, G. Wei, and M. Yingying, “Retrieving the aerosol lidar ratio profile by combining ground- and space-based elastic lidars,” Opt. Lett. |

3. | F. G. Fernald, “Analysis of atmospheric lidar observations: some comments,” Appl. Opt. |

4. | W. Gong, F. Mao, and J. Li, “OFLID: Simple method of overlap factor calculation with laser intensity distribution for biaxial lidar,” Opt. Commun. |

5. | R. Collis, “Lidar: a new atmospheric probe,” Q. J. R. Meteorol. Soc. |

6. | J. D. Klett, “Stable analytical inversion solution for processing lidar returns,” Appl. Opt. |

7. | F. Rocadenbosch, C. Soriano, A. Comerón, and J. M. Baldasano, “Lidar inversion of atmospheric backscatter and extinction-to-backscatter ratios by use of a Kalman filter,” Appl. Opt. |

8. | V. A. Kovalev, “Stable near-end solution of the lidar equation for clear atmospheres,” Appl. Opt. |

9. | F. Rocadenbosch, M. N. Reba, M. Sicard, and A. Comerón, “Practical analytical backscatter error bars for elastic one-component lidar inversion algorithm,” Appl. Opt. |

10. | H. T. Fang and D. S. Huang, “Noise reduction in lidar signal based on discrete wavelet transform,” Opt. Commun. |

11. | H. T. Fang, D. S. Huang, and Y. H. Wu, “Antinoise approximation of the lidar signal with wavelet neural networks,” Appl. Opt. |

12. | W. Gong, J. Li, F. Mao, and J. Zhang, “Comparison of simultaneous signals obtained from a dual-field-of-view lidar and its application to noise reduction based on empirical mode decomposition,” Chin. Opt. Lett. |

13. | R. E. Kalman, “A new approach to linear filtering and prediction problems,” J. Basic Eng. |

14. | R. N. Miller, M. Ghil, and F. Gauthiez, “Advanced data assimilation in strongly nonlinear dynamical systems,” J. Atmos. Sci. |

15. | G. Evensen, “Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics,” J. Geophys. Res. |

16. | G. Evensen, “The ensemble Kalman filter: Theoretical formulation and practical implementation,” Ocean Dyn. |

17. | J. L. Anderson and S. L. Anderson, “A Monte Carlo implementation of the nonlinear filtering problem to produce ensemble assimilations and forecasts,” Mon. Weather Rev. |

18. | Y. Sasano, “Tropospheric aerosol extinction coefficient profiles derived from scanning lidar measurements over Tsukuba, Japan, from 1990 to 1993,” Appl. Opt. |

19. | P. Sakov and P. R. Oke, “A deterministic formulation of the ensemble Kalman filter: an alternative to ensemble square root filters,” Tellus, Ser. A, Dyn. Meterol. Oceanogr. |

20. | F. Mao, W. Gong, and Z. Zhu, “Simple multiscale algorithm for layer detection with lidar,” Appl. Opt. |

**OCIS Codes**

(280.1100) Remote sensing and sensors : Aerosol detection

(280.3640) Remote sensing and sensors : Lidar

**ToC Category:**

Remote Sensing

**History**

Original Manuscript: January 14, 2013

Revised Manuscript: March 7, 2013

Manuscript Accepted: March 8, 2013

Published: March 28, 2013

**Citation**

Feiyue Mao, Wei Gong, and Chen Li, "Anti-noise algorithm of lidar data retrieval by combining the ensemble Kalman filter and the Fernald method," Opt. Express **21**, 8286-8297 (2013)

http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-21-7-8286

Sort: Year | Journal | Reset

### References

- V. A. Kovalev and W. E. Eichinger, Elastic lidar: theory, practice, and analysis methods (Wiley-Interscience, 2004).
- M. Feiyue, G. Wei, and M. Yingying, “Retrieving the aerosol lidar ratio profile by combining ground- and space-based elastic lidars,” Opt. Lett.37(4), 617–619 (2012). [CrossRef] [PubMed]
- F. G. Fernald, “Analysis of atmospheric lidar observations: some comments,” Appl. Opt.23(5), 652–653 (1984). [CrossRef] [PubMed]
- W. Gong, F. Mao, and J. Li, “OFLID: Simple method of overlap factor calculation with laser intensity distribution for biaxial lidar,” Opt. Commun.284(12), 2966–2971 (2011). [CrossRef]
- R. Collis, “Lidar: a new atmospheric probe,” Q. J. R. Meteorol. Soc.92(392), 220–230 (1966). [CrossRef]
- J. D. Klett, “Stable analytical inversion solution for processing lidar returns,” Appl. Opt.20(2), 211–220 (1981). [CrossRef] [PubMed]
- F. Rocadenbosch, C. Soriano, A. Comerón, and J. M. Baldasano, “Lidar inversion of atmospheric backscatter and extinction-to-backscatter ratios by use of a Kalman filter,” Appl. Opt.38(15), 3175–3189 (1999). [CrossRef] [PubMed]
- V. A. Kovalev, “Stable near-end solution of the lidar equation for clear atmospheres,” Appl. Opt.42(3), 585–591 (2003). [CrossRef] [PubMed]
- F. Rocadenbosch, M. N. Reba, M. Sicard, and A. Comerón, “Practical analytical backscatter error bars for elastic one-component lidar inversion algorithm,” Appl. Opt.49(17), 3380–3393 (2010). [CrossRef] [PubMed]
- H. T. Fang and D. S. Huang, “Noise reduction in lidar signal based on discrete wavelet transform,” Opt. Commun.233(1-3), 67–76 (2004). [CrossRef]
- H. T. Fang, D. S. Huang, and Y. H. Wu, “Antinoise approximation of the lidar signal with wavelet neural networks,” Appl. Opt.44(6), 1077–1083 (2005). [CrossRef] [PubMed]
- W. Gong, J. Li, F. Mao, and J. Zhang, “Comparison of simultaneous signals obtained from a dual-field-of-view lidar and its application to noise reduction based on empirical mode decomposition,” Chin. Opt. Lett.9(5), 050101–050104 (2011). [CrossRef]
- R. E. Kalman, “A new approach to linear filtering and prediction problems,” J. Basic Eng.82(1), 35–45 (1960). [CrossRef]
- R. N. Miller, M. Ghil, and F. Gauthiez, “Advanced data assimilation in strongly nonlinear dynamical systems,” J. Atmos. Sci.51(8), 1037–1056 (1994). [CrossRef]
- G. Evensen, “Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics,” J. Geophys. Res.99(C5), 143–162 (1994). [CrossRef]
- G. Evensen, “The ensemble Kalman filter: Theoretical formulation and practical implementation,” Ocean Dyn.53(4), 343–367 (2003). [CrossRef]
- J. L. Anderson and S. L. Anderson, “A Monte Carlo implementation of the nonlinear filtering problem to produce ensemble assimilations and forecasts,” Mon. Weather Rev.127(12), 2741–2758 (1999). [CrossRef]
- Y. Sasano, “Tropospheric aerosol extinction coefficient profiles derived from scanning lidar measurements over Tsukuba, Japan, from 1990 to 1993,” Appl. Opt.35(24), 4941–4952 (1996). [CrossRef] [PubMed]
- P. Sakov and P. R. Oke, “A deterministic formulation of the ensemble Kalman filter: an alternative to ensemble square root filters,” Tellus, Ser. A, Dyn. Meterol. Oceanogr.60(2), 361–371 (2008). [CrossRef]
- F. Mao, W. Gong, and Z. Zhu, “Simple multiscale algorithm for layer detection with lidar,” Appl. Opt.50(36), 6591–6598 (2011). [CrossRef] [PubMed]

## Cited By |
Alert me when this paper is cited |

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

« Previous Article | Next Article »

OSA is a member of CrossRef.