## Scene-Based Nonuniformity Correction with Reduced Ghosting Using a Gated LMS Algorithm

Optics Express, Vol. 17, Issue 17, pp. 14918-14933 (2009)

http://dx.doi.org/10.1364/OE.17.014918

Acrobat PDF (598 KB)

### Abstract

In this paper, we present a scene-based nouniformity correction (NUC) method using a modified adaptive least mean square (LMS) algorithm with a novel gating operation on the updates. The gating is designed to significantly reduce ghosting artifacts produced by many scene-based NUC algorithms by halting updates when temporal variation is lacking. We define the algorithm and present a number of experimental results to demonstrate the efficacy of the proposed method in comparison to several previously published methods including other LMS and constant statistics based methods. The experimental results include simulated imagery and a real infrared image sequence. We show that the proposed method significantly reduces ghosting artifacts, but has a slightly longer convergence time.

© 2009 Optical Society of America

## 1. Introduction

2. Y. M. Chiang and J. G. Harris, “An Analog Integrated Circuit for Continuous-time Gain and Offset Calibration of Sensor Arrays,” Journal of Analog Integrated Circuits and Signal Processing **12**, 231–238 (1997). [CrossRef]

12. B. M. Ratliff, M. M. Hayat, and R. C. Hardie, “An Algebraic Algorithm for Nonuniformity Correction in Focal Plane Arrays,” The Journal of the Optical Society of America A **19(9)**, 1737–1747 (2002). [CrossRef]

6. D. A. Scribner, K. A. Sarkady, M. R. Kruer, J. T. Caulfield, J. D. Hunt, and C. Herman, “Adaptive Nonuniformity Correction for IR Focal Plane Arrays using Neural Networks,” in *Proceedings of the SPIE: Infrared Sensors: Detectors, Electronics, and Signal Processing*,
T. S. Jayadev, ed., vol. 1541, pp. 100–109 (1991).

6. D. A. Scribner, K. A. Sarkady, M. R. Kruer, J. T. Caulfield, J. D. Hunt, and C. Herman, “Adaptive Nonuniformity Correction for IR Focal Plane Arrays using Neural Networks,” in *Proceedings of the SPIE: Infrared Sensors: Detectors, Electronics, and Signal Processing*,
T. S. Jayadev, ed., vol. 1541, pp. 100–109 (1991).

10. B. Narayanan, R. C. Hardie, and R. A. Muse, “Scene-based nonuniformity correction technique that exploits knowledge of the focal-plane array readout architecture,” Applied Optics **44(17)**, 3482–3491 (2005). [CrossRef]

11. R. C. Hardie, M. M. Hayat, E. E. Armstrong, and B. J. Yasuda, “Scene-based Nonuniformity Correction with Video Sequences and Registration,” Applied Optics **39(8)**, 1241–1250 (2000). [CrossRef]

13. R. C. Hardie and D. R. Droege, “A MAP Estimator for Simultaneous Super-Resolution and Detector Nonuniformity Correction,” EURASIP Journal on Advances in Signal Processing, Article ID 89354 2007 (2007). [CrossRef]

## 2. Scene Based Nonuniformity Correction

### 2.1. Observation Model

2. Y. M. Chiang and J. G. Harris, “An Analog Integrated Circuit for Continuous-time Gain and Offset Calibration of Sensor Arrays,” Journal of Analog Integrated Circuits and Signal Processing **12**, 231–238 (1997). [CrossRef]

10. B. Narayanan, R. C. Hardie, and R. A. Muse, “Scene-based nonuniformity correction technique that exploits knowledge of the focal-plane array readout architecture,” Applied Optics **44(17)**, 3482–3491 (2005). [CrossRef]

13. R. C. Hardie and D. R. Droege, “A MAP Estimator for Simultaneous Super-Resolution and Detector Nonuniformity Correction,” EURASIP Journal on Advances in Signal Processing, Article ID 89354 2007 (2007). [CrossRef]

*i*,

*j*are the spatial detector coordinates and

*n*indicates the frame number. The true scene irradiance is given by

*X*(

_{i j}*n*), and

*a*(

_{i j}*n*) and

*b*(

_{i j}*n*) are the detector scale and biases, respectively. The temporal noise is given by

*η*(

_{ij}*n*) and the observed pixel value is given by

*Y*(

_{i j}*n*). Note that the scales and biases are functions of frame number as well as spatial location. However, we assume that the scales and biases drift very slowly in time and are almost fixed with respect to frame index.

*n*=1,2,3, …,

*N*. The gain and offset corrections are given by

*ĝ*(

_{ij}*n*) and

*ô*(

_{i j}*n*), respectively. In many applications, the estimated scene irradiance does not need to be radiometrically accurate. A global gain and offset error is usually acceptable, so long as the detectors appear to be operating uniformly.

### 2.2. Constant Statistics SBNUC

2. Y. M. Chiang and J. G. Harris, “An Analog Integrated Circuit for Continuous-time Gain and Offset Calibration of Sensor Arrays,” Journal of Analog Integrated Circuits and Signal Processing **12**, 231–238 (1997). [CrossRef]

*Ŝ*(

_{ij}*n*) is the estimated temporal standard deviation estimate for detector

*i*,

*j*for frame

*n*. The effective offset correction is given by

*M̂*(n) is the estimated temporal mean estimate. Note that the image will be effectively scaled so that the pixels have a zero temporal mean and unit temporal standard deviation. Thus, a global gain and offset may be required to scale the image back to the desired dynamic range.

_{i j}*M̂*(0) and

_{ij}*Ŝ*(0) being the global spatial mean and mean absolute deviation of the first frame, {

_{ij}*Y*(1)}, respectively. We also define

_{ij}*Y*(0)=∞ to ensure that |

_{ij}*Y*(1)-

_{ij}*Y*(0)|>

_{ij}*T*for all

*i,j*. Note that

*α*controls the effective number of frames making a significant contribution to the current estimate. An

*α*close to 1 produces a wide window incorporating many frames. This gives the algorithm a long convergence time, but with the potential for a more robust estimate of the statistics. Note that after

*𝓝*=log(0.37)/log(

*α*) frames, the first frame is given a weight of 0.37 of that of the current frame. Thus,

*𝓝*serves as a type of time constant to help in selecting and interpreting

*α*. The change threshold

*T*controls the minimum amount of change between frames required to trigger an update of the estimates for that detector. We refer to the CS method using the estimates above as the gated CS method.

### 2.3. SBNUC Using the LMS

*n*=1,2,3, …,

*N*. The image

*B*(

_{ij}*n*) is the “desired” image (here a blurred version of the observed frame) and

*X̂*(

_{ij}*n*) is the current corrected image estimate. A stochastic gradient descent algorithm can be applied to the correction parameters to seek to minimize the mean squared error [6

6. D. A. Scribner, K. A. Sarkady, M. R. Kruer, J. T. Caulfield, J. D. Hunt, and C. Herman, “Adaptive Nonuniformity Correction for IR Focal Plane Arrays using Neural Networks,” in *Proceedings of the SPIE: Infrared Sensors: Detectors, Electronics, and Signal Processing*,
T. S. Jayadev, ed., vol. 1541, pp. 100–109 (1991).

*n*=1,2,3, …,

*N*-1. The parameter

*ε*(

_{ij}*n*) is a step size that governs the convergence behavior of the algorithm. The standard LMS SBNUC uses a fixed value,

*ε*(

_{ij}*n*)=

*ε*. We initialize the gain and bias corrections with

*ĝ*(1)=1 and

_{ij}*ô*(1)=0. Note that to obtain good convergence, we have found it necessary to scale the input data to lie within the interval [0,1]. This allows the gain and offsets to converge with a common step size.

_{ij}*σ*

^{2}

_{Yij(n)}is an estimate of the local spatial variance centered at pixel

*i, j*in frame

*n*. The parameter

*K*is the maximum step size, and

*M*is the scaling constant used to normalize the data to the interval [0,1]. We refer to the LMS using the step size in Eq. (10) as the adaptive LMS algorithm. We have observed that this modification significantly reduces ghosting and actually increases convergence speed, since fewer big gradient steps are taken in an erroneous direction. However, because the step size is never actually set to zero with the adaptive LMS algorithm, it will not eliminate burn-in ghosting altogether. To eliminate burn-in from lack of motion, the LMS updates could be modified to include a change threshold like those in Eqs. (5) and (6). However, we have observed that better results can be obtained using the following change gating

*n*=1,2,3, …,

*N*-1. We define

*Z*(1)=∞ to ensure that |

_{ij}*B*(1)-

_{ij}*Z*(1)|>

_{ij}*T*for all

*i, j*. Note that here we are detecting change in the desired image at a given pixel location relative to the value of the desired image at the last frame used to update that pixel. We are not simply looking for frame-to-frame change. Detecting only significant frame-to-frame change will tend to exclude slowly varying image regions where the LMS does best and limit us to mostly sharp edges where the gradient error tends to be the largest. A similar change statistic could be defined using the observed image, rather than the low-pass filtered image. However, the “desired” image provides the additional benefit of temporal noise smoothing from the Gaussian low-pass filtering. We refer to the LMS using the step size in Eq. (11) as the gated adaptive LMS algorithm.

### 3. Experimental Results

### 3.1. Simulated Data

*α*=0.992 (

*𝓝*≈124). The gated CS method uses the same

*α*and a change threshold of

*T*=20. All of the LMS methods use a step size of

*ε*=0.05 and an FIR Gaussian low-pass filter with a standard deviation of 5 pixels and kernel size of 21×21. The adaptive LMS methods use

*K*=50 and

*M*=255, and the gated adaptive LMS uses a change threshold of

*T*=20.

### 3.2. Real Infrared Imagery

*µ*m producing 14 bit data. The optics have a focal length of 120 mm and f-number of 2.3. The video is acquired at 8 Hz. The sensor has been calibrated with a laboratory blackbody correction prior to the data collection. Residual low frequency nonuniformity has been corrected using a regression algorithm with an offset circularly-symmetric polynomial model and bad pixels have been replaced. Subtle residual high spatial frequency nonuniformity remains.

*α*=0.995 (

*𝓝*≈198). The gated CS method uses the same

*α*and a change threshold of

*T*=100. The LMS methods use a step size of

*η*=0.05 and an FIR Gaussian low-pass filter with a standard deviation of 5 pixels and kernel size of 21×21. The adaptive LMS methods use

*K*=100,

*M*=2

_{14}-1, and the gated adaptive LMS uses a change threshold of

*T*=100. Notice the obvious ghosting in the nongated LMS and CS outputs. The gated CS and gated adaptive LMS images do not have this ghosting artifact. It does appear, however, that the gated adaptive LMS has done a better job reducing the high spatial frequency nonuniformity. We also applied an offset only version of the SBNUC algorithms to the real infrared imagery. The outputs for offset-only gated CS and gated adaptive LMS are shown in Fig. 5.

5. S. N. Torres and M. M. Hayat, “Kalman Filtering for Adaptive Nonuniformity Correction in Infrared Focal Plane Arrays,” The Journal of the Optical Society of America A **20(3)**, 470–480 (2003). [CrossRef]

*h*is a discrete Laplacian convolution kernel and ‖·‖

_{1}refers to an

*L*

^{1}norm. These results for the image estimates used in Fig. 4 are shown in Table 1. Also shown in the table are the corresponding hysteresis results from above.

10. B. Narayanan, R. C. Hardie, and R. A. Muse, “Scene-based nonuniformity correction technique that exploits knowledge of the focal-plane array readout architecture,” Applied Optics **44(17)**, 3482–3491 (2005). [CrossRef]

## 4. Conclusions

## Acknowledgments

## References and links

1. | A. F. Milton, F. R. Barone, and M. R. Kruer, “Influence of non-uniformity on infrared focal plane arrays performance,” Optical Engineering |

2. | Y. M. Chiang and J. G. Harris, “An Analog Integrated Circuit for Continuous-time Gain and Offset Calibration of Sensor Arrays,” Journal of Analog Integrated Circuits and Signal Processing |

3. | J. G. Harris and Y.-M. Chiang, “Minimizing the Ghosting Artifact in Scene-Based Nonuniformity Correction,” in |

4. | M. M. Hayat, S. N. Torres, E. E. Armstrong, S. C. Cain, and B. J. Yasuda, “Statistical Algorithm for Nonuniformity Correction in Focal-plane Arrays,” Applied Optics |

5. | S. N. Torres and M. M. Hayat, “Kalman Filtering for Adaptive Nonuniformity Correction in Infrared Focal Plane Arrays,” The Journal of the Optical Society of America A |

6. | D. A. Scribner, K. A. Sarkady, M. R. Kruer, J. T. Caulfield, J. D. Hunt, and C. Herman, “Adaptive Nonuniformity Correction for IR Focal Plane Arrays using Neural Networks,” in |

7. | D. A. Scribner, K. A. Sarkady, M. R. Kruer, J. T. Caulfield, J. Hunt, M. Colbert, and M. Descour, “Adaptive Retina-like Preprocessing for Imaging Detector Arrays,” vol. 3, pp. 1955–1960 (IEEE International Conference on Neural Networks, San Francisco, CA, 1993). |

8. | S. N. Torres, E. M. Vera, R. A. Reeves, and S. K. Sobarzo, “Adaptive Scene-Based Nonuniformity Correction Method for Infrared Focal Plane Arrays,” in |

9. | E. M. Vera and S. N. Torres, “Fast Adaptive Nonuniformity Correction for Infrared Focal-Plane Array Detectors,” EURASIP Journal on Applied Signal Processing |

10. | B. Narayanan, R. C. Hardie, and R. A. Muse, “Scene-based nonuniformity correction technique that exploits knowledge of the focal-plane array readout architecture,” Applied Optics |

11. | R. C. Hardie, M. M. Hayat, E. E. Armstrong, and B. J. Yasuda, “Scene-based Nonuniformity Correction with Video Sequences and Registration,” Applied Optics |

12. | B. M. Ratliff, M. M. Hayat, and R. C. Hardie, “An Algebraic Algorithm for Nonuniformity Correction in Focal Plane Arrays,” The Journal of the Optical Society of America A |

13. | R. C. Hardie and D. R. Droege, “A MAP Estimator for Simultaneous Super-Resolution and Detector Nonuniformity Correction,” EURASIP Journal on Advances in Signal Processing, Article ID 89354 2007 (2007). [CrossRef] |

**OCIS Codes**

(100.0100) Image processing : Image processing

**ToC Category:**

Image Processing

**History**

Original Manuscript: May 8, 2009

Revised Manuscript: July 22, 2009

Manuscript Accepted: July 30, 2009

Published: August 7, 2009

**Citation**

Russell C. Hardie, Frank Baxley, Brandon Brys, and Patrick Hytla, "Scene-based nonuniformity correction with reduced ghosting using a gated LMS algorithm," Opt. Express **17**, 14918-14933 (2009)

http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-17-17-14918

Sort: Year | Journal | Reset

### References

- A. F. Milton, F. R. Barone, and M. R. Kruer, "Influence of non-uniformity on infrared focal plane arrays performance," Opt. Eng. 24(5), 855-862 (1985).
- Y. M. Chiang and J. G. Harris, "An Analog Integrated Circuit for Continuous-time Gain and Offset Calibration of Sensor Arrays," J. Analog Integrated Circuits Signal Proc. 12, 231-238 (1997). [CrossRef]
- J. G. Harris and Y.-M. Chiang, "Minimizing the Ghosting Artifact in Scene-Based Nonuniformity Correction," in SPIE Conference on Infrared Imaging Systems: Design Analysis, Modeling, and Testing IX, vol. 3377 (Orlando, Florida, 1998).
- M. M. Hayat, S. N. Torres, E. E. Armstrong, S. C. Cain, and B. J. Yasuda, "Statistical Algorithm for Nonuniformity Correction in Focal-plane Arrays," Appl. Opt. 38(5), 772-780 (1999). [CrossRef]
- S. N. Torres and M. M. Hayat, "Kalman Filtering for Adaptive Nonuniformity Correction in Infrared Focal Plane Arrays," J. Opt. Soc. Am. A 20(3), 470-480 (2003). [CrossRef]
- D. A. Scribner, K. A. Sarkady, M. R. Kruer, J. T. Caulfield, J. D. Hunt, and C. Herman, "Adaptive Nonuniformity Correction for IR Focal Plane Arrays using Neural Networks," in Proceedings of the SPIE: Infrared Sensors: Detectors, Electronics, and Signal Processing, T. S. Jayadev, ed., vol. 1541, pp. 100-109 (1991).
- D. A. Scribner, K. A. Sarkady, M. R. Kruer, J. T. Caulfield, J. Hunt, M. Colbert, and M. Descour, "Adaptive Retina-like Preprocessing for Imaging Detector Arrays," vol. 3, pp. 1955-1960 (IEEE International Conference on Neural Networks, San Francisco, CA, 1993).
- S. N. Torres, E. M. Vera, R. A. Reeves, and S. K. Sobarzo, "Adaptive Scene-Based Nonuniformity Correction Method for Infrared Focal Plane Arrays," in SPIE Conference on Infrared Imaging Systems: Design Analysis, Modeling, and Testing XIV, vol. 5076 (Orlando, Florida, 2003).
- E. M. Vera and S. N. Torres, "Fast Adaptive Nonuniformity Correction for Infrared Focal-Plane Array Detectors," EURASIP Journal on Applied Signal Processing 13, 1994-2004 (2005).
- B. Narayanan, R. C. Hardie, and R. A. Muse, "Scene-based nonuniformity correction technique that exploits knowledge of the focal-plane array readout architecture," Appl. Opt. 44(17), 3482-3491 (2005). [CrossRef]
- R. C. Hardie, M. M. Hayat, E. E. Armstrong, and B. J. Yasuda, "Scene-based Nonuniformity Correction with Video Sequences and Registration," Appl. Opt. 39(8), 1241-1250 (2000). [CrossRef]
- B. M. Ratliff, M. M. Hayat, and R. C. Hardie, "An Algebraic Algorithm for Nonuniformity Correction in Focal Plane Arrays," J.Opt. Soc. Am. A 19(9), 1737-1747 (2002). [CrossRef]
- R. C. Hardie and D. R. Droege, "A MAP Estimator for Simultaneous Super-Resolution and Detector Nonuniformity Correction," EURASIP Journal on Advances in Signal Processing, Article ID 893542007 (2007). [CrossRef]

## Cited By |
Alert me when this paper is cited |

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

« Previous Article | Next Article »

OSA is a member of CrossRef.