Dead pixel replacement in LWIR microgrid polarimeters

doi:10.1364/OE.15.007596

Dead pixel replacement in LWIR microgrid polarimeters

Bradley M. Ratliff, J. Scott Tyo, James K. Boger, Wiley T. Black, David L. Bowers, and Matthew P. Fetrow »View Author Affiliations

Optics Express, Vol. 15, Issue 12, pp. 7596-7609 (2007)
http://dx.doi.org/10.1364/OE.15.007596

View Full Text Article

Acrobat PDF (394 KB)

Journal Search
Article Lookup

Article Tools

Citations

Alert me when this article is cited
Export Citation/Save

Article Navigation

▸ Article Outline
– Abstract
1. Introduction
2. Nearest like-polarization neighbor replacement scheme
3. Redundancy estimation replacement scheme
4. Discussion
5. Conclusion
– References and links

Article Tools
Full Text
Article Info
References
Cited By
Figures
Metrics
Download PDF

Viewing Equations in the
HTML Article

Optimal Viewing Recommendations

Full Text
Article Info
References (17)
Cited By
Figures (11)
Metrics

Abstract

LWIR imaging arrays are often affected by nonresponsive pixels, or “dead pixels.” These dead pixels can severely degrade the quality of imagery and often have to be replaced before subsequent image processing and display of the imagery data. For LWIR arrays that are integrated with arrays of micropolarizers, the problem of dead pixels is amplified. Conventional dead pixel replacement (DPR) strategies cannot be employed since neighboring pixels are of different polarizations. In this paper we present two DPR schemes. The first is a modified nearest-neighbor replacement method. The second is a method based on redundancy in the polarization measurements. We find that the redundancy-based DPR scheme provides an order-of-magnitude better performance for typical LWIR polarimetric data.

1. Introduction

Imaging polarimetry in the long-wave infrared (LWIR, 8 – 12 μm) has been developed as a remote sensing method that can help to detect and identify objects of interest that may not have significant thermal contrast. The LWIR is an especially interesting part of the optical spectrum because the source of the detected radiation is usually emission from the object that is being imaged. The emissive nature of the optical signature makes thermal LWIR images largely independent of external environmental parameters such as illumination and view angle. Polarization images in the LWIR are more dependent on the illumination geometry. Even objects that are highly emissive will have polarization signatures that depend on the reflected background, especially when the object is near thermal equilibrium [1

J. A. Shaw, “Degree of linear polarization in spectral radiances from water-viewing infrared polarimeters,” Appl. Opt. 38, 3157–3165 (1999). [CrossRef]

]. In spite of this complexity, LWIR polarization does provide the potential for day/night operation that is not afforded in other wavelength ranges.

The general problem of emissive polarimetry dates back more than a century [2

R. A. Millikan, “A study of the polarization of the light emitted by incandescnet solid and liquid surfaces. I.” Phys. Rev. 3, 81–99 (1895).

, 3

R. A. Millikan, “A study of the polarization of the light emitted by incandescnet solid and liquid surfaces. II.” Phys. Rev. 3, 177–192 (1895).

]. It is well known that emitted radiation can be polarized upon refraction out of the material through Fresnel transmission [4

O. Sandus, “A review of emission polarization,” Appl. Opt. 4, 1634–1642 (1965). [CrossRef]

]. LWIR imaging polarimetry has been studied for almost as long as LWIR cameras have been available [5

T. J. Rogne, “Passive detection using polarized components of infrared signatures,” in Proceedings of SPIE vol. 1317: Polarimetry: Radar, infrared visible, ultraviolet and X-ray, R. A. Chipman and J. W. Morris, eds., pp. 242 – 251 (SPIE, Bellingham, WA, 1990).

]. Numerous systems have been built that demonstrate the ability of LWIR polarimetry to aid in applications ranging from object detection and identification to imaging in scattering media [6

J. S. Tyo, D. H. Goldstein, D. B. Chenault, and J. A. Shaw, “Review of Passive Imaging Polarimetry for Remote Sensing Applications,” Appl. Opt. 45, 5453 – 5469 (2006). and references therein. [CrossRef] [PubMed]

Recent advances in the technology of LWIR sensing have enabled the development of integrated linear polarimeters that combine a LWIR focal plane array (FPA) with an array of wire grid micropolarizers in order to produce a division of focal plane (DoFP) polarimeter as depicted in Fig. 1. DoFP devices have been used in all regions of the optical spectrum and have been made with many different combinations of polarization patterns [7

A. G. Andreou and Z. K. Kalayjian, “Polarization imaging: principles and integrated polarimeters,” IEEE Sens. J. 2, 566 – 576 (2002). [CrossRef]

]. Importantly, these devices are usually mechanically rugged, inherently aligned optomechanically, and temporally synchronized, making them ideal for deployment.

Focal plane array (FPA) sensors all suffer from the common problem of unresponsive pixels. These pixels are often termed “dead pixels” and affect the quality of both the visual image and the underlying data. Dead pixels are those pixels whose measurement does not have any correlation with the true scene that is being measured. They severely degrade the quality of measured imagery and require that their measurements be replaced with more appropriate values to improve image quality to an acceptable level that depends on the subsequent processing and display that the image is subjected to. Dead pixels impact the performance of all FPA sensors, but are especially problematic in the LWIR regime. Though the problem of dead pixel replacement (DPR) is well understood for existing FPA imagery [8

D. L. Perry and E. L. Dereniak, “Linear theory of nonuniformity correction in infrared staring sensors,” Opt. Eng. 32, 1854–1859 (1993). [CrossRef]

], these techniques do not directly apply to imagery obtained from microgrid polarimeters because neighboring pixels are masked with different polarization filters.

Fig. 1. Layout of the microgrid FPA depicting the 2 × 2 superpixels that contain all four micro-polarizer orientations.

Polarization imagery estimated from microgrid arrays is particularly sensitive to dead pixels, largely due to the spatial differencing that is required to obtain the polarization estimates. Standard FPA DPR techniques, such as nearest-neighbor, mean or median value replacement[8

D. L. Perry and E. L. Dereniak, “Linear theory of nonuniformity correction in infrared staring sensors,” Opt. Eng. 32, 1854–1859 (1993). [CrossRef]

], cannot be directly applied to microgrid imagery due to the modulated polarizer measurements; a dead pixel within a microgrid-acquired image cannot be reliably replaced with a pixel from a different polarizer orientation. This complication requires that alternative schemes be developed to properly handle DPR for microgrid instruments. Such microgrid DPR strategies are presented in this paper. In particular, we present two techniques: 1) a modified nearest-neighbor approach; and 2) a microgrid-unique approach that relies on the inherent redundancy in the polarization measurements to estimate the missing intensity values.

It is important to note that determination of whether a pixel is dead or not is a significant topic in and of itself. The criteria for making such a determination can be defined in many ways and is not discussed here. For the purposes of dead pixel replacement, we assume that we have a binary image of the same dimensions as the focal plane image such that a dead pixel is indicated with a “1” and a properly functioning pixel is indicated by a “0”. This binary image is called the dead pixel map and it is assumed that this map already exists in all replacement strategy discussions below.

The remainder of this paper is organized as follows. Section 2 discusses a DPR method based on nearest-neighbor replacement. Section 3 presents a more advanced method that allows dead pixels to be replaced by exploiting redundancy. Section 4 discusses and compares the performance of these two methods. Conclusions are presented in section 5.

2. Nearest like-polarization neighbor replacement scheme

One of the fastest and simplest DPR schemes that is commonly used on standard FPA imagery is nearest neighbor replacement. In this scheme a dead pixel’s value is simply replaced with the value of one of its neighboring pixels. The only condition is that the replacing pixel not be a dead pixel itself. This method cannot be directly applied to the microgrid sensor because, for a given pixel, all of its adjacent neighbors contain measurements that were obtained through different polarization filters as shown in Fig. 1. We must therefore put the additional constraint that the dead pixel’s value only be replaced with the value of a pixel with the same polarization. This so-called nearest like-polarization neighbor (NLPN) replacement is visualized in Fig. 2. Notice that only pixels of the same polarizer orientation as the dead pixel are colored blue. The darker blue pixels are the best candidates for replacement because they are the closest to the dead pixel. In the case that the nearest pixels are themselves dead, we would then continue examining pixels of like polarization until we find the closest pixel of like polarization that is not dead itself.

Fig. 2. Candidate pixels used for replacement of a dead pixel in the nearest like-polarization neighbor (NLPN) replacement scheme. The darker blue pixels are chosen first because they have the closest Euclidean distance to the dead pixel. The algorithm will select pixels farther away only when the closest ones are themselves dead.

The NLPN method has similar advantages and disadvantages to nearest-neighbor methods for conventional, non-polarized arrays. The NLPN scheme is advantageous in that it is very fast from an implementation standpoint. One reason for this is that the closest not-dead-like-polarization pixel to be used for replacement can be determined a priori. Thus, a given dead pixel map can be algorithmically analyzed and all replacement pixels determined. This only has to be done once so that fewer computations are needed at run-time. In our implementation we replace each “1” in the dead pixel map with the relative offset from the dead pixel to the index of the replacement pixel. It is also worth noting that images can be represented as 1D arrays and we thus compute only a 1D offset. This is also useful in that we can represent each replacement pixel with a single number, rather than keeping track of two separate index offsets. Again, these relative offsets are computed apriori so that these computations do not have to be made at run-time, and is a clear advantage of the NLPN technique.

The scheme can be disadvantageous in regions were there are large clusters of dead pixels, resulting in poor replacements due to the scheme having to replace these pixels with values that are far away. When the image is relatively flat in intensity content, NLPN replacement will still work well, but when there are large intensity gradients in the scene dead pixel clusters tend to persist in the imagery due to the significant deviation in the replacement value. An obvious disadvantage of the NLPN method compared with nearest-neighbor techniques for conventional FPAs is that the replacement pixel is always at least √2 pixels away from the dead pixel.

In addition to arbitrary choice of the NLPN, it is possible to implement a replacement that averages over like polarization neighbors or randomly selects among equal-distant like polarization neighbors. These alterations impact the performance of the DPR method in predictable ways using linear systems analysis. We prefer to not use NLPN schemes based on averaging because of the superior performance of the method presented in Sec. 3 below.

3. Redundancy estimation replacement scheme

Another DPR approach that yields performance superior to the NLPN technique is a scheme that we call redundancy estimation (RE). Estimation of the three linear Stokes parameters requires that three appropriate intensity measurements be made. In the case of the microgrid depicted in Fig. 1 we have four intensity measurements (at 0°,45°,90° and 135°), and is one more measurement than is required to estimate the first three elements of the Stokes vector. It is common to use four linear polarization measurements to estimate the linear Stokes parameters [9

R. Walraven, “Polarization Imagery,” Opt. Eng. 20, 14 – 18 (1981).

], and the additional measurement – if chosen optimally [10

J. S. Tyo, “Optimum Linear Combination Strategy For A N-Channel Polarization Sensitive Vision Or Imaging System,” J. Opt. Soc. Am. A 15, 359–366 (1998). [CrossRef]

] – can enhance the robustness of the polarization estimate [11

J. S. Tyo, “Design of optimal polarimers: maximization of SNR and minimization of systematic errors,” Appl. Opt. 41, 619–630 (2002). [CrossRef] [PubMed]

, 12

D. S. Sabatke, M. R. Descour, E. Dereniak, W. C. Sweatt, S. A. Kemme, and G. S. Phipps, “Optimization of Retardance for a Complete Stokes Polarimeter,” Opt. Lett. 25, 802–804 (2000). [CrossRef]

]. Furthermore, four measurements fit conveniently within the rectangular layout of our FPA.

Because of our four linear polarization measurements, we have one more measurement than is required to estimate the Stokes vector. This inherent redundancy in microgrid imagery can be exploited to our advantage. We do this by realizing that a given intensity measurement can be estimated from three of its neighboring pixels, each of a different polarizer orientation.

3.1. Determining redundancy

The device that we are working with only measures the linear polarization parameters. We define the reduced dimensionality Stokes vector as

\hat{S} = [\begin{matrix} {\hat{s}}_{0} \\ {\hat{s}}_{1} \\ {\hat{s}}_{2} \end{matrix}] = [\begin{matrix} \frac{1}{2} (I_{0} + I_{90} + I_{45} + I_{135}) \\ I_{0} - I_{90} \\ I_{45} - I_{135} \end{matrix}],

(1)

where I_x is the intensity estimated for a polarizer oriented in direction x at the specific spatial location where the Stokes vector is being estimated. The estimation scheme can be one of many that have been designed to mitigate IFOV errors [13

B. M. Ratliff, J. K. Boger, M. P. Fetrow, J. S. Tyo, and W. T. Black, “Image processing methods to compensate for IFOV errors in microgrid imaging polarimeters,” in Proc. SPIE vol. 6240: Polarization: Measurement, Analysis, and Remote Sensing VII, D. H. Goldstein and D. B. Chenault, eds., p. 6240OE (SPIE, Bellingham, WA, 2006).

], and the choice of interpolation method is not discussed here.

For the purposes of this discussion, we will assume that the micropolarizers are ideal. This means that their extinction ratios are much greater than 1. To begin the derivation of the RE technique, we start with the generalized Mueller matrix for a linear polarizer,

M_{LP} = \frac{1}{2} [\begin{matrix} A & B \cos D & B \sin D & 0 \\ B \cos D & A \cos^{2} D + C \sin^{2} D & (A - C) \sin D \cos D & 0 \\ B \sin D & (A - C) \sin D \cos D & A \sin^{2} D + C \cos^{2} D & 0 \\ 0 & 0 & 0 & 2 C \end{matrix}]

(2)

where A = q+r, B = q - r, C =

2 \sqrt{q r}

, D = 2θ, q is the major transmittance, r is the minor transmittance and θ is the orientation angle of the linear polarizer. Since each micropolarizer is assumed ideal, q = 1, r = 0 and θ = 0°,45°,90° and 135°. If we substitute these parameters into Eq. (2) we get the four Mueller matrices corresponding to each polarizer orientation,

M_{0} = \frac{1}{2} [\begin{matrix} 1 & 1 & 0 & 0 \\ 1 & 1 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{matrix}] M_{45} = \frac{1}{2} [\begin{matrix} 1 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 \\ 1 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 \end{matrix}]

M_{90} = \frac{1}{2} [\begin{matrix} 1 & - 1 & 0 & 0 \\ - 1 & 1 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{matrix}] M_{135} = \frac{1}{2} [\begin{matrix} 1 & 0 & - 1 & 0 \\ 0 & 0 & 0 & 0 \\ - 1 & 0 & - 1 & 0 \\ 0 & 0 & 0 & 0 \end{matrix}] .

(3)

For each point in the scene we can associate a Stokes vector S = [s ₀ s ₁ S ₂ S ₃]^T. To determine the measured intensity by the FPA detector at a particular polarizer orientation we take I _θ = M _θ S, where I _θ = [I_θ ¹ I_θ ² I_θ ³ I_θ ⁴]. Under these assumptions, we can write the intensity measured by each pixel as

I_{0} = \frac{1}{2} ({\hat{s}}_{0} + {\hat{s}}_{1}); I_{45} = \frac{1}{2} ({\hat{s}}_{0} + {\hat{s}}_{2})

I_{90} = \frac{1}{2} ({\hat{s}}_{0} - {\hat{s}}_{1}); I_{135} = \frac{1}{2} ({\hat{s}}_{0} - {\hat{s}}_{2}) .

(4)

Using the four above relationships we find that we can solve for ŝ ₀, ŝ ₁ and ŝ ₂ in multiple ways to yield the following set of equations:

{\hat{s}}_{0} = I_{0} + I_{90}; {\hat{s}}_{0} = I_{45} + I_{135}

{\hat{s}}_{1} = I_{0} - I_{90}; {\hat{s}}_{0} = 2 I_{0} - I_{45} - I_{135}; {\hat{s}}_{1} = I_{45} - 2 I_{90} + I_{135}

{\hat{s}}_{2} = I_{45} - I_{135}; {\hat{s}}_{2} = 2 I_{45} - I_{0} - I_{90}; {\hat{s}}_{2} = I_{0} - 2 I_{45} + I_{90} .

(5)

Thus, there are two ways to compute s ₀ and three ways to compute both s ₁ and s ₂. Notice in each case that all four intensity measurements are not used. Rearranging these equations we find that each intensity term can be expressed in terms of the intensities of the three other orientations such that

{\bar{I}}_{0} = I_{45} - I_{90} + I_{135}; {\bar{I}}_{45} = I_{0} + I_{90} - I_{135}

{\bar{I}}_{90} = - I_{0} + I_{45} + I_{135}; {\bar{I}}_{135} = I_{0} - I_{45} + I_{90} .

(6)

With the above relationships we now have a means for estimating a given pixel’s value from three of its neighbors of differing orientation. The utility of this for DPR is clear.

3.2. DPR implementation

To begin the development of a DPR replacement algorithm using the above redundancy relationships, we first discuss the case of a dead pixel where all of its neighboring pixels are not dead. The first observation to make is that a given pixel always has neighbors that consist of four pixels of one orientation, two pixels of another orientation with the final two being of yet another orientation, as shown in Fig. 1. The polarizer orientation of all of these pixels are different from the orientation of the dead pixel itself. When all of these neighbors are not dead we first average each group of pixels of like orientation. Then, we simply use these averaged pixel values with the appropriate case in Eq. (6) to estimate the dead pixel’s value. In the case when some of the neighboring pixels are dead themselves we must first analyze the neighbors to determine if there is at least one pixel from each of the three required orientations. If this is true, then we average all pixels of like orientation that are not dead and estimate the missing value. When pixels from the three orientations are not available we cannot yet estimate the dead pixel’s value, but will be able to at a later time once the respective neighboring dead pixels are replaced. Thus, this DPR scheme may require that multiple passes be made to estimate all dead pixel values. Fig. 3 demonstrates some example cases that show when estimation is both possible and not possible when neighboring dead pixels are present.

When there are dead pixels that cannot be replaced on a given iteration of the algorithm we must continually reapply the algorithm until all dead pixels are replaced. In order to guarantee that this happens we must keep track of which pixels are replaced during a given iteration and then update the current dead pixel map at the end of the pass. Updating the dead pixel map at the end of the current iteration (rather than at the time when the dead pixel is replaced) will cause the algorithm to replace dead pixels in a particular order. For example, say that there is a cluster of dead pixels that is 3×3 in size and the algorithm iterates in a row/column fashion beginning at the top-left corner of the image. If the dead pixel map is updated at the time when replacement occurs then this entire cluster would be replaced in a single iteration. While this is desirable computationally it tends to be less accurate for replacement because it causes the replacement values to be biased with information from pixels above and to the left of the dead pixel cluster. If the dead pixel map is updated at the end of the iteration, this cluster would instead require three iterations to correct. In the first iteration only the four corner pixels would be corrected. The second iteration would then estimate the four other outer pixels and the third would then estimate the center pixel. This approach is more desirable in that it works in an outside/inward way that incorporates values from all pixels surrounding the cluster and yields better results in the replacement. Figure 4 shows some sample dead pixel patterns and the iteration on which each pixel is replaced using this latter update rule.

Fig. 3. Example cases illustrating when RE can/cannot estimate the value of a given dead pixel (indicated with an “x”). The requirement is that there be at least one good neighboring pixel from each of the three polarizer orientations opposite the dead pixel’s.

The RE strategy is advantageous in that it results in highly accurate estimates of the missing dead pixel values and is particularly superior to the NLPN technique. Its disadvantage is that it requires that the estimates be computed at run-time and therefore is not as computationally efficient as the NLPN technique. Though we don’t discuss the details here, there are a number of computations that can be made a priori to reduce the number of decisions that must be made at run-time, such as which iteration a pixel will be updated on and what not-dead neighbors it should use in the computation.

4. Discussion

4.1. DPR scheme comparison

In this section we apply both the NLPN and RE techniques to real microgrid data and compare their performance. Each technique is demonstrated on the multi-point calibrated image of Fig. 5(b). This image was acquired with a LWIR microgrid polarimeter. The FPA was manufactured by DRS Sensors and Tracking Systems, and operated in the 8 – 10 μm range. Details of the development, calibration, and operation of the sensor are presented elsewhere [14

J. K. Boger, J. S. Tyo, B. M. Ratliff, M. P. Fetrow, W. Black, and R. Kumar, “Modeling precision and acuracy of a LWIR microgrid array imaging polarimeter,” in Proc. SPIE vol. 5888: Polarization Science and Remote Sensing II, J. A. Shaw and J. S. Tyo, eds. (SPIE, Bellingham, WA, 2005). In Press.

, 15

D. Bowers, J. K. Boger, L. D. Wellens, W. T. Black, S. E. Ortega, B. M. Ratliff, M. P. Fetrow, J. E. Hubbs, and J. S. Tyo, “Evaluation and display of polarimetric image data using long-wave cooled microgrid focal plane arrays,” in Proc. SPIE vol. 6240: Polarization: Measurement, Analysis, and Remote Sensing VII, D. H. Goldstein and D. B. Chenault, eds., p. 6240OF (SPIE, Bellingham, WA, 2006).

]. We have a dead pixel map associated with this sensor obtained from the calibration procedure as shown in Fig. 6. Notice that in addition to there being a significant number of dead pixels throughout the image that there are several large clusters of dead pixels as well as a column where nearly half of the pixels are dead. For this sensor 2.9% of the pixels are flagged as being dead.

Fig. 4. Sample dead pixel patterns and the iteration that the dead pixel is replaced under the RE DPR scheme.

Fig. 5. Test images: (a) raw uncalibrated microgrid image; (b) image after multi-point calibration but before dead pixel replacement. The red-outlined regions are sub-images that are investigated in greater detail below.

Fig. 6. Dead pixel map for the sensor.

Fig. 7. Regions of the multi-point calibrated image of Fig. 5.b: (a) Image region (1,200)×(230,430) and (b) image region (401,590)×(351,480).

To better observe the dead pixels after calibration Fig. 7 shows two regions of this image as indicated by the boxes in Fig. 5(b). In each region there are a significant number of dead pixels that are observed. Figure 8 shows the images that result from applying the NLPN and RE techniques to the image of Fig. 5(b). In both images notice that the results appear quite similar at first glance. To better see the resulting images Fig. 9 shows the same regions as Fig. 7 for each DPR technique. Once again, the resulting images from each technique appear quite similar, though in the NLPN-applied image we see that there are some values near edges that are not well-replaced. In particular, there are still dead pixels that can be observed near the truck grill and bumper at regions where there is strong thermal contrast in Fig. 9(a) and near the edges of the positioned plates in Fig. 9(c). In the RE-applied image there are no remaining dead pixels that are noticeable.

Fig. 8. The multi-point calibrated image of Fig. 5.b after application of the (a) NLPN and (b) RE replacement schemes.

Fig. 9. Zoomed regions of the NLPN and RE corrected images of Fig. 8: (a) NLPN and (b) RE image region (1,200)×(230,430); (c) NLPN and (d) RE image region (401,590)×(351,480).

Fig. 10. Histogram of the relative error of the replaced pixels using the two methods discussed in this paper. The relative error was computed by replacing good pixels with the values predicted using the NLPN and RE schemes, then comparing to the actual value at that pixel.

In order to quantify the performance of the two methods, we performed a simulation experiment. Starting with the calibrated image in Fig. 5(b), we randomly eliminated 5.8% of the good pixels, resulting in a total of 8.7% dead pixels in the image. For all pixels that were not dead, we can then compare the estimated intensity to the original intensity. Histograms of the performance of the two methods are presented in Fig. 10, and statistical data is tabulated in Table 1.

Table 1. Statistics of the normalized error of the DPR schemes.

method	mean	St. Dev.
NLPN	-0.005%	3.6%
RE	-0.002%	0.43%

We see from Fig. 10 and Table 1 that both DPR methods produce unbiased estimators of the actual missing measured intensity. However, the RE scheme results in almost an order of magnitude improvement in performance over the NLPN method. It should be noted that these results are for a particular measured image. However, this image is diverse, containing both low- and high spatial frequency variations throughout the scene. We expect that both methods will perform equally well in regions of low spatial frequency, with the NLPN performing worse near strong gradients.

4.2. Impact on polarimetric images

While both techniques perform well at replacing the dead pixels in the raw multi-point calibrated image, poorly replaced dead pixels can reemerge in the polarization images. This is due to the polarization images being more sensitive to incorrect values because of the differencing that is required in their estimation. To see this the DoLP images are computed for each of the zoomed regions of Fig. 9 and displayed in Fig. 11. In each NLPN-DoLP image notice that many of the dead pixels are clearly visible whereas in the RE-DoLP image the dead pixels are not. We find that in general the RE DPR scheme yields better replacement values for dead pixels than the NLPN technique at the cost of being slightly more computationally expensive. The RE technique should thus always be employed when possible with the NLPN technique being reserved for cases when computational speed is paramount. It is also worth noting the strong edge artifacts that appear in each DoLP image. These are manifestations of IFOV error, as mentioned above. For simplicity of discussion we made no attempt to remove the artifacts within these images, but the interested reader should reference our other work on this topic [13

Fig. 11. Zoomed regions of the DoLP images computed using the no-interpolation method from the NLPN and RE corrected images of Fig. 8: (a) NLPN and (b) RE DoLP image region (1,200)×(230,430); (c) NLPN and (d) RE DoLP image region (401,590)×(351,480).

4.3. Alternate interpretations

It is important to note that the DPR schemes presented above can not add information to the image that was lost due to the dead pixel in question. The methods that we have presented can be thought of as ways of minimizing the impact of the dead pixels on the polarization products that are computed from the raw data. Consider the case of a superpixel that has three good pixels and one dead pixel. Without loss of generality, we assume that the dead pixel is horizontally polarized. If we compute ŝ ₁ at the center of this super pixel using our replaced value of I ₀, we have

{\hat{s}}_{1} = {\bar{I}}_{0} - I_{90}

(7)

= I_{45} + I_{135} - 2 I_{90}

(8)

= {\hat{s}}_{0} - 2 I_{90},

(9)

where we have estimated s ₀ in (9) using only I ₄₅ and I ₁₃₅. This is the standard method for computing the Stokes vector from a sub-optimal selection of three linear polarization states, and is widely used [16

L. B. Wolff, “Polarization Camera For Computer Vision With A Beam Splitter,” J. Opt. Soc. Am. A 11, 2935–2945 (1994). [CrossRef]

, 10

J. S. Tyo, “Optimum Linear Combination Strategy For A N-Channel Polarization Sensitive Vision Or Imaging System,” J. Opt. Soc. Am. A 15, 359–366 (1998). [CrossRef]

We see then that our DPR scheme is functionally equivalent to identifying the dead pixels and then changing the computation method for estimating the Stokes vector in the vicinity of the dead pixels. We prefer the present implementation of Sec. 3.2 for the following reasons. First of all, it is numerically more efficient to use the same mathematical operator to compute the Stokes vector at every pixel in the scene. Use of a pixel-by-pixel variation would require more memory and more complicated processing algorithms. Second, it is often advantageous to use more advanced interpolation schemes that compensate for IFOV error in computing the Stokes vector[13

]. The multi-pass processing scheme allows the choice of the interpolation method to be changed quickly without impacting other features of the image processing system. These considerations are important when generating real-time polarimetric products from the microgrid polarimeter.

5. Conclusion

The problem of dead pixels affects all LWIR imagers, and is a difficult problem in the calibration of these devices. The dead pixel represents a fundamental loss of information that cannot be recovered. DPR methods have been developed for LWIR thermal imagers that make assumptions about the spatial variation within the scene, and a nearest-neighbor replacement method is widely used for conventional imagery. When the LWIR imager is integrated with a microgrid polarizer array to form a DoFP polarimeter, the problem of dead pixels is exacerbated. The assumptions about the similarity of neighboring pixels are now affected by the fact that the closest neighbors are operating at a different polarization state as indicated in Fig. 1.

In this paper we discuss two possible DPR methods that recognize the difficulties associated with the DoFP imager strategy. The NLPN scheme replaces a dead pixel with the value at the nearest non-dead pixel of the same polarization state (or an average of equal-distant pixels). This method is easy to visualize and fast to run. However, the fidelity of the replacement is expected to be harmed by the fact that the replacing value is no nearer than √2 pixels away, as shown in Fig. 2. This greater distance can provide significant problems in areas of rapid spatial variation.

Linear DoFP polarimeters that use four measurements to estimate the three linear Stokes paramaters have the advantage of redundancy. The loss of a single pixel is not overly detrimental, as the intensity at that pixel can be estimated by using the other three measurements in the superpixel. We developed the expressions for this RE method, and demonstrated the superior performance when compared with the NLPN scheme for a typical LWIR polarimetric image.

The polarimetric redundancy of this polarimeter device enabled the RE scheme to be developed. If only three measurements had been made instead of four, then “complete” replacement could not be accomplished (complete replacement is really only possible in areas of the image that are spatially flat). If instead of a linear polarimeter we had used the four pixels to make four independent measurements in an attempt to reconstruct the full Stokes vector[7

A. G. Andreou and Z. K. Kalayjian, “Polarization imaging: principles and integrated polarimeters,” IEEE Sens. J. 2, 566 – 576 (2002). [CrossRef]

, 17

G. P. Nordin, J. T. Meier, P. C. Deguzman, and M. . Jones, “Diffractive optical element for Stokes vector measurement with a focal plane array,” in Proceedings of SPIE vol. 3754, Polarization Measurement, Analysis, and Remote Sensing II, D. H. Goldstein and D. B. Chenault, eds., pp. 169–177 (SPIE, Bellingham, WA, 1999).

], then we would no longer be overdetermined. Such a case is not completely hopeless. It has been shown that four independent polarization measurements within the Poincarè sphere cannot be completely orthogonal [11

J. S. Tyo, “Design of optimal polarimers: maximization of SNR and minimization of systematic errors,” Appl. Opt. 41, 619–630 (2002). [CrossRef] [PubMed]

]. This means that the missing measurement may still be partially correlated with the other three measurements if the states are chosen properly.

The RE DPR method described here has another potential impact. One of the inherent difficulties associated with the DoFP method is the IFOV error that is inherent in the design. Many methods have been developed to mitigate these effects that are based on interpolation strategies [7

A. G. Andreou and Z. K. Kalayjian, “Polarization imaging: principles and integrated polarimeters,” IEEE Sens. J. 2, 566 – 576 (2002). [CrossRef]

, 13

]. The RE DPR scheme might be adapted to allow the IFOV problems to be corrected using nearby pixels that are not of the same polarization, unlike previous interpolation methods that have been presented. This is the subject of ongoing work.

Acknowledgments

J. S. Tyo and J. K. Boger were supported in part by the Air Force Office of Scientific Research under award #FA9550-05-1-0090 and the National Science Foundation under award # 0238309.

References and links

1.	J. A. Shaw, “Degree of linear polarization in spectral radiances from water-viewing infrared polarimeters,” Appl. Opt. 38, 3157–3165 (1999). [CrossRef]
2.	R. A. Millikan, “A study of the polarization of the light emitted by incandescnet solid and liquid surfaces. I.” Phys. Rev. 3, 81–99 (1895).
3.	R. A. Millikan, “A study of the polarization of the light emitted by incandescnet solid and liquid surfaces. II.” Phys. Rev. 3, 177–192 (1895).
4.	O. Sandus, “A review of emission polarization,” Appl. Opt. 4, 1634–1642 (1965). [CrossRef]
5.	T. J. Rogne, “Passive detection using polarized components of infrared signatures,” in Proceedings of SPIE vol. 1317: Polarimetry: Radar, infrared visible, ultraviolet and X-ray, R. A. Chipman and J. W. Morris, eds., pp. 242 – 251 (SPIE, Bellingham, WA, 1990).
6.	J. S. Tyo, D. H. Goldstein, D. B. Chenault, and J. A. Shaw, “Review of Passive Imaging Polarimetry for Remote Sensing Applications,” Appl. Opt. 45, 5453 – 5469 (2006). and references therein. [CrossRef] [PubMed]
7.	A. G. Andreou and Z. K. Kalayjian, “Polarization imaging: principles and integrated polarimeters,” IEEE Sens. J. 2, 566 – 576 (2002). [CrossRef]
8.	D. L. Perry and E. L. Dereniak, “Linear theory of nonuniformity correction in infrared staring sensors,” Opt. Eng. 32, 1854–1859 (1993). [CrossRef]
9.	R. Walraven, “Polarization Imagery,” Opt. Eng. 20, 14 – 18 (1981).
10.	J. S. Tyo, “Optimum Linear Combination Strategy For A N-Channel Polarization Sensitive Vision Or Imaging System,” J. Opt. Soc. Am. A 15, 359–366 (1998). [CrossRef]
11.	J. S. Tyo, “Design of optimal polarimers: maximization of SNR and minimization of systematic errors,” Appl. Opt. 41, 619–630 (2002). [CrossRef] [PubMed]
12.	D. S. Sabatke, M. R. Descour, E. Dereniak, W. C. Sweatt, S. A. Kemme, and G. S. Phipps, “Optimization of Retardance for a Complete Stokes Polarimeter,” Opt. Lett. 25, 802–804 (2000). [CrossRef]
13.	B. M. Ratliff, J. K. Boger, M. P. Fetrow, J. S. Tyo, and W. T. Black, “Image processing methods to compensate for IFOV errors in microgrid imaging polarimeters,” in Proc. SPIE vol. 6240: Polarization: Measurement, Analysis, and Remote Sensing VII, D. H. Goldstein and D. B. Chenault, eds., p. 6240OE (SPIE, Bellingham, WA, 2006).
14.	J. K. Boger, J. S. Tyo, B. M. Ratliff, M. P. Fetrow, W. Black, and R. Kumar, “Modeling precision and acuracy of a LWIR microgrid array imaging polarimeter,” in Proc. SPIE vol. 5888: Polarization Science and Remote Sensing II, J. A. Shaw and J. S. Tyo, eds. (SPIE, Bellingham, WA, 2005). In Press.
15.	D. Bowers, J. K. Boger, L. D. Wellens, W. T. Black, S. E. Ortega, B. M. Ratliff, M. P. Fetrow, J. E. Hubbs, and J. S. Tyo, “Evaluation and display of polarimetric image data using long-wave cooled microgrid focal plane arrays,” in Proc. SPIE vol. 6240: Polarization: Measurement, Analysis, and Remote Sensing VII, D. H. Goldstein and D. B. Chenault, eds., p. 6240OF (SPIE, Bellingham, WA, 2006).
16.	L. B. Wolff, “Polarization Camera For Computer Vision With A Beam Splitter,” J. Opt. Soc. Am. A 11, 2935–2945 (1994). [CrossRef]
17.	G. P. Nordin, J. T. Meier, P. C. Deguzman, and M. . Jones, “Diffractive optical element for Stokes vector measurement with a focal plane array,” in Proceedings of SPIE vol. 3754, Polarization Measurement, Analysis, and Remote Sensing II, D. H. Goldstein and D. B. Chenault, eds., pp. 169–177 (SPIE, Bellingham, WA, 1999).

OCIS Codes
(110.3080) Imaging systems : Infrared imaging
(120.2130) Instrumentation, measurement, and metrology : Ellipsometry and polarimetry
(120.5410) Instrumentation, measurement, and metrology : Polarimetry

ToC Category:
Instrumentation, Measurement, and Metrology

History
Original Manuscript: April 2, 2007
Revised Manuscript: May 25, 2007
Manuscript Accepted: June 1, 2007
Published: June 6, 2007

Citation
Bradley M. Ratliff, J. Scott Tyo, James K. Boger, Wiley T. Black, David L. Bowers, and Matthew P. Fetrow, "Dead pixel replacement in LWIR microgrid polarimeters," Opt. Express 15, 7596-7609 (2007)
http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-15-12-7596

Sort: Author | Year | Journal | Reset

References

J. A. Shaw, "Degree of linear polarization in spectral radiances from water-viewing infrared polarimeters," Appl. Opt. 38, 3157-3165 (1999). [CrossRef]
R. A. Millikan, "A study of the polarization of the light emitted by incandescnet solid and liquid surfaces. I," Phys. Rev. 3, 81-99 (1895).
R. A. Millikan, "A study of the polarization of the light emitted by incandescnet solid and liquid surfaces. II," Phys. Rev. 3, 177-192 (1895).
O. Sandus, "A review of emission polarization," Appl. Opt. 4, 1634-1642 (1965). [CrossRef]
T. J. Rogne, "Passive detection using polarized components of infrared signatures," Proc. SPIE 1317, 242 - 251 (1990).
J. S. Tyo, D. H. Goldstein, D. B. Chenault, and J. A. Shaw, "Review of passive imaging polarimetry for remote sensing applications," Appl. Opt. 45, 5453 - 5469 (2006). and references therein. [CrossRef] [PubMed]
A. G. Andreou and Z. K. Kalayjian, "Polarization imaging: principles and integrated polarimeters," IEEE Sens. J. 2, 566 - 576 (2002). [CrossRef]
D. L. Perry and E. L. Dereniak, "Linear theory of nonuniformity correction in infrared staring sensors," Opt. Eng. 32, 1854-1859 (1993). [CrossRef]
R. Walraven, "Polarization Imagery," Opt. Eng. 20, 14 - 18 (1981).
J. S. Tyo, "Optimum linear combination strategy for an N-channel polarization sensitive vision or imaging system," J. Opt. Soc. Am. A 15, 359-366 (1998). [CrossRef]
J. S. Tyo, "Design of optimal polarimers: maximization of SNR and minimization of systematic errors," Appl. Opt. 41, 619-630 (2002). [CrossRef] [PubMed]
D. S. Sabatke, M. R. Descour, E. Dereniak, W. C. Sweatt, S. A. Kemme, and G. S. Phipps, "Optimization of retardance for a complete Stokes Polarimeter," Opt. Lett. 25, 802-804 (2000). [CrossRef]
B. M. Ratliff, J. K. Boger, M. P. Fetrow, J. S. Tyo, and W. T. Black, "Image processing methods to compensate for IFOV errors in microgrid imaging polarimeters," Proc. SPIE 6240, 6240OE (2006).
J. K. Boger, J. S. Tyo, B. M. Ratliff, M. P. Fetrow, W. Black, and R. Kumar, "Modeling precision and acuracy of a LWIR microgrid array imaging polarimeter," Proc. SPIE 5888, 227-238 (2005).
D. Bowers, J. K. Boger, L. D. Wellens, W. T. Black, S. E. Ortega, B. M. Ratliff, M. P. Fetrow, J. E. Hubbs, and J. S. Tyo, "Evaluation and display of polarimetric image data using long-wave cooled microgrid focal plane arrays," Proc. SPIE 6240, 6240OF (2006).
L. B. Wolff, "Polarization camera for computer vision with a beam splitter," J. Opt. Soc. Am. A 11, 2935-2945 (1994). [CrossRef]
G. P. Nordin, J. T. Meier, P. C. Deguzman, and M. Jones, "Diffractive optical element for Stokes vector measurement with a focal plane array," Proc. SPIE 3754, 169-177 (1999).

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

Click here to see a list of articles that cite this paper