Performance of infrared systems in swimmer detection for maritime security

doi:10.1364/OE.15.012296

« Show journal navigation

Performance of infrared systems in swimmer detection for maritime security

Keith Krapels, Ronald G. Driggers, and Jose F. Garcia III »View Author Affiliations

Optics Express, Vol. 15, Issue 19, pp. 12296-12305 (2007)
http://dx.doi.org/10.1364/OE.15.012296

View Full Text Article

Acrobat PDF (398 KB)

Journal Search
Article Lookup

Article Tools

Citations

Alert me when this article is cited
Export Citation/Save

Full Text
Article Info
References (5)
Cited By
Figures (8)
Metrics

Abstract

The detection of swimmer activity in harbor areas around piers and ships is an important aspect of Anti-Terrorism/Force Protection (AT/FP) sensing efforts in the U.S. Navy and Coast Guard. A series of data collections and perception experiments were conducted to validate the use of thermal target acquisition models against swimmer targets. The results were analyzed to derive the discrimination criteria necessary for sensor design for maritime force protection.

1. Introduction

The Bombing of the USS Cole in Aden harbor typifies the basis of the need for maritime force protection in the US Navy. The Navy has numerous ship assets which could be the target of terrorist action. Unfortunately, the combat systems and sensor suites of Naval ships were intended to detect, track and engage classical symmetric targets such as other combatant ships, submarines or aircraft.

While the sensors on Navy ships have capabilities against targets other than combatants, their performance is not well understood against targets such as swimmers. Also, the current capabilities to detect and track swimmers or small objects in water are manpower intensive. As a result, the Navy is desirous of developing sensor technologies intended to perform well against asymmetric threat targets with minimal manpower/manning required. The use of infrared sensors for near ship surveillance and force protection is a relatively new application. Historically, infrared sensors on surface combatants have been of two types. The first type is an optical weapon sight/targeting system for naval gunfire direction or missile direction. An example of this is the Mk57 Director for the NATO Sea Sparrow Missile System. The system has narrow field of view TV and LLLTV sensors for day and night targeting respectively. There is currently and upgrade development to replace the LLLTV sensor with a midwave infrared sensor for better all weather and night performance. The second application of infrared sensors has been the infrared search and track (IRST) system. This type of sensor was intended to detect and track aircraft and anti-ship cruise missiles at or beyond the horizon. An example is the Sirius LR-IRST being installed on Canadian and Dutch Ships. These systems are currently second generation FLIR arrays in the mid wave and long wave mechanically scanned around 360 degrees. Applications using staring arrays are under development in various countries. These systems were intended to operate with automatic detection and tracking, but not intended for surveillance of the area near the ship.

The application of staring FLIRs to near ship search and targeting is relatively new. As a result, there is not a mature, verified and validated model for performance of this sensor type at the task of detecting swimmers in the water and discriminating them from other objects. The US Army Night Vision and Electronic Sensors Directorate has a series of validated generic sensor and target acquisition models. In order to prevent duplication of effort and save the Navy considerable time and expense, it was proposed to conduct experiments to validate using the Army models for this particular Navy task. [1

M. Self, B. Miller, and D. Dixon, “Acquisition Level Definitions and Observables for Human Targets, Urban Operations, and The Global War on Terrorism,” US Army AMSRD-CER-NV-TR-235 , (2005).

]

This empirical effort was to acquire the signatures (size and ΔT) of swimmers in the water and collect infrared video clips of swimmer/diver in the water at a series of different ranges, both day and night. The video clips were for use in observer perception experiments to determine probabilities of detection of the swimmer. The signatures of the swimmers were for use in calibrating the infrared target detection models to this particular target set/task and determining the discrimination criteria.

2. Background

The performance of a military imager is currently described by the system contrast threshold function (CTF). This is the human eye CTF while the human is looking through the imager. The CTF is the visual threshold at which a sinusoid can be seen where the sinusoid extends across the field-of-view. CTF is still too difficult to measure in the laboratory, but can be related to the currently measured Minimum Resolvable Temperature (MRT) measurement. The system CTF, for a thermal imager, can be described by:

{CTF}_{sys} = \frac{{CTF}_{eye} (ξ)}{MTF (ξ)} {(1 + \frac{α^{2} σ^{2} (ξ)}{{S_{tmp}}^{2}})}^{1 ⁄ 2} [unitless]

(1)

where MTF(ξ) is the system modulation transfer function, σ(ξ) is the noise filtered in kelvin by the display and visual system in Kelvin-root seconds, S_tmp is the scene temperature difference that corresponds to half the display brightness (from zero brightness), and α is a calibration factor with units of root hertz. In the current military imager models, all systems are considered to be separable in the horizontal and vertical directions. Equation (1) provides the CTF calculation in its basic form. This general equation includes optical, detector, electronic, and display characteristics.

The system CTF provides the performance with respect to two primary conditions under which sensors operate. The first condition is a noise-limited realm where

α^{2} σ^{2} \frac{(ξ)}{{S_{tmp}}^{2}}

is larger than 1. Under these conditions, the observer can see the noise in the image as can be demonstrated in uncooled microbolometers, first generation FLIR, and second generation FLIR systems. The second condition is where

α^{2} σ^{2} \frac{(ξ)}{{S_{tmp}}^{2}}

is smaller than 1 and the noise is not visible to the observer. Under this condition, the information in the image is limited by the eye contrast threshold function and the image blur, as these are the only components of Eq. (1) that are important. Systems that operate in this realm are staring InSb sensors, MCT staring sensors and other photon detector systems that have medium to low f-numbers with an associated medium to wide field-of-view (FOV). Plenty of signal and/or integration time (resulting in many photo-electrons) is a requirement for this condition.

The procedure for calculating/predicting field range performance is shown in Fig. 1. A target is characterized by the characteristic dimension (square root of target area) in meters, source contrast, and task discrimination difficulty (N₅₀ or V₅₀). The contrast is propagated through the atmosphere and an apparent contrast is determined at the sensor. The intersection of sensor CTF and the apparent target contrast is called the “limiting frequency,” or the highest frequency resolved through the sensor at that particular contrast. In the ACQUIRE method, this limiting frequency, in cycles per milliradian, is converted to “cycles on target” by multiplying the limiting frequency by the target angular subtense (characteristic dimension divided by range). The cycles on target are compared to the N50, or discrimination criterion--sometimes called the Johnson criterion, to determine the probability. The N₅₀ is different for detection, recognition, and identification. N₅₀ gets larger as the task gets more difficult (identification requires more resolution on the target). The ratio of cycles on target to N₅₀ is input to the target transfer probability function (TTPF) and is used to determine a probability value for a specific range. This process is performed iteratively for various ranges and the probability is plotted as a function of range.

Fig. 1. ACQUIRE and TTP Process.

The TTP [2

R. Vollmerhausen, E. Jacobs, and R. Driggers, “New metric for predicting target acquisition performance,” Opt Eng. 43, 2806–2818 (2004). [CrossRef]

] approach replaces the ACQUIRE approach for most applications. It varies from the ACQUIRE approach in that a systems resolution and sensitivity is accounted for by integrating the systems CTF, as shown in Eq. (2)

V = \int_{ξ_{low}}^{ξ_{lim}} \sqrt{\frac{C_{tgt}}{{CTF}_{sys} (ξ)}} d ξ [cycles]

(2)

where the ratio of the target contrast to the system CTF is considered the excess contrast that the human can see on the target. The limits on the integral start and end where the target contrast intersects the system CTF. This integration is performed twice, once on the horizontal CTF of the system and then again on the vertical CTF of the system; Eq. (1) is assumed to be separable. The geometric mean is then taken of the results. The result is compared to the discrimination criterion, V₅₀, to determine the probability. A different TTPF is used for the ratio of V to V₅₀ than is used in the ACQUIRE process.

The TTPF is the function that converts the ratio of N to N₅₀ or the ratio of V to V₅₀ into a probability. The TTPF takes the form of

P = \frac{{(\frac{N}{N 50})}^{β}}{1 + {(\frac{N}{N 50})}^{β}} (ACQUIRE) or P = \frac{{(\frac{V}{V 50})}^{β}}{1 + {(\frac{V}{V 50})}^{β}} (TTP)

(3)

where the coefficient, β, for the TTP is 1.51+0.24(V/V₅₀). The coefficient for ACQUIRE traditionally is 2.7+0.7(N/N₅₀), but a single coefficient value of 3.8 gives very close results. The traditional ACQUIRE coefficient has been used in many sensor specifications and combat simulations, however, a great deal of real field data has been shown to match a more gradual probability curve with coefficient equal to 1.75+0.35(N/N₅₀) or a single coefficient of 2.7.

The TTP process provides a much more accurate prediction of field performance than the ACQUIRE process as has been demonstrated in numerous recognition and identification experiments. [3

J. Ratches, R. Vollmerhausen, and R. Driggers, “Target Acquisition Performance Modeling of Infrared Imaging Systems: Past, Present, and Future,” IEEE Sens. J. 1, 31–40 (2001). [CrossRef]

] Search and detection is more complicated (under clutter-limited conditions) and will be discussed in the following section.

When performing the search and detection process, the difficulty in detecting the target is highly dependent on target contrast and the competing clutter level. ACQUIRE-LC [4

B. O’ Kane, G. Page, D. Wilson, and D. Bohan, “Cycle criteria for detection of camouflaged targets,” Proceedings of the NATO Panel on Sensors and Sensor Denial by Camouflage, Concealment and Deception. (NATO, 2004).

B. O’Kane, G. Page, M. Cook, D. Bennett, D. Wilson, and D. Bohan, “Modeling the Detection of Moving Thermal Targets,” Proceedings of the Parallel Military Sensing Symposium (SENSIAC, 2004).

] was developed to predict detection probability against camouflaged targets, but has been recently modified to work against conventional targets.

Fig. 2. ACQUIRE-LC (left) and Detect05 (right) (ΔTRSS in Kelvin).

The N₅₀ required for the detection of a target in various backgrounds (and clutter levels) is shown, left side of Fig. 2, as a function of target contrast (RSS) differential temperature in Kelvin and for different background environments.

N 50 = \frac{6}{Δ T_{RSS}} + 1.5 (woodland) N 50 = \frac{0.75}{Δ T_{RSS}} + 0.75 (littoral)

(4)

The N₅₀ is the number of cycles on target required for a 50 percent probability of detection. This N₅₀ is used in the ACQUIRE process described in the previous section to convert sensor CTF and the target detection task into a probability of detection as a function of range.

The ACQUIRE-LC curve is really a signal-to-clutter detection model, where the performance of a human/sensor pair can be characterized as a function of “complexity,” which might be a soft term for clutter. The Detect05 curve is on the right in Fig. 2 and the equation is a single equation.

N 50 = 0.75 C [{(\frac{C}{Δ T_{RSS}})}^{2} + 1]

(5)

where C is the complexity. C is 1, 1.5, 2.0, and 2.7 for low, medium-low, medium, and high complexities, respectively. Figure 3 shows some examples of complexity background levels.

Fig. 3. Clutter complexity levels.

It is the intention of the military imager modeling community to transition from ACQUIRE-LC to Detect05 for both low contrast and conventional targets and for both sensor design and combat simulations. The models presented so far allow for the ACQUIRE process implementation of both ACQUIRE-LC and Detect05, but implementation in the TTP process requires the relationship between V₅₀ and N₅₀. For either curve, the relationship between V₅₀ and N₅₀ is:

V_{50} = N_{50} * 2.7 .

(6)

This equation allows the implementation of either ACQUIRE-LC or Detect05 in the TTP process.

3. Data collection

The data collection was conducted at Naval Surface Warfare Center Dahlgren Division’s waterfront. The collection ran from 6–8 June 2006. The data collection was in four parts. There was day and night up close, high resolution collection of swimmer signature data. And then there were day and night collection of video clips of the swimmer in the water at different ranges and azimuth angles. Additionally, video clips were collected with no swimmer in the field of view.

The testing area consisted of the pier, on which the sensors and data collection equipment were placed, and an inlet off of the Potomac River. The water area facing the pier had an extent of approximately 800m to the opposing shoreline. There was a channel through the center of this inlet. A channel marker was used to delineate the far right side of the test space (and is visible just inside the right side of the sensor FOV). Ranges were marked in the water with buoys on the left side of the water space. These buoys also served as the left edge marker of the test space. Figure 4 is a picture of the test range from the aspect of the sensor.

Fig. 4. Imagery Collection Site Layout

The swimmer signature data was collected at close range (25m) from a number of aspect angles. It was also both swimming (breast stroke) and treading water. The infrared imagery data was calibrated through the use of two black bodies set at 22 and 42 degree Celsius respectively.

The swimmer was ferried to a drop site by boat left in the water. The boat moved out of the field of view and the wake of the boat was allowed to subside until not visible in the FLIR imagery. Digital video clips were then captured using the camera’s digital output and a framegrabber. The safety boat would then return to the swimmer and the range/bearing data from sensor to swimmer were noted. Additionally, a single frame, with the swimmer location was captured as a location ground truth image. The swimmer location could vary dynamically due to wind and current at an observed rate of up to 1 degree/minute during portions of the data collection.

4. Sensor description

The infrared sensor used for the data collection was a Merlin midwave InSb sensor with good sensitivity (15 milliKelvin). Table 1 lists the primary sensor parameters of the sensor. Table 1 lists some of this camera’s key features and performance parameters.

Table 1. Sensor features and specifications

Feature/Parameter Description	Specification
Spectral Range	3.5–5.0 µm
Field of View (FOV)	25 mm lens: 22 by 16 degrees
Camera f/#	2.2
Measured Thermal Sensitivity	15 mK
Array Format	320×256
Detector Size	30 µm×30 µm
Detector Pitch	30 µm
Digital Video	Up to 60 Hz/12-bit resolution

The sensitivity of the sensor was measured in the laboratory where the temporal noise was measured at 15 milliKelvin. The sensitivity of the sensor was primarily limited by fixed pattern noise with a spatial noise of 46 milliKelvin. All the other noise parameters (e.g., three-dimensional noise) were negligible. The system intensity transfer function yielded a transfer value at small signals of 64.6 gray levels per Kelvin.

5. Target signatures

The swimmer used as a target was a certified diver, but he attempted to maintain the same profile for all target engagements. The swimmer wore a wet-suit, but his face was not obstructed. Figure 5(a) shows a night signature and Fig. 5(b) shows a day signature. Close signatures were analyzed for their contrast and the night contrast was calculated to be 6.4 Kelvin and the day signature was calculated to be 7.1 Kelvin. The apparent target size was estimated to be 750 square centimeters or 0.075 square meters. This was the area of the swimmer silhouette that could be seen out of the water.

Fig. 5. (a)/(b): Left picture is a night signature and right picture is a day signature.

6. Perception experiment

The imagery was processed for presentation to US Navy personnel in a search/detection observer perception experiment. It consisted of two parts. The two parts contained the day series and the night portion. The video clips were presented to the observers on a high performance, large screen laptop using a graphic user interface. Each of them was trained on the GUI use and sample imagery to learn what swimmers looked like in the FLIR video. The GUI presented observers with the video clip and they were instructed to put the cursor over the target and click on it. If there was no target present, they were instructed to select “no target.”

The video clips where 9 seconds in duration at 60 hz frame rate (540 consecutive frames). They were stored without compression. The sensors digital readout was in 320×240 format, with pixel intensity values in 12 bits. The day version had 93 imagery clips and the night version had 78 clips. There were 10 US Navy observers. Examples of the GUI are shown in Figs. 5(a) and (b) are from the day series and the night series respectively.

The imagery was processed to ensure that the display pixel MTF was not a limiting factor by interpolation using bilinear interpolation to increase their size by a factor of 2. The images on the display for the perception testing were 5.4×7.2 inches. The observer viewing distance was approximately 18 inches.

Fig. 6. (a)/(b). Screen Captures of Perception Experiment GUI from Day and Night Respectively

7. Modeling results

The military thermal imaging system performance model, NVTherm, was used to model the sensor detection performance against the swimmer at both night and day. The night data fit is shown in Fig. 7. The model was fit to the data points by placing the sensor parameters (see the sensor section) into NVTherm along with the target parameters (from the signature section) and adjusting the V₅₀ (number of integrated cycles on target for a 50% probability of detection) to fit the data. The 50% probability of detection point for the observers occurred at 832m. These results are compared in the figure. The data point at 450 meters was not used since this data point corresponded to the first range data collected and there was significant wave signatures (i.e., clutter) that competed with the target. By the time the other four data points were taken, the inlet had become calm with very little wave clutter competing with the target signature. The V₅₀ corresponding to the fit was 0.85 cycles on target, an extremely low value compared to other detection scenarios. This was a bounding fit, where a bland background is extremely low in clutter, so there is nothing competing with the target. That is, the target signature in the search process is “contrast-limited” where the contrast threshold of the eye is taxed.

Fig. 7. Night Data and NVTherm Model Fit.

The daytime results are shown in Fig. 8. In this case, the sensor model does not work as well and the target transfer probability function (TTPF) is very smooth compared to the rapid drop of performance with range of the data. Note, that in this case, the background associated with the solar reflections provided for an extremely high clutter level that made it very difficult to find the target. This realm could be considered “clutter-limited.” The V₅₀ associated with the NVTherm model fit was 4.0 integrated cycles on target. This value drives the 50% probability of detection point with range. Note that the data and the model both have the same 50% probability of detection range of 193 meters. The high clutter value could account for the dramatic drop in performance with range and future versions of NVTherm may allow for the coefficients of the TTPF to adjust for the performance rate of decline.

Fig. 8. Day Data and NVTherm Model Fit.

The swimmer detection task’s discrimination criteria, V₅₀, from the analysis of the results is a V₅₀ that ranges from 0.85 under extremely low clutter conditions (no waves or solar reflection) to 4.0 under clutter conditions (choppy waves with solar reflections). While this is a wide range of calibration values, it bounds the range performance and the conditions associated with the detection of swimmer targets.

Table 2. Results Summary

	DAY	NIGHT
Target Size (m^2)	0.075	0.075
DeltaT RSS (K)	7.1	6.4
V50	4	0.85
N50	1.5	0.31
C (Clutter Complexity)	1.9	0.41

8. Conclusions

For low clutter scenes, the target signature and the sensor noise (or contrast limit) combine to determine detection ranges. For moderate clutter scenes, the signal to clutter drives the range performance. For the night cases, the wave clutter began very small but continued to become smaller and almost negligible by the end of the night data collection. The sun and the waves during the day made for a moderate clutter environment where the target was extremely difficult to detect. Even though the conditions made for a moderate clutter case, it was obvious that the clutter could approach even higher levels. It is also worth mentioning that daytime clutter is much higher in the midwave infrared (3–5 micrometers) than in the longwave infrared (8 to 12 micrometers). This is a result of solar reflections dominating clutter in the MWIR. This is not the case in the LWIR band.

A day and night MWIR swimmer detection experiment was conducted to determine the target characteristics, task difficulty criteria, and clutter complexity value. It was demonstrated that unresolved and resolved swimmer detection is very easy in the MWIR when there is negligible clutter and is well predicted by the existing model. The swimmer in the presence of solar clutter was a much more difficult detection target. The probabilities of detection results were much steeper than the model predicts. Performance in this environment is dependent on the signal to clutter ratio. In this effort we characterized the daytime solar clutter with the DETECT05 complexity value. It is likely that detection occurred when the shape/target extent is discriminable (actually more like recognition than detection). This explains the steepness of the probability results. For the daytime case here, the resolved cycles on target at 193m (50% Probability of detection) is 3.0. This is the same as the Johnson’s criteria recognition discrimination level.

References

1.	M. Self, B. Miller, and D. Dixon, “Acquisition Level Definitions and Observables for Human Targets, Urban Operations, and The Global War on Terrorism,” US Army AMSRD-CER-NV-TR-235 , (2005).
2.	R. Vollmerhausen, E. Jacobs, and R. Driggers, “New metric for predicting target acquisition performance,” Opt Eng. 43, 2806–2818 (2004). [CrossRef]
3.	J. Ratches, R. Vollmerhausen, and R. Driggers, “Target Acquisition Performance Modeling of Infrared Imaging Systems: Past, Present, and Future,” IEEE Sens. J. 1, 31–40 (2001). [CrossRef]
4.	B. O’ Kane, G. Page, D. Wilson, and D. Bohan, “Cycle criteria for detection of camouflaged targets,” Proceedings of the NATO Panel on Sensors and Sensor Denial by Camouflage, Concealment and Deception. (NATO, 2004).
5.	B. O’Kane, G. Page, M. Cook, D. Bennett, D. Wilson, and D. Bohan, “Modeling the Detection of Moving Thermal Targets,” Proceedings of the Parallel Military Sensing Symposium (SENSIAC, 2004).

OCIS Codes
(040.2480) Detectors : FLIR, forward-looking infrared
(110.0110) Imaging systems : Imaging systems

ToC Category:
Imaging Systems

History
Original Manuscript: July 9, 2007
Revised Manuscript: August 1, 2007
Manuscript Accepted: July 31, 2007
Published: September 12, 2007

Virtual Issues
Vol. 2, Iss. 10 Virtual Journal for Biomedical Optics

Citation
Keith Krapels, Ronald G. Driggers, and Jose F. Garcia III, "Performance of infrared systems in swimmer detection for maritime security," Opt. Express 15, 12296-12305 (2007)
http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-15-19-12296

Sort: Year | Journal | Reset

References

M. Self, B. Miller, D. Dixon, "Acquisition Level Definitions and Observables for Human Targets, Urban Operations, and The Global War on Terrorism," US Army AMSRD-CER-NV-TR-235, (2005).
R. Vollmerhausen, E. Jacobs, R. Driggers, "New metric for predicting target acquisition performance," Opt Eng. 43,2806-2818 (2004). [CrossRef]
J. Ratches, R. Vollmerhausen, R. Driggers, "Target Acquisition Performance Modeling of Infrared Imaging Systems: Past, Present, and Future," IEEE Sens. J. 1, 31-40 (2001). [CrossRef]
B. O’ Kane, G. Page, D. Wilson, D. Bohan, "Cycle criteria for detection of camouflaged targets," Proceedings of the NATO Panel on Sensors and Sensor Denial by Camouflage, Concealment and Deception. (NATO, 2004).
B. O'Kane, G. Page, M. Cook, D. Bennett, D. Wilson, D. Bohan, "Modeling the Detection of Moving Thermal Targets," Proceedings of the Parallel Military Sensing Symposium (SENSIAC, 2004).

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

Click here to see a list of articles that cite this paper