## Static compressive tracking |

Optics Express, Vol. 20, Issue 19, pp. 21160-21172 (2012)

http://dx.doi.org/10.1364/OE.20.021160

Acrobat PDF (1610 KB)

### Abstract

This paper presents the Static Computational Optical Undersampled Tracker (SCOUT), an architecture for compressive motion tracking systems. The architecture uses compressive sensing techniques to track moving targets at significantly higher resolution than the detector array, allowing for low cost, low weight design and a significant reduction in data storage and bandwidth requirements. Using two amplitude masks and a standard focal plane array, the system captures many projections simultaneously, avoiding the need for time-sequential measurements of a single scene. Scenes with few moving targets on static backgrounds have frame differences that can be reconstructed using sparse signal reconstruction techniques in order to track moving targets. Simulations demonstrate theoretical performance and help to inform the choice of design parameters. We use the coherence parameter of the system matrix as an efficient predictor of reconstruction error to avoid performing computationally intensive reconstructions over the entire design space. An experimental SCOUT system demonstrates excellent reconstruction performance with 16X compression tracking movers on scenes with zero and nonzero backgrounds.

© 2012 OSA

## 1. Introduction

2. M. Lustig, D. Donoho, and J. Pauly, “Sparse MRI: The application of compressed sensing for rapid MR imaging,” Magn. Reson. Med. **58**, 1182–1195 (2007). [CrossRef] [PubMed]

## 2. Challenges in traditional optical CS architectures

3. R. Willett, R. Marcia, and J. Nichols, “Compressed sensing for practical optical imaging systems: a tutorial,” Opt. Eng. **50**, 072601 (2011). [CrossRef]

4. M. Neifeld and J. Ke, “Optical architectures for compressive imaging,” Appl. Opt. **46**, 5293, (2007). [CrossRef] [PubMed]

*M*of the projections simultaneously requires using

*M*spatial light modulators and

*M*detector elements.

## 3. The SCOUT architecture

**H**. The typical mathematical formulation for a compressive sensing system is shown in Eq. (1) and represented graphically in Fig. 3. In order to adhere to this model in an imaging context, the 2D scene and measurement arrays are lexicographically reordered into the column vectors

**f**and

**g**, respectively. Thus, any

*M*× 1 compressed measurement

**g**is obtained as: where

**f**represents the

*N*× 1 scene, and

**H**is the

*M*×

*N*system matrix, being

*M*<

*N*in order to the system to be considered compressive. The

*i*th column of

**H**describes the PSF of the

*i*th image element, whereas the

*j*th row of

**H**describes the weights of each scene location’s contribution to the

*j*th measurement. In this way, the resulting system matrix

**H**typically acts as a space-variant optical system and presents a

*block*structure, as seen in the example shown in Fig. 4.

8. W. Bajwa, J. Haupt, G. Raz, S. Wright, and R. Nowak, “Toeplitz-structured compressed sensing matrices,” in *Proceedings of IEEE Workshop on Statistical Signal Processing*, (IEEE, 2007), pp. 294–298. [CrossRef]

10. J. Romberg, “Compressive sensing by random convolution,” SIAM J. Imaging Sci. **2**, 1098–1128 (2009). [CrossRef]

11. F. Sebert, Y. Zou, and L. Ying, “Toeplitz block matrices in compressed sensing and their applications in imaging,” in Proceedings of IEEE International Conference on Information Technology and Applications in Biomedicine, (IEEE, 2008), pp. 47–50. [CrossRef]

*f*is focused at infinity and placed some distance

*f*+

*d*from the sensor. Two binary occlusion masks,

_{im}*m*

_{1}and

*m*

_{2}, are placed at distances

*d*

_{m1}and

*d*

_{m2}from the sensor. Masks

*m*

_{1}and

*m*

_{2}achieve fill factors

*f*

_{1}and

*f*

_{2}using randomly-patterned occluders of pitch

*p*

_{1}and

*p*

_{2}, respectively. The sensor captures images at resolution

*r*×

_{x}*r*; adjacent frames are subtracted, and frame differences are reconstructed at some higher resolution

_{y}*R*×

_{x}*R*using a sparsity-favoring reconstruction algorithm. Such algorithms require knowledge of the

_{y}*r*×

_{x}r_{y}*R*system matrix

_{x}R_{y}**H**, which is determined experimentally.

## 4. Simulation performance

### 4.1. Simulating a SCOUT System

*R*×

_{x}*R*; the lens is modeled as a single thin lens with transmittance function

_{y}**t**

*and the two masks have transmittance functions*

_{f}**t**

_{1}and

**t**

_{2}. The mask transmittances are 0 where the mask is black and 0.88 where the mask is clear based on measurements. The lateral magnification of the scene and the two masks is calculated using similar triangles.

*r*×

_{x}*r*PSF from each scene location in order to obtain the system matrix

_{y}**H**. Once

**H**is known, we use it to simulate the low resolution measurements,

**g**, of the higher resolution scenes,

**f**. Subsequent simulated measurements are subtracted to find Δ

**g**.

**g**and

**H**, the reconstruction algorithm finds an estimate, denoted Δ

**f**̂, of the original scene’s sparse difference frames. In our simulations, we use the

*ℓ*

_{1}-norm minimization software package [13

13. S. Kim, K. Koh, M. Lustig, S. Boyd, and D. Gorinevsky, “An interior-point method for large-scale l1-regularized least squares,” IEEE J. Sel. Top. Sig. Proc. **1**, 606–617 (2007). [CrossRef]

14. J. Tropp, “Just relax: Convex programming methods for identifying sparse signals in noise,” IEEE Trans. Inf. Theory **52**, 1030–1051 (2006). [CrossRef]

### 4.2. Quantifying reconstruction error

**e**is the difference between the true and reconstructed difference frames, and

*n*is the number of movers in the scene. The error frame is convolved with a three pixel averaging kernel

**a**, which reduces the penalty for one-pixel shift errors. The absolute value is taken in order to count positive and negative errors equally, and the error is divided by 2

*n*to make the metric independent of the number of movers. From now on, we will refer to

*P*from Eq. (2) as the reconstruction error.

### 4.3. Identifying optimal design parameters

**H**for minimum coherence would encourage small PSF magnitudes and drive total system throughput down. To eliminate this effect we normalize

**H**by the sum of the basis vector magnitudes: where

*M*and

*N*are the total number of rows and columns in the system matrix. Physically, this normalization represents division by the sum of each PSF’s light throughput. The coherence of a system matrix normalized in this way cannot be biased by reducing throughput. An unfortunate consequence of this normalization step is that mask fill factor—one of the SCOUT architecture’s parameters—cannot be optimized based on the coherence parameter because the effect of different mask fill factors are normalized out during the calculation.

*p*

_{2}.

*d*,

_{im}*p*

_{1}, and

*p*

_{2}. We compare a jointly optimized set of parameters based on the coherence parameter to a set of parameters found by varying each parameter individually. Reconstructions were performed in order to compare reconstruction error for scenes with one through six movers. Note also that reconstruction error is much higher for a set of parameters that result in high coherence.

## 5. Experimental results

*m*

_{1}has pitch

*p*

_{1}= 30 microns and fill factor

*f*

_{1}= 0.4 and is located at a distance

*d*

_{m1}= 14mm from the sensor. Mask

*m*

_{2}has pitch

*p*

_{2}= 500 microns and fill factor

*f*

_{2}= 0.2 and is located at a distance

*d*

_{m2}= 57mm from the sensor. These parameters were chosen based on simulated and experimental results. The plasma monitor was used because it provides a higher contrast compared to the traditional liquid crystal displays (LCD). Note that even though the plasma monitor was chosen for its high contrast compared to LCDs, the black background still produces a small amount of irradiance, which is a source of noise in our experiment.

*r*×

_{x}*r*is 8 × 8, while the ground-truth and reconstructed frame differences have a resolution

_{y}*R*×

_{x}*R*of 32 × 32. To simulate a low-resolution detector, the camera captures the scenes at 128 × 128 sensor pixels and the images are binned down to 8 × 8 before being used in reconstruction. The system response matrix is determined experimentally by measuring the point spread function of each scene location one at a time, as discussed in detail in Sec. 3.

_{y}*t*

_{exp}and

*t*

_{cal}are the experiment and calibration exposure times, respectively. This scaling accounts for the physical effect of increased photon collection (and hence photodetector counts) as a function of increased exposure time. The resulting peaks are easily identified against background noise.

*ℓ*

_{1}-norm minimization algorithm [13

13. S. Kim, K. Koh, M. Lustig, S. Boyd, and D. Gorinevsky, “An interior-point method for large-scale l1-regularized least squares,” IEEE J. Sel. Top. Sig. Proc. **1**, 606–617 (2007). [CrossRef]

## 6. Future work and conclusion

*ℓ*

_{1}-norm minimization algorithm and we believe that further research on sparse reconstruction with block-circulant system matrices may decrease reconstruction error. We also believe that non-isomorphic calibration techniques and adding further degrees of freedom in the design parameters could result in significant performance gains.

## References and links

1. | M. Wakin, J. Laska, M. Duarte, D. Baron, S. Sarvotham, D. Takhar, K. Kelly, and R. Baraniuk, “An architecture for compressive imaging,” in Proceedings of IEEE Intl. Conference on Image Processing, (IEEE, 2006), pp. 1273–1276. |

2. | M. Lustig, D. Donoho, and J. Pauly, “Sparse MRI: The application of compressed sensing for rapid MR imaging,” Magn. Reson. Med. |

3. | R. Willett, R. Marcia, and J. Nichols, “Compressed sensing for practical optical imaging systems: a tutorial,” Opt. Eng. |

4. | M. Neifeld and J. Ke, “Optical architectures for compressive imaging,” Appl. Opt. |

5. | M. Stenner, D. Townsend, and M. Gehm, “Static architecture for compressive motion detection in persistent, pervasive surveillance applications,” in Imaging Systems, OSA Technical Digest Series (Optical Society of America, 2010), paper IMB2. |

6. | Y. Rivenson, A. Stern, and B. Javidi, “Single exposure super-resolution compressive imaging by double phase encoding,” Opt. Express |

7. | Y. Kashter, O. Levi, and A. Stern, “Optical compressive change and motion detection,” Appl. Opt. |

8. | W. Bajwa, J. Haupt, G. Raz, S. Wright, and R. Nowak, “Toeplitz-structured compressed sensing matrices,” in |

9. | H. Rauhut, “Circulant and Toeplitz matrices in compressed sensing,” http://arxiv.org/abs/0902.4394. |

10. | J. Romberg, “Compressive sensing by random convolution,” SIAM J. Imaging Sci. |

11. | F. Sebert, Y. Zou, and L. Ying, “Toeplitz block matrices in compressed sensing and their applications in imaging,” in Proceedings of IEEE International Conference on Information Technology and Applications in Biomedicine, (IEEE, 2008), pp. 47–50. [CrossRef] |

12. | B. Liu, F. Sebert, Y. Zou, and L. Ying, “SparseSENSE: randomly-sampled parallel imaging using compressed sensing,” in Proceedings of the 16th Annual Meeting of ISMRM |

13. | S. Kim, K. Koh, M. Lustig, S. Boyd, and D. Gorinevsky, “An interior-point method for large-scale l1-regularized least squares,” IEEE J. Sel. Top. Sig. Proc. |

14. | J. Tropp, “Just relax: Convex programming methods for identifying sparse signals in noise,” IEEE Trans. Inf. Theory |

**OCIS Codes**

(100.3190) Image processing : Inverse problems

(110.1758) Imaging systems : Computational imaging

(100.4999) Image processing : Pattern recognition, target tracking

**ToC Category:**

Image Processing

**History**

Original Manuscript: May 25, 2012

Revised Manuscript: August 26, 2012

Manuscript Accepted: August 27, 2012

Published: August 31, 2012

**Citation**

D. J. Townsend, P. K. Poon, S. Wehrwein, T. Osman, A. V. Mariano, E. M. Vera, M. D. Stenner, and M. E. Gehm, "Static compressive tracking," Opt. Express **20**, 21160-21172 (2012)

http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-20-19-21160

Sort: Year | Journal | Reset

### References

- M. Wakin, J. Laska, M. Duarte, D. Baron, S. Sarvotham, D. Takhar, K. Kelly, and R. Baraniuk, “An architecture for compressive imaging,” in Proceedings of IEEE Intl. Conference on Image Processing, (IEEE, 2006), pp. 1273–1276.
- M. Lustig, D. Donoho, and J. Pauly, “Sparse MRI: The application of compressed sensing for rapid MR imaging,” Magn. Reson. Med.58, 1182–1195 (2007). [CrossRef] [PubMed]
- R. Willett, R. Marcia, and J. Nichols, “Compressed sensing for practical optical imaging systems: a tutorial,” Opt. Eng.50, 072601 (2011). [CrossRef]
- M. Neifeld and J. Ke, “Optical architectures for compressive imaging,” Appl. Opt.46, 5293, (2007). [CrossRef] [PubMed]
- M. Stenner, D. Townsend, and M. Gehm, “Static architecture for compressive motion detection in persistent, pervasive surveillance applications,” in Imaging Systems, OSA Technical Digest Series (Optical Society of America, 2010), paper IMB2.
- Y. Rivenson, A. Stern, and B. Javidi, “Single exposure super-resolution compressive imaging by double phase encoding,” Opt. Express18, 15094–15103 (2010). [CrossRef] [PubMed]
- Y. Kashter, O. Levi, and A. Stern, “Optical compressive change and motion detection,” Appl. Opt.51, 2491–2496 (2012). [CrossRef] [PubMed]
- W. Bajwa, J. Haupt, G. Raz, S. Wright, and R. Nowak, “Toeplitz-structured compressed sensing matrices,” in Proceedings of IEEE Workshop on Statistical Signal Processing, (IEEE, 2007), pp. 294–298. [CrossRef]
- H. Rauhut, “Circulant and Toeplitz matrices in compressed sensing,” http://arxiv.org/abs/0902.4394 .
- J. Romberg, “Compressive sensing by random convolution,” SIAM J. Imaging Sci.2, 1098–1128 (2009). [CrossRef]
- F. Sebert, Y. Zou, and L. Ying, “Toeplitz block matrices in compressed sensing and their applications in imaging,” in Proceedings of IEEE International Conference on Information Technology and Applications in Biomedicine, (IEEE, 2008), pp. 47–50. [CrossRef]
- B. Liu, F. Sebert, Y. Zou, and L. Ying, “SparseSENSE: randomly-sampled parallel imaging using compressed sensing,” in Proceedings of the 16th Annual Meeting of ISMRM3154 (2008).
- S. Kim, K. Koh, M. Lustig, S. Boyd, and D. Gorinevsky, “An interior-point method for large-scale l1-regularized least squares,” IEEE J. Sel. Top. Sig. Proc.1, 606–617 (2007). [CrossRef]
- J. Tropp, “Just relax: Convex programming methods for identifying sparse signals in noise,” IEEE Trans. Inf. Theory52, 1030–1051 (2006). [CrossRef]

## Cited By |
Alert me when this paper is cited |

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

« Previous Article | Next Article »

OSA is a member of CrossRef.