## Tracking of multiple objects in unknown background using Bayesian estimation in 3D space |

JOSA A, Vol. 28, Issue 9, pp. 1935-1940 (2011)

http://dx.doi.org/10.1364/JOSAA.28.001935

Acrobat PDF (925 KB)

### Abstract

We present a three-dimensional (3D) object tracking method based on a Bayesian framework for tracking multiple, occluded objects in a complex scene. The 3D passive capture of scene data is based on integral imaging. The statistical characteristics of the objects versus the background are exploited to analyze each frame. The algorithm can work with objects with unknown position, rotation, scale, and illumination. Posterior probabilities of the reconstructed scene background and the 3D objects are calculated by defining their pixel intensities as Gaussian and gamma distributions, respectively, and by assuming appropriate prior distributions for estimated parameters. Multiobject tracking is achieved by maximizing the geodesic distance between the log-posteriors of the background and the objects. Experimental results are presented.

© 2011 Optical Society of America

## 1. INTRODUCTION

2. A. Stern and B. Javidi, “3D image sensing, visualization, and processing using integral imaging,” Proc. IEEE **94**, 591–608 (2006). [CrossRef]

3. F. Okano, J. Arai, K. Mitani, and M. Okui, “Real-time integral imaging based on extremely high resolution video system,” Proc. IEEE **94**, 490–501 (2006). [CrossRef]

4. J. S. Jang and B. Javidi, “Three-dimensional synthetic aperture integral imaging,” Opt. Lett. **27**, 1144–1146 (2002). [CrossRef]

5. S. Hong and B. Javidi, “Improved resolution 3D object reconstruction using computational integral imaging with time multiplexing,” Opt. Express **12**, 4579–4588 (2004). [CrossRef] [PubMed]

6. B. Javidi, R. Ponce-Diaz, and S.-H. Hong, “Three-dimensional recognition of occluded objects by using computational integral imaging,” Opt. Lett. **31**, 1106–1108 (2006). [CrossRef] [PubMed]

7. M. Cho and B. Javidi, “Three-dimensional tracking of occluded objects using integral imaging,” Opt. Lett. **33**, 2737–2739 (2008). [CrossRef] [PubMed]

8. M. Pollefeys, L. Van Gool, M. Vergauwen, F. Verbiest, K. Cornelis, J. Tops, and R. Koch, “Visual modeling with a hand-held camera,” Int. J. Comput. Vis. **59**, 207–232 (2004). [CrossRef]

9. M. DaneshPanah and B. Javidi, “Segmentation of 3D holographic images using bivariate jointly distributed region snake,” Opt. Express **14**, 5143–5153 (2006). [CrossRef] [PubMed]

10. M. DaneshPanah and B. Javidi, “Tracking biological microorganisms in sequence of 3D holographic microscopy images,” Opt. Express **15**, 10761–10766 (2007). [CrossRef] [PubMed]

11. C. Chesnaud, V. Page, and P. Réfrégier, “Improvement in robustness of the statistically independent region snake-based segmentation method of target-shape tracking,” Opt. Lett. **23**, 488–490 (1998). [CrossRef]

12. A. Yilmaz, X. Li, and M. Shah, “Contour based object tracking with occlusion handling in video acquired using mobile cameras,” IEEE Trans. Pattern Anal. Mach. Intell. **26**, 1531–1536 (2004). [CrossRef] [PubMed]

13. M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: active contour models,” in Proceedings of the International Conference on Computer Vision (IEEE, 1987), pp. 259–268. [CrossRef]

13. M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: active contour models,” in Proceedings of the International Conference on Computer Vision (IEEE, 1987), pp. 259–268. [CrossRef]

10. M. DaneshPanah and B. Javidi, “Tracking biological microorganisms in sequence of 3D holographic microscopy images,” Opt. Express **15**, 10761–10766 (2007). [CrossRef] [PubMed]

11. C. Chesnaud, V. Page, and P. Réfrégier, “Improvement in robustness of the statistically independent region snake-based segmentation method of target-shape tracking,” Opt. Lett. **23**, 488–490 (1998). [CrossRef]

14. T. Georgiou, “Distances and Riemannian metrics for spectral density functions,” IEEE Trans. Signal Process. **55**, 3995–4003 (2007). [CrossRef]

## 2. SYNTHETIC APERTURE INTEGRAL IMAGING AND COMPUTATIONAL RECONSTRUCTION

6. B. Javidi, R. Ponce-Diaz, and S.-H. Hong, “Three-dimensional recognition of occluded objects by using computational integral imaging,” Opt. Lett. **31**, 1106–1108 (2006). [CrossRef] [PubMed]

*d*is the distance from the image sensor to the 3D object and

*f*is the focal length of the image sensor, respectively. This enables visualization of the partially occluded objects, because only the reconstruction plane with the object of interest is in focus, while the occlusion and background are out of focus.

## 3. TRACKING WITH THE BAYESIAN ALGORITHM

18. C. Chesnaud, P. Réfrégier, and V. Boulet, “Statistical region snake-based segmentation adapted to different physical noise models,” IEEE Trans. Pattern Anal. Mach. Intell. **21**, 1145–1157 (1999). [CrossRef]

### 3A. Background Region Statistics

*μ*and

*μ*and

*a posteriori*(MAP) sense. According to Bayes’s rule [19], the conditional probability can be rewritten as

*μ*and

*N*is the total number of input image pixels.

### 3B. Object Region Statistics

*j*as follows: where

*j*denotes the index of objects to be tracked,

*α*is known, and that the rate parameter

*β*has known gamma prior distribution,

*posterior*distribution of each object region:

*β*under the squared error loss is achieved as the posterior mean [19]:

### 3C. 3D Tracking with the Bayesian Algorithm

*p*between the occlusion and the background, we are first seeking to locate the objects individually, which is analogous to maxi mizing the geodesic distance [14

14. T. Georgiou, “Distances and Riemannian metrics for spectral density functions,” IEEE Trans. Signal Process. **55**, 3995–4003 (2007). [CrossRef]

*j*and the background: where

*p*is the reconstructed plane and

*j*is the index of ob jects to be tracked. Thus for an object

*j*at the reconstructed plane

*p*,

**w**is the optimal binary window and the optimal segmentation.

13. M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: active contour models,” in Proceedings of the International Conference on Computer Vision (IEEE, 1987), pp. 259–268. [CrossRef]

*t*, the embedding level-set is defined as

**q**denotes a point in the level-set, such that

10. M. DaneshPanah and B. Javidi, “Tracking biological microorganisms in sequence of 3D holographic microscopy images,” Opt. Express **15**, 10761–10766 (2007). [CrossRef] [PubMed]

**s**. The corresponding Euler–Lagrange equation result is

**15**, 10761–10766 (2007). [CrossRef] [PubMed]

## 4. EXPERIMENTAL RESULTS

*μ*and variance

*α*may be different from object to object, and usually varies between 2 and 5. For our case, we assume that

*β*follows

*β*varies in between 0.015 to 0.05 from frame to frame, because the pixel sta tistics change for different object positions, rotations, and illuminations.

17. F. Goudail and P. Réfrégier, “Optimal target tracking on image sequences with a deterministic background,” J. Opt. Soc. Am. A **14**, 3197–3207 (1997). [CrossRef]

## 5. CONCLUSIONS

## ACKNOWLEDGMENTS

1. | G. Lippmann, “La photographic intégrale,” C. R. Acad. Sci. |

2. | A. Stern and B. Javidi, “3D image sensing, visualization, and processing using integral imaging,” Proc. IEEE |

3. | F. Okano, J. Arai, K. Mitani, and M. Okui, “Real-time integral imaging based on extremely high resolution video system,” Proc. IEEE |

4. | J. S. Jang and B. Javidi, “Three-dimensional synthetic aperture integral imaging,” Opt. Lett. |

5. | S. Hong and B. Javidi, “Improved resolution 3D object reconstruction using computational integral imaging with time multiplexing,” Opt. Express |

6. | B. Javidi, R. Ponce-Diaz, and S.-H. Hong, “Three-dimensional recognition of occluded objects by using computational integral imaging,” Opt. Lett. |

7. | M. Cho and B. Javidi, “Three-dimensional tracking of occluded objects using integral imaging,” Opt. Lett. |

8. | M. Pollefeys, L. Van Gool, M. Vergauwen, F. Verbiest, K. Cornelis, J. Tops, and R. Koch, “Visual modeling with a hand-held camera,” Int. J. Comput. Vis. |

9. | M. DaneshPanah and B. Javidi, “Segmentation of 3D holographic images using bivariate jointly distributed region snake,” Opt. Express |

10. | M. DaneshPanah and B. Javidi, “Tracking biological microorganisms in sequence of 3D holographic microscopy images,” Opt. Express |

11. | C. Chesnaud, V. Page, and P. Réfrégier, “Improvement in robustness of the statistically independent region snake-based segmentation method of target-shape tracking,” Opt. Lett. |

12. | A. Yilmaz, X. Li, and M. Shah, “Contour based object tracking with occlusion handling in video acquired using mobile cameras,” IEEE Trans. Pattern Anal. Mach. Intell. |

13. | M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: active contour models,” in Proceedings of the International Conference on Computer Vision (IEEE, 1987), pp. 259–268. [CrossRef] |

14. | T. Georgiou, “Distances and Riemannian metrics for spectral density functions,” IEEE Trans. Signal Process. |

15. | O. Germain and P. Réfrégier, “Optimal snake-based seg mentation of a random luminance target on a spatially disjoint background,” Opt. Lett. |

16. | B. Javidi, P. Réfrégier, and P. Willett, “Optimum receiver design for pattern recognition with nonoverlapping signal and scene noise,” Opt. Lett. |

17. | F. Goudail and P. Réfrégier, “Optimal target tracking on image sequences with a deterministic background,” J. Opt. Soc. Am. A |

18. | C. Chesnaud, P. Réfrégier, and V. Boulet, “Statistical region snake-based segmentation adapted to different physical noise models,” IEEE Trans. Pattern Anal. Mach. Intell. |

19. | N. Mukhopadhyay, |

20. | J. Sethian, |

**OCIS Codes**

(110.6880) Imaging systems : Three-dimensional image acquisition

(150.6910) Machine vision : Three-dimensional sensing

(280.4991) Remote sensing and sensors : Passive remote sensing

**ToC Category:**

Imaging Systems

**History**

Original Manuscript: January 24, 2011

Revised Manuscript: July 13, 2011

Manuscript Accepted: July 14, 2011

Published: August 29, 2011

**Virtual Issues**

February 13, 2012 *Spotlight on Optics*

**Citation**

Yige Zhao, Xiao Xiao, Myungjin Cho, and Bahram Javidi, "Tracking of multiple objects in unknown background using Bayesian estimation in 3D space," J. Opt. Soc. Am. A **28**, 1935-1940 (2011)

http://www.opticsinfobase.org/josaa/abstract.cfm?URI=josaa-28-9-1935

Sort: Year | Journal | Reset

### References

- G. Lippmann, “La photographic intégrale,” C. R. Acad. Sci. 146, 446–451 (1908).
- A. Stern and B. Javidi, “3D image sensing, visualization, and processing using integral imaging,” Proc. IEEE 94, 591–608 (2006). [CrossRef]
- F. Okano, J. Arai, K. Mitani, and M. Okui, “Real-time integral imaging based on extremely high resolution video system,” Proc. IEEE 94, 490–501 (2006). [CrossRef]
- J. S. Jang and B. Javidi, “Three-dimensional synthetic aperture integral imaging,” Opt. Lett. 27, 1144–1146 (2002). [CrossRef]
- S. Hong and B. Javidi, “Improved resolution 3D object reconstruction using computational integral imaging with time multiplexing,” Opt. Express 12, 4579–4588 (2004). [CrossRef] [PubMed]
- B. Javidi, R. Ponce-Diaz, and S.-H. Hong, “Three-dimensional recognition of occluded objects by using computational integral imaging,” Opt. Lett. 31, 1106–1108 (2006). [CrossRef] [PubMed]
- M. Cho and B. Javidi, “Three-dimensional tracking of occluded objects using integral imaging,” Opt. Lett. 33, 2737–2739 (2008). [CrossRef] [PubMed]
- M. Pollefeys, L. Van Gool, M. Vergauwen, F. Verbiest, K. Cornelis, J. Tops, and R. Koch, “Visual modeling with a hand-held camera,” Int. J. Comput. Vis. 59, 207–232 (2004). [CrossRef]
- M. DaneshPanah and B. Javidi, “Segmentation of 3D holographic images using bivariate jointly distributed region snake,” Opt. Express 14, 5143–5153 (2006). [CrossRef] [PubMed]
- M. DaneshPanah and B. Javidi, “Tracking biological microorganisms in sequence of 3D holographic microscopy images,” Opt. Express 15, 10761–10766 (2007). [CrossRef] [PubMed]
- C. Chesnaud, V. Page, and P. Réfrégier, “Improvement in robustness of the statistically independent region snake-based segmentation method of target-shape tracking,” Opt. Lett. 23, 488–490 (1998). [CrossRef]
- A. Yilmaz, X. Li, and M. Shah, “Contour based object tracking with occlusion handling in video acquired using mobile cameras,” IEEE Trans. Pattern Anal. Mach. Intell. 26, 1531–1536(2004). [CrossRef] [PubMed]
- M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: active contour models,” in Proceedings of the International Conference on Computer Vision (IEEE, 1987), pp. 259–268. [CrossRef]
- T. Georgiou, “Distances and Riemannian metrics for spectral density functions,” IEEE Trans. Signal Process. 55, 3995–4003(2007). [CrossRef]
- O. Germain and P. Réfrégier, “Optimal snake-based segmentation of a random luminance target on a spatially disjoint background,” Opt. Lett. 21, 1845–1847 (1996). [CrossRef] [PubMed]
- B. Javidi, P. Réfrégier, and P. Willett, “Optimum receiver design for pattern recognition with nonoverlapping signal and scene noise,” Opt. Lett. 18, 1660–1662 (1993). [CrossRef] [PubMed]
- F. Goudail and P. Réfrégier, “Optimal target tracking on image sequences with a deterministic background,” J. Opt. Soc. Am. A 14, 3197–3207 (1997). [CrossRef]
- C. Chesnaud, P. Réfrégier, and V. Boulet, “Statistical region snake-based segmentation adapted to different physical noise models,” IEEE Trans. Pattern Anal. Mach. Intell. 21, 1145–1157(1999). [CrossRef]
- N. Mukhopadhyay, Probability and Statistical Inference(Marcel Dekker, 2000).
- J. Sethian, Level Set Methods: Evolving Interfaces in Computational Geometry, Fluid Mechanics, Computer Vision, and Material Sciences (Cambridge University Press, 1999).

## Cited By |
Alert me when this paper is cited |

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

### Supplementary Material

» Media 1: MPG (812 KB)

» Media 2: MPG (634 KB)

» Media 3: MPG (380 KB)

» Media 4: MPG (382 KB)

» Media 5: MPG (986 KB)

» Media 6: MPG (1076 KB)

« Previous Article | Next Article »

OSA is a member of CrossRef.