Abstract

We present a three-dimensional (3D) object tracking method based on a Bayesian framework for tracking multiple, occluded objects in a complex scene. The 3D passive capture of scene data is based on integral imaging. The statistical characteristics of the objects versus the background are exploited to analyze each frame. The algorithm can work with objects with unknown position, rotation, scale, and illumination. Posterior probabilities of the reconstructed scene background and the 3D objects are calculated by defining their pixel intensities as Gaussian and gamma distributions, respectively, and by assuming appropriate prior distributions for estimated parameters. Multiobject tracking is achieved by maximizing the geodesic distance between the log-posteriors of the background and the objects. Experimental results are presented.

© 2011 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. G. Lippmann, “La photographic intégrale,” C. R. Acad. Sci. 146, 446–451 (1908).
  2. A. Stern and B. Javidi, “3D image sensing, visualization, and processing using integral imaging,” Proc. IEEE 94, 591–608 (2006).
    [CrossRef]
  3. F. Okano, J. Arai, K. Mitani, and M. Okui, “Real-time integral imaging based on extremely high resolution video system,” Proc. IEEE 94, 490–501 (2006).
    [CrossRef]
  4. J. S. Jang and B. Javidi, “Three-dimensional synthetic aperture integral imaging,” Opt. Lett. 27, 1144–1146 (2002).
    [CrossRef]
  5. S. Hong and B. Javidi, “Improved resolution 3D object reconstruction using computational integral imaging with time multiplexing,” Opt. Express 12, 4579–4588 (2004).
    [CrossRef] [PubMed]
  6. B. Javidi, R. Ponce-Diaz, and S.-H. Hong, “Three-dimensional recognition of occluded objects by using computational integral imaging,” Opt. Lett. 31, 1106–1108 (2006).
    [CrossRef] [PubMed]
  7. M. Cho and B. Javidi, “Three-dimensional tracking of occluded objects using integral imaging,” Opt. Lett. 33, 2737–2739 (2008).
    [CrossRef] [PubMed]
  8. M. Pollefeys, L. Van Gool, M. Vergauwen, F. Verbiest, K. Cornelis, J. Tops, and R. Koch, “Visual modeling with a hand-held camera,” Int. J. Comput. Vis. 59, 207–232 (2004).
    [CrossRef]
  9. M. DaneshPanah and B. Javidi, “Segmentation of 3D holographic images using bivariate jointly distributed region snake,” Opt. Express 14, 5143–5153 (2006).
    [CrossRef] [PubMed]
  10. M. DaneshPanah and B. Javidi, “Tracking biological microorganisms in sequence of 3D holographic microscopy images,” Opt. Express 15, 10761–10766 (2007).
    [CrossRef] [PubMed]
  11. C. Chesnaud, V. Page, and P. Réfrégier, “Improvement in robustness of the statistically independent region snake-based segmentation method of target-shape tracking,” Opt. Lett. 23, 488–490 (1998).
    [CrossRef]
  12. A. Yilmaz, X. Li, and M. Shah, “Contour based object tracking with occlusion handling in video acquired using mobile cameras,” IEEE Trans. Pattern Anal. Mach. Intell. 26, 1531–1536(2004).
    [CrossRef] [PubMed]
  13. M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: active contour models,” in Proceedings of the International Conference on Computer Vision (IEEE, 1987), pp. 259–268.
    [CrossRef]
  14. T. Georgiou, “Distances and Riemannian metrics for spectral density functions,” IEEE Trans. Signal Process. 55, 3995–4003(2007).
    [CrossRef]
  15. O. Germain and P. Réfrégier, “Optimal snake-based segmentation of a random luminance target on a spatially disjoint background,” Opt. Lett. 21, 1845–1847 (1996).
    [CrossRef] [PubMed]
  16. B. Javidi, P. Réfrégier, and P. Willett, “Optimum receiver design for pattern recognition with nonoverlapping signal and scene noise,” Opt. Lett. 18, 1660–1662 (1993).
    [CrossRef] [PubMed]
  17. F. Goudail and P. Réfrégier, “Optimal target tracking on image sequences with a deterministic background,” J. Opt. Soc. Am. A 14, 3197–3207 (1997).
    [CrossRef]
  18. C. Chesnaud, P. Réfrégier, and V. Boulet, “Statistical region snake-based segmentation adapted to different physical noise models,” IEEE Trans. Pattern Anal. Mach. Intell. 21, 1145–1157(1999).
    [CrossRef]
  19. N. Mukhopadhyay, Probability and Statistical Inference(Marcel Dekker, 2000).
  20. J. Sethian, Level Set Methods: Evolving Interfaces in Computational Geometry, Fluid Mechanics, Computer Vision, and Material Sciences (Cambridge University Press, 1999).

2008

2007

M. DaneshPanah and B. Javidi, “Tracking biological microorganisms in sequence of 3D holographic microscopy images,” Opt. Express 15, 10761–10766 (2007).
[CrossRef] [PubMed]

T. Georgiou, “Distances and Riemannian metrics for spectral density functions,” IEEE Trans. Signal Process. 55, 3995–4003(2007).
[CrossRef]

2006

B. Javidi, R. Ponce-Diaz, and S.-H. Hong, “Three-dimensional recognition of occluded objects by using computational integral imaging,” Opt. Lett. 31, 1106–1108 (2006).
[CrossRef] [PubMed]

M. DaneshPanah and B. Javidi, “Segmentation of 3D holographic images using bivariate jointly distributed region snake,” Opt. Express 14, 5143–5153 (2006).
[CrossRef] [PubMed]

A. Stern and B. Javidi, “3D image sensing, visualization, and processing using integral imaging,” Proc. IEEE 94, 591–608 (2006).
[CrossRef]

F. Okano, J. Arai, K. Mitani, and M. Okui, “Real-time integral imaging based on extremely high resolution video system,” Proc. IEEE 94, 490–501 (2006).
[CrossRef]

2004

S. Hong and B. Javidi, “Improved resolution 3D object reconstruction using computational integral imaging with time multiplexing,” Opt. Express 12, 4579–4588 (2004).
[CrossRef] [PubMed]

M. Pollefeys, L. Van Gool, M. Vergauwen, F. Verbiest, K. Cornelis, J. Tops, and R. Koch, “Visual modeling with a hand-held camera,” Int. J. Comput. Vis. 59, 207–232 (2004).
[CrossRef]

A. Yilmaz, X. Li, and M. Shah, “Contour based object tracking with occlusion handling in video acquired using mobile cameras,” IEEE Trans. Pattern Anal. Mach. Intell. 26, 1531–1536(2004).
[CrossRef] [PubMed]

2002

1999

C. Chesnaud, P. Réfrégier, and V. Boulet, “Statistical region snake-based segmentation adapted to different physical noise models,” IEEE Trans. Pattern Anal. Mach. Intell. 21, 1145–1157(1999).
[CrossRef]

1998

1997

1996

1993

1908

G. Lippmann, “La photographic intégrale,” C. R. Acad. Sci. 146, 446–451 (1908).

Arai, J.

F. Okano, J. Arai, K. Mitani, and M. Okui, “Real-time integral imaging based on extremely high resolution video system,” Proc. IEEE 94, 490–501 (2006).
[CrossRef]

Boulet, V.

C. Chesnaud, P. Réfrégier, and V. Boulet, “Statistical region snake-based segmentation adapted to different physical noise models,” IEEE Trans. Pattern Anal. Mach. Intell. 21, 1145–1157(1999).
[CrossRef]

Chesnaud, C.

C. Chesnaud, P. Réfrégier, and V. Boulet, “Statistical region snake-based segmentation adapted to different physical noise models,” IEEE Trans. Pattern Anal. Mach. Intell. 21, 1145–1157(1999).
[CrossRef]

C. Chesnaud, V. Page, and P. Réfrégier, “Improvement in robustness of the statistically independent region snake-based segmentation method of target-shape tracking,” Opt. Lett. 23, 488–490 (1998).
[CrossRef]

Cho, M.

Cornelis, K.

M. Pollefeys, L. Van Gool, M. Vergauwen, F. Verbiest, K. Cornelis, J. Tops, and R. Koch, “Visual modeling with a hand-held camera,” Int. J. Comput. Vis. 59, 207–232 (2004).
[CrossRef]

DaneshPanah, M.

Georgiou, T.

T. Georgiou, “Distances and Riemannian metrics for spectral density functions,” IEEE Trans. Signal Process. 55, 3995–4003(2007).
[CrossRef]

Germain, O.

Goudail, F.

Hong, S.

Hong, S.-H.

Jang, J. S.

Javidi, B.

Kass, M.

M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: active contour models,” in Proceedings of the International Conference on Computer Vision (IEEE, 1987), pp. 259–268.
[CrossRef]

Koch, R.

M. Pollefeys, L. Van Gool, M. Vergauwen, F. Verbiest, K. Cornelis, J. Tops, and R. Koch, “Visual modeling with a hand-held camera,” Int. J. Comput. Vis. 59, 207–232 (2004).
[CrossRef]

Li, X.

A. Yilmaz, X. Li, and M. Shah, “Contour based object tracking with occlusion handling in video acquired using mobile cameras,” IEEE Trans. Pattern Anal. Mach. Intell. 26, 1531–1536(2004).
[CrossRef] [PubMed]

Lippmann, G.

G. Lippmann, “La photographic intégrale,” C. R. Acad. Sci. 146, 446–451 (1908).

Mitani, K.

F. Okano, J. Arai, K. Mitani, and M. Okui, “Real-time integral imaging based on extremely high resolution video system,” Proc. IEEE 94, 490–501 (2006).
[CrossRef]

Mukhopadhyay, N.

N. Mukhopadhyay, Probability and Statistical Inference(Marcel Dekker, 2000).

Okano, F.

F. Okano, J. Arai, K. Mitani, and M. Okui, “Real-time integral imaging based on extremely high resolution video system,” Proc. IEEE 94, 490–501 (2006).
[CrossRef]

Okui, M.

F. Okano, J. Arai, K. Mitani, and M. Okui, “Real-time integral imaging based on extremely high resolution video system,” Proc. IEEE 94, 490–501 (2006).
[CrossRef]

Page, V.

Pollefeys, M.

M. Pollefeys, L. Van Gool, M. Vergauwen, F. Verbiest, K. Cornelis, J. Tops, and R. Koch, “Visual modeling with a hand-held camera,” Int. J. Comput. Vis. 59, 207–232 (2004).
[CrossRef]

Ponce-Diaz, R.

Réfrégier, P.

Sethian, J.

J. Sethian, Level Set Methods: Evolving Interfaces in Computational Geometry, Fluid Mechanics, Computer Vision, and Material Sciences (Cambridge University Press, 1999).

Shah, M.

A. Yilmaz, X. Li, and M. Shah, “Contour based object tracking with occlusion handling in video acquired using mobile cameras,” IEEE Trans. Pattern Anal. Mach. Intell. 26, 1531–1536(2004).
[CrossRef] [PubMed]

Stern, A.

A. Stern and B. Javidi, “3D image sensing, visualization, and processing using integral imaging,” Proc. IEEE 94, 591–608 (2006).
[CrossRef]

Terzopoulos, D.

M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: active contour models,” in Proceedings of the International Conference on Computer Vision (IEEE, 1987), pp. 259–268.
[CrossRef]

Tops, J.

M. Pollefeys, L. Van Gool, M. Vergauwen, F. Verbiest, K. Cornelis, J. Tops, and R. Koch, “Visual modeling with a hand-held camera,” Int. J. Comput. Vis. 59, 207–232 (2004).
[CrossRef]

Van Gool, L.

M. Pollefeys, L. Van Gool, M. Vergauwen, F. Verbiest, K. Cornelis, J. Tops, and R. Koch, “Visual modeling with a hand-held camera,” Int. J. Comput. Vis. 59, 207–232 (2004).
[CrossRef]

Verbiest, F.

M. Pollefeys, L. Van Gool, M. Vergauwen, F. Verbiest, K. Cornelis, J. Tops, and R. Koch, “Visual modeling with a hand-held camera,” Int. J. Comput. Vis. 59, 207–232 (2004).
[CrossRef]

Vergauwen, M.

M. Pollefeys, L. Van Gool, M. Vergauwen, F. Verbiest, K. Cornelis, J. Tops, and R. Koch, “Visual modeling with a hand-held camera,” Int. J. Comput. Vis. 59, 207–232 (2004).
[CrossRef]

Willett, P.

Witkin, A.

M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: active contour models,” in Proceedings of the International Conference on Computer Vision (IEEE, 1987), pp. 259–268.
[CrossRef]

Yilmaz, A.

A. Yilmaz, X. Li, and M. Shah, “Contour based object tracking with occlusion handling in video acquired using mobile cameras,” IEEE Trans. Pattern Anal. Mach. Intell. 26, 1531–1536(2004).
[CrossRef] [PubMed]

C. R. Acad. Sci.

G. Lippmann, “La photographic intégrale,” C. R. Acad. Sci. 146, 446–451 (1908).

IEEE Trans. Pattern Anal. Mach. Intell.

A. Yilmaz, X. Li, and M. Shah, “Contour based object tracking with occlusion handling in video acquired using mobile cameras,” IEEE Trans. Pattern Anal. Mach. Intell. 26, 1531–1536(2004).
[CrossRef] [PubMed]

C. Chesnaud, P. Réfrégier, and V. Boulet, “Statistical region snake-based segmentation adapted to different physical noise models,” IEEE Trans. Pattern Anal. Mach. Intell. 21, 1145–1157(1999).
[CrossRef]

IEEE Trans. Signal Process.

T. Georgiou, “Distances and Riemannian metrics for spectral density functions,” IEEE Trans. Signal Process. 55, 3995–4003(2007).
[CrossRef]

Int. J. Comput. Vis.

M. Pollefeys, L. Van Gool, M. Vergauwen, F. Verbiest, K. Cornelis, J. Tops, and R. Koch, “Visual modeling with a hand-held camera,” Int. J. Comput. Vis. 59, 207–232 (2004).
[CrossRef]

J. Opt. Soc. Am. A

Opt. Express

Opt. Lett.

Proc. IEEE

A. Stern and B. Javidi, “3D image sensing, visualization, and processing using integral imaging,” Proc. IEEE 94, 591–608 (2006).
[CrossRef]

F. Okano, J. Arai, K. Mitani, and M. Okui, “Real-time integral imaging based on extremely high resolution video system,” Proc. IEEE 94, 490–501 (2006).
[CrossRef]

Other

M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: active contour models,” in Proceedings of the International Conference on Computer Vision (IEEE, 1987), pp. 259–268.
[CrossRef]

N. Mukhopadhyay, Probability and Statistical Inference(Marcel Dekker, 2000).

J. Sethian, Level Set Methods: Evolving Interfaces in Computational Geometry, Fluid Mechanics, Computer Vision, and Material Sciences (Cambridge University Press, 1999).

Supplementary Material (6)

» Media 1: MPG (812 KB)     
» Media 2: MPG (634 KB)     
» Media 3: MPG (380 KB)     
» Media 4: MPG (382 KB)     
» Media 5: MPG (986 KB)     
» Media 6: MPG (1076 KB)     

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (7)

Fig. 1
Fig. 1

3D integral imaging sensing and reconstruction: (a) scene capture process and (b) 3D reconstruction of the scene in Fig. 2.

Fig. 2
Fig. 2

Experimental setup and objects with unknown occlusion and background used in the scene: (a) experimental setup, (b) objects to be tracked (two cars), (c) background, and (d) elemental images.

Fig. 3
Fig. 3

3D plot of objects positions to be tracked (3D movements).

Fig. 4
Fig. 4

2D tracking results of optimal object tracking algorithm presented in Ref. [17] with objects rotated and illumination changed for scenes in (a) and (b): (a) two occluded objects in frame two, (b) two occluded objects in frame three, and (c) tracking results of frame three.

Fig. 5
Fig. 5

3D reconstruction with occluded objects (a) occluded objects (cars behind tree branches) are located at Z = 380 mm and Z = 410 mm from the sensor, respectively. (b)–(f) 3D reconstructed image sequences at Z = 190 , Z = 380 , Z = 410 , Z = 570 , and Z = 690 mm , respectively. (b) Reconstructed plane of occlusion (tree branches).

Fig. 6
Fig. 6

3D tracking results for the first frame: (a) car 1 with occlusion (Media 1), (b) car 2 with occlusion (Media 2), (c) car 1 without occlusion (Media 3), and (d) car 2 without occlusion (Media 4)

Fig. 7
Fig. 7

3D tracking results of moving cars (Media 5 and Media 6) in unknown background (see Fig. 2). The objects’ movements are shown in Fig. 3. (a) Frame 2, (b) frame 9 (illumination reduced to one-half and with car 1 rotated), (c) frame 14 (with both cars rotated), (d) frame 17 (with car 2 rotated), and (e) frame 27 (scene illumination doubled and with car 1 rotated).

Equations (12)

Equations on this page are rendered with MathJax. Learn more.

s i = o i w i + b i ( 1 w i ) .
f b ( s ) = 1 2 π σ e ( s μ ) 2 2 σ 2 ,
( μ ^ , σ ^ 2 ) = arg max ( μ , σ 2 ) P b ( μ , σ 2 | w , s ) ,
P b ( μ , σ 2 | w , s ) = P b ( s | w , μ , σ 2 ) P b ( μ , σ 2 ) P b ( s ) = P b ( s | w , μ , σ 2 ) P b ( μ ) P b ( σ 2 | μ ) P b ( s ) .
P b ( s | w , μ , σ 2 ) = i = 1 N [ 1 2 π σ e ( s i μ ) 2 2 σ 2 ] ( 1 w i ) .
log P b μ = 0 μ ^ | s , w = 1 N ( 1 w ) i = 1 N s i ( 1 w i ) , log P b σ 2 = 0 σ ^ 2 | μ ^ , s , w = 1 N ( 1 w ) + 2 i = 1 N ( s i μ ^ ) 2 ( 1 w i ) ,
f o j ( s ) = β α Γ ( α ) s α 1 e β s ,
P o j ( β | α , w , s ) P o j ( s | w , α , β ) π ( α 0 , β 0 ) = i = 1 N [ β α Γ ( α ) s i α 1 e β s i ] w i β 0 α 0 Γ ( α 0 ) β α 0 1 e β 0 β = β α N ( w ) Γ ( α ) N ( w ) ( i = 1 N s i w i ) α 1 e β i = 1 N s i w i β 0 α 0 Γ ( α 0 ) β α 0 1 e β 0 β β ( α N ( w ) + α 0 ) 1 e β ( i = 1 N s i w i + β 0 ) gamma ( α N ( w ) + α 0 , i = 1 N s i w i + β 0 ) .
β ^ | α , w , s , α 0 , β 0 = α N ( w ) + α 0 i = 1 N s i w i + β 0 .
ε p j ( s , w ) = E [ log ( P o j ( s | w , α , β ^ ) P b ( s | w , μ ^ , σ ^ 2 ) ) 2 ] { E [ log ( P o j ( s | w , α , β ^ ) P b ( s | w , μ ^ , σ ^ 2 ) ) ] } 2 ,
F ( q ) = φ ε p j ( s , w ) + div ( ( φ ( q , t ) ) | ( φ ( q , t ) ) | ) .
ε j ( s , w ^ , p ^ ) = arg max ( p , w ) ε p j ( s , w ) .

Metrics