Abstract

A system that performs three-dimensional (3D) tracking of multiple skin-colored regions (SCRs) in images acquired by a calibrated, possibly moving stereoscopic rig is described. The system consists of a collection of techniques that permit the modeling and detection of SCRs, the determination of their temporal association in monocular image sequences, the establishment of their correspondence between stereo images, and the extraction of their 3D positions in a world-centered coordinate system. The development of these techniques has been motivated by the need for robust, near-real-time tracking performance. SCRs are detected by use of a Bayesian classifier that is trained with the aid of a novel technique. More specifically, the classifier is bootstrapped with a small set of training data. Then, as new images are being processed, an iterative training procedure is employed to refine the classifier. Furthermore, a technique is proposed to enable the classifier to cope with changes in illumination. Tracking of SCRs in time as well as matching of SCRs in the images of the employed stereo rig is performed through computationally inexpensive and robust techniques. One of the main characteristics of the skin-colored region tracker (SCRT) instrument is its ability to report the 3D positions of SCRs in a world-centered coordinate system by employing a possibly moving stereo rig with independently verging CCD cameras. The system operates on images of dimensions 640 × 480 pixels at a rate of 13 Hz on a conventional Pentium 4 processor at 1.8 GHz. Representative experimental results from the application of the SCRT to image sequences are also provided.

© 2004 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. K. Meyer, H. L. Applewhite, F. A. Biocca, “A survey of position trackers,” Presence 1, 173–200 (1992).
  2. D. M. Gavrila, “The visual analysis of human movement: a survey,” Comput. Vis. Image Underst. 73, 82–98 (1999).
    [CrossRef]
  3. Q. Delamarre, O. Faugeras, “3D articulated models and multi-view tracking with physical forces,” Comput. (Vis. Image Underst. 81, 328–357 (2001).
    [CrossRef]
  4. T. S. Jebara, A. Pentland, “Parameterized structure from motion for 3D adaptive feedback tracking of faces,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1997), pp. 144–150.
    [CrossRef]
  5. S. H. Kim, N. K. Kim, S. C. Ahn, H. G. Kim, “Object oriented face detection using range and color information,” in Proceedings of the 3rd IEEE International Conference on Automatic Face and Gesture Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1998), pp. 76–81.
  6. M. J. Jones, J. M. Rehg, “Statistical color models with application to skin detection,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1999), pp. 274–280.
  7. D. Saxe, R. Foulds, “Toward robust skin identification in video images,” 2nd International Conference on Automatic Face and Gesture Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1996), pp. 379–384.
  8. D. Chai, K. N. Ngan, “Locating the facial region of a head-and-shoulders color image,” in Proceedings of the 3rd IEEE International Conference on Automatic Face and Gesture Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1998), pp. 124–129.
    [CrossRef]
  9. M. H. Yang, N. Ahuja, “Detecting Human Faces in Color Images,” in Proceedings of the IEEE International Conference on Image Processing (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1998), pp. 127–130.
  10. J. C. Terrillon, M. N. Shirazi, H. Fukamachi, S. Akamatsu, “Comparative performance of different skin chrominance models and chrominance spaces for the automatic detection of human faces in color images,” in Proceedings of IEEE International Conference on Face and Gesture Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 2000), pp. 54–61.
    [CrossRef]
  11. J. Cai, A. Goshtasby, “Detecting human faces in color images,” Image Vis. Comput. 18, 63–75 (1999).
    [CrossRef]
  12. S. McKenna, Y. Raja, S. Gong, “Tracking color objects using adaptive mixture models,” Image Vis. Comput. 17, 225–231 (1999).
    [CrossRef]
  13. Y. Raja, S. McKenna, G. Gong, “Tracking and segmenting people in varying lighting conditions using color,” in Proceedings of the 3rd IEEE International Conference on Automatic Face and Gesture Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1998), pp. 228–233.
    [CrossRef]
  14. T. S. Jebara, A. Pentland, “Parametrized structure from motion for 3D adaptive feedback tracking of faces,” in Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1997), pp. 144–150.
    [CrossRef]
  15. T. S. Jebara, K. Russel, A. Pentland, “Mixture of eigenfeatures for real-time structure from texture,” in Proceedings of the Sixth International Conference on Computer Vision (Narosa, Bombay, 1998), pp. 128–135.
  16. M. H. Yang, N. Ahuja, Face Detection and Gesture Recognition for Human-Computer Interaction (Kluwer Academic, Dordrecht, The Netherlands, 2001).
    [CrossRef]
  17. M. H. Yang, D. J. Kriegman, N. Ahuja, “Detecting faces in images: a survey,” IEEE Trans. Pattern Anal. Mach. Intell. 24, 34–58 (2002).
    [CrossRef]
  18. R. E. Kalman, “A new approach to linear filtering and prediction problems,” Trans. ASME Ser. D. 82, 35–45 (1960).
    [CrossRef]
  19. D. Comaniciu, V. Ramesh, P. Meer, “Real-time tracking of non-rigid objects using mean shift,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 2000), pp. 142–151.
  20. O. Javed, M. Shah, “Tracking and object classification for automated surveillance,” in European Conference on Computer Vision (Springer-Verlag, Berlin, 2002), pp. 343–357.
  21. N. T. Siebel, S. Maybank, “Fusion of multiple tracking algorithms for robust people tracking,” in European Conference on Computer Vision (Springer-Verlag, Berlin, 2002), pp. 373–387.
  22. M. Spengler, B. Schiele, “Multi-object tracking based on a modular knowledge hierarchy,” in International Conference on Computer Vision Systems (Springer-VerlagHeidelberg, 2003), pp. 376–385.
    [CrossRef]
  23. C. Stauffer, W. Grimson, “Adaptive background mixture models for real-time tracking,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1999), pp. 246–252.
  24. J. Triesch, C. von der Malsburg, “Democratic integration: self-organized integration of adaptive cues,” Neural Comput. 13, 2049–2074 (2001).
    [CrossRef] [PubMed]
  25. R. Fablet, M. J. Black, “Automatic detection and tracking of human motion with a view-based representation,” in European Conference on Computer Vision (Springer-Verlag, Berlin, 2002), pp. 476–491.
  26. M. Isard, A. Blake, “Icondensation: unifying low-level and high-level tracking in a stochastic framework,” in European Conference on Computer Vision (Springer-Verlag, Berlin, 1998), pp. 893–908.
  27. J. Vermaak, P. Perez, M. Gangnet, A. Blake, “Towards improved observation models for visual tracking: selective adaptation,” in European Conference on Computer Vision (Springer-Verlag, Berlin, 2002), pp. 645–660.
  28. C. Hue, J.-P. Le Cadre, P. Pérez, “Sequential Monte Carlo methods for multiple target tracking and data fusion,” IEEE Trans. Signal Process. 50, 309–325 (2002).
    [CrossRef]
  29. M. Isard, J. MacCormick, “Bramble: a Bayesian multiple-blob tracker,” in Proceedings of the International Conference on Computer Vision ICCV (IEEE Computer Society, Los Alamitos, Calif., 2001).
  30. E. Koller-Meier, F. Ade, “Tracking multiple objects using the condensation algorithm,” J. Robot. Auton. Syst. 34(2–3), 93–105 (2001).
    [CrossRef]
  31. P. Perez, C. Hue, J. Vermaak, M. Gangnet, “Color-based probabilistic tracking,” Proceedings of the European Conference on Computer Vision (Springer-Verlag, Berlin, 2002), pp. 661–675.
  32. Y. Li, A. Hilton, J. Illingworth, “A relaxation algorithm for real-time multiple view 3D-tracking,” Image Vis. Comput. 20, 841–859 (2002).
    [CrossRef]
  33. T. Inaguma, K. Oomura, H. Saji, H. Nakatani, “Efficient Search Technique for Hand Gesture Tracking in Three Dimensions”, in International Workshop on Biologically Motivated Computer Vision (Springer-Verlag, Berlin, 2000), pp. 594–601.
    [CrossRef]
  34. R. Hartley, P. Sturm, “Triangulation,” Compu. Vis. Image Underst. 68, 146–157 (1997).
    [CrossRef]
  35. O. Faugeras, Q.-T. Luong, T. Papadopoulo, The Geometry of Multiple Images (MIT Press, Cambridge, Mass., 2001).
  36. S. O. Orphanoudakis, A. A. Argyros, M. Vincze, “Towards a cognitive vision methodology: understanding and interpreting activities of experts,” ERCIM News, No. 53 (ERCIM EEIG, Sophia-Antipolis, France, 2003); http://www.ercim.org .
  37. K. Jack, Video Demystified: A Handbook for the Digital Engineer (HighText, Solana Beach, Calif., 1993).
  38. D. A. Forsyth, J. Ponce, Computer Vision: A Modern Approach (Prentice-Hall, Englewood Cliffs, N.J., 2003).
  39. J. F. Canny, “A computational approach to edge detection,” IEEE Trans. Pattern Anal. Mach. Intell. 8, 769–798 (1986).
  40. L. Robert, C. Zeller, O. D. Faugeras, M. Hebert, “Applications of non-metric vision to some visually guided robotic tasks,” in Visual Navigation: From Biological Systems to Unmanned Ground Vehicles, Y. Aloimonos, ed. (Erlbaum, Hillsdale, N.J., 1997), Chap. 5, pp. 89–134.
  41. Z. Zhang, “Determining the epipolar geometry and its uncertainty: a review,” Int. J. Comput. Vision 27, 161–195 (1998).
    [CrossRef]
  42. H. Hirschmüller, “Improvements in real-time correlation-based stereo vision,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 2001), pp. 141–148.
  43. R. Goldman, “Intersection of two lines in three-space,” in Graphics Gems, A. S. Glassner, ed. (Academic, San Diego, Calif., 1990), Vol. 1, p. 304.
  44. M. I. A. Lourakis, A. A. Argyros, “Efficient 3D camera matchmoving using markerless, segmentation-free plane tracking,” Technical Report ICS/FORTH-TR-324 (Institute of Computer Science, Foundation for Research and Technology—Hellas, Heraklion, Greece, Sept.2003).

2002

M. H. Yang, D. J. Kriegman, N. Ahuja, “Detecting faces in images: a survey,” IEEE Trans. Pattern Anal. Mach. Intell. 24, 34–58 (2002).
[CrossRef]

C. Hue, J.-P. Le Cadre, P. Pérez, “Sequential Monte Carlo methods for multiple target tracking and data fusion,” IEEE Trans. Signal Process. 50, 309–325 (2002).
[CrossRef]

Y. Li, A. Hilton, J. Illingworth, “A relaxation algorithm for real-time multiple view 3D-tracking,” Image Vis. Comput. 20, 841–859 (2002).
[CrossRef]

2001

J. Triesch, C. von der Malsburg, “Democratic integration: self-organized integration of adaptive cues,” Neural Comput. 13, 2049–2074 (2001).
[CrossRef] [PubMed]

E. Koller-Meier, F. Ade, “Tracking multiple objects using the condensation algorithm,” J. Robot. Auton. Syst. 34(2–3), 93–105 (2001).
[CrossRef]

Q. Delamarre, O. Faugeras, “3D articulated models and multi-view tracking with physical forces,” Comput. (Vis. Image Underst. 81, 328–357 (2001).
[CrossRef]

1999

J. Cai, A. Goshtasby, “Detecting human faces in color images,” Image Vis. Comput. 18, 63–75 (1999).
[CrossRef]

S. McKenna, Y. Raja, S. Gong, “Tracking color objects using adaptive mixture models,” Image Vis. Comput. 17, 225–231 (1999).
[CrossRef]

D. M. Gavrila, “The visual analysis of human movement: a survey,” Comput. Vis. Image Underst. 73, 82–98 (1999).
[CrossRef]

1998

Z. Zhang, “Determining the epipolar geometry and its uncertainty: a review,” Int. J. Comput. Vision 27, 161–195 (1998).
[CrossRef]

1997

R. Hartley, P. Sturm, “Triangulation,” Compu. Vis. Image Underst. 68, 146–157 (1997).
[CrossRef]

1992

K. Meyer, H. L. Applewhite, F. A. Biocca, “A survey of position trackers,” Presence 1, 173–200 (1992).

1986

J. F. Canny, “A computational approach to edge detection,” IEEE Trans. Pattern Anal. Mach. Intell. 8, 769–798 (1986).

1960

R. E. Kalman, “A new approach to linear filtering and prediction problems,” Trans. ASME Ser. D. 82, 35–45 (1960).
[CrossRef]

Ade, F.

E. Koller-Meier, F. Ade, “Tracking multiple objects using the condensation algorithm,” J. Robot. Auton. Syst. 34(2–3), 93–105 (2001).
[CrossRef]

Ahn, S. C.

S. H. Kim, N. K. Kim, S. C. Ahn, H. G. Kim, “Object oriented face detection using range and color information,” in Proceedings of the 3rd IEEE International Conference on Automatic Face and Gesture Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1998), pp. 76–81.

Ahuja, N.

M. H. Yang, D. J. Kriegman, N. Ahuja, “Detecting faces in images: a survey,” IEEE Trans. Pattern Anal. Mach. Intell. 24, 34–58 (2002).
[CrossRef]

M. H. Yang, N. Ahuja, “Detecting Human Faces in Color Images,” in Proceedings of the IEEE International Conference on Image Processing (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1998), pp. 127–130.

M. H. Yang, N. Ahuja, Face Detection and Gesture Recognition for Human-Computer Interaction (Kluwer Academic, Dordrecht, The Netherlands, 2001).
[CrossRef]

Akamatsu, S.

J. C. Terrillon, M. N. Shirazi, H. Fukamachi, S. Akamatsu, “Comparative performance of different skin chrominance models and chrominance spaces for the automatic detection of human faces in color images,” in Proceedings of IEEE International Conference on Face and Gesture Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 2000), pp. 54–61.
[CrossRef]

Applewhite, H. L.

K. Meyer, H. L. Applewhite, F. A. Biocca, “A survey of position trackers,” Presence 1, 173–200 (1992).

Argyros, A. A.

M. I. A. Lourakis, A. A. Argyros, “Efficient 3D camera matchmoving using markerless, segmentation-free plane tracking,” Technical Report ICS/FORTH-TR-324 (Institute of Computer Science, Foundation for Research and Technology—Hellas, Heraklion, Greece, Sept.2003).

S. O. Orphanoudakis, A. A. Argyros, M. Vincze, “Towards a cognitive vision methodology: understanding and interpreting activities of experts,” ERCIM News, No. 53 (ERCIM EEIG, Sophia-Antipolis, France, 2003); http://www.ercim.org .

Biocca, F. A.

K. Meyer, H. L. Applewhite, F. A. Biocca, “A survey of position trackers,” Presence 1, 173–200 (1992).

Black, M. J.

R. Fablet, M. J. Black, “Automatic detection and tracking of human motion with a view-based representation,” in European Conference on Computer Vision (Springer-Verlag, Berlin, 2002), pp. 476–491.

Blake, A.

J. Vermaak, P. Perez, M. Gangnet, A. Blake, “Towards improved observation models for visual tracking: selective adaptation,” in European Conference on Computer Vision (Springer-Verlag, Berlin, 2002), pp. 645–660.

M. Isard, A. Blake, “Icondensation: unifying low-level and high-level tracking in a stochastic framework,” in European Conference on Computer Vision (Springer-Verlag, Berlin, 1998), pp. 893–908.

Cai, J.

J. Cai, A. Goshtasby, “Detecting human faces in color images,” Image Vis. Comput. 18, 63–75 (1999).
[CrossRef]

Canny, J. F.

J. F. Canny, “A computational approach to edge detection,” IEEE Trans. Pattern Anal. Mach. Intell. 8, 769–798 (1986).

Chai, D.

D. Chai, K. N. Ngan, “Locating the facial region of a head-and-shoulders color image,” in Proceedings of the 3rd IEEE International Conference on Automatic Face and Gesture Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1998), pp. 124–129.
[CrossRef]

Comaniciu, D.

D. Comaniciu, V. Ramesh, P. Meer, “Real-time tracking of non-rigid objects using mean shift,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 2000), pp. 142–151.

Delamarre, Q.

Q. Delamarre, O. Faugeras, “3D articulated models and multi-view tracking with physical forces,” Comput. (Vis. Image Underst. 81, 328–357 (2001).
[CrossRef]

Fablet, R.

R. Fablet, M. J. Black, “Automatic detection and tracking of human motion with a view-based representation,” in European Conference on Computer Vision (Springer-Verlag, Berlin, 2002), pp. 476–491.

Faugeras, O.

Q. Delamarre, O. Faugeras, “3D articulated models and multi-view tracking with physical forces,” Comput. (Vis. Image Underst. 81, 328–357 (2001).
[CrossRef]

O. Faugeras, Q.-T. Luong, T. Papadopoulo, The Geometry of Multiple Images (MIT Press, Cambridge, Mass., 2001).

Faugeras, O. D.

L. Robert, C. Zeller, O. D. Faugeras, M. Hebert, “Applications of non-metric vision to some visually guided robotic tasks,” in Visual Navigation: From Biological Systems to Unmanned Ground Vehicles, Y. Aloimonos, ed. (Erlbaum, Hillsdale, N.J., 1997), Chap. 5, pp. 89–134.

Forsyth, D. A.

D. A. Forsyth, J. Ponce, Computer Vision: A Modern Approach (Prentice-Hall, Englewood Cliffs, N.J., 2003).

Foulds, R.

D. Saxe, R. Foulds, “Toward robust skin identification in video images,” 2nd International Conference on Automatic Face and Gesture Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1996), pp. 379–384.

Fukamachi, H.

J. C. Terrillon, M. N. Shirazi, H. Fukamachi, S. Akamatsu, “Comparative performance of different skin chrominance models and chrominance spaces for the automatic detection of human faces in color images,” in Proceedings of IEEE International Conference on Face and Gesture Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 2000), pp. 54–61.
[CrossRef]

Gangnet, M.

P. Perez, C. Hue, J. Vermaak, M. Gangnet, “Color-based probabilistic tracking,” Proceedings of the European Conference on Computer Vision (Springer-Verlag, Berlin, 2002), pp. 661–675.

J. Vermaak, P. Perez, M. Gangnet, A. Blake, “Towards improved observation models for visual tracking: selective adaptation,” in European Conference on Computer Vision (Springer-Verlag, Berlin, 2002), pp. 645–660.

Gavrila, D. M.

D. M. Gavrila, “The visual analysis of human movement: a survey,” Comput. Vis. Image Underst. 73, 82–98 (1999).
[CrossRef]

Goldman, R.

R. Goldman, “Intersection of two lines in three-space,” in Graphics Gems, A. S. Glassner, ed. (Academic, San Diego, Calif., 1990), Vol. 1, p. 304.

Gong, G.

Y. Raja, S. McKenna, G. Gong, “Tracking and segmenting people in varying lighting conditions using color,” in Proceedings of the 3rd IEEE International Conference on Automatic Face and Gesture Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1998), pp. 228–233.
[CrossRef]

Gong, S.

S. McKenna, Y. Raja, S. Gong, “Tracking color objects using adaptive mixture models,” Image Vis. Comput. 17, 225–231 (1999).
[CrossRef]

Goshtasby, A.

J. Cai, A. Goshtasby, “Detecting human faces in color images,” Image Vis. Comput. 18, 63–75 (1999).
[CrossRef]

Grimson, W.

C. Stauffer, W. Grimson, “Adaptive background mixture models for real-time tracking,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1999), pp. 246–252.

Hartley, R.

R. Hartley, P. Sturm, “Triangulation,” Compu. Vis. Image Underst. 68, 146–157 (1997).
[CrossRef]

Hebert, M.

L. Robert, C. Zeller, O. D. Faugeras, M. Hebert, “Applications of non-metric vision to some visually guided robotic tasks,” in Visual Navigation: From Biological Systems to Unmanned Ground Vehicles, Y. Aloimonos, ed. (Erlbaum, Hillsdale, N.J., 1997), Chap. 5, pp. 89–134.

Hilton, A.

Y. Li, A. Hilton, J. Illingworth, “A relaxation algorithm for real-time multiple view 3D-tracking,” Image Vis. Comput. 20, 841–859 (2002).
[CrossRef]

Hirschmüller, H.

H. Hirschmüller, “Improvements in real-time correlation-based stereo vision,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 2001), pp. 141–148.

Hue, C.

C. Hue, J.-P. Le Cadre, P. Pérez, “Sequential Monte Carlo methods for multiple target tracking and data fusion,” IEEE Trans. Signal Process. 50, 309–325 (2002).
[CrossRef]

P. Perez, C. Hue, J. Vermaak, M. Gangnet, “Color-based probabilistic tracking,” Proceedings of the European Conference on Computer Vision (Springer-Verlag, Berlin, 2002), pp. 661–675.

Illingworth, J.

Y. Li, A. Hilton, J. Illingworth, “A relaxation algorithm for real-time multiple view 3D-tracking,” Image Vis. Comput. 20, 841–859 (2002).
[CrossRef]

Inaguma, T.

T. Inaguma, K. Oomura, H. Saji, H. Nakatani, “Efficient Search Technique for Hand Gesture Tracking in Three Dimensions”, in International Workshop on Biologically Motivated Computer Vision (Springer-Verlag, Berlin, 2000), pp. 594–601.
[CrossRef]

Isard, M.

M. Isard, J. MacCormick, “Bramble: a Bayesian multiple-blob tracker,” in Proceedings of the International Conference on Computer Vision ICCV (IEEE Computer Society, Los Alamitos, Calif., 2001).

M. Isard, A. Blake, “Icondensation: unifying low-level and high-level tracking in a stochastic framework,” in European Conference on Computer Vision (Springer-Verlag, Berlin, 1998), pp. 893–908.

Jack, K.

K. Jack, Video Demystified: A Handbook for the Digital Engineer (HighText, Solana Beach, Calif., 1993).

Javed, O.

O. Javed, M. Shah, “Tracking and object classification for automated surveillance,” in European Conference on Computer Vision (Springer-Verlag, Berlin, 2002), pp. 343–357.

Jebara, T. S.

T. S. Jebara, K. Russel, A. Pentland, “Mixture of eigenfeatures for real-time structure from texture,” in Proceedings of the Sixth International Conference on Computer Vision (Narosa, Bombay, 1998), pp. 128–135.

T. S. Jebara, A. Pentland, “Parametrized structure from motion for 3D adaptive feedback tracking of faces,” in Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1997), pp. 144–150.
[CrossRef]

T. S. Jebara, A. Pentland, “Parameterized structure from motion for 3D adaptive feedback tracking of faces,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1997), pp. 144–150.
[CrossRef]

Jones, M. J.

M. J. Jones, J. M. Rehg, “Statistical color models with application to skin detection,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1999), pp. 274–280.

Kalman, R. E.

R. E. Kalman, “A new approach to linear filtering and prediction problems,” Trans. ASME Ser. D. 82, 35–45 (1960).
[CrossRef]

Kim, H. G.

S. H. Kim, N. K. Kim, S. C. Ahn, H. G. Kim, “Object oriented face detection using range and color information,” in Proceedings of the 3rd IEEE International Conference on Automatic Face and Gesture Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1998), pp. 76–81.

Kim, N. K.

S. H. Kim, N. K. Kim, S. C. Ahn, H. G. Kim, “Object oriented face detection using range and color information,” in Proceedings of the 3rd IEEE International Conference on Automatic Face and Gesture Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1998), pp. 76–81.

Kim, S. H.

S. H. Kim, N. K. Kim, S. C. Ahn, H. G. Kim, “Object oriented face detection using range and color information,” in Proceedings of the 3rd IEEE International Conference on Automatic Face and Gesture Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1998), pp. 76–81.

Koller-Meier, E.

E. Koller-Meier, F. Ade, “Tracking multiple objects using the condensation algorithm,” J. Robot. Auton. Syst. 34(2–3), 93–105 (2001).
[CrossRef]

Kriegman, D. J.

M. H. Yang, D. J. Kriegman, N. Ahuja, “Detecting faces in images: a survey,” IEEE Trans. Pattern Anal. Mach. Intell. 24, 34–58 (2002).
[CrossRef]

Le Cadre, J.-P.

C. Hue, J.-P. Le Cadre, P. Pérez, “Sequential Monte Carlo methods for multiple target tracking and data fusion,” IEEE Trans. Signal Process. 50, 309–325 (2002).
[CrossRef]

Li, Y.

Y. Li, A. Hilton, J. Illingworth, “A relaxation algorithm for real-time multiple view 3D-tracking,” Image Vis. Comput. 20, 841–859 (2002).
[CrossRef]

Lourakis, M. I. A.

M. I. A. Lourakis, A. A. Argyros, “Efficient 3D camera matchmoving using markerless, segmentation-free plane tracking,” Technical Report ICS/FORTH-TR-324 (Institute of Computer Science, Foundation for Research and Technology—Hellas, Heraklion, Greece, Sept.2003).

Luong, Q.-T.

O. Faugeras, Q.-T. Luong, T. Papadopoulo, The Geometry of Multiple Images (MIT Press, Cambridge, Mass., 2001).

MacCormick, J.

M. Isard, J. MacCormick, “Bramble: a Bayesian multiple-blob tracker,” in Proceedings of the International Conference on Computer Vision ICCV (IEEE Computer Society, Los Alamitos, Calif., 2001).

Maybank, S.

N. T. Siebel, S. Maybank, “Fusion of multiple tracking algorithms for robust people tracking,” in European Conference on Computer Vision (Springer-Verlag, Berlin, 2002), pp. 373–387.

McKenna, S.

S. McKenna, Y. Raja, S. Gong, “Tracking color objects using adaptive mixture models,” Image Vis. Comput. 17, 225–231 (1999).
[CrossRef]

Y. Raja, S. McKenna, G. Gong, “Tracking and segmenting people in varying lighting conditions using color,” in Proceedings of the 3rd IEEE International Conference on Automatic Face and Gesture Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1998), pp. 228–233.
[CrossRef]

Meer, P.

D. Comaniciu, V. Ramesh, P. Meer, “Real-time tracking of non-rigid objects using mean shift,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 2000), pp. 142–151.

Meyer, K.

K. Meyer, H. L. Applewhite, F. A. Biocca, “A survey of position trackers,” Presence 1, 173–200 (1992).

Nakatani, H.

T. Inaguma, K. Oomura, H. Saji, H. Nakatani, “Efficient Search Technique for Hand Gesture Tracking in Three Dimensions”, in International Workshop on Biologically Motivated Computer Vision (Springer-Verlag, Berlin, 2000), pp. 594–601.
[CrossRef]

Ngan, K. N.

D. Chai, K. N. Ngan, “Locating the facial region of a head-and-shoulders color image,” in Proceedings of the 3rd IEEE International Conference on Automatic Face and Gesture Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1998), pp. 124–129.
[CrossRef]

Oomura, K.

T. Inaguma, K. Oomura, H. Saji, H. Nakatani, “Efficient Search Technique for Hand Gesture Tracking in Three Dimensions”, in International Workshop on Biologically Motivated Computer Vision (Springer-Verlag, Berlin, 2000), pp. 594–601.
[CrossRef]

Orphanoudakis, S. O.

S. O. Orphanoudakis, A. A. Argyros, M. Vincze, “Towards a cognitive vision methodology: understanding and interpreting activities of experts,” ERCIM News, No. 53 (ERCIM EEIG, Sophia-Antipolis, France, 2003); http://www.ercim.org .

Papadopoulo, T.

O. Faugeras, Q.-T. Luong, T. Papadopoulo, The Geometry of Multiple Images (MIT Press, Cambridge, Mass., 2001).

Pentland, A.

T. S. Jebara, A. Pentland, “Parametrized structure from motion for 3D adaptive feedback tracking of faces,” in Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1997), pp. 144–150.
[CrossRef]

T. S. Jebara, A. Pentland, “Parameterized structure from motion for 3D adaptive feedback tracking of faces,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1997), pp. 144–150.
[CrossRef]

T. S. Jebara, K. Russel, A. Pentland, “Mixture of eigenfeatures for real-time structure from texture,” in Proceedings of the Sixth International Conference on Computer Vision (Narosa, Bombay, 1998), pp. 128–135.

Perez, P.

J. Vermaak, P. Perez, M. Gangnet, A. Blake, “Towards improved observation models for visual tracking: selective adaptation,” in European Conference on Computer Vision (Springer-Verlag, Berlin, 2002), pp. 645–660.

P. Perez, C. Hue, J. Vermaak, M. Gangnet, “Color-based probabilistic tracking,” Proceedings of the European Conference on Computer Vision (Springer-Verlag, Berlin, 2002), pp. 661–675.

Pérez, P.

C. Hue, J.-P. Le Cadre, P. Pérez, “Sequential Monte Carlo methods for multiple target tracking and data fusion,” IEEE Trans. Signal Process. 50, 309–325 (2002).
[CrossRef]

Ponce, J.

D. A. Forsyth, J. Ponce, Computer Vision: A Modern Approach (Prentice-Hall, Englewood Cliffs, N.J., 2003).

Raja, Y.

S. McKenna, Y. Raja, S. Gong, “Tracking color objects using adaptive mixture models,” Image Vis. Comput. 17, 225–231 (1999).
[CrossRef]

Y. Raja, S. McKenna, G. Gong, “Tracking and segmenting people in varying lighting conditions using color,” in Proceedings of the 3rd IEEE International Conference on Automatic Face and Gesture Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1998), pp. 228–233.
[CrossRef]

Ramesh, V.

D. Comaniciu, V. Ramesh, P. Meer, “Real-time tracking of non-rigid objects using mean shift,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 2000), pp. 142–151.

Rehg, J. M.

M. J. Jones, J. M. Rehg, “Statistical color models with application to skin detection,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1999), pp. 274–280.

Robert, L.

L. Robert, C. Zeller, O. D. Faugeras, M. Hebert, “Applications of non-metric vision to some visually guided robotic tasks,” in Visual Navigation: From Biological Systems to Unmanned Ground Vehicles, Y. Aloimonos, ed. (Erlbaum, Hillsdale, N.J., 1997), Chap. 5, pp. 89–134.

Russel, K.

T. S. Jebara, K. Russel, A. Pentland, “Mixture of eigenfeatures for real-time structure from texture,” in Proceedings of the Sixth International Conference on Computer Vision (Narosa, Bombay, 1998), pp. 128–135.

Saji, H.

T. Inaguma, K. Oomura, H. Saji, H. Nakatani, “Efficient Search Technique for Hand Gesture Tracking in Three Dimensions”, in International Workshop on Biologically Motivated Computer Vision (Springer-Verlag, Berlin, 2000), pp. 594–601.
[CrossRef]

Saxe, D.

D. Saxe, R. Foulds, “Toward robust skin identification in video images,” 2nd International Conference on Automatic Face and Gesture Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1996), pp. 379–384.

Schiele, B.

M. Spengler, B. Schiele, “Multi-object tracking based on a modular knowledge hierarchy,” in International Conference on Computer Vision Systems (Springer-VerlagHeidelberg, 2003), pp. 376–385.
[CrossRef]

Shah, M.

O. Javed, M. Shah, “Tracking and object classification for automated surveillance,” in European Conference on Computer Vision (Springer-Verlag, Berlin, 2002), pp. 343–357.

Shirazi, M. N.

J. C. Terrillon, M. N. Shirazi, H. Fukamachi, S. Akamatsu, “Comparative performance of different skin chrominance models and chrominance spaces for the automatic detection of human faces in color images,” in Proceedings of IEEE International Conference on Face and Gesture Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 2000), pp. 54–61.
[CrossRef]

Siebel, N. T.

N. T. Siebel, S. Maybank, “Fusion of multiple tracking algorithms for robust people tracking,” in European Conference on Computer Vision (Springer-Verlag, Berlin, 2002), pp. 373–387.

Spengler, M.

M. Spengler, B. Schiele, “Multi-object tracking based on a modular knowledge hierarchy,” in International Conference on Computer Vision Systems (Springer-VerlagHeidelberg, 2003), pp. 376–385.
[CrossRef]

Stauffer, C.

C. Stauffer, W. Grimson, “Adaptive background mixture models for real-time tracking,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1999), pp. 246–252.

Sturm, P.

R. Hartley, P. Sturm, “Triangulation,” Compu. Vis. Image Underst. 68, 146–157 (1997).
[CrossRef]

Terrillon, J. C.

J. C. Terrillon, M. N. Shirazi, H. Fukamachi, S. Akamatsu, “Comparative performance of different skin chrominance models and chrominance spaces for the automatic detection of human faces in color images,” in Proceedings of IEEE International Conference on Face and Gesture Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 2000), pp. 54–61.
[CrossRef]

Triesch, J.

J. Triesch, C. von der Malsburg, “Democratic integration: self-organized integration of adaptive cues,” Neural Comput. 13, 2049–2074 (2001).
[CrossRef] [PubMed]

Vermaak, J.

J. Vermaak, P. Perez, M. Gangnet, A. Blake, “Towards improved observation models for visual tracking: selective adaptation,” in European Conference on Computer Vision (Springer-Verlag, Berlin, 2002), pp. 645–660.

P. Perez, C. Hue, J. Vermaak, M. Gangnet, “Color-based probabilistic tracking,” Proceedings of the European Conference on Computer Vision (Springer-Verlag, Berlin, 2002), pp. 661–675.

Vincze, M.

S. O. Orphanoudakis, A. A. Argyros, M. Vincze, “Towards a cognitive vision methodology: understanding and interpreting activities of experts,” ERCIM News, No. 53 (ERCIM EEIG, Sophia-Antipolis, France, 2003); http://www.ercim.org .

von der Malsburg, C.

J. Triesch, C. von der Malsburg, “Democratic integration: self-organized integration of adaptive cues,” Neural Comput. 13, 2049–2074 (2001).
[CrossRef] [PubMed]

Yang, M. H.

M. H. Yang, D. J. Kriegman, N. Ahuja, “Detecting faces in images: a survey,” IEEE Trans. Pattern Anal. Mach. Intell. 24, 34–58 (2002).
[CrossRef]

M. H. Yang, N. Ahuja, “Detecting Human Faces in Color Images,” in Proceedings of the IEEE International Conference on Image Processing (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1998), pp. 127–130.

M. H. Yang, N. Ahuja, Face Detection and Gesture Recognition for Human-Computer Interaction (Kluwer Academic, Dordrecht, The Netherlands, 2001).
[CrossRef]

Zeller, C.

L. Robert, C. Zeller, O. D. Faugeras, M. Hebert, “Applications of non-metric vision to some visually guided robotic tasks,” in Visual Navigation: From Biological Systems to Unmanned Ground Vehicles, Y. Aloimonos, ed. (Erlbaum, Hillsdale, N.J., 1997), Chap. 5, pp. 89–134.

Zhang, Z.

Z. Zhang, “Determining the epipolar geometry and its uncertainty: a review,” Int. J. Comput. Vision 27, 161–195 (1998).
[CrossRef]

Compu. Vis. Image Underst.

R. Hartley, P. Sturm, “Triangulation,” Compu. Vis. Image Underst. 68, 146–157 (1997).
[CrossRef]

Comput. (Vis. Image Underst.

Q. Delamarre, O. Faugeras, “3D articulated models and multi-view tracking with physical forces,” Comput. (Vis. Image Underst. 81, 328–357 (2001).
[CrossRef]

Comput. Vis. Image Underst.

D. M. Gavrila, “The visual analysis of human movement: a survey,” Comput. Vis. Image Underst. 73, 82–98 (1999).
[CrossRef]

IEEE Trans. Pattern Anal. Mach. Intell.

M. H. Yang, D. J. Kriegman, N. Ahuja, “Detecting faces in images: a survey,” IEEE Trans. Pattern Anal. Mach. Intell. 24, 34–58 (2002).
[CrossRef]

J. F. Canny, “A computational approach to edge detection,” IEEE Trans. Pattern Anal. Mach. Intell. 8, 769–798 (1986).

IEEE Trans. Signal Process.

C. Hue, J.-P. Le Cadre, P. Pérez, “Sequential Monte Carlo methods for multiple target tracking and data fusion,” IEEE Trans. Signal Process. 50, 309–325 (2002).
[CrossRef]

Image Vis. Comput.

Y. Li, A. Hilton, J. Illingworth, “A relaxation algorithm for real-time multiple view 3D-tracking,” Image Vis. Comput. 20, 841–859 (2002).
[CrossRef]

J. Cai, A. Goshtasby, “Detecting human faces in color images,” Image Vis. Comput. 18, 63–75 (1999).
[CrossRef]

S. McKenna, Y. Raja, S. Gong, “Tracking color objects using adaptive mixture models,” Image Vis. Comput. 17, 225–231 (1999).
[CrossRef]

Int. J. Comput. Vision

Z. Zhang, “Determining the epipolar geometry and its uncertainty: a review,” Int. J. Comput. Vision 27, 161–195 (1998).
[CrossRef]

J. Robot. Auton. Syst.

E. Koller-Meier, F. Ade, “Tracking multiple objects using the condensation algorithm,” J. Robot. Auton. Syst. 34(2–3), 93–105 (2001).
[CrossRef]

Neural Comput.

J. Triesch, C. von der Malsburg, “Democratic integration: self-organized integration of adaptive cues,” Neural Comput. 13, 2049–2074 (2001).
[CrossRef] [PubMed]

Presence

K. Meyer, H. L. Applewhite, F. A. Biocca, “A survey of position trackers,” Presence 1, 173–200 (1992).

Trans. ASME Ser. D.

R. E. Kalman, “A new approach to linear filtering and prediction problems,” Trans. ASME Ser. D. 82, 35–45 (1960).
[CrossRef]

Other

D. Comaniciu, V. Ramesh, P. Meer, “Real-time tracking of non-rigid objects using mean shift,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 2000), pp. 142–151.

O. Javed, M. Shah, “Tracking and object classification for automated surveillance,” in European Conference on Computer Vision (Springer-Verlag, Berlin, 2002), pp. 343–357.

N. T. Siebel, S. Maybank, “Fusion of multiple tracking algorithms for robust people tracking,” in European Conference on Computer Vision (Springer-Verlag, Berlin, 2002), pp. 373–387.

M. Spengler, B. Schiele, “Multi-object tracking based on a modular knowledge hierarchy,” in International Conference on Computer Vision Systems (Springer-VerlagHeidelberg, 2003), pp. 376–385.
[CrossRef]

C. Stauffer, W. Grimson, “Adaptive background mixture models for real-time tracking,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1999), pp. 246–252.

Y. Raja, S. McKenna, G. Gong, “Tracking and segmenting people in varying lighting conditions using color,” in Proceedings of the 3rd IEEE International Conference on Automatic Face and Gesture Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1998), pp. 228–233.
[CrossRef]

T. S. Jebara, A. Pentland, “Parametrized structure from motion for 3D adaptive feedback tracking of faces,” in Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1997), pp. 144–150.
[CrossRef]

T. S. Jebara, K. Russel, A. Pentland, “Mixture of eigenfeatures for real-time structure from texture,” in Proceedings of the Sixth International Conference on Computer Vision (Narosa, Bombay, 1998), pp. 128–135.

M. H. Yang, N. Ahuja, Face Detection and Gesture Recognition for Human-Computer Interaction (Kluwer Academic, Dordrecht, The Netherlands, 2001).
[CrossRef]

T. S. Jebara, A. Pentland, “Parameterized structure from motion for 3D adaptive feedback tracking of faces,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1997), pp. 144–150.
[CrossRef]

S. H. Kim, N. K. Kim, S. C. Ahn, H. G. Kim, “Object oriented face detection using range and color information,” in Proceedings of the 3rd IEEE International Conference on Automatic Face and Gesture Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1998), pp. 76–81.

M. J. Jones, J. M. Rehg, “Statistical color models with application to skin detection,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1999), pp. 274–280.

D. Saxe, R. Foulds, “Toward robust skin identification in video images,” 2nd International Conference on Automatic Face and Gesture Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1996), pp. 379–384.

D. Chai, K. N. Ngan, “Locating the facial region of a head-and-shoulders color image,” in Proceedings of the 3rd IEEE International Conference on Automatic Face and Gesture Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1998), pp. 124–129.
[CrossRef]

M. H. Yang, N. Ahuja, “Detecting Human Faces in Color Images,” in Proceedings of the IEEE International Conference on Image Processing (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1998), pp. 127–130.

J. C. Terrillon, M. N. Shirazi, H. Fukamachi, S. Akamatsu, “Comparative performance of different skin chrominance models and chrominance spaces for the automatic detection of human faces in color images,” in Proceedings of IEEE International Conference on Face and Gesture Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 2000), pp. 54–61.
[CrossRef]

M. Isard, J. MacCormick, “Bramble: a Bayesian multiple-blob tracker,” in Proceedings of the International Conference on Computer Vision ICCV (IEEE Computer Society, Los Alamitos, Calif., 2001).

R. Fablet, M. J. Black, “Automatic detection and tracking of human motion with a view-based representation,” in European Conference on Computer Vision (Springer-Verlag, Berlin, 2002), pp. 476–491.

M. Isard, A. Blake, “Icondensation: unifying low-level and high-level tracking in a stochastic framework,” in European Conference on Computer Vision (Springer-Verlag, Berlin, 1998), pp. 893–908.

J. Vermaak, P. Perez, M. Gangnet, A. Blake, “Towards improved observation models for visual tracking: selective adaptation,” in European Conference on Computer Vision (Springer-Verlag, Berlin, 2002), pp. 645–660.

P. Perez, C. Hue, J. Vermaak, M. Gangnet, “Color-based probabilistic tracking,” Proceedings of the European Conference on Computer Vision (Springer-Verlag, Berlin, 2002), pp. 661–675.

T. Inaguma, K. Oomura, H. Saji, H. Nakatani, “Efficient Search Technique for Hand Gesture Tracking in Three Dimensions”, in International Workshop on Biologically Motivated Computer Vision (Springer-Verlag, Berlin, 2000), pp. 594–601.
[CrossRef]

O. Faugeras, Q.-T. Luong, T. Papadopoulo, The Geometry of Multiple Images (MIT Press, Cambridge, Mass., 2001).

S. O. Orphanoudakis, A. A. Argyros, M. Vincze, “Towards a cognitive vision methodology: understanding and interpreting activities of experts,” ERCIM News, No. 53 (ERCIM EEIG, Sophia-Antipolis, France, 2003); http://www.ercim.org .

K. Jack, Video Demystified: A Handbook for the Digital Engineer (HighText, Solana Beach, Calif., 1993).

D. A. Forsyth, J. Ponce, Computer Vision: A Modern Approach (Prentice-Hall, Englewood Cliffs, N.J., 2003).

H. Hirschmüller, “Improvements in real-time correlation-based stereo vision,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 2001), pp. 141–148.

R. Goldman, “Intersection of two lines in three-space,” in Graphics Gems, A. S. Glassner, ed. (Academic, San Diego, Calif., 1990), Vol. 1, p. 304.

M. I. A. Lourakis, A. A. Argyros, “Efficient 3D camera matchmoving using markerless, segmentation-free plane tracking,” Technical Report ICS/FORTH-TR-324 (Institute of Computer Science, Foundation for Research and Technology—Hellas, Heraklion, Greece, Sept.2003).

L. Robert, C. Zeller, O. D. Faugeras, M. Hebert, “Applications of non-metric vision to some visually guided robotic tasks,” in Visual Navigation: From Biological Systems to Unmanned Ground Vehicles, Y. Aloimonos, ed. (Erlbaum, Hillsdale, N.J., 1997), Chap. 5, pp. 89–134.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (9)

Fig. 1
Fig. 1

Stereoscopic head (courtesy of Profactor GmbH) that is used to acquire stereo image pairs that are fed to the SCRT system.

Fig. 2
Fig. 2

Block diagram of the SCRT system.

Fig. 3
Fig. 3

Graphic illustration of the epipolar geometry of a stereo pair. Epipolar plane C L MC R intersects the two image planes along epipolar lines l L and l R .

Fig. 4
Fig. 4

Two methods of achieving the propagation of labels of SCRs both in time and between two stereo views. (a) The AS module is used to match SCRs of the left and the right images of the stereo pair at each point in time. The AT module is used to propagate labels in time in the image sequence of the left camera only. (b) The AS module is used to match SCRs only when a new SCR appears in the field of view. Separate AT modules are then used to propagate the SCR labels in time independently for the left and the right image sequences. As the AT module is typically more robust than the AS module, the second approach is adopted.

Fig. 5
Fig. 5

Configuration of overlapping windows used in the correlation method proposed by Hirschmüller.42

Fig. 6
Fig. 6

Tracking results for Sequence1_head. Each SCR appears as a color blob superimposed upon the right image of the stereo pair.

Fig. 7
Fig. 7

3D trajectories of the SCRs tracked in the experiment of Fig. 6. The top-left and middle-left isolated spots correspond to the motion of the operator’s head and of the armchair’s arm, respectively. The trajectory in the center of the image corresponds to the hand trajectory. The upper facet of the CD player has also been reconstructed, to serve as a reference.

Fig. 8
Fig. 8

Tracking results for Sequence2_arm. Each SCR appears as a color blob superimposed upon the right image of the stereo pair.

Fig. 9
Fig. 9

3D trajectory of the hand detected in the experiment of Fig. 8. The small straight-line segment that appears at the right corresponds to the tray of the CD player.

Equations (17)

Equations on this page are rendered with MathJax. Learn more.

Ps|c=Pc|sPsPc.
PAs|c=aPs|c+1-aPws|c,
DTSCt-1i, SCtj=|mCt-1i-mCtj|.
j=arg min1kNCtDTSCt-1i, SCtk.
TD>|UmaxC|+|UmaxO|fZminν.
F=1detAReRxH,
eR=ARt,
H=ARRAL-1.
DSSLti, SRtj=maxdFmLti, mRtj,dFTmRtj, mLti.
j=arg min1kNRtDSSLti, SRtk.
i=arg min1kNLtDSSRtj, SLtk.
Δ=ΔC+Δmax1+Δmax2.
Z=-mR×eRmR×HmLmR×HmL2,X=ZAL-10mL,Y=ZAL-11mL.
P=½CL+νˆLsL+CR+νˆRsR,
sL=detMR-MLνˆRνLR|νLR|2, sR=detMR-MLνˆLνLR|νLR|2,
νˆL=ML-CL|ML-CL|, νˆR=MR-CR|MR-CR|, νLR=νL×νR.
P=0.6Pt+0.3Pt-1+0.1Pt-2.

Metrics