Abstract

We present an image-based method for face recognition across different illuminations and poses, where the term image-based means that no explicit prior three-dimensional models are needed. As face recognition under illumination and pose variations involves three factors, namely, identity, illumination, and pose, generalizations in all these three factors are desired. We present a recognition approach that is able to generalize in the identity and illumination dimensions and handle a given set of poses. Specifically, the proposed approach derives an identity signature that is illumination- and pose-invariant, where the identity is tackled by means of subspace encoding, the illumination is characterized with a Lambertian reflectance model, and the given set of poses is treated as a whole. Experimental results using the Pose, Illumination, and Expression (PIE) database demonstrate the effectiveness of the proposed approach.

© 2005 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. W. Zhao, R. Chellappa, A. Rosenfeld, J. Phillips, “Face recognition: a literature survey,” ACM Comput. Surv. 12, 399–458 (2003).
    [CrossRef]
  2. P. Phillips, H. Moon, S. Rizvi, P. Rauss, “The FERET evaluation methodology for face-recognition algorithms,” IEEE Trans. Pattern Anal. Mach. Intell. 22, 1090–1104 (2000).
    [CrossRef]
  3. R. Basri, D. Jacobs, “Lambertian reflectance and linear subspaces,” IEEE Trans. Pattern Anal. Mach. Intell. 25, 218–233 (2003).
    [CrossRef]
  4. A. Georghiades, P. Belhumeur, D. Kriegman, “From few to many: illumination cone models for face recognition under variable lighting and pose,” IEEE Trans. Pattern Anal. Mach. Intell. 23, 643–660 (2001).
    [CrossRef]
  5. A. Shashua, T. R. Raviv, “The quotient image: class based re-rendering and recognition with varying illumination,” IEEE Trans. Pattern Anal. Mach. Intell. 23, 129–139 (2001).
    [CrossRef]
  6. A. Yuille, D. Snow, R. Epstein, P. Belhumeur, “Determining generative models for objects under varying illumination: shape and albedo from multiple images using SVD and integrability,” Int. J. Comput. Vision 35, 203–222 (1999).
    [CrossRef]
  7. S. Zhou, R. Chellappa, D. Jacobs, “Characterization of human faces under illumination variations using rank, integrability, and symmetry constraints,” in Proceedings of the European Conference on Computer Vision (Springer-Verlag, Berlin, 2004), Vol. 1, pp. 588–601.
  8. V. Blanz, T. Vetter, “Face identification across different poses and illumination with a 3D morphable model,” IEEE Trans. Pattern Anal. Mach. Intell. 25, 1063–1074 (2003).
    [CrossRef]
  9. P. Fua, “Regularized bundle adjustment to model heads from image sequences without calibrated data,” Int. J. Comput. Vision 38, 153–157 (2000).
    [CrossRef]
  10. A. Roy Chowdhury, R. Chellappa, “Face reconstruction from video using uncertainty analysis and a generic model,” Comput. Vision Image Underst. 91, 188–213 (2003).
    [CrossRef]
  11. Y. Shan, Z. Liu, Z. Zhang, “Model-based bundle adjustment with application to face modeling,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE Press, Piscataway, N.J., 2001), Vol. 2, pp. 644–651.
  12. A. Laurentini, “The visual hull concept for silhouette-based image understanding,” IEEE Trans. Pattern Anal. Mach. Intell. 16, 150–162 (1994).
    [CrossRef]
  13. M. Levoy, P. Hanrahan, “Light field rendering,” in Proceedings of SIGGRAPH ( www.siggraph.org , 1996), pp. 31–42.
  14. W. Matusik, C. Buehler, R. Raskar, S. Gortler, L. McMillan, “Image-based visual hulls,” in Proceedings of SIGGRAPH ( www.siggraph.org , 2000), pp. 369–374.
  15. G. Qian, R. Chellappa, “Structure from motion using sequential Monte Carlo methods,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE Press, Piscataway, N.J., 2001), Vol. 2, pp. 614–621.
  16. S. J. Gortler, R. Grzeszczuk, R. Szeliski, M. Cohen, “The lumigraph,” in Proceedings of SIGGRAPH ( www.siggraph.org , 1996), pp. 43–54.
  17. A. Roy Chowdhury, R. Chellappa, “Stochastic approximation and rate distortion analysis for robust structure and motion estimation,” Int. J. Comput. Vis. 55, 27–53 (2003).
    [CrossRef]
  18. M. Kirby, L. Sirovich, “Application of Karhunen–Loeve procedure of the characterization of human faces,” IEEE Trans. Pattern Anal. Mach. Intell. 12, 103–108 (1990).
    [CrossRef]
  19. M. Turk, A. Pentland, “Eigenfaces for recognition,” J. Cogn Neurosci. 3, 72–86 (1991).
    [CrossRef]
  20. R. Gross, I. Matthews, S. Baker, “Eigen light-fields and face recognition across pose,” in Proceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition (IEEE Press, Piscataway, N.J., 2002), pp. 3–9.
  21. R. Gross, I. Matthews, S. Baker, “Fisher light-fields for face recognition across pose and illumination,” in Proceedings of the 24th Symposium of the German Association for Pattern Recognition (DAGM) (Springer-Verlag, Berlin, 2002), pp. 481–489.
  22. T. Sim, S. Baker, M. Bsat, “The CMU pose, illumination, and expression (PIE) database,” in Proceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition (IEEE Press, Piscataway, N.J., 2002), pp. 53–58.
  23. S. Zhou, R. Chellappa, “Rank constrained recognition under unknown illumination,” in IEEE International Workshop on Analysis and Modeling of Faces and Gestures (IEEE Press, Piscataway, N.J., 2003), pp. 11–18.
  24. A. Shashua, “On photometric issues in 3D visual recognition from a single 2D image,” Int. J. Comput. Vision 21, 99–122 (1997).
    [CrossRef]
  25. A. Pentland, B. Moghaddam, T. Starner, “View-based and modular eigenspaces for face recognition,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE Computer Society, Los Alamitos, Calif., 1994), pp. 84–91.
  26. M. A. O. Vasilescu, D. Terzopoulos, “Multilinear analysis of image ensembles: tensorfaces,” in European Conference on Computer Vision (Springer-Verlag, Berlin, 2002), Vol. 1, pp. 447–460.
  27. S. Romdhani, T. Vetter, “Efficient, robust and accurate fitting of a 3D morphable model,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE Press, Piscataway, N.J., 2003), pp. 59–66.
  28. A. Lanitis, C. J. Taylor, T. F. Cootes, “Automatic interpretation and coding of face images using flexible models,” IEEE Trans. Pattern Anal. Mach. Intell. 19, 743–756 (1997).
    [CrossRef]
  29. T. Vetter, T. Poggio, “Linear object classes and image synthesis from a single example image,” IEEE Trans. Pattern Anal. Mach. Intell. 19, 733–742 (1997).
    [CrossRef]
  30. In our current implementation, the L vector is not completely shape free, since we use a simple normalization operation.
  31. W. T. Freeman, J. B. Tenenbaum, “Learning bilinear models for two-factor problems in vision,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE Computer Society, Los Alamitos, Calif., 1997), pp. 554–560.
  32. M. Brand, “Incremental singular value decomposition of uncertain data with missing values,” in European Conference on Computer Vision (Springer-Verlag, Berlin, 2002), Vol. 1, pp. 707–720.
  33. H. Gupta, “Contour based 3D face modeling from monocular video,” M.S. dissertation (University of Maryland, College Park, Md., 2003).

2003 (5)

R. Basri, D. Jacobs, “Lambertian reflectance and linear subspaces,” IEEE Trans. Pattern Anal. Mach. Intell. 25, 218–233 (2003).
[CrossRef]

W. Zhao, R. Chellappa, A. Rosenfeld, J. Phillips, “Face recognition: a literature survey,” ACM Comput. Surv. 12, 399–458 (2003).
[CrossRef]

V. Blanz, T. Vetter, “Face identification across different poses and illumination with a 3D morphable model,” IEEE Trans. Pattern Anal. Mach. Intell. 25, 1063–1074 (2003).
[CrossRef]

A. Roy Chowdhury, R. Chellappa, “Face reconstruction from video using uncertainty analysis and a generic model,” Comput. Vision Image Underst. 91, 188–213 (2003).
[CrossRef]

A. Roy Chowdhury, R. Chellappa, “Stochastic approximation and rate distortion analysis for robust structure and motion estimation,” Int. J. Comput. Vis. 55, 27–53 (2003).
[CrossRef]

2001 (2)

A. Georghiades, P. Belhumeur, D. Kriegman, “From few to many: illumination cone models for face recognition under variable lighting and pose,” IEEE Trans. Pattern Anal. Mach. Intell. 23, 643–660 (2001).
[CrossRef]

A. Shashua, T. R. Raviv, “The quotient image: class based re-rendering and recognition with varying illumination,” IEEE Trans. Pattern Anal. Mach. Intell. 23, 129–139 (2001).
[CrossRef]

2000 (2)

P. Fua, “Regularized bundle adjustment to model heads from image sequences without calibrated data,” Int. J. Comput. Vision 38, 153–157 (2000).
[CrossRef]

P. Phillips, H. Moon, S. Rizvi, P. Rauss, “The FERET evaluation methodology for face-recognition algorithms,” IEEE Trans. Pattern Anal. Mach. Intell. 22, 1090–1104 (2000).
[CrossRef]

1999 (1)

A. Yuille, D. Snow, R. Epstein, P. Belhumeur, “Determining generative models for objects under varying illumination: shape and albedo from multiple images using SVD and integrability,” Int. J. Comput. Vision 35, 203–222 (1999).
[CrossRef]

1997 (3)

A. Shashua, “On photometric issues in 3D visual recognition from a single 2D image,” Int. J. Comput. Vision 21, 99–122 (1997).
[CrossRef]

A. Lanitis, C. J. Taylor, T. F. Cootes, “Automatic interpretation and coding of face images using flexible models,” IEEE Trans. Pattern Anal. Mach. Intell. 19, 743–756 (1997).
[CrossRef]

T. Vetter, T. Poggio, “Linear object classes and image synthesis from a single example image,” IEEE Trans. Pattern Anal. Mach. Intell. 19, 733–742 (1997).
[CrossRef]

1994 (1)

A. Laurentini, “The visual hull concept for silhouette-based image understanding,” IEEE Trans. Pattern Anal. Mach. Intell. 16, 150–162 (1994).
[CrossRef]

1991 (1)

M. Turk, A. Pentland, “Eigenfaces for recognition,” J. Cogn Neurosci. 3, 72–86 (1991).
[CrossRef]

1990 (1)

M. Kirby, L. Sirovich, “Application of Karhunen–Loeve procedure of the characterization of human faces,” IEEE Trans. Pattern Anal. Mach. Intell. 12, 103–108 (1990).
[CrossRef]

Baker, S.

R. Gross, I. Matthews, S. Baker, “Fisher light-fields for face recognition across pose and illumination,” in Proceedings of the 24th Symposium of the German Association for Pattern Recognition (DAGM) (Springer-Verlag, Berlin, 2002), pp. 481–489.

R. Gross, I. Matthews, S. Baker, “Eigen light-fields and face recognition across pose,” in Proceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition (IEEE Press, Piscataway, N.J., 2002), pp. 3–9.

T. Sim, S. Baker, M. Bsat, “The CMU pose, illumination, and expression (PIE) database,” in Proceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition (IEEE Press, Piscataway, N.J., 2002), pp. 53–58.

Basri, R.

R. Basri, D. Jacobs, “Lambertian reflectance and linear subspaces,” IEEE Trans. Pattern Anal. Mach. Intell. 25, 218–233 (2003).
[CrossRef]

Belhumeur, P.

A. Georghiades, P. Belhumeur, D. Kriegman, “From few to many: illumination cone models for face recognition under variable lighting and pose,” IEEE Trans. Pattern Anal. Mach. Intell. 23, 643–660 (2001).
[CrossRef]

A. Yuille, D. Snow, R. Epstein, P. Belhumeur, “Determining generative models for objects under varying illumination: shape and albedo from multiple images using SVD and integrability,” Int. J. Comput. Vision 35, 203–222 (1999).
[CrossRef]

Blanz, V.

V. Blanz, T. Vetter, “Face identification across different poses and illumination with a 3D morphable model,” IEEE Trans. Pattern Anal. Mach. Intell. 25, 1063–1074 (2003).
[CrossRef]

Brand, M.

M. Brand, “Incremental singular value decomposition of uncertain data with missing values,” in European Conference on Computer Vision (Springer-Verlag, Berlin, 2002), Vol. 1, pp. 707–720.

Bsat, M.

T. Sim, S. Baker, M. Bsat, “The CMU pose, illumination, and expression (PIE) database,” in Proceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition (IEEE Press, Piscataway, N.J., 2002), pp. 53–58.

Buehler, C.

W. Matusik, C. Buehler, R. Raskar, S. Gortler, L. McMillan, “Image-based visual hulls,” in Proceedings of SIGGRAPH ( www.siggraph.org , 2000), pp. 369–374.

Chellappa, R.

A. Roy Chowdhury, R. Chellappa, “Face reconstruction from video using uncertainty analysis and a generic model,” Comput. Vision Image Underst. 91, 188–213 (2003).
[CrossRef]

W. Zhao, R. Chellappa, A. Rosenfeld, J. Phillips, “Face recognition: a literature survey,” ACM Comput. Surv. 12, 399–458 (2003).
[CrossRef]

A. Roy Chowdhury, R. Chellappa, “Stochastic approximation and rate distortion analysis for robust structure and motion estimation,” Int. J. Comput. Vis. 55, 27–53 (2003).
[CrossRef]

G. Qian, R. Chellappa, “Structure from motion using sequential Monte Carlo methods,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE Press, Piscataway, N.J., 2001), Vol. 2, pp. 614–621.

S. Zhou, R. Chellappa, “Rank constrained recognition under unknown illumination,” in IEEE International Workshop on Analysis and Modeling of Faces and Gestures (IEEE Press, Piscataway, N.J., 2003), pp. 11–18.

S. Zhou, R. Chellappa, D. Jacobs, “Characterization of human faces under illumination variations using rank, integrability, and symmetry constraints,” in Proceedings of the European Conference on Computer Vision (Springer-Verlag, Berlin, 2004), Vol. 1, pp. 588–601.

Chowdhury, A. Roy

A. Roy Chowdhury, R. Chellappa, “Face reconstruction from video using uncertainty analysis and a generic model,” Comput. Vision Image Underst. 91, 188–213 (2003).
[CrossRef]

A. Roy Chowdhury, R. Chellappa, “Stochastic approximation and rate distortion analysis for robust structure and motion estimation,” Int. J. Comput. Vis. 55, 27–53 (2003).
[CrossRef]

Cohen, M.

S. J. Gortler, R. Grzeszczuk, R. Szeliski, M. Cohen, “The lumigraph,” in Proceedings of SIGGRAPH ( www.siggraph.org , 1996), pp. 43–54.

Cootes, T. F.

A. Lanitis, C. J. Taylor, T. F. Cootes, “Automatic interpretation and coding of face images using flexible models,” IEEE Trans. Pattern Anal. Mach. Intell. 19, 743–756 (1997).
[CrossRef]

Epstein, R.

A. Yuille, D. Snow, R. Epstein, P. Belhumeur, “Determining generative models for objects under varying illumination: shape and albedo from multiple images using SVD and integrability,” Int. J. Comput. Vision 35, 203–222 (1999).
[CrossRef]

Freeman, W. T.

W. T. Freeman, J. B. Tenenbaum, “Learning bilinear models for two-factor problems in vision,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE Computer Society, Los Alamitos, Calif., 1997), pp. 554–560.

Fua, P.

P. Fua, “Regularized bundle adjustment to model heads from image sequences without calibrated data,” Int. J. Comput. Vision 38, 153–157 (2000).
[CrossRef]

Georghiades, A.

A. Georghiades, P. Belhumeur, D. Kriegman, “From few to many: illumination cone models for face recognition under variable lighting and pose,” IEEE Trans. Pattern Anal. Mach. Intell. 23, 643–660 (2001).
[CrossRef]

Gortler, S.

W. Matusik, C. Buehler, R. Raskar, S. Gortler, L. McMillan, “Image-based visual hulls,” in Proceedings of SIGGRAPH ( www.siggraph.org , 2000), pp. 369–374.

Gortler, S. J.

S. J. Gortler, R. Grzeszczuk, R. Szeliski, M. Cohen, “The lumigraph,” in Proceedings of SIGGRAPH ( www.siggraph.org , 1996), pp. 43–54.

Gross, R.

R. Gross, I. Matthews, S. Baker, “Eigen light-fields and face recognition across pose,” in Proceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition (IEEE Press, Piscataway, N.J., 2002), pp. 3–9.

R. Gross, I. Matthews, S. Baker, “Fisher light-fields for face recognition across pose and illumination,” in Proceedings of the 24th Symposium of the German Association for Pattern Recognition (DAGM) (Springer-Verlag, Berlin, 2002), pp. 481–489.

Grzeszczuk, R.

S. J. Gortler, R. Grzeszczuk, R. Szeliski, M. Cohen, “The lumigraph,” in Proceedings of SIGGRAPH ( www.siggraph.org , 1996), pp. 43–54.

Gupta, H.

H. Gupta, “Contour based 3D face modeling from monocular video,” M.S. dissertation (University of Maryland, College Park, Md., 2003).

Hanrahan, P.

M. Levoy, P. Hanrahan, “Light field rendering,” in Proceedings of SIGGRAPH ( www.siggraph.org , 1996), pp. 31–42.

Jacobs, D.

R. Basri, D. Jacobs, “Lambertian reflectance and linear subspaces,” IEEE Trans. Pattern Anal. Mach. Intell. 25, 218–233 (2003).
[CrossRef]

S. Zhou, R. Chellappa, D. Jacobs, “Characterization of human faces under illumination variations using rank, integrability, and symmetry constraints,” in Proceedings of the European Conference on Computer Vision (Springer-Verlag, Berlin, 2004), Vol. 1, pp. 588–601.

Kirby, M.

M. Kirby, L. Sirovich, “Application of Karhunen–Loeve procedure of the characterization of human faces,” IEEE Trans. Pattern Anal. Mach. Intell. 12, 103–108 (1990).
[CrossRef]

Kriegman, D.

A. Georghiades, P. Belhumeur, D. Kriegman, “From few to many: illumination cone models for face recognition under variable lighting and pose,” IEEE Trans. Pattern Anal. Mach. Intell. 23, 643–660 (2001).
[CrossRef]

Lanitis, A.

A. Lanitis, C. J. Taylor, T. F. Cootes, “Automatic interpretation and coding of face images using flexible models,” IEEE Trans. Pattern Anal. Mach. Intell. 19, 743–756 (1997).
[CrossRef]

Laurentini, A.

A. Laurentini, “The visual hull concept for silhouette-based image understanding,” IEEE Trans. Pattern Anal. Mach. Intell. 16, 150–162 (1994).
[CrossRef]

Levoy, M.

M. Levoy, P. Hanrahan, “Light field rendering,” in Proceedings of SIGGRAPH ( www.siggraph.org , 1996), pp. 31–42.

Liu, Z.

Y. Shan, Z. Liu, Z. Zhang, “Model-based bundle adjustment with application to face modeling,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE Press, Piscataway, N.J., 2001), Vol. 2, pp. 644–651.

Matthews, I.

R. Gross, I. Matthews, S. Baker, “Fisher light-fields for face recognition across pose and illumination,” in Proceedings of the 24th Symposium of the German Association for Pattern Recognition (DAGM) (Springer-Verlag, Berlin, 2002), pp. 481–489.

R. Gross, I. Matthews, S. Baker, “Eigen light-fields and face recognition across pose,” in Proceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition (IEEE Press, Piscataway, N.J., 2002), pp. 3–9.

Matusik, W.

W. Matusik, C. Buehler, R. Raskar, S. Gortler, L. McMillan, “Image-based visual hulls,” in Proceedings of SIGGRAPH ( www.siggraph.org , 2000), pp. 369–374.

McMillan, L.

W. Matusik, C. Buehler, R. Raskar, S. Gortler, L. McMillan, “Image-based visual hulls,” in Proceedings of SIGGRAPH ( www.siggraph.org , 2000), pp. 369–374.

Moghaddam, B.

A. Pentland, B. Moghaddam, T. Starner, “View-based and modular eigenspaces for face recognition,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE Computer Society, Los Alamitos, Calif., 1994), pp. 84–91.

Moon, H.

P. Phillips, H. Moon, S. Rizvi, P. Rauss, “The FERET evaluation methodology for face-recognition algorithms,” IEEE Trans. Pattern Anal. Mach. Intell. 22, 1090–1104 (2000).
[CrossRef]

Pentland, A.

M. Turk, A. Pentland, “Eigenfaces for recognition,” J. Cogn Neurosci. 3, 72–86 (1991).
[CrossRef]

A. Pentland, B. Moghaddam, T. Starner, “View-based and modular eigenspaces for face recognition,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE Computer Society, Los Alamitos, Calif., 1994), pp. 84–91.

Phillips, J.

W. Zhao, R. Chellappa, A. Rosenfeld, J. Phillips, “Face recognition: a literature survey,” ACM Comput. Surv. 12, 399–458 (2003).
[CrossRef]

Phillips, P.

P. Phillips, H. Moon, S. Rizvi, P. Rauss, “The FERET evaluation methodology for face-recognition algorithms,” IEEE Trans. Pattern Anal. Mach. Intell. 22, 1090–1104 (2000).
[CrossRef]

Poggio, T.

T. Vetter, T. Poggio, “Linear object classes and image synthesis from a single example image,” IEEE Trans. Pattern Anal. Mach. Intell. 19, 733–742 (1997).
[CrossRef]

Qian, G.

G. Qian, R. Chellappa, “Structure from motion using sequential Monte Carlo methods,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE Press, Piscataway, N.J., 2001), Vol. 2, pp. 614–621.

Raskar, R.

W. Matusik, C. Buehler, R. Raskar, S. Gortler, L. McMillan, “Image-based visual hulls,” in Proceedings of SIGGRAPH ( www.siggraph.org , 2000), pp. 369–374.

Rauss, P.

P. Phillips, H. Moon, S. Rizvi, P. Rauss, “The FERET evaluation methodology for face-recognition algorithms,” IEEE Trans. Pattern Anal. Mach. Intell. 22, 1090–1104 (2000).
[CrossRef]

Raviv, T. R.

A. Shashua, T. R. Raviv, “The quotient image: class based re-rendering and recognition with varying illumination,” IEEE Trans. Pattern Anal. Mach. Intell. 23, 129–139 (2001).
[CrossRef]

Rizvi, S.

P. Phillips, H. Moon, S. Rizvi, P. Rauss, “The FERET evaluation methodology for face-recognition algorithms,” IEEE Trans. Pattern Anal. Mach. Intell. 22, 1090–1104 (2000).
[CrossRef]

Romdhani, S.

S. Romdhani, T. Vetter, “Efficient, robust and accurate fitting of a 3D morphable model,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE Press, Piscataway, N.J., 2003), pp. 59–66.

Rosenfeld, A.

W. Zhao, R. Chellappa, A. Rosenfeld, J. Phillips, “Face recognition: a literature survey,” ACM Comput. Surv. 12, 399–458 (2003).
[CrossRef]

Shan, Y.

Y. Shan, Z. Liu, Z. Zhang, “Model-based bundle adjustment with application to face modeling,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE Press, Piscataway, N.J., 2001), Vol. 2, pp. 644–651.

Shashua, A.

A. Shashua, T. R. Raviv, “The quotient image: class based re-rendering and recognition with varying illumination,” IEEE Trans. Pattern Anal. Mach. Intell. 23, 129–139 (2001).
[CrossRef]

A. Shashua, “On photometric issues in 3D visual recognition from a single 2D image,” Int. J. Comput. Vision 21, 99–122 (1997).
[CrossRef]

Sim, T.

T. Sim, S. Baker, M. Bsat, “The CMU pose, illumination, and expression (PIE) database,” in Proceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition (IEEE Press, Piscataway, N.J., 2002), pp. 53–58.

Sirovich, L.

M. Kirby, L. Sirovich, “Application of Karhunen–Loeve procedure of the characterization of human faces,” IEEE Trans. Pattern Anal. Mach. Intell. 12, 103–108 (1990).
[CrossRef]

Snow, D.

A. Yuille, D. Snow, R. Epstein, P. Belhumeur, “Determining generative models for objects under varying illumination: shape and albedo from multiple images using SVD and integrability,” Int. J. Comput. Vision 35, 203–222 (1999).
[CrossRef]

Starner, T.

A. Pentland, B. Moghaddam, T. Starner, “View-based and modular eigenspaces for face recognition,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE Computer Society, Los Alamitos, Calif., 1994), pp. 84–91.

Szeliski, R.

S. J. Gortler, R. Grzeszczuk, R. Szeliski, M. Cohen, “The lumigraph,” in Proceedings of SIGGRAPH ( www.siggraph.org , 1996), pp. 43–54.

Taylor, C. J.

A. Lanitis, C. J. Taylor, T. F. Cootes, “Automatic interpretation and coding of face images using flexible models,” IEEE Trans. Pattern Anal. Mach. Intell. 19, 743–756 (1997).
[CrossRef]

Tenenbaum, J. B.

W. T. Freeman, J. B. Tenenbaum, “Learning bilinear models for two-factor problems in vision,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE Computer Society, Los Alamitos, Calif., 1997), pp. 554–560.

Terzopoulos, D.

M. A. O. Vasilescu, D. Terzopoulos, “Multilinear analysis of image ensembles: tensorfaces,” in European Conference on Computer Vision (Springer-Verlag, Berlin, 2002), Vol. 1, pp. 447–460.

Turk, M.

M. Turk, A. Pentland, “Eigenfaces for recognition,” J. Cogn Neurosci. 3, 72–86 (1991).
[CrossRef]

Vasilescu, M. A. O.

M. A. O. Vasilescu, D. Terzopoulos, “Multilinear analysis of image ensembles: tensorfaces,” in European Conference on Computer Vision (Springer-Verlag, Berlin, 2002), Vol. 1, pp. 447–460.

Vetter, T.

V. Blanz, T. Vetter, “Face identification across different poses and illumination with a 3D morphable model,” IEEE Trans. Pattern Anal. Mach. Intell. 25, 1063–1074 (2003).
[CrossRef]

T. Vetter, T. Poggio, “Linear object classes and image synthesis from a single example image,” IEEE Trans. Pattern Anal. Mach. Intell. 19, 733–742 (1997).
[CrossRef]

S. Romdhani, T. Vetter, “Efficient, robust and accurate fitting of a 3D morphable model,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE Press, Piscataway, N.J., 2003), pp. 59–66.

Yuille, A.

A. Yuille, D. Snow, R. Epstein, P. Belhumeur, “Determining generative models for objects under varying illumination: shape and albedo from multiple images using SVD and integrability,” Int. J. Comput. Vision 35, 203–222 (1999).
[CrossRef]

Zhang, Z.

Y. Shan, Z. Liu, Z. Zhang, “Model-based bundle adjustment with application to face modeling,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE Press, Piscataway, N.J., 2001), Vol. 2, pp. 644–651.

Zhao, W.

W. Zhao, R. Chellappa, A. Rosenfeld, J. Phillips, “Face recognition: a literature survey,” ACM Comput. Surv. 12, 399–458 (2003).
[CrossRef]

Zhou, S.

S. Zhou, R. Chellappa, “Rank constrained recognition under unknown illumination,” in IEEE International Workshop on Analysis and Modeling of Faces and Gestures (IEEE Press, Piscataway, N.J., 2003), pp. 11–18.

S. Zhou, R. Chellappa, D. Jacobs, “Characterization of human faces under illumination variations using rank, integrability, and symmetry constraints,” in Proceedings of the European Conference on Computer Vision (Springer-Verlag, Berlin, 2004), Vol. 1, pp. 588–601.

ACM Comput. Surv. (1)

W. Zhao, R. Chellappa, A. Rosenfeld, J. Phillips, “Face recognition: a literature survey,” ACM Comput. Surv. 12, 399–458 (2003).
[CrossRef]

Comput. Vision Image Underst. (1)

A. Roy Chowdhury, R. Chellappa, “Face reconstruction from video using uncertainty analysis and a generic model,” Comput. Vision Image Underst. 91, 188–213 (2003).
[CrossRef]

IEEE Trans. Pattern Anal. Mach. Intell. (9)

V. Blanz, T. Vetter, “Face identification across different poses and illumination with a 3D morphable model,” IEEE Trans. Pattern Anal. Mach. Intell. 25, 1063–1074 (2003).
[CrossRef]

P. Phillips, H. Moon, S. Rizvi, P. Rauss, “The FERET evaluation methodology for face-recognition algorithms,” IEEE Trans. Pattern Anal. Mach. Intell. 22, 1090–1104 (2000).
[CrossRef]

R. Basri, D. Jacobs, “Lambertian reflectance and linear subspaces,” IEEE Trans. Pattern Anal. Mach. Intell. 25, 218–233 (2003).
[CrossRef]

A. Georghiades, P. Belhumeur, D. Kriegman, “From few to many: illumination cone models for face recognition under variable lighting and pose,” IEEE Trans. Pattern Anal. Mach. Intell. 23, 643–660 (2001).
[CrossRef]

A. Shashua, T. R. Raviv, “The quotient image: class based re-rendering and recognition with varying illumination,” IEEE Trans. Pattern Anal. Mach. Intell. 23, 129–139 (2001).
[CrossRef]

A. Laurentini, “The visual hull concept for silhouette-based image understanding,” IEEE Trans. Pattern Anal. Mach. Intell. 16, 150–162 (1994).
[CrossRef]

M. Kirby, L. Sirovich, “Application of Karhunen–Loeve procedure of the characterization of human faces,” IEEE Trans. Pattern Anal. Mach. Intell. 12, 103–108 (1990).
[CrossRef]

A. Lanitis, C. J. Taylor, T. F. Cootes, “Automatic interpretation and coding of face images using flexible models,” IEEE Trans. Pattern Anal. Mach. Intell. 19, 743–756 (1997).
[CrossRef]

T. Vetter, T. Poggio, “Linear object classes and image synthesis from a single example image,” IEEE Trans. Pattern Anal. Mach. Intell. 19, 733–742 (1997).
[CrossRef]

Int. J. Comput. Vis. (1)

A. Roy Chowdhury, R. Chellappa, “Stochastic approximation and rate distortion analysis for robust structure and motion estimation,” Int. J. Comput. Vis. 55, 27–53 (2003).
[CrossRef]

Int. J. Comput. Vision (3)

A. Shashua, “On photometric issues in 3D visual recognition from a single 2D image,” Int. J. Comput. Vision 21, 99–122 (1997).
[CrossRef]

A. Yuille, D. Snow, R. Epstein, P. Belhumeur, “Determining generative models for objects under varying illumination: shape and albedo from multiple images using SVD and integrability,” Int. J. Comput. Vision 35, 203–222 (1999).
[CrossRef]

P. Fua, “Regularized bundle adjustment to model heads from image sequences without calibrated data,” Int. J. Comput. Vision 38, 153–157 (2000).
[CrossRef]

J. Cogn Neurosci. (1)

M. Turk, A. Pentland, “Eigenfaces for recognition,” J. Cogn Neurosci. 3, 72–86 (1991).
[CrossRef]

Other (17)

R. Gross, I. Matthews, S. Baker, “Eigen light-fields and face recognition across pose,” in Proceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition (IEEE Press, Piscataway, N.J., 2002), pp. 3–9.

R. Gross, I. Matthews, S. Baker, “Fisher light-fields for face recognition across pose and illumination,” in Proceedings of the 24th Symposium of the German Association for Pattern Recognition (DAGM) (Springer-Verlag, Berlin, 2002), pp. 481–489.

T. Sim, S. Baker, M. Bsat, “The CMU pose, illumination, and expression (PIE) database,” in Proceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition (IEEE Press, Piscataway, N.J., 2002), pp. 53–58.

S. Zhou, R. Chellappa, “Rank constrained recognition under unknown illumination,” in IEEE International Workshop on Analysis and Modeling of Faces and Gestures (IEEE Press, Piscataway, N.J., 2003), pp. 11–18.

M. Levoy, P. Hanrahan, “Light field rendering,” in Proceedings of SIGGRAPH ( www.siggraph.org , 1996), pp. 31–42.

W. Matusik, C. Buehler, R. Raskar, S. Gortler, L. McMillan, “Image-based visual hulls,” in Proceedings of SIGGRAPH ( www.siggraph.org , 2000), pp. 369–374.

G. Qian, R. Chellappa, “Structure from motion using sequential Monte Carlo methods,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE Press, Piscataway, N.J., 2001), Vol. 2, pp. 614–621.

S. J. Gortler, R. Grzeszczuk, R. Szeliski, M. Cohen, “The lumigraph,” in Proceedings of SIGGRAPH ( www.siggraph.org , 1996), pp. 43–54.

Y. Shan, Z. Liu, Z. Zhang, “Model-based bundle adjustment with application to face modeling,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE Press, Piscataway, N.J., 2001), Vol. 2, pp. 644–651.

S. Zhou, R. Chellappa, D. Jacobs, “Characterization of human faces under illumination variations using rank, integrability, and symmetry constraints,” in Proceedings of the European Conference on Computer Vision (Springer-Verlag, Berlin, 2004), Vol. 1, pp. 588–601.

A. Pentland, B. Moghaddam, T. Starner, “View-based and modular eigenspaces for face recognition,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE Computer Society, Los Alamitos, Calif., 1994), pp. 84–91.

M. A. O. Vasilescu, D. Terzopoulos, “Multilinear analysis of image ensembles: tensorfaces,” in European Conference on Computer Vision (Springer-Verlag, Berlin, 2002), Vol. 1, pp. 447–460.

S. Romdhani, T. Vetter, “Efficient, robust and accurate fitting of a 3D morphable model,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE Press, Piscataway, N.J., 2003), pp. 59–66.

In our current implementation, the L vector is not completely shape free, since we use a simple normalization operation.

W. T. Freeman, J. B. Tenenbaum, “Learning bilinear models for two-factor problems in vision,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE Computer Society, Los Alamitos, Calif., 1997), pp. 554–560.

M. Brand, “Incremental singular value decomposition of uncertain data with missing values,” in European Conference on Computer Vision (Springer-Verlag, Berlin, 2002), Vol. 1, pp. 707–720.

H. Gupta, “Contour based 3D face modeling from monocular video,” M.S. dissertation (University of Maryland, College Park, Md., 2003).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (7)

Fig. 1
Fig. 1

PIE object under different illuminations and poses.

Fig. 2
Fig. 2

Illustration of the 2D light field of a 2D object (a square with four differently colored sides), which is placed within a circle. The angles θ and ϕ are used to relate the viewpoint to the radiance from the object. The right-hand image shows the actual light field for the square object. See another illustration in Ref. 20.

Fig. 3
Fig. 3

Examples of the face images of PIE object 4056 under selected illuminations and poses. This object is used in the testing stage.

Fig. 4
Fig. 4

First nine columns of the learned W matrix.

Fig. 5
Fig. 5

Reconstruction results of PIE object 4056 in Fig. 3. Note that only the f’s and the s’s for row c27 are used for reconstructing all images here.

Fig. 6
Fig. 6

Original image of PIE object 4056 at (c05, f18), reconstructed image rendered by using the f and s coefficients recovered from the original image, and detected τ image with the threshold a in Eq. (23) set to 10.

Fig. 7
Fig. 7

Average recognition rates across illuminations (top row) and across poses (bottom row) for three cases. Case (a) shows the average recognition rate (averaging over all illuminations/poses and all gallery sets) obtained by the proposed algorithm with use of the top n matches. Case (b) shows the average recognition rate [averaging over all illuminations/poses for the gallery set (c27, f11) only] obtained by the proposed algorithm with use of the top n matches. Case (c) shows the average recognition rate (averaging over all illuminations/poses and all gallery sets) obtained by the eigenface algorithm with use of the top n matches.

Tables (3)

Tables Icon

Table 1 Recognition Rates for All the Probe Sets with a Fixed Gallery Set (c27, f11)

Tables Icon

Table 2 Average Recognition Rates for All the Gallery Setsa

Tables Icon

Table 3 Recognition Rates for Test Scenario B

Equations (26)

Equations on this page are rendered with MathJax. Learn more.

hs=pnTs=tTs,n3×1[nx, ny, nz]T,
t3×1pn,
hs[i=1dhis]=[i=1dtiT]s=Ts,
Hd×n[i=1nhi]=T[i=1nsi]=Td×3S3×n,
T=i=1mfiTi=[i=1mTi](fI3)=W(fI3),
hs=Ts=W(fI3)s=W(fs)=Wk,
Hd×n=W[i=1n(fisi)]=W[i=1nki]=WK,
hd×1=W(fs˜)=W˜d×mfm×1,
Hd×n=[i=1nhi]=W˜[i=1nfi]=W˜d×mFm×n,
ei=nainLn.
L=i=1mfiei=Ef,
hv=Rv[L]=Rv[Ef]=Rv[E]f=Evf,
Ls=tTs,
eis=naintnTs=teiTs,
Ls=[i=1mteiTs]Tf=W(fs),
hvs=Rv[Ls]=Rv[W(fs)]=Wv(fs),
A=[n=1N[s=1SLns],B=[s=1S[n=1NLns]],
s=[j=1r([i=1mWijv]f)]hvs,
f=[i=1m([j=1rWijv]s)]1Thvs1,
sq=[j=1r([i=1mWijvq]f)]hvqsq,q=1, 2,, Q,
f=[q=1Q[i=1m([j=1rWijvq]sq)]]1T[q=1Qhvqsq]1.
s=[j=1r([i=1mWijv]f)][τ ° hvs],
f=[i=1m([j=1rWijv]s)]1Tτ ° hvs1,
τ=[|hvs-Wv(fs)|<a],
τ=[a1<hvs<a2],
cc(p, g)=(fp, fg)/[(fp, fp)(fg, fg)]1/2,

Metrics