Abstract

We introduce a useful tool for controlling the skin melanin texture of facial photographs. Controlling the skin melanin texture is an important task in the reproduction of posters, TV commercials, movies, and so on. We used component maps of melanin, which were obtained by a previous method [J. Opt. Soc. Am. A 16, 2169 (1999)] as the first processing step. We propose to control the melanin texture continuously and physiologically, based on the analysis of 123 skin textures in our database. The physiological validity for the change of the melanin texture is confirmed by comparing the synthesized image with an ultraviolet image, which can be used to predict the change of melanin texture due to aging. The control processes are implemented on programmable graphics hardware, and real-time processing is achieved for a facial videostream.

© 2006 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. N. Tsumura, N. Ojima, K. Sato, M. Shiraishi, H. Shimizu, H. Nabeshima, S. Akazaki, K. Hori, and Y. Miyake, "Image-based skin color and texture analysis/synthesis by extracting hemoglobin and melanin information in the skin," ACM (Assoc. Comput. Mach.) Trans. Graphics 22, 770-779 (2003).
  2. N. Tsumura, H. Haneishi, and Y. Miyake, "Independent component analysis of skin color image," J. Opt. Soc. Am. A 16, 9, 2169-2176 (1999).
    [Crossref]
  3. K. Perlin, "An image synthesizer," in Proceedings of the 12th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 1985), pp. 287-296.
  4. A. Witkin and M. Kass, "Reaction-diffusion textures," in Proceedings of the 18th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 1991), pp. 299-308.
  5. G. Turk, "Generating textures on arbitrary surfaces using reaction-diffusion," in Proceedings of the 18th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 1991), pp. 289-298.
  6. K. W. Fleischer, D. H. Laidlaw, B. L. Currin, and A. H. Barr, "Cellular texture generation," in Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 1995), pp. 239-248.
  7. S. Worley, "A cellular texture basis function," in Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 1996), pp. 291-294.
  8. M. Walter, A. Fournier, and M. Reimers, "Clonal mosaic model for the synthesis of mammalian coat patterns," in Proceedings of Graphics Interface '98, pp. 82-91 (Canadian Human-Computer Communication Society, June 1998).
  9. D. J. Heeger and J. R. Bergen, "Pyramid-based texture analysis/synthesis," in Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 1995), pp. 229-238.
  10. J. S. De Bonet, "Multiresolution sampling procedure for analysis and synthesis of texture images," in Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 1997), pp. 361-368.
  11. E. Simoncelli and J. Portilla, "Texture characterization via joint statistics of wavelet coefficient magnitudes," in Proceedings of Fifth International Conference on Image Processing1,62-66 (Institute of Electrical and Electronic Engineers, 1998).
  12. A. Efros and T. Leung, "Texture synthesis by non-parametric sampling," in Proceedings of Seventh International Conference on Computer Vision, pp. 1033-1038 (Institute of Electrical and Electronic Engineers, 1999).
    [Crossref]
  13. L.-Y. Wei and M. Levoy, "Fast texture synthesis using tree-structured vector quantization," in Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2000), pp. 479-488.
  14. M. Ashikhmin, "Synthesizing natural textures," in Proceedings of the 2001 Symposium on Interactive 3D Graphics (ACM Press, 2001), pp. 217-226.
    [Crossref]
  15. L. Liang, C. Liu, Y.-Q. Xu, B. Guo, and H.-Y. Shum, "Real-time texture synthesis by patch-based sampling," ACM (Assoc. Comput. Mach.) Trans. Graphics 20, 127-150 (2001).
  16. A. Hertzmann, C. E. Jacobs, N. Oliver, B. Curless, and D. H. Salesin, "Image analogies," in Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2001), pp. 327-340.
  17. A. A. Efros and W. T. Freeman, "Image quilting for texture synthesis and transfer," in Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2001), pp. 341-346.
  18. J. Zhang, K. Zhou, L. Velho, B. Guo, and H.-Y. Shum, "Synthesis of progressively-variant textures on arbitrary surfaces," ACM (Assoc. Comput. Mach.) Trans. Graphics 22, 295-302 (2003).
  19. V. Kwatra, A. Schödl, I. Essa, G. Turk, and A. Bobick, "Graphcut textures: image and video synthesis using graph cuts," (Assoc. Comput. Mach.) Trans. Graphics 22, 277-286 (2003).
  20. Q. Wu and Y. Yu, "Feature matching and deformation for texture synthesis," ACM (Assoc. Comput. Mach.) Trans. Graphics 23, 364-367 (2004).
  21. Y. Tsin, Y. Liu, and V. Ramesh, "Texture replacement in real images," in Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, 539-544 (IEEE, 2001).
  22. J.-M. Dischler and D. Ghazanfarpour, "Interactive image-based modeling of macrostructured textures," IEEE Comput. Graphics Appl. 19, 66-74 (1999).
    [Crossref]
  23. Y. Liu and W. Lin, "Deformable texture: the irregular-regular-irregular cycle," in Proceedings of the Third International Workshop on Texture Analysis and Synthesis, pp. 65-70 (Heriot-Watt University, 2003).
  24. S. Lefebvre and H. Hoppe, "Parallel controllable texture synthesis," ACM (Assoc. Comput. Math.) Trans. Graphics 24, 787-794 (2005).
  25. W. Matusik, M. Zwicker, and F. Durand, "Texture design using a simplicial complex of Morphable Textures," ACM (Assoc. Comput. Math.) Trans. Graphics 24, 777-786 (2005).
  26. O. G. Cula and K. J. Dana, "3D texture recognition using bidirectional feature histograms," Int. J. Comput. Vision 59, 33-60 (2004).
    [Crossref]
  27. O. G. Cula, K. J. Dana, F. P. Murphy, and B. K. Rao, "Bidirectional imaging and modeling of skin texture," IEEE Trans. Biomed. Eng. 51, 2148-2159 (2004).
    [Crossref] [PubMed]
  28. T. Leung and J. Malik, "Representing and recognizing the visual appearance of materials using three dimensional textons," Int. J. Comput. Vision 43, 29-44 (2001).
    [Crossref]
  29. W. Matusik, H. Pfister, M. Brand, and L. McMillan, "A data-driven reflectance model, ACM (Assoc. Comput. Mach.)Trans. Graphics 22, 759-769 (2003).
    [Crossref]
  30. P. Debevec, T. Hawkins, C. Tchou, H.-P. Duiker, W. Sarokin, and M. Sagar, "Acquiring the reflectance field of a human face," in Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2000), pp. 145-156.
  31. W. Matusik and H. Pfister, "3D TV: a scalable system for real-time acquisition, transmission, and autostereoscopic display of dynamic scenes," ACM (Assoc. Comput. Mach.) Trans. Graphics 23, 814-824 (2004).
  32. P. J. Burt and E. H. Adelson, "A multiresolution spline with application to image mosaics," ACM (Assoc. Comput. Mach.) Trans. Graphics 2, 217-236 (1983).
  33. G. Turk, "Texture synthesis on surfaces," in Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2001), pp. 345-354.
  34. L.-Y. Wei and M. Levoy, "Texture synthesis over arbitrary manifold surfaces," in Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2001), pp. 355-360.
  35. T. S. Kuroda and M. Fukuda, "Rab27A-binding protein Slp2-a is required for peripheral melanosome distribution and elongated cell shape in melanocytes," Nature Cell Biol. 6, 1195-1203 (1994).
    [Crossref]

2005 (2)

S. Lefebvre and H. Hoppe, "Parallel controllable texture synthesis," ACM (Assoc. Comput. Math.) Trans. Graphics 24, 787-794 (2005).

W. Matusik, M. Zwicker, and F. Durand, "Texture design using a simplicial complex of Morphable Textures," ACM (Assoc. Comput. Math.) Trans. Graphics 24, 777-786 (2005).

2004 (4)

O. G. Cula and K. J. Dana, "3D texture recognition using bidirectional feature histograms," Int. J. Comput. Vision 59, 33-60 (2004).
[Crossref]

O. G. Cula, K. J. Dana, F. P. Murphy, and B. K. Rao, "Bidirectional imaging and modeling of skin texture," IEEE Trans. Biomed. Eng. 51, 2148-2159 (2004).
[Crossref] [PubMed]

Q. Wu and Y. Yu, "Feature matching and deformation for texture synthesis," ACM (Assoc. Comput. Mach.) Trans. Graphics 23, 364-367 (2004).

W. Matusik and H. Pfister, "3D TV: a scalable system for real-time acquisition, transmission, and autostereoscopic display of dynamic scenes," ACM (Assoc. Comput. Mach.) Trans. Graphics 23, 814-824 (2004).

2003 (4)

W. Matusik, H. Pfister, M. Brand, and L. McMillan, "A data-driven reflectance model, ACM (Assoc. Comput. Mach.)Trans. Graphics 22, 759-769 (2003).
[Crossref]

N. Tsumura, N. Ojima, K. Sato, M. Shiraishi, H. Shimizu, H. Nabeshima, S. Akazaki, K. Hori, and Y. Miyake, "Image-based skin color and texture analysis/synthesis by extracting hemoglobin and melanin information in the skin," ACM (Assoc. Comput. Mach.) Trans. Graphics 22, 770-779 (2003).

J. Zhang, K. Zhou, L. Velho, B. Guo, and H.-Y. Shum, "Synthesis of progressively-variant textures on arbitrary surfaces," ACM (Assoc. Comput. Mach.) Trans. Graphics 22, 295-302 (2003).

V. Kwatra, A. Schödl, I. Essa, G. Turk, and A. Bobick, "Graphcut textures: image and video synthesis using graph cuts," (Assoc. Comput. Mach.) Trans. Graphics 22, 277-286 (2003).

2001 (2)

T. Leung and J. Malik, "Representing and recognizing the visual appearance of materials using three dimensional textons," Int. J. Comput. Vision 43, 29-44 (2001).
[Crossref]

L. Liang, C. Liu, Y.-Q. Xu, B. Guo, and H.-Y. Shum, "Real-time texture synthesis by patch-based sampling," ACM (Assoc. Comput. Mach.) Trans. Graphics 20, 127-150 (2001).

1999 (2)

J.-M. Dischler and D. Ghazanfarpour, "Interactive image-based modeling of macrostructured textures," IEEE Comput. Graphics Appl. 19, 66-74 (1999).
[Crossref]

N. Tsumura, H. Haneishi, and Y. Miyake, "Independent component analysis of skin color image," J. Opt. Soc. Am. A 16, 9, 2169-2176 (1999).
[Crossref]

1994 (1)

T. S. Kuroda and M. Fukuda, "Rab27A-binding protein Slp2-a is required for peripheral melanosome distribution and elongated cell shape in melanocytes," Nature Cell Biol. 6, 1195-1203 (1994).
[Crossref]

1983 (1)

P. J. Burt and E. H. Adelson, "A multiresolution spline with application to image mosaics," ACM (Assoc. Comput. Mach.) Trans. Graphics 2, 217-236 (1983).

Adelson, E. H.

P. J. Burt and E. H. Adelson, "A multiresolution spline with application to image mosaics," ACM (Assoc. Comput. Mach.) Trans. Graphics 2, 217-236 (1983).

Akazaki, S.

N. Tsumura, N. Ojima, K. Sato, M. Shiraishi, H. Shimizu, H. Nabeshima, S. Akazaki, K. Hori, and Y. Miyake, "Image-based skin color and texture analysis/synthesis by extracting hemoglobin and melanin information in the skin," ACM (Assoc. Comput. Mach.) Trans. Graphics 22, 770-779 (2003).

Ashikhmin, M.

M. Ashikhmin, "Synthesizing natural textures," in Proceedings of the 2001 Symposium on Interactive 3D Graphics (ACM Press, 2001), pp. 217-226.
[Crossref]

Barr, A. H.

K. W. Fleischer, D. H. Laidlaw, B. L. Currin, and A. H. Barr, "Cellular texture generation," in Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 1995), pp. 239-248.

Bergen, J. R.

D. J. Heeger and J. R. Bergen, "Pyramid-based texture analysis/synthesis," in Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 1995), pp. 229-238.

Bobick, A.

V. Kwatra, A. Schödl, I. Essa, G. Turk, and A. Bobick, "Graphcut textures: image and video synthesis using graph cuts," (Assoc. Comput. Mach.) Trans. Graphics 22, 277-286 (2003).

Brand, M.

W. Matusik, H. Pfister, M. Brand, and L. McMillan, "A data-driven reflectance model, ACM (Assoc. Comput. Mach.)Trans. Graphics 22, 759-769 (2003).
[Crossref]

Burt, P. J.

P. J. Burt and E. H. Adelson, "A multiresolution spline with application to image mosaics," ACM (Assoc. Comput. Mach.) Trans. Graphics 2, 217-236 (1983).

Cula, O. G.

O. G. Cula and K. J. Dana, "3D texture recognition using bidirectional feature histograms," Int. J. Comput. Vision 59, 33-60 (2004).
[Crossref]

O. G. Cula, K. J. Dana, F. P. Murphy, and B. K. Rao, "Bidirectional imaging and modeling of skin texture," IEEE Trans. Biomed. Eng. 51, 2148-2159 (2004).
[Crossref] [PubMed]

Curless, B.

A. Hertzmann, C. E. Jacobs, N. Oliver, B. Curless, and D. H. Salesin, "Image analogies," in Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2001), pp. 327-340.

Currin, B. L.

K. W. Fleischer, D. H. Laidlaw, B. L. Currin, and A. H. Barr, "Cellular texture generation," in Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 1995), pp. 239-248.

Dana, K. J.

O. G. Cula, K. J. Dana, F. P. Murphy, and B. K. Rao, "Bidirectional imaging and modeling of skin texture," IEEE Trans. Biomed. Eng. 51, 2148-2159 (2004).
[Crossref] [PubMed]

O. G. Cula and K. J. Dana, "3D texture recognition using bidirectional feature histograms," Int. J. Comput. Vision 59, 33-60 (2004).
[Crossref]

De Bonet, J. S.

J. S. De Bonet, "Multiresolution sampling procedure for analysis and synthesis of texture images," in Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 1997), pp. 361-368.

Debevec, P.

P. Debevec, T. Hawkins, C. Tchou, H.-P. Duiker, W. Sarokin, and M. Sagar, "Acquiring the reflectance field of a human face," in Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2000), pp. 145-156.

Dischler, J.-M.

J.-M. Dischler and D. Ghazanfarpour, "Interactive image-based modeling of macrostructured textures," IEEE Comput. Graphics Appl. 19, 66-74 (1999).
[Crossref]

Duiker, H.-P.

P. Debevec, T. Hawkins, C. Tchou, H.-P. Duiker, W. Sarokin, and M. Sagar, "Acquiring the reflectance field of a human face," in Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2000), pp. 145-156.

Durand, F.

W. Matusik, M. Zwicker, and F. Durand, "Texture design using a simplicial complex of Morphable Textures," ACM (Assoc. Comput. Math.) Trans. Graphics 24, 777-786 (2005).

Efros, A.

A. Efros and T. Leung, "Texture synthesis by non-parametric sampling," in Proceedings of Seventh International Conference on Computer Vision, pp. 1033-1038 (Institute of Electrical and Electronic Engineers, 1999).
[Crossref]

Efros, A. A.

A. A. Efros and W. T. Freeman, "Image quilting for texture synthesis and transfer," in Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2001), pp. 341-346.

Essa, I.

V. Kwatra, A. Schödl, I. Essa, G. Turk, and A. Bobick, "Graphcut textures: image and video synthesis using graph cuts," (Assoc. Comput. Mach.) Trans. Graphics 22, 277-286 (2003).

Fleischer, K. W.

K. W. Fleischer, D. H. Laidlaw, B. L. Currin, and A. H. Barr, "Cellular texture generation," in Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 1995), pp. 239-248.

Fournier, A.

M. Walter, A. Fournier, and M. Reimers, "Clonal mosaic model for the synthesis of mammalian coat patterns," in Proceedings of Graphics Interface '98, pp. 82-91 (Canadian Human-Computer Communication Society, June 1998).

Freeman, W. T.

A. A. Efros and W. T. Freeman, "Image quilting for texture synthesis and transfer," in Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2001), pp. 341-346.

Fukuda, M.

T. S. Kuroda and M. Fukuda, "Rab27A-binding protein Slp2-a is required for peripheral melanosome distribution and elongated cell shape in melanocytes," Nature Cell Biol. 6, 1195-1203 (1994).
[Crossref]

Ghazanfarpour, D.

J.-M. Dischler and D. Ghazanfarpour, "Interactive image-based modeling of macrostructured textures," IEEE Comput. Graphics Appl. 19, 66-74 (1999).
[Crossref]

Guo, B.

J. Zhang, K. Zhou, L. Velho, B. Guo, and H.-Y. Shum, "Synthesis of progressively-variant textures on arbitrary surfaces," ACM (Assoc. Comput. Mach.) Trans. Graphics 22, 295-302 (2003).

L. Liang, C. Liu, Y.-Q. Xu, B. Guo, and H.-Y. Shum, "Real-time texture synthesis by patch-based sampling," ACM (Assoc. Comput. Mach.) Trans. Graphics 20, 127-150 (2001).

Haneishi, H.

Hawkins, T.

P. Debevec, T. Hawkins, C. Tchou, H.-P. Duiker, W. Sarokin, and M. Sagar, "Acquiring the reflectance field of a human face," in Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2000), pp. 145-156.

Heeger, D. J.

D. J. Heeger and J. R. Bergen, "Pyramid-based texture analysis/synthesis," in Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 1995), pp. 229-238.

Hertzmann, A.

A. Hertzmann, C. E. Jacobs, N. Oliver, B. Curless, and D. H. Salesin, "Image analogies," in Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2001), pp. 327-340.

Hoppe, H.

S. Lefebvre and H. Hoppe, "Parallel controllable texture synthesis," ACM (Assoc. Comput. Math.) Trans. Graphics 24, 787-794 (2005).

Hori, K.

N. Tsumura, N. Ojima, K. Sato, M. Shiraishi, H. Shimizu, H. Nabeshima, S. Akazaki, K. Hori, and Y. Miyake, "Image-based skin color and texture analysis/synthesis by extracting hemoglobin and melanin information in the skin," ACM (Assoc. Comput. Mach.) Trans. Graphics 22, 770-779 (2003).

Jacobs, C. E.

A. Hertzmann, C. E. Jacobs, N. Oliver, B. Curless, and D. H. Salesin, "Image analogies," in Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2001), pp. 327-340.

Kass, M.

A. Witkin and M. Kass, "Reaction-diffusion textures," in Proceedings of the 18th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 1991), pp. 299-308.

Kuroda, T. S.

T. S. Kuroda and M. Fukuda, "Rab27A-binding protein Slp2-a is required for peripheral melanosome distribution and elongated cell shape in melanocytes," Nature Cell Biol. 6, 1195-1203 (1994).
[Crossref]

Kwatra, V.

V. Kwatra, A. Schödl, I. Essa, G. Turk, and A. Bobick, "Graphcut textures: image and video synthesis using graph cuts," (Assoc. Comput. Mach.) Trans. Graphics 22, 277-286 (2003).

Laidlaw, D. H.

K. W. Fleischer, D. H. Laidlaw, B. L. Currin, and A. H. Barr, "Cellular texture generation," in Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 1995), pp. 239-248.

Lefebvre, S.

S. Lefebvre and H. Hoppe, "Parallel controllable texture synthesis," ACM (Assoc. Comput. Math.) Trans. Graphics 24, 787-794 (2005).

Leung, T.

T. Leung and J. Malik, "Representing and recognizing the visual appearance of materials using three dimensional textons," Int. J. Comput. Vision 43, 29-44 (2001).
[Crossref]

A. Efros and T. Leung, "Texture synthesis by non-parametric sampling," in Proceedings of Seventh International Conference on Computer Vision, pp. 1033-1038 (Institute of Electrical and Electronic Engineers, 1999).
[Crossref]

Levoy, M.

L.-Y. Wei and M. Levoy, "Fast texture synthesis using tree-structured vector quantization," in Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2000), pp. 479-488.

L.-Y. Wei and M. Levoy, "Texture synthesis over arbitrary manifold surfaces," in Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2001), pp. 355-360.

Liang, L.

L. Liang, C. Liu, Y.-Q. Xu, B. Guo, and H.-Y. Shum, "Real-time texture synthesis by patch-based sampling," ACM (Assoc. Comput. Mach.) Trans. Graphics 20, 127-150 (2001).

Lin, W.

Y. Liu and W. Lin, "Deformable texture: the irregular-regular-irregular cycle," in Proceedings of the Third International Workshop on Texture Analysis and Synthesis, pp. 65-70 (Heriot-Watt University, 2003).

Liu, C.

L. Liang, C. Liu, Y.-Q. Xu, B. Guo, and H.-Y. Shum, "Real-time texture synthesis by patch-based sampling," ACM (Assoc. Comput. Mach.) Trans. Graphics 20, 127-150 (2001).

Liu, Y.

Y. Liu and W. Lin, "Deformable texture: the irregular-regular-irregular cycle," in Proceedings of the Third International Workshop on Texture Analysis and Synthesis, pp. 65-70 (Heriot-Watt University, 2003).

Y. Tsin, Y. Liu, and V. Ramesh, "Texture replacement in real images," in Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, 539-544 (IEEE, 2001).

Malik, J.

T. Leung and J. Malik, "Representing and recognizing the visual appearance of materials using three dimensional textons," Int. J. Comput. Vision 43, 29-44 (2001).
[Crossref]

Matusik, W.

W. Matusik, M. Zwicker, and F. Durand, "Texture design using a simplicial complex of Morphable Textures," ACM (Assoc. Comput. Math.) Trans. Graphics 24, 777-786 (2005).

W. Matusik and H. Pfister, "3D TV: a scalable system for real-time acquisition, transmission, and autostereoscopic display of dynamic scenes," ACM (Assoc. Comput. Mach.) Trans. Graphics 23, 814-824 (2004).

W. Matusik, H. Pfister, M. Brand, and L. McMillan, "A data-driven reflectance model, ACM (Assoc. Comput. Mach.)Trans. Graphics 22, 759-769 (2003).
[Crossref]

McMillan, L.

W. Matusik, H. Pfister, M. Brand, and L. McMillan, "A data-driven reflectance model, ACM (Assoc. Comput. Mach.)Trans. Graphics 22, 759-769 (2003).
[Crossref]

Miyake, Y.

N. Tsumura, N. Ojima, K. Sato, M. Shiraishi, H. Shimizu, H. Nabeshima, S. Akazaki, K. Hori, and Y. Miyake, "Image-based skin color and texture analysis/synthesis by extracting hemoglobin and melanin information in the skin," ACM (Assoc. Comput. Mach.) Trans. Graphics 22, 770-779 (2003).

N. Tsumura, H. Haneishi, and Y. Miyake, "Independent component analysis of skin color image," J. Opt. Soc. Am. A 16, 9, 2169-2176 (1999).
[Crossref]

Murphy, F. P.

O. G. Cula, K. J. Dana, F. P. Murphy, and B. K. Rao, "Bidirectional imaging and modeling of skin texture," IEEE Trans. Biomed. Eng. 51, 2148-2159 (2004).
[Crossref] [PubMed]

Nabeshima, H.

N. Tsumura, N. Ojima, K. Sato, M. Shiraishi, H. Shimizu, H. Nabeshima, S. Akazaki, K. Hori, and Y. Miyake, "Image-based skin color and texture analysis/synthesis by extracting hemoglobin and melanin information in the skin," ACM (Assoc. Comput. Mach.) Trans. Graphics 22, 770-779 (2003).

Ojima, N.

N. Tsumura, N. Ojima, K. Sato, M. Shiraishi, H. Shimizu, H. Nabeshima, S. Akazaki, K. Hori, and Y. Miyake, "Image-based skin color and texture analysis/synthesis by extracting hemoglobin and melanin information in the skin," ACM (Assoc. Comput. Mach.) Trans. Graphics 22, 770-779 (2003).

Oliver, N.

A. Hertzmann, C. E. Jacobs, N. Oliver, B. Curless, and D. H. Salesin, "Image analogies," in Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2001), pp. 327-340.

Perlin, K.

K. Perlin, "An image synthesizer," in Proceedings of the 12th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 1985), pp. 287-296.

Pfister, H.

W. Matusik and H. Pfister, "3D TV: a scalable system for real-time acquisition, transmission, and autostereoscopic display of dynamic scenes," ACM (Assoc. Comput. Mach.) Trans. Graphics 23, 814-824 (2004).

W. Matusik, H. Pfister, M. Brand, and L. McMillan, "A data-driven reflectance model, ACM (Assoc. Comput. Mach.)Trans. Graphics 22, 759-769 (2003).
[Crossref]

Portilla, J.

E. Simoncelli and J. Portilla, "Texture characterization via joint statistics of wavelet coefficient magnitudes," in Proceedings of Fifth International Conference on Image Processing1,62-66 (Institute of Electrical and Electronic Engineers, 1998).

Ramesh, V.

Y. Tsin, Y. Liu, and V. Ramesh, "Texture replacement in real images," in Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, 539-544 (IEEE, 2001).

Rao, B. K.

O. G. Cula, K. J. Dana, F. P. Murphy, and B. K. Rao, "Bidirectional imaging and modeling of skin texture," IEEE Trans. Biomed. Eng. 51, 2148-2159 (2004).
[Crossref] [PubMed]

Reimers, M.

M. Walter, A. Fournier, and M. Reimers, "Clonal mosaic model for the synthesis of mammalian coat patterns," in Proceedings of Graphics Interface '98, pp. 82-91 (Canadian Human-Computer Communication Society, June 1998).

Sagar, M.

P. Debevec, T. Hawkins, C. Tchou, H.-P. Duiker, W. Sarokin, and M. Sagar, "Acquiring the reflectance field of a human face," in Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2000), pp. 145-156.

Salesin, D. H.

A. Hertzmann, C. E. Jacobs, N. Oliver, B. Curless, and D. H. Salesin, "Image analogies," in Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2001), pp. 327-340.

Sarokin, W.

P. Debevec, T. Hawkins, C. Tchou, H.-P. Duiker, W. Sarokin, and M. Sagar, "Acquiring the reflectance field of a human face," in Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2000), pp. 145-156.

Sato, K.

N. Tsumura, N. Ojima, K. Sato, M. Shiraishi, H. Shimizu, H. Nabeshima, S. Akazaki, K. Hori, and Y. Miyake, "Image-based skin color and texture analysis/synthesis by extracting hemoglobin and melanin information in the skin," ACM (Assoc. Comput. Mach.) Trans. Graphics 22, 770-779 (2003).

Schödl, A.

V. Kwatra, A. Schödl, I. Essa, G. Turk, and A. Bobick, "Graphcut textures: image and video synthesis using graph cuts," (Assoc. Comput. Mach.) Trans. Graphics 22, 277-286 (2003).

Shimizu, H.

N. Tsumura, N. Ojima, K. Sato, M. Shiraishi, H. Shimizu, H. Nabeshima, S. Akazaki, K. Hori, and Y. Miyake, "Image-based skin color and texture analysis/synthesis by extracting hemoglobin and melanin information in the skin," ACM (Assoc. Comput. Mach.) Trans. Graphics 22, 770-779 (2003).

Shiraishi, M.

N. Tsumura, N. Ojima, K. Sato, M. Shiraishi, H. Shimizu, H. Nabeshima, S. Akazaki, K. Hori, and Y. Miyake, "Image-based skin color and texture analysis/synthesis by extracting hemoglobin and melanin information in the skin," ACM (Assoc. Comput. Mach.) Trans. Graphics 22, 770-779 (2003).

Shum, H.-Y.

J. Zhang, K. Zhou, L. Velho, B. Guo, and H.-Y. Shum, "Synthesis of progressively-variant textures on arbitrary surfaces," ACM (Assoc. Comput. Mach.) Trans. Graphics 22, 295-302 (2003).

L. Liang, C. Liu, Y.-Q. Xu, B. Guo, and H.-Y. Shum, "Real-time texture synthesis by patch-based sampling," ACM (Assoc. Comput. Mach.) Trans. Graphics 20, 127-150 (2001).

Simoncelli, E.

E. Simoncelli and J. Portilla, "Texture characterization via joint statistics of wavelet coefficient magnitudes," in Proceedings of Fifth International Conference on Image Processing1,62-66 (Institute of Electrical and Electronic Engineers, 1998).

Tchou, C.

P. Debevec, T. Hawkins, C. Tchou, H.-P. Duiker, W. Sarokin, and M. Sagar, "Acquiring the reflectance field of a human face," in Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2000), pp. 145-156.

Tsin, Y.

Y. Tsin, Y. Liu, and V. Ramesh, "Texture replacement in real images," in Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, 539-544 (IEEE, 2001).

Tsumura, N.

N. Tsumura, N. Ojima, K. Sato, M. Shiraishi, H. Shimizu, H. Nabeshima, S. Akazaki, K. Hori, and Y. Miyake, "Image-based skin color and texture analysis/synthesis by extracting hemoglobin and melanin information in the skin," ACM (Assoc. Comput. Mach.) Trans. Graphics 22, 770-779 (2003).

N. Tsumura, H. Haneishi, and Y. Miyake, "Independent component analysis of skin color image," J. Opt. Soc. Am. A 16, 9, 2169-2176 (1999).
[Crossref]

Turk, G.

V. Kwatra, A. Schödl, I. Essa, G. Turk, and A. Bobick, "Graphcut textures: image and video synthesis using graph cuts," (Assoc. Comput. Mach.) Trans. Graphics 22, 277-286 (2003).

G. Turk, "Generating textures on arbitrary surfaces using reaction-diffusion," in Proceedings of the 18th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 1991), pp. 289-298.

G. Turk, "Texture synthesis on surfaces," in Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2001), pp. 345-354.

Velho, L.

J. Zhang, K. Zhou, L. Velho, B. Guo, and H.-Y. Shum, "Synthesis of progressively-variant textures on arbitrary surfaces," ACM (Assoc. Comput. Mach.) Trans. Graphics 22, 295-302 (2003).

Walter, M.

M. Walter, A. Fournier, and M. Reimers, "Clonal mosaic model for the synthesis of mammalian coat patterns," in Proceedings of Graphics Interface '98, pp. 82-91 (Canadian Human-Computer Communication Society, June 1998).

Wei, L.-Y.

L.-Y. Wei and M. Levoy, "Fast texture synthesis using tree-structured vector quantization," in Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2000), pp. 479-488.

L.-Y. Wei and M. Levoy, "Texture synthesis over arbitrary manifold surfaces," in Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2001), pp. 355-360.

Witkin, A.

A. Witkin and M. Kass, "Reaction-diffusion textures," in Proceedings of the 18th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 1991), pp. 299-308.

Worley, S.

S. Worley, "A cellular texture basis function," in Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 1996), pp. 291-294.

Wu, Q.

Q. Wu and Y. Yu, "Feature matching and deformation for texture synthesis," ACM (Assoc. Comput. Mach.) Trans. Graphics 23, 364-367 (2004).

Xu, Y.-Q.

L. Liang, C. Liu, Y.-Q. Xu, B. Guo, and H.-Y. Shum, "Real-time texture synthesis by patch-based sampling," ACM (Assoc. Comput. Mach.) Trans. Graphics 20, 127-150 (2001).

Yu, Y.

Q. Wu and Y. Yu, "Feature matching and deformation for texture synthesis," ACM (Assoc. Comput. Mach.) Trans. Graphics 23, 364-367 (2004).

Zhang, J.

J. Zhang, K. Zhou, L. Velho, B. Guo, and H.-Y. Shum, "Synthesis of progressively-variant textures on arbitrary surfaces," ACM (Assoc. Comput. Mach.) Trans. Graphics 22, 295-302 (2003).

Zhou, K.

J. Zhang, K. Zhou, L. Velho, B. Guo, and H.-Y. Shum, "Synthesis of progressively-variant textures on arbitrary surfaces," ACM (Assoc. Comput. Mach.) Trans. Graphics 22, 295-302 (2003).

Zwicker, M.

W. Matusik, M. Zwicker, and F. Durand, "Texture design using a simplicial complex of Morphable Textures," ACM (Assoc. Comput. Math.) Trans. Graphics 24, 777-786 (2005).

(Assoc. Comput. Mach.) Trans. Graphics (1)

V. Kwatra, A. Schödl, I. Essa, G. Turk, and A. Bobick, "Graphcut textures: image and video synthesis using graph cuts," (Assoc. Comput. Mach.) Trans. Graphics 22, 277-286 (2003).

ACM (8)

Q. Wu and Y. Yu, "Feature matching and deformation for texture synthesis," ACM (Assoc. Comput. Mach.) Trans. Graphics 23, 364-367 (2004).

L. Liang, C. Liu, Y.-Q. Xu, B. Guo, and H.-Y. Shum, "Real-time texture synthesis by patch-based sampling," ACM (Assoc. Comput. Mach.) Trans. Graphics 20, 127-150 (2001).

S. Lefebvre and H. Hoppe, "Parallel controllable texture synthesis," ACM (Assoc. Comput. Math.) Trans. Graphics 24, 787-794 (2005).

W. Matusik, M. Zwicker, and F. Durand, "Texture design using a simplicial complex of Morphable Textures," ACM (Assoc. Comput. Math.) Trans. Graphics 24, 777-786 (2005).

N. Tsumura, N. Ojima, K. Sato, M. Shiraishi, H. Shimizu, H. Nabeshima, S. Akazaki, K. Hori, and Y. Miyake, "Image-based skin color and texture analysis/synthesis by extracting hemoglobin and melanin information in the skin," ACM (Assoc. Comput. Mach.) Trans. Graphics 22, 770-779 (2003).

W. Matusik and H. Pfister, "3D TV: a scalable system for real-time acquisition, transmission, and autostereoscopic display of dynamic scenes," ACM (Assoc. Comput. Mach.) Trans. Graphics 23, 814-824 (2004).

P. J. Burt and E. H. Adelson, "A multiresolution spline with application to image mosaics," ACM (Assoc. Comput. Mach.) Trans. Graphics 2, 217-236 (1983).

J. Zhang, K. Zhou, L. Velho, B. Guo, and H.-Y. Shum, "Synthesis of progressively-variant textures on arbitrary surfaces," ACM (Assoc. Comput. Mach.) Trans. Graphics 22, 295-302 (2003).

IEEE Comput. Graphics Appl. (1)

J.-M. Dischler and D. Ghazanfarpour, "Interactive image-based modeling of macrostructured textures," IEEE Comput. Graphics Appl. 19, 66-74 (1999).
[Crossref]

IEEE Trans. Biomed. Eng. (1)

O. G. Cula, K. J. Dana, F. P. Murphy, and B. K. Rao, "Bidirectional imaging and modeling of skin texture," IEEE Trans. Biomed. Eng. 51, 2148-2159 (2004).
[Crossref] [PubMed]

Int. J. Comput. Vision (2)

T. Leung and J. Malik, "Representing and recognizing the visual appearance of materials using three dimensional textons," Int. J. Comput. Vision 43, 29-44 (2001).
[Crossref]

O. G. Cula and K. J. Dana, "3D texture recognition using bidirectional feature histograms," Int. J. Comput. Vision 59, 33-60 (2004).
[Crossref]

J. Opt. Soc. Am. A (1)

Nature Cell Biol. (1)

T. S. Kuroda and M. Fukuda, "Rab27A-binding protein Slp2-a is required for peripheral melanosome distribution and elongated cell shape in melanocytes," Nature Cell Biol. 6, 1195-1203 (1994).
[Crossref]

Trans. Graphics (1)

W. Matusik, H. Pfister, M. Brand, and L. McMillan, "A data-driven reflectance model, ACM (Assoc. Comput. Mach.)Trans. Graphics 22, 759-769 (2003).
[Crossref]

Other (19)

P. Debevec, T. Hawkins, C. Tchou, H.-P. Duiker, W. Sarokin, and M. Sagar, "Acquiring the reflectance field of a human face," in Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2000), pp. 145-156.

G. Turk, "Texture synthesis on surfaces," in Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2001), pp. 345-354.

L.-Y. Wei and M. Levoy, "Texture synthesis over arbitrary manifold surfaces," in Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2001), pp. 355-360.

K. Perlin, "An image synthesizer," in Proceedings of the 12th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 1985), pp. 287-296.

A. Witkin and M. Kass, "Reaction-diffusion textures," in Proceedings of the 18th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 1991), pp. 299-308.

G. Turk, "Generating textures on arbitrary surfaces using reaction-diffusion," in Proceedings of the 18th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 1991), pp. 289-298.

K. W. Fleischer, D. H. Laidlaw, B. L. Currin, and A. H. Barr, "Cellular texture generation," in Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 1995), pp. 239-248.

S. Worley, "A cellular texture basis function," in Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 1996), pp. 291-294.

M. Walter, A. Fournier, and M. Reimers, "Clonal mosaic model for the synthesis of mammalian coat patterns," in Proceedings of Graphics Interface '98, pp. 82-91 (Canadian Human-Computer Communication Society, June 1998).

D. J. Heeger and J. R. Bergen, "Pyramid-based texture analysis/synthesis," in Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 1995), pp. 229-238.

J. S. De Bonet, "Multiresolution sampling procedure for analysis and synthesis of texture images," in Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 1997), pp. 361-368.

E. Simoncelli and J. Portilla, "Texture characterization via joint statistics of wavelet coefficient magnitudes," in Proceedings of Fifth International Conference on Image Processing1,62-66 (Institute of Electrical and Electronic Engineers, 1998).

A. Efros and T. Leung, "Texture synthesis by non-parametric sampling," in Proceedings of Seventh International Conference on Computer Vision, pp. 1033-1038 (Institute of Electrical and Electronic Engineers, 1999).
[Crossref]

L.-Y. Wei and M. Levoy, "Fast texture synthesis using tree-structured vector quantization," in Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2000), pp. 479-488.

M. Ashikhmin, "Synthesizing natural textures," in Proceedings of the 2001 Symposium on Interactive 3D Graphics (ACM Press, 2001), pp. 217-226.
[Crossref]

Y. Liu and W. Lin, "Deformable texture: the irregular-regular-irregular cycle," in Proceedings of the Third International Workshop on Texture Analysis and Synthesis, pp. 65-70 (Heriot-Watt University, 2003).

A. Hertzmann, C. E. Jacobs, N. Oliver, B. Curless, and D. H. Salesin, "Image analogies," in Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2001), pp. 327-340.

A. A. Efros and W. T. Freeman, "Image quilting for texture synthesis and transfer," in Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2001), pp. 341-346.

Y. Tsin, Y. Liu, and V. Ramesh, "Texture replacement in real images," in Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, 539-544 (IEEE, 2001).

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (10)

Fig. 1
Fig. 1

(Color online) Separation of two images with different polarizing filters into specular, shading, melanin, and hemoglobin components. We introduced interactive separation into the original method to improve the separated results.

Fig. 2
Fig. 2

(Color online) Process of controlling the proposed skin melanin texture.

Fig. 3
Fig. 3

(Color online) Skin melanin texture database (18 of 123 samples). The values written below each texture image are yellow unevenness A and age Y.

Fig. 4
Fig. 4

(Color online) Various types of skin melanin texture and their histograms in layers of the Laplacian pyramid decomposition.

Fig. 5
Fig. 5

(Color online) Plots of the principal space of 123 skin melanin texture and the control direction for aging and yellow unevenness. The colors of each plot are related to (a) yellow unevenness or (b) age. The arrows indicate the principal direction of change.

Fig. 6
Fig. 6

(Color online) Results of the skin appearance change for three original images by the data-driven physiologically based melanin texture control.

Fig. 7
Fig. 7

(Color online) Continuous melanin texture controls based on the data-driven physiological model from the skin texture database. The second image from the left is the original facial image and texture.

Fig. 8
Fig. 8

(Color online) Comparison of the synthesized image with the UV image, which can be used to predict the change of melanin texture in aging: (a) original image, (b) UV image, (c) controlled image.

Fig. 9
Fig. 9

(Color online) Real-time processing system for the facial appearance analysis and continuous synthesis that we developed.

Fig. 10
Fig. 10

(Color online) Real-time processing by the developed system.

Equations (12)

Equations on this page are rendered with MathJax. Learn more.

+ h i ( v ) d v = 1 .
σ i = [ + { h i ( v ) } 2 d v ] 1 / 2 , i = 1   or   2 .
σ i = 1 2 r left [ 2 0 { h i ( v ) } 2 d v ] 1 / 2 , i = 3   or   4 ,
r left = 0 h i ( v ) d v .
r i = 1 thresh_ i 2 r left N ( v : 0 , σ i ) d v
thresh_ i a i exp { u i ( v thresh_ i ) } d v = r i .
f ^ = [ 1 JND_ σ 1 0 0 0           0 0 0 1 JND_ r 2 ] f
p = Σ 1 B ( f ^ f ^ ¯ ) ,
[ ρ b ] = [ P 1 ] ( [ P 1 ] [ P 1 ] t ) 1 a ,
p = p + s ρ ,
f ^ = Σ B t p i + f ^ ¯ ,
c = B 1 ν d       log ,

Metrics