Abstract

Integral photography (IP), which is one of the ideal 3-D photographic technologies, can be regarded as a method of capturing and displaying light rays passing through a plane. The NHK Science and Technical Research Laboratories have developed a real-time IP system using an HDTV camera and an optical fiber array. In this paper, the authors propose a method of synthesizing arbitrary views from IP images captured by the HDTV camera. This is a kind of image-based rendering system, founded on the 4-D data space representation of light rays. Experimental results show the potential to improve the quality of images rendered by computer graphics techniques.

©2001 Optical Society of America

Full Article  |  PDF Article
OSA Recommended Articles
Real-time capture and reconstruction system with multiple GPUs for a 3D live scene by a generation from 4K IP images to 8K holograms

Yasuyuki Ichihashi, Ryutaro Oi, Takanori Senoh, Kenji Yamamoto, and Taiichiro Kurita
Opt. Express 20(19) 21645-21655 (2012)

Gradient-index lens-array method based on real-time integral photography for three-dimensional images

Jun Arai, Fumio Okano, Haruo Hoshino, and Ichiro Yuyama
Appl. Opt. 37(11) 2034-2045 (1998)

Computational integral-imaging reconstruction-based 3-D volumetric target object recognition by using a 3-D reference object

Seung-Cheol Kim, Seok-Chan Park, and Eun-Soo Kim
Appl. Opt. 48(34) H95-H104 (2009)

More Recommended Articles

References

  • View by:
  • Article Order
  • |
  • Year
  • |
  • Author
  • |
  • Publication
  1. S. E. Chen, “QuickTime VR: an image-based approach to virtual environment navigation,” in SIG-GRAPH’95, (Association for Computing Machinery, Los Angeles, 1995), 29–38, http://www.acm.org/pubs/articles/proceedings/graph/218380/p29-chen/p29-chen.pdf.
  2. L. McMillan and G. Bishop, “Plenoptic modeling: an image-based rendering system,” in SIG-GRAPH’ 95, (Association for Computing Machinery, Los Angeles, 1995), 39–46, http://www.acm.org/pubs/articles/proceedings/graph/218380/p39-mcmillan/p39-mcmillan.pdf.
  3. T. Naemura, M. Kaneko, and H. Harashima, “Orthographic approach to representing 3-D images and interpolating light rays for 3-D image communication and virtual environment,” Signal Processing: Image Communication,  14, 21–37 (1998), http://www.elsevier.nl/gej-ng/10/22/18/35/17/18/article.pdf.
    [Crossref]
  4. T. Naemura and H. Harashima, “Ray-based approach to the integrated 3D visual communication,” in Three-Dimensional Video and Display: Devices and Systems, B. Javidi and F. Okano, eds., Proc. SPIECR76, (to be published).
  5. F. Okano, H. Hoshino, J. Arai, and I. Yuyama, “Real-time pickup method for a three-dimensional image based on Integral Photography”, Appl. Opt. ,  36, 1598–1603 (1997), http://www.opticsinfobase.org/ViewMedia.cfm?id=42907&seq=0.
    [Crossref] [PubMed]
  6. J. Arai, F. Okano, H. Hoshino, and I. Yuyama, “Gradient-index lens-array method based on real-time integral photography for three-dimensional images,” Appl. Opt.,  37, 2034–2045 (1998), http://www.opticsinfobase.org/ViewMedia.cfm?id=43189&seq=0.
    [Crossref]
  7. H. Hoshino, F. Okano, H. Isono, and I. Yuyama, “Analysis of resolution limitation of integral photography,” J. Opt. Soc. Am. A,  15, 2058–2065 (1998), http://www.opticsinfobase.org/ViewMedia.cfm?id=1567&seq=0.
    [Crossref]
  8. T. Yoshida, T. Naemura, and H. Harashima, “3-D computer graphics based on integral photography,” in 3-D Image Conference 2000, (Tokyo, 2000), 39–42, in Japanese http://www.3d-conf.org/00/abstracts/1-9.html#English.
  9. T. Yanagisawa, T. Naemura, M. Kaneko, and H. Harashima, “Handling of 3-dimensional objects in ray space,” in 1995 Information and System Society Conference, (Institute of Electronics, Information and Communication Engineers, Tokyo, 1995), D-169, in Japanese.
  10. M. Levoy and P. Hanrahan, “Light field rendering,” in SIGGRAPH’96, (Association for Computing Machinery, New Orleans, 1996), 31–42, http://www.acm.org/pubs/articles/proceedings/graph/237170/p31-levoy/p31-levoy.pdf.
  11. S. Gortler, R. Grzeszczuk, R. Szeliski, and M. Cohen, “The Lumigraph,” in SIGGRAPH’96, (Association for Computing Machinery, New Orleans, 1996), 43–54, http://www.acm.org/pubs/articles/proceedings/graph/237170/p43-gortler/p43-gortler.pdf.
  12. T. Naemura and H. Harashima, “Real-time video-based rendering for augmented spatial communication,” in Visual Communication and Image Processing ’99, K. Aizawa, R. Stevenson, and Y. Zhang, eds., Proc. SPIE3653, 620–631 (1999), http://bookstore.spie.org/cgi-bin/abstract.pl?bibcode=1998SPIE%2e3653%2e%2e620N&page=1&qs=spie.

1998 (3)

T. Naemura, M. Kaneko, and H. Harashima, “Orthographic approach to representing 3-D images and interpolating light rays for 3-D image communication and virtual environment,” Signal Processing: Image Communication,  14, 21–37 (1998), http://www.elsevier.nl/gej-ng/10/22/18/35/17/18/article.pdf.
[Crossref]

H. Hoshino, F. Okano, H. Isono, and I. Yuyama, “Analysis of resolution limitation of integral photography,” J. Opt. Soc. Am. A,  15, 2058–2065 (1998), http://www.opticsinfobase.org/ViewMedia.cfm?id=1567&seq=0.
[Crossref]

J. Arai, F. Okano, H. Hoshino, and I. Yuyama, “Gradient-index lens-array method based on real-time integral photography for three-dimensional images,” Appl. Opt.,  37, 2034–2045 (1998), http://www.opticsinfobase.org/ViewMedia.cfm?id=43189&seq=0.
[Crossref]

1997 (1)

Arai, J.

Bishop, G.

L. McMillan and G. Bishop, “Plenoptic modeling: an image-based rendering system,” in SIG-GRAPH’ 95, (Association for Computing Machinery, Los Angeles, 1995), 39–46, http://www.acm.org/pubs/articles/proceedings/graph/218380/p39-mcmillan/p39-mcmillan.pdf.

Chen, S. E.

S. E. Chen, “QuickTime VR: an image-based approach to virtual environment navigation,” in SIG-GRAPH’95, (Association for Computing Machinery, Los Angeles, 1995), 29–38, http://www.acm.org/pubs/articles/proceedings/graph/218380/p29-chen/p29-chen.pdf.

Cohen, M.

S. Gortler, R. Grzeszczuk, R. Szeliski, and M. Cohen, “The Lumigraph,” in SIGGRAPH’96, (Association for Computing Machinery, New Orleans, 1996), 43–54, http://www.acm.org/pubs/articles/proceedings/graph/237170/p43-gortler/p43-gortler.pdf.

Gortler, S.

S. Gortler, R. Grzeszczuk, R. Szeliski, and M. Cohen, “The Lumigraph,” in SIGGRAPH’96, (Association for Computing Machinery, New Orleans, 1996), 43–54, http://www.acm.org/pubs/articles/proceedings/graph/237170/p43-gortler/p43-gortler.pdf.

Grzeszczuk, R.

S. Gortler, R. Grzeszczuk, R. Szeliski, and M. Cohen, “The Lumigraph,” in SIGGRAPH’96, (Association for Computing Machinery, New Orleans, 1996), 43–54, http://www.acm.org/pubs/articles/proceedings/graph/237170/p43-gortler/p43-gortler.pdf.

Hanrahan, P.

M. Levoy and P. Hanrahan, “Light field rendering,” in SIGGRAPH’96, (Association for Computing Machinery, New Orleans, 1996), 31–42, http://www.acm.org/pubs/articles/proceedings/graph/237170/p31-levoy/p31-levoy.pdf.

Harashima, H.

T. Naemura, M. Kaneko, and H. Harashima, “Orthographic approach to representing 3-D images and interpolating light rays for 3-D image communication and virtual environment,” Signal Processing: Image Communication,  14, 21–37 (1998), http://www.elsevier.nl/gej-ng/10/22/18/35/17/18/article.pdf.
[Crossref]

T. Naemura and H. Harashima, “Ray-based approach to the integrated 3D visual communication,” in Three-Dimensional Video and Display: Devices and Systems, B. Javidi and F. Okano, eds., Proc. SPIECR76, (to be published).

T. Naemura and H. Harashima, “Real-time video-based rendering for augmented spatial communication,” in Visual Communication and Image Processing ’99, K. Aizawa, R. Stevenson, and Y. Zhang, eds., Proc. SPIE3653, 620–631 (1999), http://bookstore.spie.org/cgi-bin/abstract.pl?bibcode=1998SPIE%2e3653%2e%2e620N&page=1&qs=spie.

T. Yanagisawa, T. Naemura, M. Kaneko, and H. Harashima, “Handling of 3-dimensional objects in ray space,” in 1995 Information and System Society Conference, (Institute of Electronics, Information and Communication Engineers, Tokyo, 1995), D-169, in Japanese.

T. Yoshida, T. Naemura, and H. Harashima, “3-D computer graphics based on integral photography,” in 3-D Image Conference 2000, (Tokyo, 2000), 39–42, in Japanese http://www.3d-conf.org/00/abstracts/1-9.html#English.

Hoshino, H.

Isono, H.

H. Hoshino, F. Okano, H. Isono, and I. Yuyama, “Analysis of resolution limitation of integral photography,” J. Opt. Soc. Am. A,  15, 2058–2065 (1998), http://www.opticsinfobase.org/ViewMedia.cfm?id=1567&seq=0.
[Crossref]

Kaneko, M.

T. Naemura, M. Kaneko, and H. Harashima, “Orthographic approach to representing 3-D images and interpolating light rays for 3-D image communication and virtual environment,” Signal Processing: Image Communication,  14, 21–37 (1998), http://www.elsevier.nl/gej-ng/10/22/18/35/17/18/article.pdf.
[Crossref]

T. Yanagisawa, T. Naemura, M. Kaneko, and H. Harashima, “Handling of 3-dimensional objects in ray space,” in 1995 Information and System Society Conference, (Institute of Electronics, Information and Communication Engineers, Tokyo, 1995), D-169, in Japanese.

Levoy, M.

M. Levoy and P. Hanrahan, “Light field rendering,” in SIGGRAPH’96, (Association for Computing Machinery, New Orleans, 1996), 31–42, http://www.acm.org/pubs/articles/proceedings/graph/237170/p31-levoy/p31-levoy.pdf.

McMillan, L.

L. McMillan and G. Bishop, “Plenoptic modeling: an image-based rendering system,” in SIG-GRAPH’ 95, (Association for Computing Machinery, Los Angeles, 1995), 39–46, http://www.acm.org/pubs/articles/proceedings/graph/218380/p39-mcmillan/p39-mcmillan.pdf.

Naemura, T.

T. Naemura, M. Kaneko, and H. Harashima, “Orthographic approach to representing 3-D images and interpolating light rays for 3-D image communication and virtual environment,” Signal Processing: Image Communication,  14, 21–37 (1998), http://www.elsevier.nl/gej-ng/10/22/18/35/17/18/article.pdf.
[Crossref]

T. Naemura and H. Harashima, “Ray-based approach to the integrated 3D visual communication,” in Three-Dimensional Video and Display: Devices and Systems, B. Javidi and F. Okano, eds., Proc. SPIECR76, (to be published).

T. Yoshida, T. Naemura, and H. Harashima, “3-D computer graphics based on integral photography,” in 3-D Image Conference 2000, (Tokyo, 2000), 39–42, in Japanese http://www.3d-conf.org/00/abstracts/1-9.html#English.

T. Yanagisawa, T. Naemura, M. Kaneko, and H. Harashima, “Handling of 3-dimensional objects in ray space,” in 1995 Information and System Society Conference, (Institute of Electronics, Information and Communication Engineers, Tokyo, 1995), D-169, in Japanese.

T. Naemura and H. Harashima, “Real-time video-based rendering for augmented spatial communication,” in Visual Communication and Image Processing ’99, K. Aizawa, R. Stevenson, and Y. Zhang, eds., Proc. SPIE3653, 620–631 (1999), http://bookstore.spie.org/cgi-bin/abstract.pl?bibcode=1998SPIE%2e3653%2e%2e620N&page=1&qs=spie.

Okano, F.

Szeliski, R.

S. Gortler, R. Grzeszczuk, R. Szeliski, and M. Cohen, “The Lumigraph,” in SIGGRAPH’96, (Association for Computing Machinery, New Orleans, 1996), 43–54, http://www.acm.org/pubs/articles/proceedings/graph/237170/p43-gortler/p43-gortler.pdf.

Yanagisawa, T.

T. Yanagisawa, T. Naemura, M. Kaneko, and H. Harashima, “Handling of 3-dimensional objects in ray space,” in 1995 Information and System Society Conference, (Institute of Electronics, Information and Communication Engineers, Tokyo, 1995), D-169, in Japanese.

Yoshida, T.

T. Yoshida, T. Naemura, and H. Harashima, “3-D computer graphics based on integral photography,” in 3-D Image Conference 2000, (Tokyo, 2000), 39–42, in Japanese http://www.3d-conf.org/00/abstracts/1-9.html#English.

Yuyama, I.

Appl. Opt. (2)

J. Opt. Soc. Am. A (1)

H. Hoshino, F. Okano, H. Isono, and I. Yuyama, “Analysis of resolution limitation of integral photography,” J. Opt. Soc. Am. A,  15, 2058–2065 (1998), http://www.opticsinfobase.org/ViewMedia.cfm?id=1567&seq=0.
[Crossref]

Signal Processing: Image Communication (1)

T. Naemura, M. Kaneko, and H. Harashima, “Orthographic approach to representing 3-D images and interpolating light rays for 3-D image communication and virtual environment,” Signal Processing: Image Communication,  14, 21–37 (1998), http://www.elsevier.nl/gej-ng/10/22/18/35/17/18/article.pdf.
[Crossref]

Other (8)

T. Naemura and H. Harashima, “Ray-based approach to the integrated 3D visual communication,” in Three-Dimensional Video and Display: Devices and Systems, B. Javidi and F. Okano, eds., Proc. SPIECR76, (to be published).

S. E. Chen, “QuickTime VR: an image-based approach to virtual environment navigation,” in SIG-GRAPH’95, (Association for Computing Machinery, Los Angeles, 1995), 29–38, http://www.acm.org/pubs/articles/proceedings/graph/218380/p29-chen/p29-chen.pdf.

L. McMillan and G. Bishop, “Plenoptic modeling: an image-based rendering system,” in SIG-GRAPH’ 95, (Association for Computing Machinery, Los Angeles, 1995), 39–46, http://www.acm.org/pubs/articles/proceedings/graph/218380/p39-mcmillan/p39-mcmillan.pdf.

T. Yoshida, T. Naemura, and H. Harashima, “3-D computer graphics based on integral photography,” in 3-D Image Conference 2000, (Tokyo, 2000), 39–42, in Japanese http://www.3d-conf.org/00/abstracts/1-9.html#English.

T. Yanagisawa, T. Naemura, M. Kaneko, and H. Harashima, “Handling of 3-dimensional objects in ray space,” in 1995 Information and System Society Conference, (Institute of Electronics, Information and Communication Engineers, Tokyo, 1995), D-169, in Japanese.

M. Levoy and P. Hanrahan, “Light field rendering,” in SIGGRAPH’96, (Association for Computing Machinery, New Orleans, 1996), 31–42, http://www.acm.org/pubs/articles/proceedings/graph/237170/p31-levoy/p31-levoy.pdf.

S. Gortler, R. Grzeszczuk, R. Szeliski, and M. Cohen, “The Lumigraph,” in SIGGRAPH’96, (Association for Computing Machinery, New Orleans, 1996), 43–54, http://www.acm.org/pubs/articles/proceedings/graph/237170/p43-gortler/p43-gortler.pdf.

T. Naemura and H. Harashima, “Real-time video-based rendering for augmented spatial communication,” in Visual Communication and Image Processing ’99, K. Aizawa, R. Stevenson, and Y. Zhang, eds., Proc. SPIE3653, 620–631 (1999), http://bookstore.spie.org/cgi-bin/abstract.pl?bibcode=1998SPIE%2e3653%2e%2e620N&page=1&qs=spie.

Supplementary Material (7)

» Media 1: MOV (616 KB)     
» Media 2: MOV (328 KB)     
» Media 3: MOV (291 KB)     
» Media 4: MOV (269 KB)     
» Media 5: MOV (359 KB)     
» Media 6: MOV (178 KB)     
» Media 7: MOV (254 KB)     

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (9)

Fig. 1.
Fig. 1. Diagram of real-time IP system.

Download Full Size | PPT Slide | PDF

Fig. 2.
Fig. 2. Captured IP image.

Download Full Size | PPT Slide | PDF

Fig. 3.
Fig. 3. Coordinate system.

Download Full Size | PPT Slide | PDF

Fig. 4.
Fig. 4. Principle of interpolation.

Download Full Size | PPT Slide | PDF

Fig. 5.
Fig. 5. Result of Method 1 (res1.mov; 617 KB).

Download Full Size | PPT Slide | PDF

Fig. 6.
Fig. 6. Result of Method 2 (res2.mov; 329 KB).

Download Full Size | PPT Slide | PDF

Fig. 7.
Fig. 7. Results of Method 3 (res3b.mov; 291 KB), (res3f.mov; 270 KB).

Download Full Size | PPT Slide | PDF

Fig. 8.
Fig. 8. Result of reflective object (crystal.mov; 360 KB).

Download Full Size | PPT Slide | PDF

Fig. 9.
Fig. 9. Results of human images (naemura.mov; 178 KB), (finger.mov; 254 KB).

Download Full Size | PPT Slide | PDF

Tables (2)

Tables Icon

Table 1. Equipment specifications [6].

View Table | View all tables in this article
Tables Icon

Table 2. Comparison of VBR and the proposed method.

View Table | View all tables in this article

Equations (4)

Equations on this page are rendered with MathJax. Learn more.

x e = x c + ( x c − x 0 ) d z 0 − d = z 0 x c − d x 0 z 0 − d
y e = y c + ( y c − y 0 ) d z 0 − d = z 0 y c − d y 0 z 0 − d
q m − x m − d = x m − x 0 d − z 0
q m + 1 − q m − z s = x m + 1 − x m d − z s

Metrics