Abstract

Integral photography (IP), which is one of the ideal 3-D photographic technologies, can be regarded as a method of capturing and displaying light rays passing through a plane. The NHK Science and Technical Research Laboratories have developed a real-time IP system using an HDTV camera and an optical fiber array. In this paper, the authors propose a method of synthesizing arbitrary views from IP images captured by the HDTV camera. This is a kind of image-based rendering system, founded on the 4-D data space representation of light rays. Experimental results show the potential to improve the quality of images rendered by computer graphics techniques.

© Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |

  1. S. E. Chen, "QuickTime VR: an image-based approach to virtual environment navigation," in SIGGRAPH'95, (Association for Computing Machinery, Los Angeles, 1995), 29-38, http://www.acm.org/pubs/articles/proceedings/graph/218380/p29-chen/p29-chen.pdf .
  2. L. McMillan and G. Bishop, "Plenoptic modeling: an image-based rendering system," in SIG-GRAPH'95, (Association for Computing Machinery, Los Angeles, 1995), 39-46, http://www.acm.org/pubs/articles/proceedings/graph/218380/p39-mcmillan/p39-mcmillan.pdf.
  3. T. Naemura, M. Kaneko and H. Harashima, "Orthographic approach to representing 3-D images and interpolating light rays for 3-D image communication and virtual environment," Signal Processing: Image Communication, 14, 21-37 (1998), http://www.elsevier.nl/gej-ng/10/22/18/35/17/18/article.pdf .
    [CrossRef]
  4. T. Naemura and H. Harashima, "Ray-based approach to the integrated 3D visual communication," in Three-Dimensional Video and Display: Devices and Systems, B. Javidi and F. Okano, eds., Proc. SPIE CR76, (to be published).
  5. F. Okano, H. Hoshino, J. Arai and I. Yuyama, "Real-time pickup method for a three-dimensional image based on Integral Photography," Appl. Opt. 36, 1598-1603 (1997), http://www.opticsinfobase.org/ViewMedia.cfm?id=42907&seq=0.
    [CrossRef] [PubMed]
  6. J. Arai, F. Okano, H. Hoshino and I. Yuyama, "Gradient-index lens-array method based on real-time integral photography for three-dimensional images," Appl. Opt. 37, 2034-2045 (1998), http://www.opticsinfobase.org/ViewMedia.cfm?id=43189&seq=0.
    [CrossRef]
  7. H. Hoshino, F. Okano, H. Isono and I. Yuyama, "Analysis of resolution limitation of integral photography," J. Opt. Soc. Am. A 15, 2058-2065 (1998), http://www.opticsinfobase.org/ViewMedia.cfm?id=1567&seq=0.
    [CrossRef]
  8. T. Yoshida, T. Naemura and H. Harashima, "3-D computer graphics based on integral photography," in 3-D Image Conference 2000, (Tokyo, 2000), 39-42, in Japanese http://www.3d-conf.org/00/abstracts/1-9.html#English.
  9. T. Yanagisawa, T. Naemura, M. Kaneko and H. Harashima, "Handling of 3-dimensional objects in ray space," in 1995 Information and System Society Conference, (Institute of Electronics, Information and Communication Engineers, Tokyo, 1995), D-169, in Japanese.
  10. M. Levoy and P. Hanrahan, "Light field rendering," in SIGGRAPH'96, (Association for Computing Machinery, New Orleans, 1996), 31-42, http://www.acm.org/pubs/articles/proceedings/graph/237170/p31-levoy/p31-levoy.pdf .
  11. S. Gortler, R. Grzeszczuk, R. Szeliski and M. Cohen, "The Lumigraph," in SIGGRAPH'96, (Association for Computing Machinery, New Orleans, 1996), 43-54, http://www.acm.org/pubs/articles/proceedings/graph/237170/p43-gortler/p43-gortler.pdf
  12. T. Naemura and H. Harashima, "Real-time video-based rendering for augmented spatial communication," in Visual Communication and Image Processing '99, K. Aizawa, R. Stevenson and Y. Zhang, eds., Proc. SPIE 3653, 620-631 (1999), http://bookstore.spie.org/cgi-bin/abstract.pl?bibcode=1998SPIE%2e3653%2e%2e620N&page=1&qs=spie.

Other

S. E. Chen, "QuickTime VR: an image-based approach to virtual environment navigation," in SIGGRAPH'95, (Association for Computing Machinery, Los Angeles, 1995), 29-38, http://www.acm.org/pubs/articles/proceedings/graph/218380/p29-chen/p29-chen.pdf .

L. McMillan and G. Bishop, "Plenoptic modeling: an image-based rendering system," in SIG-GRAPH'95, (Association for Computing Machinery, Los Angeles, 1995), 39-46, http://www.acm.org/pubs/articles/proceedings/graph/218380/p39-mcmillan/p39-mcmillan.pdf.

T. Naemura, M. Kaneko and H. Harashima, "Orthographic approach to representing 3-D images and interpolating light rays for 3-D image communication and virtual environment," Signal Processing: Image Communication, 14, 21-37 (1998), http://www.elsevier.nl/gej-ng/10/22/18/35/17/18/article.pdf .
[CrossRef]

T. Naemura and H. Harashima, "Ray-based approach to the integrated 3D visual communication," in Three-Dimensional Video and Display: Devices and Systems, B. Javidi and F. Okano, eds., Proc. SPIE CR76, (to be published).

F. Okano, H. Hoshino, J. Arai and I. Yuyama, "Real-time pickup method for a three-dimensional image based on Integral Photography," Appl. Opt. 36, 1598-1603 (1997), http://www.opticsinfobase.org/ViewMedia.cfm?id=42907&seq=0.
[CrossRef] [PubMed]

J. Arai, F. Okano, H. Hoshino and I. Yuyama, "Gradient-index lens-array method based on real-time integral photography for three-dimensional images," Appl. Opt. 37, 2034-2045 (1998), http://www.opticsinfobase.org/ViewMedia.cfm?id=43189&seq=0.
[CrossRef]

H. Hoshino, F. Okano, H. Isono and I. Yuyama, "Analysis of resolution limitation of integral photography," J. Opt. Soc. Am. A 15, 2058-2065 (1998), http://www.opticsinfobase.org/ViewMedia.cfm?id=1567&seq=0.
[CrossRef]

T. Yoshida, T. Naemura and H. Harashima, "3-D computer graphics based on integral photography," in 3-D Image Conference 2000, (Tokyo, 2000), 39-42, in Japanese http://www.3d-conf.org/00/abstracts/1-9.html#English.

T. Yanagisawa, T. Naemura, M. Kaneko and H. Harashima, "Handling of 3-dimensional objects in ray space," in 1995 Information and System Society Conference, (Institute of Electronics, Information and Communication Engineers, Tokyo, 1995), D-169, in Japanese.

M. Levoy and P. Hanrahan, "Light field rendering," in SIGGRAPH'96, (Association for Computing Machinery, New Orleans, 1996), 31-42, http://www.acm.org/pubs/articles/proceedings/graph/237170/p31-levoy/p31-levoy.pdf .

S. Gortler, R. Grzeszczuk, R. Szeliski and M. Cohen, "The Lumigraph," in SIGGRAPH'96, (Association for Computing Machinery, New Orleans, 1996), 43-54, http://www.acm.org/pubs/articles/proceedings/graph/237170/p43-gortler/p43-gortler.pdf

T. Naemura and H. Harashima, "Real-time video-based rendering for augmented spatial communication," in Visual Communication and Image Processing '99, K. Aizawa, R. Stevenson and Y. Zhang, eds., Proc. SPIE 3653, 620-631 (1999), http://bookstore.spie.org/cgi-bin/abstract.pl?bibcode=1998SPIE%2e3653%2e%2e620N&page=1&qs=spie.

Supplementary Material (7)

» Media 1: MOV (616 KB)     
» Media 2: MOV (328 KB)     
» Media 3: MOV (291 KB)     
» Media 4: MOV (269 KB)     
» Media 5: MOV (359 KB)     
» Media 6: MOV (178 KB)     
» Media 7: MOV (254 KB)     

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (9)

Fig. 1.
Fig. 1.

Diagram of real-time IP system.

Fig. 2.
Fig. 2.

Captured IP image.

Fig. 3.
Fig. 3.

Coordinate system.

Fig. 4.
Fig. 4.

Principle of interpolation.

Fig. 5.
Fig. 5.

Result of Method 1 (res1.mov; 617 KB).

Fig. 6.
Fig. 6.

Result of Method 2 (res2.mov; 329 KB).

Fig. 7.
Fig. 7.

Results of Method 3 (res3b.mov; 291 KB), (res3f.mov; 270 KB).

Fig. 8.
Fig. 8.

Result of reflective object (crystal.mov; 360 KB).

Fig. 9.
Fig. 9.

Results of human images (naemura.mov; 178 KB), (finger.mov; 254 KB).

Tables (2)

Tables Icon

Table 1. Equipment specifications [6].

Tables Icon

Table 2. Comparison of VBR and the proposed method.

Equations (4)

Equations on this page are rendered with MathJax. Learn more.

x e = x c + ( x c x 0 ) d z 0 d = z 0 x c d x 0 z 0 d
y e = y c + ( y c y 0 ) d z 0 d = z 0 y c d y 0 z 0 d
q m x m d = x m x 0 d z 0
q m + 1 q m z s = x m + 1 x m d z s

Metrics