Abstract

Computational integral imaging method can digitally provide a series of plane images of three- dimensional (3D) objects. However, the resolution of 3D reconstructed images is dramatically degraded as the distance from the lenslet array increases. In this paper, to overcome this problem, we propose a novel computational integral imaging reconstruction (CIIR) method based on smart pixel mapping (SPM). Since SPM is a computational process in which elemental images recorded at long distances are convertible to ones recorded near lenslet array, this can give us the improved resolution of plane images for 3D objects located at a long distance range from a lenslet array. For the effective use of the SPM-based CIIR method, we design a novel two-stage CIIR method by the combined use of the conventional CIIR and the SPM-based one. The conventional CIIR method is applied over a short distance range, while the SPM-based CIIR is used over a long distance range. We carry out some experiments to verify the performance of the two-stage CIIR system. From the experimental results, the proposed system can provide a substantial gain over a long distance range in terms of the resolution of reconstructed plane images.

© 2008 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. G. Lippmann, “La photographic intergrale,” C. R. Acad. Sci. 146, 446-451 (1908).
  2. A. Stern and B. Javidi, “Three-dimensional image sensing, visualization, and processing using integral imaging,” Proc. IEEE 94, 591-607 (2006).
    [Crossref]
  3. B. Javidi and S.-H. Hong, “Three-dimensional holographic image sensing and integral imaging display,” J. Display Technol. 1, 341-346 (2005).
    [Crossref]
  4. F. Okano, H. Hoshino, J. Arai, and I. Yuyama, “Real-time pickup method for a three-dimensional image based on integral photography,” Appl. Opt. 36, 1598-1603 (1997).
    [Crossref] [PubMed]
  5. J.-S. Jang and B. Javidi, “Improved viewing resolution of three-dimensional integral imaging by use of nonstationary micro-optics,” Opt. Lett. 27, 324-326 (2002).
    [Crossref]
  6. B. Lee, S. Jung, and J.-H. Park, “Viewing-angle-enhanced integral imaging by lens switching,” Opt. Lett. 27, 818-820 (2002).
    [Crossref]
  7. D.-H. Shin, B. Lee, and E.-S. Kim, “Multidirectional curved integral imaging with large depth by additional use of a large-aperture lens,” Appl. Opt. 45, 7375-7381 (2006).
    [Crossref] [PubMed]
  8. R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a handheld plenoptic camera,” Computer Science Technical Report CSTR 2005-02 (Stanford University, 2005).
  9. H. Arimoto and B. Javidi, “Integral three-dimensional imaging with digital reconstruction,” Opt. Lett. 26, 157-159(2001).
    [Crossref]
  10. Y. Frauel and B. Javidi, “Digital three-dimensional image correlation by use of computer-reconstructed integral imaging,” Appl. Opt. 41, 5488-5496 (2002).
    [Crossref] [PubMed]
  11. Y. Frauel and B. Javidi, “Digital three-dimensional object reconstruction and correlation based on integral imaging,” Proc. SPIE 5006, 83-91 (2003).
    [Crossref]
  12. S. Kishk and B. Javidi, “Improved resolution 3D object sensing and recognition using time multiplexed computational integral imaging,” Opt. Express 11, 3528-3541 (2003).
    [Crossref] [PubMed]
  13. S.-H. Hong, J.-S. Jang, and B. Javidi, “Three-dimensional volumetric object reconstruction using computational integral imaging,” Opt. Express 12, 483-491 (2004).
    [Crossref] [PubMed]
  14. S.-H. Hong and B. Javidi, “Improved resolution 3D object reconstruction using computational integral imaging with time multiplexing,” Opt. Express 12, 4579-4588 (2004).
    [Crossref] [PubMed]
  15. Y.-W. Song, B. Javidi, and F. Jin, “3D object scaling in integral imaging display by varying the spatial ray sampling rate,” Opt. Express 13, 3242-3251 (2005).
    [Crossref] [PubMed]
  16. B. Javidi, R. Ponce-Díaz, and S.-H. Hong, “Three-dimensional recognition of occluded objects by using computational integral imaging,” Opt. Lett. 31, 1106-1108 (2006).
    [Crossref] [PubMed]
  17. S.-H. Hong and B. Javidi, “Distortion-tolerant 3D recognition of occluded objects using computational integral imaging,” Opt. Express 14, 12085-12095 (2006).
    [Crossref] [PubMed]
  18. H. Yoo and D.-H. Shin, “Improved analysis on the signal property of computational integral imaging system,” Opt. Express 15, 14107-14114 (2007).
    [Crossref] [PubMed]
  19. D.-H. Shin and H. Yoo, “Image quality enhancement in 3D computational integral imaging by use of interpolation methods,” Opt. Express 15, 12039-12049 (2007).
    [Crossref] [PubMed]
  20. J.-S. Park, D.-C. Hwang, D.-H. Shin, and E.-S. Kim, “Resolution-enhanced 3D image correlator using computationally reconstructed integral images,” Opt. Commun. 276, 72-79(2007).
    [Crossref]
  21. S. Yeom, B. Javidi, and E. Watson, “Three-dimensional distortion-tolerant object recognition using photon-counting integral imaging,” Opt. Express 15, 1513-1533(2007).
    [Crossref] [PubMed]
  22. D.-C. Hwang, K.-J. Lee, S.-C. Kim, and E.-S. Kim, “Extraction of location coordinates of 3-D objects from computationally reconstructed integral images basing on a blur metric,” Opt. Express 16, 3623-3635 (2008).
    [Crossref] [PubMed]
  23. D.-H. Shin and H. Yoo, “Scale-variant magnification for computational integral imaging and its application to 3D object correlator,” Opt. Express 16, 8855-8867 (2008).
    [Crossref] [PubMed]
  24. M. Martinez-Corral, B. Javidi, R. Martínez-Cuenca, and G. Saavedra, “Formation of real, orthoscopic integral images by smart pixel mapping,” Opt. Express 13, 9175-9180(2005).
    [Crossref] [PubMed]

2008 (2)

2007 (4)

2006 (4)

2005 (3)

2004 (2)

2003 (2)

Y. Frauel and B. Javidi, “Digital three-dimensional object reconstruction and correlation based on integral imaging,” Proc. SPIE 5006, 83-91 (2003).
[Crossref]

S. Kishk and B. Javidi, “Improved resolution 3D object sensing and recognition using time multiplexed computational integral imaging,” Opt. Express 11, 3528-3541 (2003).
[Crossref] [PubMed]

2002 (3)

2001 (1)

1997 (1)

1908 (1)

G. Lippmann, “La photographic intergrale,” C. R. Acad. Sci. 146, 446-451 (1908).

Arai, J.

Arimoto, H.

Bredif, M.

R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a handheld plenoptic camera,” Computer Science Technical Report CSTR 2005-02 (Stanford University, 2005).

Duval, G.

R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a handheld plenoptic camera,” Computer Science Technical Report CSTR 2005-02 (Stanford University, 2005).

Frauel, Y.

Y. Frauel and B. Javidi, “Digital three-dimensional object reconstruction and correlation based on integral imaging,” Proc. SPIE 5006, 83-91 (2003).
[Crossref]

Y. Frauel and B. Javidi, “Digital three-dimensional image correlation by use of computer-reconstructed integral imaging,” Appl. Opt. 41, 5488-5496 (2002).
[Crossref] [PubMed]

Hanrahan, P.

R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a handheld plenoptic camera,” Computer Science Technical Report CSTR 2005-02 (Stanford University, 2005).

Hong, S.-H.

Horowitz, M.

R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a handheld plenoptic camera,” Computer Science Technical Report CSTR 2005-02 (Stanford University, 2005).

Hoshino, H.

Hwang, D.-C.

D.-C. Hwang, K.-J. Lee, S.-C. Kim, and E.-S. Kim, “Extraction of location coordinates of 3-D objects from computationally reconstructed integral images basing on a blur metric,” Opt. Express 16, 3623-3635 (2008).
[Crossref] [PubMed]

J.-S. Park, D.-C. Hwang, D.-H. Shin, and E.-S. Kim, “Resolution-enhanced 3D image correlator using computationally reconstructed integral images,” Opt. Commun. 276, 72-79(2007).
[Crossref]

Jang, J.-S.

Javidi, B.

S. Yeom, B. Javidi, and E. Watson, “Three-dimensional distortion-tolerant object recognition using photon-counting integral imaging,” Opt. Express 15, 1513-1533(2007).
[Crossref] [PubMed]

S.-H. Hong and B. Javidi, “Distortion-tolerant 3D recognition of occluded objects using computational integral imaging,” Opt. Express 14, 12085-12095 (2006).
[Crossref] [PubMed]

B. Javidi, R. Ponce-Díaz, and S.-H. Hong, “Three-dimensional recognition of occluded objects by using computational integral imaging,” Opt. Lett. 31, 1106-1108 (2006).
[Crossref] [PubMed]

A. Stern and B. Javidi, “Three-dimensional image sensing, visualization, and processing using integral imaging,” Proc. IEEE 94, 591-607 (2006).
[Crossref]

Y.-W. Song, B. Javidi, and F. Jin, “3D object scaling in integral imaging display by varying the spatial ray sampling rate,” Opt. Express 13, 3242-3251 (2005).
[Crossref] [PubMed]

M. Martinez-Corral, B. Javidi, R. Martínez-Cuenca, and G. Saavedra, “Formation of real, orthoscopic integral images by smart pixel mapping,” Opt. Express 13, 9175-9180(2005).
[Crossref] [PubMed]

B. Javidi and S.-H. Hong, “Three-dimensional holographic image sensing and integral imaging display,” J. Display Technol. 1, 341-346 (2005).
[Crossref]

S.-H. Hong, J.-S. Jang, and B. Javidi, “Three-dimensional volumetric object reconstruction using computational integral imaging,” Opt. Express 12, 483-491 (2004).
[Crossref] [PubMed]

S.-H. Hong and B. Javidi, “Improved resolution 3D object reconstruction using computational integral imaging with time multiplexing,” Opt. Express 12, 4579-4588 (2004).
[Crossref] [PubMed]

Y. Frauel and B. Javidi, “Digital three-dimensional object reconstruction and correlation based on integral imaging,” Proc. SPIE 5006, 83-91 (2003).
[Crossref]

S. Kishk and B. Javidi, “Improved resolution 3D object sensing and recognition using time multiplexed computational integral imaging,” Opt. Express 11, 3528-3541 (2003).
[Crossref] [PubMed]

J.-S. Jang and B. Javidi, “Improved viewing resolution of three-dimensional integral imaging by use of nonstationary micro-optics,” Opt. Lett. 27, 324-326 (2002).
[Crossref]

Y. Frauel and B. Javidi, “Digital three-dimensional image correlation by use of computer-reconstructed integral imaging,” Appl. Opt. 41, 5488-5496 (2002).
[Crossref] [PubMed]

H. Arimoto and B. Javidi, “Integral three-dimensional imaging with digital reconstruction,” Opt. Lett. 26, 157-159(2001).
[Crossref]

Jin, F.

Jung, S.

Kim, E.-S.

Kim, S.-C.

Kishk, S.

Lee, B.

Lee, K.-J.

Levoy, M.

R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a handheld plenoptic camera,” Computer Science Technical Report CSTR 2005-02 (Stanford University, 2005).

Lippmann, G.

G. Lippmann, “La photographic intergrale,” C. R. Acad. Sci. 146, 446-451 (1908).

Martinez-Corral, M.

Martínez-Cuenca, R.

Ng, R.

R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a handheld plenoptic camera,” Computer Science Technical Report CSTR 2005-02 (Stanford University, 2005).

Okano, F.

Park, J.-H.

Park, J.-S.

J.-S. Park, D.-C. Hwang, D.-H. Shin, and E.-S. Kim, “Resolution-enhanced 3D image correlator using computationally reconstructed integral images,” Opt. Commun. 276, 72-79(2007).
[Crossref]

Ponce-Díaz, R.

Saavedra, G.

Shin, D.-H.

Song, Y.-W.

Stern, A.

A. Stern and B. Javidi, “Three-dimensional image sensing, visualization, and processing using integral imaging,” Proc. IEEE 94, 591-607 (2006).
[Crossref]

Watson, E.

Yeom, S.

Yoo, H.

Yuyama, I.

Appl. Opt. (3)

C. R. Acad. Sci. (1)

G. Lippmann, “La photographic intergrale,” C. R. Acad. Sci. 146, 446-451 (1908).

J. Display Technol. (1)

Opt. Commun. (1)

J.-S. Park, D.-C. Hwang, D.-H. Shin, and E.-S. Kim, “Resolution-enhanced 3D image correlator using computationally reconstructed integral images,” Opt. Commun. 276, 72-79(2007).
[Crossref]

Opt. Express (11)

S. Yeom, B. Javidi, and E. Watson, “Three-dimensional distortion-tolerant object recognition using photon-counting integral imaging,” Opt. Express 15, 1513-1533(2007).
[Crossref] [PubMed]

D.-C. Hwang, K.-J. Lee, S.-C. Kim, and E.-S. Kim, “Extraction of location coordinates of 3-D objects from computationally reconstructed integral images basing on a blur metric,” Opt. Express 16, 3623-3635 (2008).
[Crossref] [PubMed]

D.-H. Shin and H. Yoo, “Scale-variant magnification for computational integral imaging and its application to 3D object correlator,” Opt. Express 16, 8855-8867 (2008).
[Crossref] [PubMed]

M. Martinez-Corral, B. Javidi, R. Martínez-Cuenca, and G. Saavedra, “Formation of real, orthoscopic integral images by smart pixel mapping,” Opt. Express 13, 9175-9180(2005).
[Crossref] [PubMed]

S.-H. Hong and B. Javidi, “Distortion-tolerant 3D recognition of occluded objects using computational integral imaging,” Opt. Express 14, 12085-12095 (2006).
[Crossref] [PubMed]

H. Yoo and D.-H. Shin, “Improved analysis on the signal property of computational integral imaging system,” Opt. Express 15, 14107-14114 (2007).
[Crossref] [PubMed]

D.-H. Shin and H. Yoo, “Image quality enhancement in 3D computational integral imaging by use of interpolation methods,” Opt. Express 15, 12039-12049 (2007).
[Crossref] [PubMed]

S. Kishk and B. Javidi, “Improved resolution 3D object sensing and recognition using time multiplexed computational integral imaging,” Opt. Express 11, 3528-3541 (2003).
[Crossref] [PubMed]

S.-H. Hong, J.-S. Jang, and B. Javidi, “Three-dimensional volumetric object reconstruction using computational integral imaging,” Opt. Express 12, 483-491 (2004).
[Crossref] [PubMed]

S.-H. Hong and B. Javidi, “Improved resolution 3D object reconstruction using computational integral imaging with time multiplexing,” Opt. Express 12, 4579-4588 (2004).
[Crossref] [PubMed]

Y.-W. Song, B. Javidi, and F. Jin, “3D object scaling in integral imaging display by varying the spatial ray sampling rate,” Opt. Express 13, 3242-3251 (2005).
[Crossref] [PubMed]

Opt. Lett. (4)

Proc. IEEE (1)

A. Stern and B. Javidi, “Three-dimensional image sensing, visualization, and processing using integral imaging,” Proc. IEEE 94, 591-607 (2006).
[Crossref]

Proc. SPIE (1)

Y. Frauel and B. Javidi, “Digital three-dimensional object reconstruction and correlation based on integral imaging,” Proc. SPIE 5006, 83-91 (2003).
[Crossref]

Other (1)

R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a handheld plenoptic camera,” Computer Science Technical Report CSTR 2005-02 (Stanford University, 2005).

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (12)

Fig. 1
Fig. 1

General integral imaging system: (a) pickup process and (b) reconstruction process.

Fig. 2
Fig. 2

CII system: (a) pickup process, (b) CIIR process, and (c) superimposition example for four EIs.

Fig. 3
Fig. 3

Principle of CIIR by use of SPM.

Fig. 4
Fig. 4

Principle of SMP with two-step recording process: (a) recording of the first EIs and (b) recording of new EIs using SPM.

Fig. 5
Fig. 5

(a) Block diagram of the conventional CIIR system and (b) block diagram of the proposed CIIR system.

Fig. 6
Fig. 6

Experimental structure for calculating PSNR.

Fig. 7
Fig. 7

Reconstructed plane images according to distance z: (a) conventional method and (b) proposed method.

Fig. 8
Fig. 8

PSNR results for reconstructed plane images.

Fig. 9
Fig. 9

Experimental setup for partially occluded 3D objects.

Fig. 10
Fig. 10

Four kinds of experimental test images.

Fig. 11
Fig. 11

EIs and enlarged images of “tree” and “house” test images: (a) original images and (b) new images after SPM.

Fig. 12
Fig. 12

Reconstructed images and PSNR results: (a) conventional method and (b) proposed two-stage method.

Equations (9)

Equations on this page are rendered with MathJax. Learn more.

R p q z ( x , y ) = ( g z ) 2 E ( g x z + ( 1 + g z ) s x p , g y z + ( 1 + g z ) s x q ) for     { s x ( p z / 2 g x s x ( p + z / 2 g ) s y ( q z / 2 g y s y ( q + z / 2 g ) ,
R z ( x , y ) = p = 0 m 1 q = 0 n 1 R p q z ( x , y ) ,
R normal z ( x , y ) = R z ( x , y ) N z ( x , y ) .
T i j = E k l ,
l = ( V + 1 ) j , k = { i + U / 2 j if     U   is even i + ( U + 1 ) / 2 j if     U   is odd .
d = U × g ,
z = d z .
PSNR = 10 log 10 255 2 MSE ,
MSE = 1 P Q x = 1 P y = 1 Q [ O ( x , y ) R normal z ( x , y ) ] 2 .

Metrics