Abstract

We present a computational scheme for removing an occlusion in a partially occluded, far object in a computational integral imaging (CII) system. In order to obtain the high resolution elemental image array (EIA) with enhanced information of the occluded image for better applying block matching, a smart pixel mapping process and a subimage transform process are adopted. Based on depth maps produced between adjacent subimages, we acquire the expected EIA without occlusion information. Theoretical analysis of the proposed scheme is given. To show the effectiveness of the proposed scheme, we carry out some experiments. As demonstrated using test images, experimental results show that the proposed scheme outperforms Shin’s scheme and the CII scheme under the same situations.

© 2010 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. C. Wheatstone, “On some remarkable, and hitherto unobserved, phenomena of binocular vision,” Philos. Trans. R. Soc. London 128, 371–394 (1838).
    [CrossRef]
  2. A. Stern and B. Javidi, “Three dimensional image sensing, visualization, and processing using integral image,” Proc. IEEE 94, 591–607 (2006).
    [CrossRef]
  3. B. Lee, S.-Y. Jung, S.-W. Min, and J.-H. Park, “Three-dimensional display by use of integral photography with dynamically variable image planes,” Opt. Lett. 26, 1481–1482 (2001).
    [CrossRef]
  4. J.-S. Jang and B. Javidi, “Improved viewing resolution of three-dimensional integral imaging by use of nonstationary micro-optics,” Opt. Lett. 27, 324–326 (2002).
    [CrossRef]
  5. D.-H. Shin, B. Lee, and E.-S. Kim, “Multidirectional curved integral imaging with large depth by additional use of a large-aperture lens,” Appl. Opt. 45, 7375–7381 (2006).
    [CrossRef]
  6. D.-H. Shin, B. Lee, and E.-S. Kim, “Optical display of true 3D objects in depth-priority integral imaging using an active sensor,” Opt. Commun. 275, 330–334 (2007).
    [CrossRef]
  7. G. Lippmann, “La photographie integrale,” C. R. Acad. Sci. 146, 446–451 (1908).
  8. H. Arimoto and B. Javidi, “Integral three-dimensional imaging with digital reconstruction,” Opt. Lett. 26, 157–159(2001).
    [CrossRef]
  9. Y. Frauel and B. Javidi, “Digital three-dimensional image correlation by use of computer-reconstructed,” Appl. Opt. 41, 5488–5496 (2002).
    [CrossRef]
  10. S.-H. Hong, J.-S. Jang, and B. Javidi, “Three-dimensional volumetric object reconstruction using computational integral image,” Opt. Express 12, 483–491 (2004).
    [CrossRef]
  11. D.-H. Shin, E.-S. Kim, and B. Lee, “Computational reconstruction technique of three-dimensional object in integral imaging using a lenslet array,” Jpn. J. Appl. Phys. 44, 8016–8018(2005).
    [CrossRef]
  12. S.-H. Hong and B. Javidi, “Improved resolution 3D object reconstruction using computational integral imaging with time multiplexing,” Opt. Express 12, 4579–4588 (2004).
    [CrossRef]
  13. S.-H. Hong and B. Javidi, “Distortion-tolerant 3D recognition of occluded objects using computational integral imaging,” Opt. Express 14, 12085–12095 (2006).
    [CrossRef]
  14. D.-H. Shin and H. Yoo, “Image quality enhancement in 3D computational integral imaging by use of interpolation methods,” Opt. Express 15, 12039–12046 (2007).
    [CrossRef]
  15. H. Yoo and D.-H. Shin, “Improved analysis on the signal property of computational integral imaging system,” Opt. Express 15, 14107–14114 (2007).
    [CrossRef]
  16. B. Javidi, R. Ponce-Diaz, and S.-H. Hong, “Three-dimensional recognition of occluded objects by using computational integral imaging,” Opt. Lett. 31, 1106–1108 (2006).
    [CrossRef]
  17. J.-S. Park, D.-C. Hwang, D.-H. Shin, and E.-S. Kim, “Resolution-enhanced 3D image correlator using computationally reconstructed integral images,” Opt. Commun. 276, 72–79(2007).
    [CrossRef]
  18. D.-H. Shin, B.-G. Lee, and J.-J. Lee, “Occlusion removal method of partially occluded 3D object using sub-image block matching in computational integral imaging,” Opt. Express 16, 16294–16304 (2008).
    [CrossRef]
  19. M. Martinez-Corral, B. Javidi, R. Martinez-Cuenca, and G. Saavedra, “Formation of real, orthoscopic integral images by smart pixel mapping,” Opt. Express 13, 9175–9180(2005).
    [CrossRef]
  20. H. E. Ives, “Optical properties of a Lippmann lenticulated sheet,” J. Opt. Soc. Am. 21, 171–176 (1931).
    [CrossRef]
  21. T. Okoshi, “Three-dimensional displays,” Proc. IEEE 68, 548–564 (1980).
    [CrossRef]
  22. N. Davies, M. McCormick, and L. Yang, “Three-dimensional imaging systems: a new development,” Appl. Opt. 27, 4520–4528 (1988).
    [CrossRef]
  23. J.-H. Park, S.-W. Min, S.-Y. Jung, and B. Lee, “Analysis of viewing parameters for two display methods based on integral photography,” Appl. Opt. 40, 5217–5232 (2001).
    [CrossRef]
  24. J.-S. Jang and B. Javidi, “Two-step integral imaging for orthoscopic three-dimensional imaging with improved viewing resolution,” Opt. Eng. 41, 2568–2571 (2002).
    [CrossRef]
  25. J.-H. Park, J. Kim, and B. Lee, “Three-dimension optical correlator using a sub-image array,” Opt. Express 13, 5116–5126 (2005).
    [CrossRef]
  26. C. Wu, M. McCormick, A. Aggoun, and S. Y. Kung, “Depth map from unidirectional integral images using a disparity algorithm based on neighbourhood constraint and relaxation,” in Proceedings of the International Conference on Visual Information Engineering (IEEE, 2003), pp. 65–68.

2008 (1)

2007 (4)

D.-H. Shin and H. Yoo, “Image quality enhancement in 3D computational integral imaging by use of interpolation methods,” Opt. Express 15, 12039–12046 (2007).
[CrossRef]

H. Yoo and D.-H. Shin, “Improved analysis on the signal property of computational integral imaging system,” Opt. Express 15, 14107–14114 (2007).
[CrossRef]

D.-H. Shin, B. Lee, and E.-S. Kim, “Optical display of true 3D objects in depth-priority integral imaging using an active sensor,” Opt. Commun. 275, 330–334 (2007).
[CrossRef]

J.-S. Park, D.-C. Hwang, D.-H. Shin, and E.-S. Kim, “Resolution-enhanced 3D image correlator using computationally reconstructed integral images,” Opt. Commun. 276, 72–79(2007).
[CrossRef]

2006 (4)

2005 (3)

2004 (2)

2003 (1)

C. Wu, M. McCormick, A. Aggoun, and S. Y. Kung, “Depth map from unidirectional integral images using a disparity algorithm based on neighbourhood constraint and relaxation,” in Proceedings of the International Conference on Visual Information Engineering (IEEE, 2003), pp. 65–68.

2002 (3)

2001 (3)

1988 (1)

1980 (1)

T. Okoshi, “Three-dimensional displays,” Proc. IEEE 68, 548–564 (1980).
[CrossRef]

1931 (1)

1908 (1)

G. Lippmann, “La photographie integrale,” C. R. Acad. Sci. 146, 446–451 (1908).

1838 (1)

C. Wheatstone, “On some remarkable, and hitherto unobserved, phenomena of binocular vision,” Philos. Trans. R. Soc. London 128, 371–394 (1838).
[CrossRef]

Aggoun, A.

C. Wu, M. McCormick, A. Aggoun, and S. Y. Kung, “Depth map from unidirectional integral images using a disparity algorithm based on neighbourhood constraint and relaxation,” in Proceedings of the International Conference on Visual Information Engineering (IEEE, 2003), pp. 65–68.

Arimoto, H.

Davies, N.

Frauel, Y.

Hong, S.-H.

Hwang, D.-C.

J.-S. Park, D.-C. Hwang, D.-H. Shin, and E.-S. Kim, “Resolution-enhanced 3D image correlator using computationally reconstructed integral images,” Opt. Commun. 276, 72–79(2007).
[CrossRef]

Ives, H. E.

Jang, J.-S.

Javidi, B.

S.-H. Hong and B. Javidi, “Distortion-tolerant 3D recognition of occluded objects using computational integral imaging,” Opt. Express 14, 12085–12095 (2006).
[CrossRef]

B. Javidi, R. Ponce-Diaz, and S.-H. Hong, “Three-dimensional recognition of occluded objects by using computational integral imaging,” Opt. Lett. 31, 1106–1108 (2006).
[CrossRef]

A. Stern and B. Javidi, “Three dimensional image sensing, visualization, and processing using integral image,” Proc. IEEE 94, 591–607 (2006).
[CrossRef]

M. Martinez-Corral, B. Javidi, R. Martinez-Cuenca, and G. Saavedra, “Formation of real, orthoscopic integral images by smart pixel mapping,” Opt. Express 13, 9175–9180(2005).
[CrossRef]

S.-H. Hong and B. Javidi, “Improved resolution 3D object reconstruction using computational integral imaging with time multiplexing,” Opt. Express 12, 4579–4588 (2004).
[CrossRef]

S.-H. Hong, J.-S. Jang, and B. Javidi, “Three-dimensional volumetric object reconstruction using computational integral image,” Opt. Express 12, 483–491 (2004).
[CrossRef]

J.-S. Jang and B. Javidi, “Improved viewing resolution of three-dimensional integral imaging by use of nonstationary micro-optics,” Opt. Lett. 27, 324–326 (2002).
[CrossRef]

J.-S. Jang and B. Javidi, “Two-step integral imaging for orthoscopic three-dimensional imaging with improved viewing resolution,” Opt. Eng. 41, 2568–2571 (2002).
[CrossRef]

Y. Frauel and B. Javidi, “Digital three-dimensional image correlation by use of computer-reconstructed,” Appl. Opt. 41, 5488–5496 (2002).
[CrossRef]

H. Arimoto and B. Javidi, “Integral three-dimensional imaging with digital reconstruction,” Opt. Lett. 26, 157–159(2001).
[CrossRef]

Jung, S.-Y.

Kim, E.-S.

J.-S. Park, D.-C. Hwang, D.-H. Shin, and E.-S. Kim, “Resolution-enhanced 3D image correlator using computationally reconstructed integral images,” Opt. Commun. 276, 72–79(2007).
[CrossRef]

D.-H. Shin, B. Lee, and E.-S. Kim, “Optical display of true 3D objects in depth-priority integral imaging using an active sensor,” Opt. Commun. 275, 330–334 (2007).
[CrossRef]

D.-H. Shin, B. Lee, and E.-S. Kim, “Multidirectional curved integral imaging with large depth by additional use of a large-aperture lens,” Appl. Opt. 45, 7375–7381 (2006).
[CrossRef]

D.-H. Shin, E.-S. Kim, and B. Lee, “Computational reconstruction technique of three-dimensional object in integral imaging using a lenslet array,” Jpn. J. Appl. Phys. 44, 8016–8018(2005).
[CrossRef]

Kim, J.

Kung, S. Y.

C. Wu, M. McCormick, A. Aggoun, and S. Y. Kung, “Depth map from unidirectional integral images using a disparity algorithm based on neighbourhood constraint and relaxation,” in Proceedings of the International Conference on Visual Information Engineering (IEEE, 2003), pp. 65–68.

Lee, B.

Lee, B.-G.

Lee, J.-J.

Lippmann, G.

G. Lippmann, “La photographie integrale,” C. R. Acad. Sci. 146, 446–451 (1908).

Martinez-Corral, M.

Martinez-Cuenca, R.

McCormick, M.

C. Wu, M. McCormick, A. Aggoun, and S. Y. Kung, “Depth map from unidirectional integral images using a disparity algorithm based on neighbourhood constraint and relaxation,” in Proceedings of the International Conference on Visual Information Engineering (IEEE, 2003), pp. 65–68.

N. Davies, M. McCormick, and L. Yang, “Three-dimensional imaging systems: a new development,” Appl. Opt. 27, 4520–4528 (1988).
[CrossRef]

Min, S.-W.

Okoshi, T.

T. Okoshi, “Three-dimensional displays,” Proc. IEEE 68, 548–564 (1980).
[CrossRef]

Park, J.-H.

Park, J.-S.

J.-S. Park, D.-C. Hwang, D.-H. Shin, and E.-S. Kim, “Resolution-enhanced 3D image correlator using computationally reconstructed integral images,” Opt. Commun. 276, 72–79(2007).
[CrossRef]

Ponce-Diaz, R.

Saavedra, G.

Shin, D.-H.

D.-H. Shin, B.-G. Lee, and J.-J. Lee, “Occlusion removal method of partially occluded 3D object using sub-image block matching in computational integral imaging,” Opt. Express 16, 16294–16304 (2008).
[CrossRef]

J.-S. Park, D.-C. Hwang, D.-H. Shin, and E.-S. Kim, “Resolution-enhanced 3D image correlator using computationally reconstructed integral images,” Opt. Commun. 276, 72–79(2007).
[CrossRef]

D.-H. Shin and H. Yoo, “Image quality enhancement in 3D computational integral imaging by use of interpolation methods,” Opt. Express 15, 12039–12046 (2007).
[CrossRef]

D.-H. Shin, B. Lee, and E.-S. Kim, “Optical display of true 3D objects in depth-priority integral imaging using an active sensor,” Opt. Commun. 275, 330–334 (2007).
[CrossRef]

H. Yoo and D.-H. Shin, “Improved analysis on the signal property of computational integral imaging system,” Opt. Express 15, 14107–14114 (2007).
[CrossRef]

D.-H. Shin, B. Lee, and E.-S. Kim, “Multidirectional curved integral imaging with large depth by additional use of a large-aperture lens,” Appl. Opt. 45, 7375–7381 (2006).
[CrossRef]

D.-H. Shin, E.-S. Kim, and B. Lee, “Computational reconstruction technique of three-dimensional object in integral imaging using a lenslet array,” Jpn. J. Appl. Phys. 44, 8016–8018(2005).
[CrossRef]

Stern, A.

A. Stern and B. Javidi, “Three dimensional image sensing, visualization, and processing using integral image,” Proc. IEEE 94, 591–607 (2006).
[CrossRef]

Wheatstone, C.

C. Wheatstone, “On some remarkable, and hitherto unobserved, phenomena of binocular vision,” Philos. Trans. R. Soc. London 128, 371–394 (1838).
[CrossRef]

Wu, C.

C. Wu, M. McCormick, A. Aggoun, and S. Y. Kung, “Depth map from unidirectional integral images using a disparity algorithm based on neighbourhood constraint and relaxation,” in Proceedings of the International Conference on Visual Information Engineering (IEEE, 2003), pp. 65–68.

Yang, L.

Yoo, H.

Appl. Opt. (4)

C. R. Acad. Sci. (1)

G. Lippmann, “La photographie integrale,” C. R. Acad. Sci. 146, 446–451 (1908).

J. Opt. Soc. Am. (1)

Jpn. J. Appl. Phys. (1)

D.-H. Shin, E.-S. Kim, and B. Lee, “Computational reconstruction technique of three-dimensional object in integral imaging using a lenslet array,” Jpn. J. Appl. Phys. 44, 8016–8018(2005).
[CrossRef]

Opt. Commun. (2)

J.-S. Park, D.-C. Hwang, D.-H. Shin, and E.-S. Kim, “Resolution-enhanced 3D image correlator using computationally reconstructed integral images,” Opt. Commun. 276, 72–79(2007).
[CrossRef]

D.-H. Shin, B. Lee, and E.-S. Kim, “Optical display of true 3D objects in depth-priority integral imaging using an active sensor,” Opt. Commun. 275, 330–334 (2007).
[CrossRef]

Opt. Eng. (1)

J.-S. Jang and B. Javidi, “Two-step integral imaging for orthoscopic three-dimensional imaging with improved viewing resolution,” Opt. Eng. 41, 2568–2571 (2002).
[CrossRef]

Opt. Express (8)

Opt. Lett. (4)

Philos. Trans. R. Soc. London (1)

C. Wheatstone, “On some remarkable, and hitherto unobserved, phenomena of binocular vision,” Philos. Trans. R. Soc. London 128, 371–394 (1838).
[CrossRef]

Proc. IEEE (2)

A. Stern and B. Javidi, “Three dimensional image sensing, visualization, and processing using integral image,” Proc. IEEE 94, 591–607 (2006).
[CrossRef]

T. Okoshi, “Three-dimensional displays,” Proc. IEEE 68, 548–564 (1980).
[CrossRef]

Other (1)

C. Wu, M. McCormick, A. Aggoun, and S. Y. Kung, “Depth map from unidirectional integral images using a disparity algorithm based on neighbourhood constraint and relaxation,” in Proceedings of the International Conference on Visual Information Engineering (IEEE, 2003), pp. 65–68.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (11)

Fig. 1
Fig. 1

Concept of VCR.

Fig. 2
Fig. 2

Analysis for SIT.

Fig. 3
Fig. 3

System using the proposed scheme.

Fig. 4
Fig. 4

Experimental setup.

Fig. 5
Fig. 5

Process of SPM.

Fig. 6
Fig. 6

Geometric diagram of the SIT process based on multiple-pixel extraction.

Fig. 7
Fig. 7

(a) Occluding plane object, (b) occluded plane object, and (c) occluded scene.

Fig. 8
Fig. 8

Process of removing occlusion.

Fig. 9
Fig. 9

(a) Modified EIA and (b) reconstructed image.

Fig. 10
Fig. 10

Reconstructed image by (a) the proposed scheme, (b) the Shin et al. scheme, and (c) the CII scheme.

Fig. 11
Fig. 11

PSNR of the unobstructed images reconstructed by the proposed scheme, the Shin scheme, and the CII scheme, respectively: (a) distance between the occluded image and the occluding image is 15 mm and (b) distance between the occluded image and the occluding image is 30 mm .

Tables (1)

Tables Icon

Table 1 List of Abbreviations

Equations (16)

Equations on this page are rendered with MathJax. Learn more.

m max = g z s = s M ,
R i , j = E k , l ,
l = ( V + 1 ) j , k = { i + U / 2 j ( U : even ) i + ( U + 1 ) / 2 j ( U : odd ) .
d = U × g ,
z = d z .
S j , n = R n , j ,
S j = R ( t s + q m + r ) ,
m max = g z s = s M ,
SAD ( x , y ) ( u , v ) = i = 1 B j = 1 B | I 1 ( x + i , y + i ) I 2 ( x + u + i , y + v + j ) | ,
( u ^ , v ^ ) = argmin SAD ( x , y ) ( u , v ) .
S _ block ( x , y ) ( u ^ , v ^ ) = j = 1 B i = 1 B S _ pixel ( x , y , u ^ , v ^ ) ( i , j ) ,
S _ pixel ( x , y , u ^ , v ^ ) ( i , j ) = { 1 , | I L ( x + i , y + j ) I R ( x + u + i , y + v + j ) | δ I 0 , otherwise } ,
P = argmin S _ block ( x , y ) ( u ^ , v ^ ) .
D = ( P ± 1 ) ψ f Δ ,
PSNR = 10 log 10 255 2 MSE ( O , O ) ,
MSE = 1 X Y x = 1 W y = 1 Y [ O ( x , y ) O ( x , y ) ] 2 ,

Metrics