Reconstruction of three-dimensional occluded object using optical flow and triangular mesh reconstruction in integral imaging

Jae-Hyun Jung; Keehoon Hong; Gilbae Park; Indeok Chung; Jae-Hyeung Park; Byoungho Lee

doi:10.1364/OE.18.026373

Reconstruction of three-dimensional occluded object using optical flow and triangular mesh reconstruction in integral imaging

Jae-Hyun Jung, Keehoon Hong, Gilbae Park, Indeok Chung, Jae-Hyeung Park, and Byoungho Lee

Optics Express
Vol. 18,
Issue 25,
pp. 26373-26387
(2010)
•https://doi.org/10.1364/OE.18.026373

Get Citation
Copy Citation Text
Jae-Hyun Jung, Keehoon Hong, Gilbae Park, Indeok Chung, Jae-Hyeung Park, and Byoungho Lee, "Reconstruction of three-dimensional occluded object using optical flow and triangular mesh reconstruction in integral imaging," Opt. Express 18, 26373-26387 (2010)

Export Citation
- BibTex
- Endnote (RIS)
- HTML
- Plain Text
Get PDF (1910 KB)
Set citation alerts
Save article

Related Topics
Table of Contents Category
- Imaging Systems
Optics & Photonics Topics
?

The topics in this list come from the OSA Optics and Photonics Topics applied to this article.
Previously assigned OCIS codes
- Three-dimensional image processing (100.6890)
- Image formation theory (110.2990)

About this Article
History
- Original Manuscript: October 12, 2010
- Revised Manuscript: November 24, 2010
- Manuscript Accepted: November 27, 2010
- Published: December 1, 2010

Accessible

Open Access

Abstract

We proposed a reconstruction method for the occluded region of three-dimensional (3D) object using the depth extraction based on the optical flow and triangular mesh reconstruction in integral imaging. The depth information of sub-images from the acquired elemental image set is extracted using the optical flow with sub-pixel accuracy, which alleviates the depth quantization problem. The extracted depth maps of sub-image array are segmented by the depth threshold from the histogram based segmentation, which is represented as the point clouds. The point clouds are projected to the viewpoint of center sub-image and reconstructed by the triangular mesh reconstruction. The experimental results support the validity of the proposed method with high accuracy of peak signal-to-noise ratio and normalized cross-correlation in 3D image recognition.

Full Article | PDF Article

OSA Recommended Articles

Occlusion removal method of partially occluded 3D object using sub-image block matching in computational integral imaging

Dong-Hak Shin, Byung-Gook Lee, and Joon-Jae Lee
Opt. Express 16(21) 16294-16304 (2008)

Depth extraction of three-dimensional objects in space by the computational integral imaging reconstruction technique

Dong-Choon Hwang, Dong-Hak Shin, Seung-Cheol Kim, and Eun-Soo Kim
Appl. Opt. 47(19) D128-D135 (2008)

Visibility-enhanced reconstruction of three-dimensional objects under a heavily scattering medium through combined use of intermediate view reconstruction, multipixel extraction, and histogram equalization methods in the conventional integral imaging system

Miao Zhang, Yongri Piao, and Eun-Soo Kim
Appl. Opt. 50(28) 5369-5381 (2011)

References

View by:
Article Order
|
Year
|
Author
|
Publication

A. Kubota, A. Smolic, M. Magnor, M. Tanimoto, T. Chen, and C. Zhang, “Multiview imaging and 3DTV,” IEEE Signal Process. Mag. 24(6), 10–21 (2007).
P. Benzie, J. Watson, P. Surman, I. Rakkolainen, K. Hopf, H. Urey, V. Sainov, and C. von Kopylow, “A survey of 3DTV displays: techniques and technologies,” IEEE Trans. Circ. Syst. Video Tech. 17(11), 1647–1658 (2007).
[Crossref]
M. Okutomi and T. Kanade, “A multiple-baseline stereo,” IEEE Trans. Pattern Anal. Mach. Intell. 15(4), 353–363 (1993).
[Crossref]
B. Lee, J.-H. Park, and S.-W. Min, Digital Holography and Three-Dimensional Display, T.-C. Poon, ed. (Springer US, 2006), Chap. 12.
G. Lippmann, “La photographie integrále,” C. R. Acad. Sci. Ser. IIc Chim. 146, 446–451 (1908).
F. Okano, H. Hoshino, J. Arai, and I. Yuyama, “Real-time pickup method for a three-dimensional image based on integral photography,” Appl. Opt. 36(7), 1598–1603 (1997).
[Crossref] [PubMed]
J.-H. Park, S. Jung, H. Choi, Y. Kim, and B. Lee, “Depth extraction by use of a rectangular lens array and one-dimensional elemental image modification,” Appl. Opt. 43(25), 4882–4895 (2004).
[Crossref] [PubMed]
M. Martínez-Corral, B. Javidi, R. Martínez-Cuenca, and G. Saavedra, “Formation of real, orthoscopic integral images by smart pixel mapping,” Opt. Express 13(23), 9175–9180 (2005).
[Crossref] [PubMed]
G. Passalis, N. Sgouros, S. Athineos, and T. Theoharis, “Enhanced reconstruction of three-dimensional shape and texture from integral photography images,” Appl. Opt. 46(22), 5311–5320 (2007).
[Crossref] [PubMed]
D.-H. Shin and E.-S. Kim, “Computational integral imaging reconstruction of 3D object using a depth conversion technique,” J. Opt. Soc. Korea 12(3), 131–135 (2008).
[Crossref]
J.-H. Park, G. Baasantseren, N. Kim, G. Park, J.-M. Kang, and B. Lee, “View image generation in perspective and orthographic projection geometry based on integral imaging,” Opt. Express 16(12), 8800–8813 (2008).
[Crossref] [PubMed]
M. Zhang, Y. Piao, and E.-S. Kim, “Occlusion-removed scheme using depth-reversed method in computational integral imaging,” Appl. Opt. 49(14), 2571–2580 (2010).
[Crossref]
S.-H. Hong and B. Javidi, “Distortion-tolerant 3D recognition of occluded objects using computational integral imaging,” Opt. Express 14(25), 12085–12095 (2006).
[Crossref] [PubMed]
D.-H. Shin, B.-G. Lee, and J.-J. Lee, “Occlusion removal method of partially occluded 3D object using sub-image block matching in computational integral imaging,” Opt. Express 16(21), 16294–16304 (2008).
[Crossref] [PubMed]
K.-J. Lee, D.-C. Hwang, S.-C. Kim, and E.-S. Kim, “Blur-metric-based resolution enhancement of computationally reconstructed integral images,” Appl. Opt. 47(15), 2859–2869 (2008).
[Crossref] [PubMed]
C. M. Do and B. Javidi, “3D integral imaging reconstruction of occluded objects using independent component analysis-based K-means clustering,” J. Disp. Technol. 6(7), 257–262 (2010).
[Crossref]
J.-H. Park, K. Hong, and B. Lee, “Recent progress in three-dimensional information processing based on integral imaging,” Appl. Opt. 48(34), H77–H94 (2009).
[Crossref] [PubMed]
M. Levoy, and P. Hanrahan, “Light field rendering,” in Proceedings of SIGGRAPH ‘96 (Association for Computing Machinery, New Orleans, 1996), pp. 31–42.
B. Lee, S. Jung, S.-W. Min, and J.-H. Park, “Three-dimensional display by use of integral photography with dynamically variable image planes,” Opt. Lett. 26(19), 1481–1482 (2001).
[Crossref]
K. Hong, J. Hong, J.-H. Jung, J.-H. Park, and B. Lee, “Rectification of elemental image set and extraction of lens lattice by projective image transformation in integral imaging,” Opt. Express 18(11), 12002–12016 (2010).
[Crossref] [PubMed]
B. Horn and B. Schunck, “Determining optical flow,” Artif. Intell. 17(1-3), 185–203 (1981).
[Crossref]
A. S. Ogale, C. Fermüller, and Y. Aloimonos, “Motion segmentation using occlusions,” IEEE Trans. Pattern Anal. Mach. Intell. 27(6), 988–992 (2005).
[Crossref] [PubMed]
D. Sun, S. Roth, and M. J. Black, “Secrets of optical flow estimation and their principles,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2010), pp. 2432–2439.
S. Baker, D. Scharstein, J. P. Lewis, S. Roth, M. J. Black, and R. Szeliski, “A database and evaluation methodology for optical flow,” in Proceedings of IEEE Conference on International Conference on Computer Vision (IEEE, 2007), pp. 1–8.
H. Kim, J. Hahn, and B. Lee, “Mathematical modeling of triangle-mesh-modeled three-dimensional surface objects for digital holography,” Appl. Opt. 47(19), D117–D127 (2008).
[Crossref] [PubMed]
J. Hahn, Y. Kim, E.-H. Kim, and B. Lee, “Undistorted pickup method of both virtual and real objects for integral imaging,” Opt. Express 16(18), 13969–13978 (2008).
[Crossref] [PubMed]
R. Martinez-Cuenca, A. Pons, G. Saavedra, M. Martinez-Corral, and B. Javidi, “Optically-corrected elemental images for undistorted Integral image display,” Opt. Express 14(21), 9657–9663 (2006).
[Crossref] [PubMed]

2010 (3)

C. M. Do and B. Javidi, “3D integral imaging reconstruction of occluded objects using independent component analysis-based K-means clustering,” J. Disp. Technol. 6(7), 257–262 (2010).
[Crossref]

M. Zhang, Y. Piao, and E.-S. Kim, “Occlusion-removed scheme using depth-reversed method in computational integral imaging,” Appl. Opt. 49(14), 2571–2580 (2010).
[Crossref]

K. Hong, J. Hong, J.-H. Jung, J.-H. Park, and B. Lee, “Rectification of elemental image set and extraction of lens lattice by projective image transformation in integral imaging,” Opt. Express 18(11), 12002–12016 (2010).
[Crossref] [PubMed]

2009 (1)

J.-H. Park, K. Hong, and B. Lee, “Recent progress in three-dimensional information processing based on integral imaging,” Appl. Opt. 48(34), H77–H94 (2009).
[Crossref] [PubMed]

2008 (6)

D.-H. Shin and E.-S. Kim, “Computational integral imaging reconstruction of 3D object using a depth conversion technique,” J. Opt. Soc. Korea 12(3), 131–135 (2008).
[Crossref]

H. Kim, J. Hahn, and B. Lee, “Mathematical modeling of triangle-mesh-modeled three-dimensional surface objects for digital holography,” Appl. Opt. 47(19), D117–D127 (2008).
[Crossref] [PubMed]

K.-J. Lee, D.-C. Hwang, S.-C. Kim, and E.-S. Kim, “Blur-metric-based resolution enhancement of computationally reconstructed integral images,” Appl. Opt. 47(15), 2859–2869 (2008).
[Crossref] [PubMed]

J.-H. Park, G. Baasantseren, N. Kim, G. Park, J.-M. Kang, and B. Lee, “View image generation in perspective and orthographic projection geometry based on integral imaging,” Opt. Express 16(12), 8800–8813 (2008).
[Crossref] [PubMed]

J. Hahn, Y. Kim, E.-H. Kim, and B. Lee, “Undistorted pickup method of both virtual and real objects for integral imaging,” Opt. Express 16(18), 13969–13978 (2008).
[Crossref] [PubMed]

D.-H. Shin, B.-G. Lee, and J.-J. Lee, “Occlusion removal method of partially occluded 3D object using sub-image block matching in computational integral imaging,” Opt. Express 16(21), 16294–16304 (2008).
[Crossref] [PubMed]

2007 (3)

A. Kubota, A. Smolic, M. Magnor, M. Tanimoto, T. Chen, and C. Zhang, “Multiview imaging and 3DTV,” IEEE Signal Process. Mag. 24(6), 10–21 (2007).

P. Benzie, J. Watson, P. Surman, I. Rakkolainen, K. Hopf, H. Urey, V. Sainov, and C. von Kopylow, “A survey of 3DTV displays: techniques and technologies,” IEEE Trans. Circ. Syst. Video Tech. 17(11), 1647–1658 (2007).
[Crossref]

G. Passalis, N. Sgouros, S. Athineos, and T. Theoharis, “Enhanced reconstruction of three-dimensional shape and texture from integral photography images,” Appl. Opt. 46(22), 5311–5320 (2007).
[Crossref] [PubMed]

2006 (2)

R. Martinez-Cuenca, A. Pons, G. Saavedra, M. Martinez-Corral, and B. Javidi, “Optically-corrected elemental images for undistorted Integral image display,” Opt. Express 14(21), 9657–9663 (2006).
[Crossref] [PubMed]

S.-H. Hong and B. Javidi, “Distortion-tolerant 3D recognition of occluded objects using computational integral imaging,” Opt. Express 14(25), 12085–12095 (2006).
[Crossref] [PubMed]

2005 (2)

A. S. Ogale, C. Fermüller, and Y. Aloimonos, “Motion segmentation using occlusions,” IEEE Trans. Pattern Anal. Mach. Intell. 27(6), 988–992 (2005).
[Crossref] [PubMed]

M. Martínez-Corral, B. Javidi, R. Martínez-Cuenca, and G. Saavedra, “Formation of real, orthoscopic integral images by smart pixel mapping,” Opt. Express 13(23), 9175–9180 (2005).
[Crossref] [PubMed]

1993 (1)

M. Okutomi and T. Kanade, “A multiple-baseline stereo,” IEEE Trans. Pattern Anal. Mach. Intell. 15(4), 353–363 (1993).
[Crossref]

1981 (1)

B. Horn and B. Schunck, “Determining optical flow,” Artif. Intell. 17(1-3), 185–203 (1981).
[Crossref]

1908 (1)

G. Lippmann, “La photographie integrále,” C. R. Acad. Sci. Ser. IIc Chim. 146, 446–451 (1908).

Aloimonos, Y.

A. S. Ogale, C. Fermüller, and Y. Aloimonos, “Motion segmentation using occlusions,” IEEE Trans. Pattern Anal. Mach. Intell. 27(6), 988–992 (2005).
[Crossref] [PubMed]

Arai, J.

F. Okano, H. Hoshino, J. Arai, and I. Yuyama, “Real-time pickup method for a three-dimensional image based on integral photography,” Appl. Opt. 36(7), 1598–1603 (1997).
[Crossref] [PubMed]

Athineos, S.

Baasantseren, G.

Benzie, P.

Chen, T.

A. Kubota, A. Smolic, M. Magnor, M. Tanimoto, T. Chen, and C. Zhang, “Multiview imaging and 3DTV,” IEEE Signal Process. Mag. 24(6), 10–21 (2007).

Choi, H.

Do, C. M.

Fermüller, C.

A. S. Ogale, C. Fermüller, and Y. Aloimonos, “Motion segmentation using occlusions,” IEEE Trans. Pattern Anal. Mach. Intell. 27(6), 988–992 (2005).
[Crossref] [PubMed]

Hahn, J.

J. Hahn, Y. Kim, E.-H. Kim, and B. Lee, “Undistorted pickup method of both virtual and real objects for integral imaging,” Opt. Express 16(18), 13969–13978 (2008).
[Crossref] [PubMed]

H. Kim, J. Hahn, and B. Lee, “Mathematical modeling of triangle-mesh-modeled three-dimensional surface objects for digital holography,” Appl. Opt. 47(19), D117–D127 (2008).
[Crossref] [PubMed]

Hong, J.

Hong, K.

J.-H. Park, K. Hong, and B. Lee, “Recent progress in three-dimensional information processing based on integral imaging,” Appl. Opt. 48(34), H77–H94 (2009).
[Crossref] [PubMed]

Hong, S.-H.

S.-H. Hong and B. Javidi, “Distortion-tolerant 3D recognition of occluded objects using computational integral imaging,” Opt. Express 14(25), 12085–12095 (2006).
[Crossref] [PubMed]

Hopf, K.

Horn, B.

B. Horn and B. Schunck, “Determining optical flow,” Artif. Intell. 17(1-3), 185–203 (1981).
[Crossref]

Hoshino, H.

F. Okano, H. Hoshino, J. Arai, and I. Yuyama, “Real-time pickup method for a three-dimensional image based on integral photography,” Appl. Opt. 36(7), 1598–1603 (1997).
[Crossref] [PubMed]

Hwang, D.-C.

Javidi, B.

S.-H. Hong and B. Javidi, “Distortion-tolerant 3D recognition of occluded objects using computational integral imaging,” Opt. Express 14(25), 12085–12095 (2006).
[Crossref] [PubMed]

Jung, J.-H.

Jung, S.

B. Lee, S. Jung, S.-W. Min, and J.-H. Park, “Three-dimensional display by use of integral photography with dynamically variable image planes,” Opt. Lett. 26(19), 1481–1482 (2001).
[Crossref]

Kanade, T.

M. Okutomi and T. Kanade, “A multiple-baseline stereo,” IEEE Trans. Pattern Anal. Mach. Intell. 15(4), 353–363 (1993).
[Crossref]

Kang, J.-M.

Kim, E.-H.

J. Hahn, Y. Kim, E.-H. Kim, and B. Lee, “Undistorted pickup method of both virtual and real objects for integral imaging,” Opt. Express 16(18), 13969–13978 (2008).
[Crossref] [PubMed]

Kim, E.-S.

M. Zhang, Y. Piao, and E.-S. Kim, “Occlusion-removed scheme using depth-reversed method in computational integral imaging,” Appl. Opt. 49(14), 2571–2580 (2010).
[Crossref]

D.-H. Shin and E.-S. Kim, “Computational integral imaging reconstruction of 3D object using a depth conversion technique,” J. Opt. Soc. Korea 12(3), 131–135 (2008).
[Crossref]

Kim, H.

H. Kim, J. Hahn, and B. Lee, “Mathematical modeling of triangle-mesh-modeled three-dimensional surface objects for digital holography,” Appl. Opt. 47(19), D117–D127 (2008).
[Crossref] [PubMed]

Kim, N.

Kim, S.-C.

Kim, Y.

J. Hahn, Y. Kim, E.-H. Kim, and B. Lee, “Undistorted pickup method of both virtual and real objects for integral imaging,” Opt. Express 16(18), 13969–13978 (2008).
[Crossref] [PubMed]

Kubota, A.

A. Kubota, A. Smolic, M. Magnor, M. Tanimoto, T. Chen, and C. Zhang, “Multiview imaging and 3DTV,” IEEE Signal Process. Mag. 24(6), 10–21 (2007).

Lee, B.

J.-H. Park, K. Hong, and B. Lee, “Recent progress in three-dimensional information processing based on integral imaging,” Appl. Opt. 48(34), H77–H94 (2009).
[Crossref] [PubMed]

H. Kim, J. Hahn, and B. Lee, “Mathematical modeling of triangle-mesh-modeled three-dimensional surface objects for digital holography,” Appl. Opt. 47(19), D117–D127 (2008).
[Crossref] [PubMed]

J. Hahn, Y. Kim, E.-H. Kim, and B. Lee, “Undistorted pickup method of both virtual and real objects for integral imaging,” Opt. Express 16(18), 13969–13978 (2008).
[Crossref] [PubMed]

B. Lee, S. Jung, S.-W. Min, and J.-H. Park, “Three-dimensional display by use of integral photography with dynamically variable image planes,” Opt. Lett. 26(19), 1481–1482 (2001).
[Crossref]

Lee, B.-G.

Lee, J.-J.

Lee, K.-J.

Lippmann, G.

G. Lippmann, “La photographie integrále,” C. R. Acad. Sci. Ser. IIc Chim. 146, 446–451 (1908).

Magnor, M.

A. Kubota, A. Smolic, M. Magnor, M. Tanimoto, T. Chen, and C. Zhang, “Multiview imaging and 3DTV,” IEEE Signal Process. Mag. 24(6), 10–21 (2007).

Martinez-Corral, M.

Martínez-Corral, M.

Martinez-Cuenca, R.

Martínez-Cuenca, R.

Min, S.-W.

B. Lee, S. Jung, S.-W. Min, and J.-H. Park, “Three-dimensional display by use of integral photography with dynamically variable image planes,” Opt. Lett. 26(19), 1481–1482 (2001).
[Crossref]

Ogale, A. S.

A. S. Ogale, C. Fermüller, and Y. Aloimonos, “Motion segmentation using occlusions,” IEEE Trans. Pattern Anal. Mach. Intell. 27(6), 988–992 (2005).
[Crossref] [PubMed]

Okano, F.

F. Okano, H. Hoshino, J. Arai, and I. Yuyama, “Real-time pickup method for a three-dimensional image based on integral photography,” Appl. Opt. 36(7), 1598–1603 (1997).
[Crossref] [PubMed]

Okutomi, M.

M. Okutomi and T. Kanade, “A multiple-baseline stereo,” IEEE Trans. Pattern Anal. Mach. Intell. 15(4), 353–363 (1993).
[Crossref]

Park, G.

Park, J.-H.

J.-H. Park, K. Hong, and B. Lee, “Recent progress in three-dimensional information processing based on integral imaging,” Appl. Opt. 48(34), H77–H94 (2009).
[Crossref] [PubMed]

B. Lee, S. Jung, S.-W. Min, and J.-H. Park, “Three-dimensional display by use of integral photography with dynamically variable image planes,” Opt. Lett. 26(19), 1481–1482 (2001).
[Crossref]

Passalis, G.

Piao, Y.

M. Zhang, Y. Piao, and E.-S. Kim, “Occlusion-removed scheme using depth-reversed method in computational integral imaging,” Appl. Opt. 49(14), 2571–2580 (2010).
[Crossref]

Pons, A.

Rakkolainen, I.

Saavedra, G.

Sainov, V.

Schunck, B.

B. Horn and B. Schunck, “Determining optical flow,” Artif. Intell. 17(1-3), 185–203 (1981).
[Crossref]

Sgouros, N.

Shin, D.-H.

D.-H. Shin and E.-S. Kim, “Computational integral imaging reconstruction of 3D object using a depth conversion technique,” J. Opt. Soc. Korea 12(3), 131–135 (2008).
[Crossref]

Smolic, A.

A. Kubota, A. Smolic, M. Magnor, M. Tanimoto, T. Chen, and C. Zhang, “Multiview imaging and 3DTV,” IEEE Signal Process. Mag. 24(6), 10–21 (2007).

Surman, P.

Tanimoto, M.

A. Kubota, A. Smolic, M. Magnor, M. Tanimoto, T. Chen, and C. Zhang, “Multiview imaging and 3DTV,” IEEE Signal Process. Mag. 24(6), 10–21 (2007).

Theoharis, T.

Urey, H.

von Kopylow, C.

Watson, J.

Yuyama, I.

F. Okano, H. Hoshino, J. Arai, and I. Yuyama, “Real-time pickup method for a three-dimensional image based on integral photography,” Appl. Opt. 36(7), 1598–1603 (1997).
[Crossref] [PubMed]

Zhang, C.

A. Kubota, A. Smolic, M. Magnor, M. Tanimoto, T. Chen, and C. Zhang, “Multiview imaging and 3DTV,” IEEE Signal Process. Mag. 24(6), 10–21 (2007).

Zhang, M.

M. Zhang, Y. Piao, and E.-S. Kim, “Occlusion-removed scheme using depth-reversed method in computational integral imaging,” Appl. Opt. 49(14), 2571–2580 (2010).
[Crossref]

Appl. Opt. (7)

F. Okano, H. Hoshino, J. Arai, and I. Yuyama, “Real-time pickup method for a three-dimensional image based on integral photography,” Appl. Opt. 36(7), 1598–1603 (1997).
[Crossref] [PubMed]

M. Zhang, Y. Piao, and E.-S. Kim, “Occlusion-removed scheme using depth-reversed method in computational integral imaging,” Appl. Opt. 49(14), 2571–2580 (2010).
[Crossref]

J.-H. Park, K. Hong, and B. Lee, “Recent progress in three-dimensional information processing based on integral imaging,” Appl. Opt. 48(34), H77–H94 (2009).
[Crossref] [PubMed]

H. Kim, J. Hahn, and B. Lee, “Mathematical modeling of triangle-mesh-modeled three-dimensional surface objects for digital holography,” Appl. Opt. 47(19), D117–D127 (2008).
[Crossref] [PubMed]

Artif. Intell. (1)

B. Horn and B. Schunck, “Determining optical flow,” Artif. Intell. 17(1-3), 185–203 (1981).
[Crossref]

C. R. Acad. Sci. Ser. IIc Chim. (1)

G. Lippmann, “La photographie integrále,” C. R. Acad. Sci. Ser. IIc Chim. 146, 446–451 (1908).

IEEE Signal Process. Mag. (1)

A. Kubota, A. Smolic, M. Magnor, M. Tanimoto, T. Chen, and C. Zhang, “Multiview imaging and 3DTV,” IEEE Signal Process. Mag. 24(6), 10–21 (2007).

IEEE Trans. Circ. Syst. Video Tech. (1)

IEEE Trans. Pattern Anal. Mach. Intell. (2)

M. Okutomi and T. Kanade, “A multiple-baseline stereo,” IEEE Trans. Pattern Anal. Mach. Intell. 15(4), 353–363 (1993).
[Crossref]

A. S. Ogale, C. Fermüller, and Y. Aloimonos, “Motion segmentation using occlusions,” IEEE Trans. Pattern Anal. Mach. Intell. 27(6), 988–992 (2005).
[Crossref] [PubMed]

J. Disp. Technol. (1)

J. Opt. Soc. Korea (1)

D.-H. Shin and E.-S. Kim, “Computational integral imaging reconstruction of 3D object using a depth conversion technique,” J. Opt. Soc. Korea 12(3), 131–135 (2008).
[Crossref]

Opt. Express (7)

S.-H. Hong and B. Javidi, “Distortion-tolerant 3D recognition of occluded objects using computational integral imaging,” Opt. Express 14(25), 12085–12095 (2006).
[Crossref] [PubMed]

J. Hahn, Y. Kim, E.-H. Kim, and B. Lee, “Undistorted pickup method of both virtual and real objects for integral imaging,” Opt. Express 16(18), 13969–13978 (2008).
[Crossref] [PubMed]

Opt. Lett. (1)

B. Lee, S. Jung, S.-W. Min, and J.-H. Park, “Three-dimensional display by use of integral photography with dynamically variable image planes,” Opt. Lett. 26(19), 1481–1482 (2001).
[Crossref]

Other (4)

M. Levoy, and P. Hanrahan, “Light field rendering,” in Proceedings of SIGGRAPH ‘96 (Association for Computing Machinery, New Orleans, 1996), pp. 31–42.

B. Lee, J.-H. Park, and S.-W. Min, Digital Holography and Three-Dimensional Display, T.-C. Poon, ed. (Springer US, 2006), Chap. 12.

D. Sun, S. Roth, and M. J. Black, “Secrets of optical flow estimation and their principles,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2010), pp. 2432–2439.

S. Baker, D. Scharstein, J. P. Lewis, S. Roth, M. J. Black, and R. Szeliski, “A database and evaluation methodology for optical flow,” in Proceedings of IEEE Conference on International Conference on Computer Vision (IEEE, 2007), pp. 1–8.

Supplementary Material (6)

» Media 1: MOV (545 KB)

» Media 2: MOV (544 KB)

» Media 3: MOV (671 KB)

» Media 4: MOV (691 KB)

» Media 5: MOV (606 KB)

» Media 6: MOV (612 KB)

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.

Click here to see a list of articles that cite this paper

Figures (17)

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 17
Fig. 16

Fig. 1 Concept of the proposed method.

Download Full Size | PPT Slide | PDF

Fig. 2 Two types of pickup process in integral imaging: (a) real mode pickup process with central depth plane and the other depth plane, (b) focal mode pickup process.

Download Full Size | PPT Slide | PDF

Fig. 3 An example of elemental image to sub-image conversion: (a) computer generated elemental image set and the center elemental image (56 × 55 lens array, 19 × 19 pixels/lens, f = 3.3), (b) sub-image array and the center sub-image (19 × 19 sub-image array, 56 × 55 pixels/sub-image).

Download Full Size | PPT Slide | PDF

Fig. 4 Principles of depth extraction and depth quantization problem in (a) elemental image based method, (b) sub-image based method.

Download Full Size | PPT Slide | PDF

Fig. 5 The process of depth extraction using optical flows between center sub-image and x-directional sub-image sequence.

Download Full Size | PPT Slide | PDF

Fig. 6 Comparison of the depth map of center sub-image using SSSD and optical flow: (a) the extracted depth map using SSSD, (b) extracted depth map using optical flow, (c) comparison of the ground truth and extracted depth maps obtained by different methods in y-z plane (x = 22).

Download Full Size | PPT Slide | PDF

Fig. 7 Histogram of the number of pixels in depth map of center sub-image and depth threshold between the inflection points.

Download Full Size | PPT Slide | PDF

Fig. 8 Point cloud representation and depth segmentation of center sub-image using depth threshold.

Download Full Size | PPT Slide | PDF

Fig. 9 Reconstruction process of occluded object in point cloud representation: (a) depth extraction and segmentation of side sub-images, (b) reconstruction of center sub-image with resolution enhancement using triangular mesh reconstruction.

Download Full Size | PPT Slide | PDF

Fig. 10 Triangular mesh reconstruction of center sub-image: (a) reconstruction of the vertex of occluded region in center sub-image (blue point) using neighbor projected points (red points) in the refinement window (S = 4), (b) the refining of all other vertices using the scanning of the window and triangular mesh reconstruction.

Download Full Size | PPT Slide | PDF

Fig. 11 Triangular mesh reconstruction of occluded region in center sub-image: (a) triangular mesh representation of occluded center sub-image, (b) triangular mesh reconstruction of occluded region in center sub-image with resolution enhancement (S = 10).

Download Full Size | PPT Slide | PDF

Fig. 12 Comparison of occluded sub-image array and reconstructed sub-image array in arbitrary viewpoints: (a) movie of occluded sub-image array (Media 1), (b) movie of reconstructed sub-image array with resolution enhancement (S = 10) (Media 2).

Download Full Size | PPT Slide | PDF

Fig. 13 Analysis of the range of reconstruction in occluded area using the proposed method (odd number n and n_eff ): w_oc in red color is the maximum range of reconstruction with n_eff lens array, and blue color is the case of n which is smaller than n_eff lens array.

Download Full Size | PPT Slide | PDF

Fig. 14 Experimental setup: (a) experimental setup for pickup process, (b) formation of target object and obstacle.

Download Full Size | PPT Slide | PDF

Fig. 15 Experimental result: (a) rectified elemental image set, (b) center sub-image, (c) depth map of center sub-image from the optical flow with sub-pixel accuracy, (d) reconstructed center sub-image, (e) reconstructed center sub-image with resolution enhancement (S = 5), (f) reconstructed depth map of center sub-image with resolution enhancement (S = 5).

Download Full Size | PPT Slide | PDF

Fig. 17 Experimental result at arbitrary viewpoints: (a) movie of occluded sub-images of box objects (Media 3), (b) movie of reconstructed sub-images of box objects with resolution enhancement (S = 5) (Media 4), (c) movie of occluded sub-images of hand object (Media 5), (d) movie of reconstructed sub-images of hand object with resolution enhancement (S = 5) (Media 6).

Download Full Size | PPT Slide | PDF

Fig. 16 Experimental result of triangular mesh reconstruction: (a) triangular mesh representation of occluded center sub-image, (b) triangular mesh reconstruction of occluded region in center sub-image with resolution enhancement using proposed method (S = 5).

Download Full Size | PPT Slide | PDF

Tables (2)

Table 1 Specification of the experimental setup

View Table | View all tables in this article

Table 2 PSNR and NCC results of reconstructed object

View Table | View all tables in this article

Equations (9)

Equations on this page are rendered with MathJax. Learn more.

D_{C D P} = \frac{f g}{g - f} .

D_{d_{l}, d_{p}} = \frac{f d_{l} p_{l}}{d_{P} p_{p}},

\frac{d I}{d t} = \frac{\partial I}{\partial x} \frac{d x}{d t} + \frac{\partial I}{\partial y} \frac{d y}{d t} + \frac{\partial I}{\partial t} = 0.

u = \frac{d x}{d t}, v = \frac{d y}{d t} .

\begin{array}{l} D_{O F (p_{c}, q_{c})} (x, y) \\ = \frac{f p_{l}}{2 p_{p}} [\frac{1}{n_{x} - 1} \sum_{p = 1}^{n_{x}} | \frac{u_{(p_{c}, q_{c}), (p, q_{c})} (x, y)}{p_{c} - p} | + \frac{1}{n_{y} - 1} \sum_{q = 1}^{n_{y}} | \frac{v_{(p_{c}, q_{c}), (p_{c}, q)} (x, y)}{q_{c} - q} |], \end{array}

(x^{'}, y^{'}) = (x + \frac{p_{p} (p_{c} - p) D_{O F} (x, y)}{f p_{l}}, y + \frac{p_{p} (q_{c} - q) D_{O F} (x, y)}{f p_{l}}),

n_{e f f} = ⌈ \frac{D_{o c}}{f} + \frac{w}{p_{l}} + 1 ⌉,

d_{p_{e f f}} = ⌊ \frac{f [(n - 1) p_{l} - w]}{2 D_{o c} p_{p}} ⌋,

w_{o c} = {\begin{matrix} \frac{D_{t} p_{l}}{2 f} - \frac{(n_{e f f} - 1) p_{l}}{2} + \frac{w}{2}, n \geq n_{e f f} \\ \frac{D_{t} d_{p_{e f f}} p_{p}}{f} - \frac{(n - 1) p_{l}}{2} + \frac{w}{2}, n < n_{e f f} \end{matrix} .

Setup	Specification	Characteristic
Lens array	Number of lenses Lens size Focal length	99 by 98 1 mm by 1mm 3.3 mm
Camera	ModelResolutionLens system	Canon 5D Mark II5616 by 3744Canon EF 28-135mm F3.5-5.6

Test image set	Occluded image		Reconstructed image
Test image set	PSNR	NCC	PSNR	NCC
CGIP	28.78 dB	0.9558	36.93 dB	0.9874
Real objects (box)	26.22 dB	0.9743	28.51 dB	0.9907
Real objects (hand)	25.42 dB	0.9782	28.84 dB	0.9909

Setup	Specification	Characteristic
Lens array	Number of lenses Lens size Focal length	99 by 98 1 mm by 1mm 3.3 mm
Camera	ModelResolutionLens system	Canon 5D Mark II5616 by 3744Canon EF 28-135mm F3.5-5.6

Test image set	Occluded image		Reconstructed image
Test image set	PSNR	NCC	PSNR	NCC
CGIP	28.78 dB	0.9558	36.93 dB	0.9874
Real objects (box)	26.22 dB	0.9743	28.51 dB	0.9907
Real objects (hand)	25.42 dB	0.9782	28.84 dB	0.9909

Abstract

References

2010 (3)

2009 (1)

2008 (6)

2007 (3)

2006 (2)

2005 (2)

2004 (1)

2001 (1)

1997 (1)

1993 (1)

1981 (1)

1908 (1)

Aloimonos, Y.

Arai, J.

Athineos, S.

Baasantseren, G.

Benzie, P.

Chen, T.

Choi, H.

Do, C. M.

Fermüller, C.

Hahn, J.

Hong, J.

Hong, K.

Hong, S.-H.

Hopf, K.

Horn, B.

Hoshino, H.

Hwang, D.-C.

Javidi, B.

Jung, J.-H.

Jung, S.

Kanade, T.

Kang, J.-M.

Kim, E.-H.

Kim, E.-S.

Kim, H.

Kim, N.

Kim, S.-C.

Kim, Y.

Kubota, A.

Lee, B.

Lee, B.-G.

Lee, J.-J.

Lee, K.-J.

Lippmann, G.

Magnor, M.

Martinez-Corral, M.

Martínez-Corral, M.

Martinez-Cuenca, R.

Martínez-Cuenca, R.

Min, S.-W.

Ogale, A. S.

Okano, F.

Okutomi, M.

Park, G.

Park, J.-H.

Passalis, G.

Piao, Y.

Pons, A.

Rakkolainen, I.

Saavedra, G.

Sainov, V.

Schunck, B.

Sgouros, N.

Shin, D.-H.

Smolic, A.

Surman, P.

Tanimoto, M.

Theoharis, T.

Urey, H.

von Kopylow, C.

Watson, J.

Yuyama, I.

Zhang, C.

Zhang, M.

Appl. Opt. (7)

Artif. Intell. (1)