Abstract

We describe a computational method for depth extraction of three-dimensional (3D) objects using block matching for slice images in synthetic aperture integral imaging (SAII). SAII is capable of provid ing high-resolution 3D slice images for 3D objects because the picked-up elemental images are high- resolution ones. In the proposed method, the high-resolution elemental images are recorded by moving a camera; a computational reconstruction algorithm based on ray backprojection generates a set of 3D slice images from the recorded elemental images. To extract depth information of the 3D objects, we propose a new block-matching algorithm between a reference elemental image and a set of 3D slice images. The property of the slices images is that the focused areas are the right location for an object, whereas the blurred areas are considered to be empty space; thus, this can extract robust and accurate depth information of the 3D objects. To demonstrate our method, we carry out the preliminary experiments of 3D objects; the results indicate that our method is superior to a conventional method in terms of depth-map quality.

© 2011 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. T. Okoshi, Three-Dimensional Imaging Techniques (Academic, 1976).
  2. S. A. Benton, Selected Papers on Three-Dimensional Displays (SPIE Press, 2001).
  3. B. Javidi and F. Okano, Three-Dimensional Television, Video, and Display Technologies (Springer, 2002).
  4. Y. Kim, K. Hong, and B. Lee, “Recent researches based on integral imaging display method,” 3D Res. 1(1), 17–27 (2010).
    [CrossRef]
  5. M. G. Lippmann, “Épreuves réversibles donnant la sensation du relief,” J. Phys. 7, 821–825 (1908).
  6. J.-S. Jang and B. Javidi, “Improved viewing resolution of three-dimensional integral imaging with nonstationary micro-optics,” Opt. Lett. 27, 324–326 (2002).
    [CrossRef]
  7. H. Yoo and D.-H. Shin, “Improved analysis on the signal property of computational integral imaging system,” Opt. Express 15, 14107–14114 (2007).
    [CrossRef] [PubMed]
  8. D.-H. Shin, B. Lee, and E.-S. Kim, “Optical display of true 3D objects in depth-priority integral imaging using an active sensor,” Opt. Commun. 275, 330–334 (2007).
    [CrossRef]
  9. A. Stern and B. Javidi, “Three-dimensional image sensing, visualization, and processing using integral imaging,” Proc. IEEE 94, 591–607 (2006).
    [CrossRef]
  10. Y. S. Hwang, S.-H. Hong, and B. Javidi, “Free view 3-D visualization of occluded objects by using computational synthetic aperture integral imaging,” J. Display Technol. 3, 64–70(2007).
    [CrossRef]
  11. M. Cho and B. Javidi, “Three-dimensional visualization of objects in turbid water using integral imaging,” J. Display Technol. 6, 544–547 (2010).
    [CrossRef]
  12. I. Moon and B. Javidi, “Three-dimensional recognition of photon-starved events using computational integral imaging and statistical sampling,” Opt. Lett. 34, 731–733 (2009).
    [CrossRef] [PubMed]
  13. J.-H. Park, S. Jung, H. Choi, Y. Kim, and B. Lee, “Depth extraction by use of a rectangular lens array and one-dimensional elemental image modification,” Appl. Opt. 43, 4882–4895 (2004).
    [CrossRef] [PubMed]
  14. J.-H. Park, Y. Kim, J. Kim, S.-W. Min, and B. Lee, “Three-dimensional display scheme based on integral imaging with three-dimensional information processing,” Opt. Express 12, 6020–6032 (2004).
    [CrossRef] [PubMed]
  15. G. Passalis, N. Sgouros, S. Athineos, and T. Theohari, “Enhanced reconstruction of three-dimensional shape and texture from integral photography images,” Appl. Opt. 46, 5311–5320 (2007).
    [CrossRef] [PubMed]
  16. D.-H. Shin, B.-G. Lee, and J.-J. Lee, “Occlusion removal method of partially occluded 3D object using sub-image block matching in computational integral imaging,” Opt. Express 16, 16294–16304 (2008).
    [CrossRef] [PubMed]
  17. B.-G. Lee, H.-H. Kang, and E.-S. Kim, “Occlusion removal method of partially occluded object using variance in computational integral imaging,” 3D Res. 1(2), 6–10 (2010).
    [CrossRef]
  18. J.-H. Jung, K. Hong, G. Park, I. Chung, J.-H. Park, and B. Lee, “Reconstruction of three-dimensional occluded object using optical flow and triangular mesh reconstruction in integral imaging,” Opt. Express 18, 26373–26387 (2010).
    [CrossRef] [PubMed]
  19. H. Yoo and D.-H. Shin, “Fast computational integral imaging reconstruction method using a fractional delay filter,” Jpn. J. Appl. Phys. 49, 022503 (2010).
    [CrossRef]
  20. A. R. Greenleaf, Photographic Optics (MacMillan, 1950), pp. 25–27.
  21. B. B. Alagoz, “Obtaining depth maps from color images by region based stereo matching algorithms,” OncuBilim Algorithm Systems Labs 8, 4 (2008) arXiv:0812.1340v2.
  22. M. Z. Brown, D. Burschka, and G. D. Hager, “Advances in computational stereo,” IEEE Trans. Pattern Anal. Mach. Intell. 25, 993–1008 (2003).
    [CrossRef]
  23. H. Yoo, “Artifact analysis and image enhancement in three-dimensional computational integral imaging using smooth windowing technique,” Opt. Lett. 36, 2107–2109 (2011).
    [CrossRef] [PubMed]
  24. J.-J. Lee, B.-G. Lee, and H. Yoo, “Image quality enhancement of computational integral imaging reconstruction for partially occluded objects using binary weighting mask on occlusion areas,” Appl. Opt. 50, 1889–1893 (2011).
    [CrossRef] [PubMed]

2011

2010

Y. Kim, K. Hong, and B. Lee, “Recent researches based on integral imaging display method,” 3D Res. 1(1), 17–27 (2010).
[CrossRef]

H. Yoo and D.-H. Shin, “Fast computational integral imaging reconstruction method using a fractional delay filter,” Jpn. J. Appl. Phys. 49, 022503 (2010).
[CrossRef]

B.-G. Lee, H.-H. Kang, and E.-S. Kim, “Occlusion removal method of partially occluded object using variance in computational integral imaging,” 3D Res. 1(2), 6–10 (2010).
[CrossRef]

M. Cho and B. Javidi, “Three-dimensional visualization of objects in turbid water using integral imaging,” J. Display Technol. 6, 544–547 (2010).
[CrossRef]

J.-H. Jung, K. Hong, G. Park, I. Chung, J.-H. Park, and B. Lee, “Reconstruction of three-dimensional occluded object using optical flow and triangular mesh reconstruction in integral imaging,” Opt. Express 18, 26373–26387 (2010).
[CrossRef] [PubMed]

2009

2008

D.-H. Shin, B.-G. Lee, and J.-J. Lee, “Occlusion removal method of partially occluded 3D object using sub-image block matching in computational integral imaging,” Opt. Express 16, 16294–16304 (2008).
[CrossRef] [PubMed]

B. B. Alagoz, “Obtaining depth maps from color images by region based stereo matching algorithms,” OncuBilim Algorithm Systems Labs 8, 4 (2008) arXiv:0812.1340v2.

2007

2006

A. Stern and B. Javidi, “Three-dimensional image sensing, visualization, and processing using integral imaging,” Proc. IEEE 94, 591–607 (2006).
[CrossRef]

2004

2003

M. Z. Brown, D. Burschka, and G. D. Hager, “Advances in computational stereo,” IEEE Trans. Pattern Anal. Mach. Intell. 25, 993–1008 (2003).
[CrossRef]

2002

1908

M. G. Lippmann, “Épreuves réversibles donnant la sensation du relief,” J. Phys. 7, 821–825 (1908).

Alagoz, B. B.

B. B. Alagoz, “Obtaining depth maps from color images by region based stereo matching algorithms,” OncuBilim Algorithm Systems Labs 8, 4 (2008) arXiv:0812.1340v2.

Athineos, S.

Benton, S. A.

S. A. Benton, Selected Papers on Three-Dimensional Displays (SPIE Press, 2001).

Brown, M. Z.

M. Z. Brown, D. Burschka, and G. D. Hager, “Advances in computational stereo,” IEEE Trans. Pattern Anal. Mach. Intell. 25, 993–1008 (2003).
[CrossRef]

Burschka, D.

M. Z. Brown, D. Burschka, and G. D. Hager, “Advances in computational stereo,” IEEE Trans. Pattern Anal. Mach. Intell. 25, 993–1008 (2003).
[CrossRef]

Cho, M.

Choi, H.

Chung, I.

Greenleaf, A. R.

A. R. Greenleaf, Photographic Optics (MacMillan, 1950), pp. 25–27.

Hager, G. D.

M. Z. Brown, D. Burschka, and G. D. Hager, “Advances in computational stereo,” IEEE Trans. Pattern Anal. Mach. Intell. 25, 993–1008 (2003).
[CrossRef]

Hong, K.

Hong, S.-H.

Hwang, Y. S.

Jang, J.-S.

Javidi, B.

Jung, J.-H.

Jung, S.

Kang, H.-H.

B.-G. Lee, H.-H. Kang, and E.-S. Kim, “Occlusion removal method of partially occluded object using variance in computational integral imaging,” 3D Res. 1(2), 6–10 (2010).
[CrossRef]

Kim, E.-S.

B.-G. Lee, H.-H. Kang, and E.-S. Kim, “Occlusion removal method of partially occluded object using variance in computational integral imaging,” 3D Res. 1(2), 6–10 (2010).
[CrossRef]

D.-H. Shin, B. Lee, and E.-S. Kim, “Optical display of true 3D objects in depth-priority integral imaging using an active sensor,” Opt. Commun. 275, 330–334 (2007).
[CrossRef]

Kim, J.

Kim, Y.

Lee, B.

Lee, B.-G.

Lee, J.-J.

Lippmann, M. G.

M. G. Lippmann, “Épreuves réversibles donnant la sensation du relief,” J. Phys. 7, 821–825 (1908).

Min, S.-W.

Moon, I.

Okano, F.

B. Javidi and F. Okano, Three-Dimensional Television, Video, and Display Technologies (Springer, 2002).

Okoshi, T.

T. Okoshi, Three-Dimensional Imaging Techniques (Academic, 1976).

Park, G.

Park, J.-H.

Passalis, G.

Sgouros, N.

Shin, D.-H.

H. Yoo and D.-H. Shin, “Fast computational integral imaging reconstruction method using a fractional delay filter,” Jpn. J. Appl. Phys. 49, 022503 (2010).
[CrossRef]

D.-H. Shin, B.-G. Lee, and J.-J. Lee, “Occlusion removal method of partially occluded 3D object using sub-image block matching in computational integral imaging,” Opt. Express 16, 16294–16304 (2008).
[CrossRef] [PubMed]

H. Yoo and D.-H. Shin, “Improved analysis on the signal property of computational integral imaging system,” Opt. Express 15, 14107–14114 (2007).
[CrossRef] [PubMed]

D.-H. Shin, B. Lee, and E.-S. Kim, “Optical display of true 3D objects in depth-priority integral imaging using an active sensor,” Opt. Commun. 275, 330–334 (2007).
[CrossRef]

Stern, A.

A. Stern and B. Javidi, “Three-dimensional image sensing, visualization, and processing using integral imaging,” Proc. IEEE 94, 591–607 (2006).
[CrossRef]

Theohari, T.

Yoo, H.

3D Res.

Y. Kim, K. Hong, and B. Lee, “Recent researches based on integral imaging display method,” 3D Res. 1(1), 17–27 (2010).
[CrossRef]

B.-G. Lee, H.-H. Kang, and E.-S. Kim, “Occlusion removal method of partially occluded object using variance in computational integral imaging,” 3D Res. 1(2), 6–10 (2010).
[CrossRef]

Appl. Opt.

IEEE Trans. Pattern Anal. Mach. Intell.

M. Z. Brown, D. Burschka, and G. D. Hager, “Advances in computational stereo,” IEEE Trans. Pattern Anal. Mach. Intell. 25, 993–1008 (2003).
[CrossRef]

J. Display Technol.

J. Phys.

M. G. Lippmann, “Épreuves réversibles donnant la sensation du relief,” J. Phys. 7, 821–825 (1908).

Jpn. J. Appl. Phys.

H. Yoo and D.-H. Shin, “Fast computational integral imaging reconstruction method using a fractional delay filter,” Jpn. J. Appl. Phys. 49, 022503 (2010).
[CrossRef]

OncuBilim Algorithm Systems Labs

B. B. Alagoz, “Obtaining depth maps from color images by region based stereo matching algorithms,” OncuBilim Algorithm Systems Labs 8, 4 (2008) arXiv:0812.1340v2.

Opt. Commun.

D.-H. Shin, B. Lee, and E.-S. Kim, “Optical display of true 3D objects in depth-priority integral imaging using an active sensor,” Opt. Commun. 275, 330–334 (2007).
[CrossRef]

Opt. Express

Opt. Lett.

Proc. IEEE

A. Stern and B. Javidi, “Three-dimensional image sensing, visualization, and processing using integral imaging,” Proc. IEEE 94, 591–607 (2006).
[CrossRef]

Other

A. R. Greenleaf, Photographic Optics (MacMillan, 1950), pp. 25–27.

T. Okoshi, Three-Dimensional Imaging Techniques (Academic, 1976).

S. A. Benton, Selected Papers on Three-Dimensional Displays (SPIE Press, 2001).

B. Javidi and F. Okano, Three-Dimensional Television, Video, and Display Technologies (Springer, 2002).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (9)

Fig. 1
Fig. 1

Principle of SAII.

Fig. 2
Fig. 2

Proposed depth extraction method using slice images.

Fig. 3
Fig. 3

(a) Elemental images (b) principle of computational reconstruction.

Fig. 4
Fig. 4

Block-matching algorithm between reference elemental image and slice images.

Fig. 5
Fig. 5

Four examples of recorded elemental images: (a) (1, 1)th elemental image, (b) (7, 1)th elemental image, (c) (4, 3)th elemental image, and (d) (1, 5)th elemental image.

Fig. 6
Fig. 6

Reconstructed slice images at (a) 980, (b) 1050, (c) 1110, and (d)  1190 mm .

Fig. 7
Fig. 7

Visual comparison for extracted depth maps of a cubic object extracted by the (a) conventional method, (b) proposed method with the center elemental image, and (c) another elemental image as the reference image.

Fig. 8
Fig. 8

(a) Character objects and reconstructed slice images at (b) 970 (c) 1070, and (d)  1200 mm .

Fig. 9
Fig. 9

Visual comparison for extracted depth maps of character objects extracted by the (a) conventional method and (b) proposed method.

Equations (4)

Equations on this page are rendered with MathJax. Learn more.

I ( x , y , z 0 ) = 1 O ( x , y ) k = 0 K 1 l = 0 L 1 I k l ( x k S , y l S ) ,
S = p δ × M ,
SAD ( x , y , z ) = i = 1 b j = 1 b | B ref ( x + i , y + j ) B z ( x + i , y + j ) | ,
z ^ ( x , y ) = argmin SAD ( x , y , z ) .

Metrics