Abstract

This paper describes a computational reconstruction method for 3-D imaging via a diffraction grating. An optical device consisting of a diffraction grating with a camera produces a parallax image array (PIA) for 3-D imaging in an efficient way according to recent researches. Unlike other capturing systems for a PIA such as a lens array with a camera and a camera array, a diffraction grating with a camera has an advantage in terms of the optical system complexity. However, since the diffraction grating is transparent, the captured raw image by the diffraction grating has no feature to detect the boundary of each parallax image. Moreover, the diffraction grating allows parallax images to overlap each other due to its optical property. Those problems prevent computational reconstruction from generating 3-D images. To remedy those problems, we propose a 3-D computational reconstruction method via a diffraction grating. The proposed method using a diffraction grating includes analyzing the PIA pickup process and converting a captured raw image into a well-defined PIA. Our analysis introduces a virtual pinhole; thus, a diffraction grating works as a camera array. Also, it defines the effective object area to segment parallax images and provides a mapping between each segmented parallax image and corresponding virtual pinhole. The minimum image area is also defined to determine the minimum field of view for our reconstruction. Optical experimental results indicated the proposed theoretical analysis and computational reconstruction in diffraction grating imaging are feasible in 3-D imaging. To our best knowledge, this is the first report on 3-D computational reconstruction via a diffraction grating.

© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

Full Article  |  PDF Article
OSA Recommended Articles
Three-dimensional volumetric object reconstruction using computational integral imaging

Seung-Hyun Hong, Ju-Seog Jang, and Bahram Javidi
Opt. Express 12(3) 483-491 (2004)

Applicability of diffraction grating to parallax image array generation in integral imaging

Jang-Il Ser, Jae-Young Jang, Sungdo Cha, and Seung-Ho Shin
Appl. Opt. 49(13) 2429-2433 (2010)

Curved computational integral imaging reconstruction technique for resolution-enhanced display of three-dimensional object images

Joo-Bong Hyun, Dong-Choon Hwang, Dong-Hak Shin, and Eun-Soo Kim
Appl. Opt. 46(31) 7697-7708 (2007)

References

  • View by:
  • |
  • |
  • |

  1. J.-H. Park, J. Kim, Y. Kim, and B. Lee, “Resolution-enhanced three-dimension/two-dimension convertible display based on integral imaging,” Opt. Express 13(6), 1875–1884 (2005).
    [Crossref]
  2. R. Martínez-Cuenca, G. Saavedra, A. Pons, B. Javidi, and M. Martínez-Corral, “Facet braiding: a fundamental problem in integral imaging,” Opt. Lett. 32(9), 1078–1080 (2007).
    [Crossref]
  3. D.-H. Shin and H. Yoo, “Image quality enhancement in 3D computational integral imaging by use of interpolation methods,” Opt. Express 15(19), 12039–12049 (2007).
    [Crossref]
  4. D.-H. Shin, B.-G. Lee, and J.-J. Lee, “Occlusion removal method of partially occluded 3D object using sub-image block matching in computational integral imaging,” Opt. Express 16(21), 16294–16304 (2008).
    [Crossref]
  5. Y. Piao, D.-H. Shin, and E.-S. Kim, “Robust image encryption by combined use of integral imaging and pixel scrambling techniques,” Opt. Lasers Eng. 47(11), 1273–1281 (2009).
    [Crossref]
  6. J.-Y. Jang, H.-S. Lee, S. Cha, and S.-H. Shin, “Viewing angle enhanced integral imaging display by using a high refractive index medium,” Appl. Opt. 50(7), B71–B76 (2011).
    [Crossref]
  7. H. Yoo, “Axially moving a lenslet array for high-resolution 3D images in computational integral imaging,” Opt. Express 21(7), 8873–8878 (2013).
    [Crossref]
  8. X. Xiao, B. Javidi, M. Martinez-Corral, and A. Stern, “Advances in three-dimensional integral imaging: sensing, display, and applications [Invited],” Appl. Opt. 52(4), 546–560 (2013).
    [Crossref]
  9. J.-Y. Jang, D. Shin, and E.-S. Kim, “Optical three-dimensional refocusing from elemental images based on a sifting property of the periodic δ-function array in integral-imaging,” Opt. Express 22(2), 1533–1550 (2014).
    [Crossref]
  10. J.-Y. Jang and M. Cho, “Orthoscopic real image reconstruction in integral imaging by rotating an elemental image based on the reference point of object space,” Appl. Opt. 54(18), 5877–5881 (2015).
    [Crossref]
  11. H. Yoo and J.-Y. Jang, “Intermediate elemental image reconstruction for refocused three-dimensional images in integral imaging by convolution with δ-function sequences,” Opt. Lasers Eng. 97, 93–99 (2017).
    [Crossref]
  12. J. Wei, S. Wang, Y. Zhao, and M. Piao, “Synthetic aperture integral imaging using edge depth maps of unstructured monocular video,” Opt. Express 26(26), 34894–34908 (2018).
    [Crossref]
  13. J.-Y. Jang, D. Shin, and E.-S. Kim, “Improved 3-D image reconstruction using the convolution property of periodic functions in curved integral-imaging,” Opt. Lasers Eng. 54, 14–20 (2014).
    [Crossref]
  14. S. Komatsu, A. Markman, and B. Javidi, “Optical sensing and detection in turbid water using multidimensional integral imaging,” Opt. Lett. 43(14), 3261–3264 (2018).
    [Crossref]
  15. K.-C. Kwon, M.-U. Erdenebat, Y.-T. Lim, K.-I. Joo, M.-K. Park, H. Park, J.-R. Jeong, H.-R. Kim, and N. Kim, “Enhancement of the depth-of-field of integral imaging microscope by using switchable bifocal liquid-crystalline polymer micro lens array,” Opt. Express 25(24), 30503–30512 (2017).
    [Crossref]
  16. H.-S. Kim, K.-M. Jeong, S.-I. Hong, N.-Y. Jo, and J.-H. Park, “Analysis of image distortion based on light ray field by multi-view and horizontal parallax only integral imaging display,” Opt. Express 20(21), 23755–23768 (2012).
    [Crossref]
  17. J.-I. Ser, J.-Y. Jang, S. Cha, and S.-H. Shin, “Applicability of diffraction grating to parallax image array generation in integral imaging,” Appl. Opt. 49(13), 2429–2433 (2010).
    [Crossref]
  18. J.-Y. Jang, J.-I. Ser, and E.-S. Kim, “Wave-optical analysis of parallax-image generation based on multiple diffraction gratings,” Opt. Lett. 38(11), 1835–1837 (2013).
    [Crossref]
  19. A. Aggoun, “Pre-processing of integral images for 3-D displays,” J. Disp. Technol. 2(4), 393–400 (2006).
    [Crossref]
  20. N. P. Sgouros, S. S. Athineos, M. S. Sangriotis, P. G. Papageorgas, and N. G. Theofanous, “Accurate lattice extraction in integral images,” Opt. Express 14(22), 10403–10409 (2006).
    [Crossref]
  21. J.-J. Lee, D.-H. Shin, and B.-G. Lee, “Simple correction method of distorted elemental images using surface markers on lenslet array for computational integral imaging reconstruction,” Opt. Express 17(20), 18026–18037 (2009).
    [Crossref]
  22. K. Hong, J. Hong, J.-H. Jung, J.-H. Park, and B. Lee, “Rectification of elemental image set and extraction of lens lattice by projective image transformation in integral imaging,” Opt. Express 18(11), 12002–12016 (2010).
    [Crossref]

2018 (2)

2017 (2)

K.-C. Kwon, M.-U. Erdenebat, Y.-T. Lim, K.-I. Joo, M.-K. Park, H. Park, J.-R. Jeong, H.-R. Kim, and N. Kim, “Enhancement of the depth-of-field of integral imaging microscope by using switchable bifocal liquid-crystalline polymer micro lens array,” Opt. Express 25(24), 30503–30512 (2017).
[Crossref]

H. Yoo and J.-Y. Jang, “Intermediate elemental image reconstruction for refocused three-dimensional images in integral imaging by convolution with δ-function sequences,” Opt. Lasers Eng. 97, 93–99 (2017).
[Crossref]

2015 (1)

2014 (2)

J.-Y. Jang, D. Shin, and E.-S. Kim, “Optical three-dimensional refocusing from elemental images based on a sifting property of the periodic δ-function array in integral-imaging,” Opt. Express 22(2), 1533–1550 (2014).
[Crossref]

J.-Y. Jang, D. Shin, and E.-S. Kim, “Improved 3-D image reconstruction using the convolution property of periodic functions in curved integral-imaging,” Opt. Lasers Eng. 54, 14–20 (2014).
[Crossref]

2013 (3)

2012 (1)

2011 (1)

2010 (2)

2009 (2)

J.-J. Lee, D.-H. Shin, and B.-G. Lee, “Simple correction method of distorted elemental images using surface markers on lenslet array for computational integral imaging reconstruction,” Opt. Express 17(20), 18026–18037 (2009).
[Crossref]

Y. Piao, D.-H. Shin, and E.-S. Kim, “Robust image encryption by combined use of integral imaging and pixel scrambling techniques,” Opt. Lasers Eng. 47(11), 1273–1281 (2009).
[Crossref]

2008 (1)

2007 (2)

2006 (2)

2005 (1)

Aggoun, A.

A. Aggoun, “Pre-processing of integral images for 3-D displays,” J. Disp. Technol. 2(4), 393–400 (2006).
[Crossref]

Athineos, S. S.

Cha, S.

Cho, M.

Erdenebat, M.-U.

Hong, J.

Hong, K.

Hong, S.-I.

Jang, J.-Y.

Javidi, B.

Jeong, J.-R.

Jeong, K.-M.

Jo, N.-Y.

Joo, K.-I.

Jung, J.-H.

Kim, E.-S.

J.-Y. Jang, D. Shin, and E.-S. Kim, “Improved 3-D image reconstruction using the convolution property of periodic functions in curved integral-imaging,” Opt. Lasers Eng. 54, 14–20 (2014).
[Crossref]

J.-Y. Jang, D. Shin, and E.-S. Kim, “Optical three-dimensional refocusing from elemental images based on a sifting property of the periodic δ-function array in integral-imaging,” Opt. Express 22(2), 1533–1550 (2014).
[Crossref]

J.-Y. Jang, J.-I. Ser, and E.-S. Kim, “Wave-optical analysis of parallax-image generation based on multiple diffraction gratings,” Opt. Lett. 38(11), 1835–1837 (2013).
[Crossref]

Y. Piao, D.-H. Shin, and E.-S. Kim, “Robust image encryption by combined use of integral imaging and pixel scrambling techniques,” Opt. Lasers Eng. 47(11), 1273–1281 (2009).
[Crossref]

Kim, H.-R.

Kim, H.-S.

Kim, J.

Kim, N.

Kim, Y.

Komatsu, S.

Kwon, K.-C.

Lee, B.

Lee, B.-G.

Lee, H.-S.

Lee, J.-J.

Lim, Y.-T.

Markman, A.

Martinez-Corral, M.

Martínez-Corral, M.

Martínez-Cuenca, R.

Papageorgas, P. G.

Park, H.

Park, J.-H.

Park, M.-K.

Piao, M.

Piao, Y.

Y. Piao, D.-H. Shin, and E.-S. Kim, “Robust image encryption by combined use of integral imaging and pixel scrambling techniques,” Opt. Lasers Eng. 47(11), 1273–1281 (2009).
[Crossref]

Pons, A.

Saavedra, G.

Sangriotis, M. S.

Ser, J.-I.

Sgouros, N. P.

Shin, D.

J.-Y. Jang, D. Shin, and E.-S. Kim, “Improved 3-D image reconstruction using the convolution property of periodic functions in curved integral-imaging,” Opt. Lasers Eng. 54, 14–20 (2014).
[Crossref]

J.-Y. Jang, D. Shin, and E.-S. Kim, “Optical three-dimensional refocusing from elemental images based on a sifting property of the periodic δ-function array in integral-imaging,” Opt. Express 22(2), 1533–1550 (2014).
[Crossref]

Shin, D.-H.

Shin, S.-H.

Stern, A.

Theofanous, N. G.

Wang, S.

Wei, J.

Xiao, X.

Yoo, H.

Zhao, Y.

Appl. Opt. (4)

J. Disp. Technol. (1)

A. Aggoun, “Pre-processing of integral images for 3-D displays,” J. Disp. Technol. 2(4), 393–400 (2006).
[Crossref]

Opt. Express (11)

N. P. Sgouros, S. S. Athineos, M. S. Sangriotis, P. G. Papageorgas, and N. G. Theofanous, “Accurate lattice extraction in integral images,” Opt. Express 14(22), 10403–10409 (2006).
[Crossref]

J.-J. Lee, D.-H. Shin, and B.-G. Lee, “Simple correction method of distorted elemental images using surface markers on lenslet array for computational integral imaging reconstruction,” Opt. Express 17(20), 18026–18037 (2009).
[Crossref]

K. Hong, J. Hong, J.-H. Jung, J.-H. Park, and B. Lee, “Rectification of elemental image set and extraction of lens lattice by projective image transformation in integral imaging,” Opt. Express 18(11), 12002–12016 (2010).
[Crossref]

K.-C. Kwon, M.-U. Erdenebat, Y.-T. Lim, K.-I. Joo, M.-K. Park, H. Park, J.-R. Jeong, H.-R. Kim, and N. Kim, “Enhancement of the depth-of-field of integral imaging microscope by using switchable bifocal liquid-crystalline polymer micro lens array,” Opt. Express 25(24), 30503–30512 (2017).
[Crossref]

H.-S. Kim, K.-M. Jeong, S.-I. Hong, N.-Y. Jo, and J.-H. Park, “Analysis of image distortion based on light ray field by multi-view and horizontal parallax only integral imaging display,” Opt. Express 20(21), 23755–23768 (2012).
[Crossref]

J. Wei, S. Wang, Y. Zhao, and M. Piao, “Synthetic aperture integral imaging using edge depth maps of unstructured monocular video,” Opt. Express 26(26), 34894–34908 (2018).
[Crossref]

J.-Y. Jang, D. Shin, and E.-S. Kim, “Optical three-dimensional refocusing from elemental images based on a sifting property of the periodic δ-function array in integral-imaging,” Opt. Express 22(2), 1533–1550 (2014).
[Crossref]

H. Yoo, “Axially moving a lenslet array for high-resolution 3D images in computational integral imaging,” Opt. Express 21(7), 8873–8878 (2013).
[Crossref]

J.-H. Park, J. Kim, Y. Kim, and B. Lee, “Resolution-enhanced three-dimension/two-dimension convertible display based on integral imaging,” Opt. Express 13(6), 1875–1884 (2005).
[Crossref]

D.-H. Shin and H. Yoo, “Image quality enhancement in 3D computational integral imaging by use of interpolation methods,” Opt. Express 15(19), 12039–12049 (2007).
[Crossref]

D.-H. Shin, B.-G. Lee, and J.-J. Lee, “Occlusion removal method of partially occluded 3D object using sub-image block matching in computational integral imaging,” Opt. Express 16(21), 16294–16304 (2008).
[Crossref]

Opt. Lasers Eng. (3)

Y. Piao, D.-H. Shin, and E.-S. Kim, “Robust image encryption by combined use of integral imaging and pixel scrambling techniques,” Opt. Lasers Eng. 47(11), 1273–1281 (2009).
[Crossref]

J.-Y. Jang, D. Shin, and E.-S. Kim, “Improved 3-D image reconstruction using the convolution property of periodic functions in curved integral-imaging,” Opt. Lasers Eng. 54, 14–20 (2014).
[Crossref]

H. Yoo and J.-Y. Jang, “Intermediate elemental image reconstruction for refocused three-dimensional images in integral imaging by convolution with δ-function sequences,” Opt. Lasers Eng. 97, 93–99 (2017).
[Crossref]

Opt. Lett. (3)

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (10)

Fig. 1.
Fig. 1. PIA capturing systems and region of parallax images according to the characteristics of each system. (a) Camera array and its parallax images. (b) Lens array and its parallax images. (c) Diffraction grating and imaging points on the pickup plane.
Fig. 2.
Fig. 2. The proposed diffraction grating imaging system consisting of PIA pickup and computational reconstruction processes, where PI stands for parallax image, VP for virtual pinhole, EOA for the effective object area, and MIA for the minimum image area.
Fig. 3.
Fig. 3. Geometrical relationship in diffraction grating imaging among a point object, parallax images (PIs), and picked up PIs.
Fig. 4.
Fig. 4. Geometrical relationship between virtual rays from PIs and real rays from a point object.
Fig. 5.
Fig. 5. Geometrical relationship among a point object, virtual pinhole (VP), virtual image (VI) plane, and I(x1st, yO, zO).
Fig. 6.
Fig. 6. (a) Geometrical relationship among EOA, virtual pinholes, and MIA. (b) Illustration of the imaging formation and mapping process using a real image of a circle object.
Fig. 7.
Fig. 7. Virtual pinhole (VP) models for computational reconstruction in diffraction grating imaging.
Fig. 8.
Fig. 8. Experimental setup of PIA pickup process and observation images of object space according to change of illumination. (a) Experimental setup. (b) Indoor lighting only. (c) Both indoor and laser lighting. (d) Laser lighting only.
Fig. 9.
Fig. 9. Objects used in the process to pick up PIAs and the extracted PIAs by the proposed method via diffraction grating imaging.
Fig. 10.
Fig. 10. 3-D objects of (a) a circle, (b) geometric shapes, (c) letters of “3DS” and (d) male and female models, and their 3-D reconstructed images.

Equations (8)

Equations on this page are rendered with MathJax. Learn more.

x m t h = x O + | z O d | tan ( s i n 1 ( m λ a ) ) ,
I ( x m t h , y n t h , z O ) = ( ( z I z O ) x m t h , ( z I z O ) y n t h , z I ) ,
G ( x m t h , y n t h , z O ) = ( ( d z O ) x m t h , ( d z O ) y n t h , d ) ,
V P ( x m t h , y n t h , z O ) = ( ( x m t h x O ) d z O d , ( y n t h y O ) d z O d , 0 ) ,
x V I m t h = ( x m t h d z O x O ) z O z I z O d + x O .
Δ x m a p p i n g = | z I d z O d ( x O x m t h ) | .
Δ x max = | z O d | tan ( s i n 1 ( λ a ) ) ,
Δ r = z I z O ( d tan ( s i n 1 ( λ a ) ) + Δ x max 2 ) Δ x max .

Metrics