Abstract

A novel approach to extract the depth data of 3D objects in space by using the computational integral imaging reconstruction (CIIR) technique is proposed. With elemental images of 3D objects captured by the CCD camera through a pinhole array, depth-dependent object images can be reconstructed on the output plane by the CIIR technique. Only the images reconstructed on the output planes where 3D objects were located are clearly focused; so the depth data of 3D objects in space can be extracted by discriminating these focused output images from the others by using an image separation technique. A feasibility test of the proposed CIIR-based depth extraction method is carried out, and its results are discussed as well.

© 2008 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. J.-I. Park and S. Inoue, “Acquisition of sharp depth map from multiple cameras,” Signal Process. 14, 7-19 (1998).
  2. J.-H. Ko, D.-C. Hwang, Y.-W. Jung, and E.-S. Kim, “Intelligent mobile robot system for path planning using stereo camera-based geometry information.” Proc. SPIE 6006, 60060L(2005).
    [CrossRef]
  3. Y. Oike, M. Ikeda, and K. Asada, “A 120×110 position sensor with the capability of sensitive and selective light detection in wide dynamic range for robust active range finding,” IEEE J Solid-State Circuits 39, 246-251 (2004).
    [CrossRef]
  4. M. Kawakita, K. Iizuka, T. Aida, H. Kikuchi, H. Fujikake, J. Yonai, and K. Takizawa, “Axi-vision camera (real-time distance-mapping camera),” Appl. Opt. 39, 3931-3939 (2000).
    [CrossRef]
  5. G. J. Iddan and G. Yahav, “Three-dimensional imaging in the studio and elsewhere,” Proc. SPIE . 4298, 48-55 (2000).
    [CrossRef]
  6. J.-H. Lee, J.-H. Ko, K.-J. Lee, J.-H. Jang, and E.-S. Kim, “Implementation of stereo camera-based automatic unmanned ground vehicle system for adaptive target detection,” Proc. SPIE . 5608, 188-197 (2004).
    [CrossRef]
  7. J.-H. Park, S. Jung, H. Choi, Y. Kim, and B. Lee, “Depth extraction by use of a rectangular lens array and one-dimensional elemental image modification,” Appl. Opt. 43, 4882-4895(2004).
    [CrossRef] [PubMed]
  8. J.-H. Park, Y. Kim, J. Kim, S.-W. Min, and B. Lee, “Three-dimensional display scheme based on integral imaging with three-dimensional information processing,” Opt. Express 12, 6020-6032 (2004).
    [CrossRef] [PubMed]
  9. G. Passalis, N. Sgouros, S. Athineos, and T. Theohari, “Enhanced reconstruction of three-dimensional shape and texture from integral photography images” Appl. Opt. 46, 5311-5320 (2007).
    [CrossRef] [PubMed]
  10. D.-C. Hwang, D.-H. Shin, and E.-S. Kim, “Extraction of depth cue of a 3-D object using a computational integral imaging reconstruction scheme,” in Adaptive Optics: Analysis and Methods/Computational Optical Sensing and Imaging/Information Photonics/Signal Recovery and Synthesis Topical Meetings on CD-ROM, OSA Technical Digest (CD) (Optical Society of America, 2007), paper DTuB8.
  11. J.-S. Jang and B. Javidi, “Improved viewing resolution of three-dimensional integral imaging with nonstationary micro-optics,” Opt. Lett. 27, 324-326 (2002).
    [CrossRef]
  12. S.-W. Min, B. Javidi, and B. Lee, “Enhanced three-dimensional integral imaging system by use of double display devices,” Appl. Opt. 42, 4186-4195 (2003).
    [CrossRef] [PubMed]
  13. H. Arimoto and B. Javidi, “Integral three-dimensional imaging with digital reconstruction,” Opt. Lett. 26, 157-159(2001).
    [CrossRef]
  14. S. Hong, J.-S. Jang, and B. Javidi, “Three-dimensional volumetric object reconstruction using computational integral imaging,” Opt. Express 12, 483-491 (2004).
    [CrossRef] [PubMed]
  15. D.-H. Shin, E.-S. Kim, and B. Lee, “Computational reconstruction technique of three-dimensional object in integral imaging using a lenslet array,” Jpn. J. Appl. Phys. 44, 8016-8018(2005).
    [CrossRef]
  16. D.-H. Shin, M. Cho, K.-C. Park, and E.-S. Kim, “Computational technique of volumetric object reconstruction in integral imaging by use of real and virtual image fields,” ETRI J. 27, 208-712 (2005).
    [CrossRef]
  17. S.-H. Hong and B. Javidi, “Improved resolution 3D object reconstruction using computational integral imaging with time multiplexing,” Opt. Express 12, 4579-4588(2004).
    [CrossRef] [PubMed]
  18. H. Yoo and D.-H. Shin, “Improved analysis on the signal property of computational integral imaging system,” Opt. Express 15, 14107-14114 (2007).
    [CrossRef] [PubMed]
  19. S. Yeom, B. Javidi, C.-w. Lee, and E. Watson, “Photon-counting passive 3D image sensing for reconstruction and recognition of partially occluded objects,” Opt. Express 15, 16189-16195(2007).
    [CrossRef] [PubMed]
  20. Y. Y. Schechner, N. Kiryati, and R. Basri, “Separation of transparent layers using focus,” EE-PUB-1086 (Technion-Israel Institute of Technology, 1997).
  21. Y. S. Hwang, S.-H. Hong, and B. Javidi, “free view 3-D visualization of occluded objects by using computational synthetic aperture integral imaging,” J. Disp. Technol. 3, 64-70(2007).
    [CrossRef]

2007 (4)

2005 (3)

D.-H. Shin, E.-S. Kim, and B. Lee, “Computational reconstruction technique of three-dimensional object in integral imaging using a lenslet array,” Jpn. J. Appl. Phys. 44, 8016-8018(2005).
[CrossRef]

D.-H. Shin, M. Cho, K.-C. Park, and E.-S. Kim, “Computational technique of volumetric object reconstruction in integral imaging by use of real and virtual image fields,” ETRI J. 27, 208-712 (2005).
[CrossRef]

J.-H. Ko, D.-C. Hwang, Y.-W. Jung, and E.-S. Kim, “Intelligent mobile robot system for path planning using stereo camera-based geometry information.” Proc. SPIE 6006, 60060L(2005).
[CrossRef]

2004 (6)

2003 (1)

2002 (1)

2001 (1)

2000 (2)

1998 (1)

J.-I. Park and S. Inoue, “Acquisition of sharp depth map from multiple cameras,” Signal Process. 14, 7-19 (1998).

Aida, T.

Arimoto, H.

Asada, K.

Y. Oike, M. Ikeda, and K. Asada, “A 120×110 position sensor with the capability of sensitive and selective light detection in wide dynamic range for robust active range finding,” IEEE J Solid-State Circuits 39, 246-251 (2004).
[CrossRef]

Athineos, S.

Basri, R.

Y. Y. Schechner, N. Kiryati, and R. Basri, “Separation of transparent layers using focus,” EE-PUB-1086 (Technion-Israel Institute of Technology, 1997).

Cho, M.

D.-H. Shin, M. Cho, K.-C. Park, and E.-S. Kim, “Computational technique of volumetric object reconstruction in integral imaging by use of real and virtual image fields,” ETRI J. 27, 208-712 (2005).
[CrossRef]

Choi, H.

Fujikake, H.

Hong, S.

Hong, S.-H.

Y. S. Hwang, S.-H. Hong, and B. Javidi, “free view 3-D visualization of occluded objects by using computational synthetic aperture integral imaging,” J. Disp. Technol. 3, 64-70(2007).
[CrossRef]

S.-H. Hong and B. Javidi, “Improved resolution 3D object reconstruction using computational integral imaging with time multiplexing,” Opt. Express 12, 4579-4588(2004).
[CrossRef] [PubMed]

Hwang, D.-C.

J.-H. Ko, D.-C. Hwang, Y.-W. Jung, and E.-S. Kim, “Intelligent mobile robot system for path planning using stereo camera-based geometry information.” Proc. SPIE 6006, 60060L(2005).
[CrossRef]

D.-C. Hwang, D.-H. Shin, and E.-S. Kim, “Extraction of depth cue of a 3-D object using a computational integral imaging reconstruction scheme,” in Adaptive Optics: Analysis and Methods/Computational Optical Sensing and Imaging/Information Photonics/Signal Recovery and Synthesis Topical Meetings on CD-ROM, OSA Technical Digest (CD) (Optical Society of America, 2007), paper DTuB8.

Hwang, Y. S.

Y. S. Hwang, S.-H. Hong, and B. Javidi, “free view 3-D visualization of occluded objects by using computational synthetic aperture integral imaging,” J. Disp. Technol. 3, 64-70(2007).
[CrossRef]

Iddan, G. J.

G. J. Iddan and G. Yahav, “Three-dimensional imaging in the studio and elsewhere,” Proc. SPIE . 4298, 48-55 (2000).
[CrossRef]

Iizuka, K.

Ikeda, M.

Y. Oike, M. Ikeda, and K. Asada, “A 120×110 position sensor with the capability of sensitive and selective light detection in wide dynamic range for robust active range finding,” IEEE J Solid-State Circuits 39, 246-251 (2004).
[CrossRef]

Inoue, S.

J.-I. Park and S. Inoue, “Acquisition of sharp depth map from multiple cameras,” Signal Process. 14, 7-19 (1998).

Jang, J.-H.

J.-H. Lee, J.-H. Ko, K.-J. Lee, J.-H. Jang, and E.-S. Kim, “Implementation of stereo camera-based automatic unmanned ground vehicle system for adaptive target detection,” Proc. SPIE . 5608, 188-197 (2004).
[CrossRef]

Jang, J.-S.

Javidi, B.

Jung, S.

Jung, Y.-W.

J.-H. Ko, D.-C. Hwang, Y.-W. Jung, and E.-S. Kim, “Intelligent mobile robot system for path planning using stereo camera-based geometry information.” Proc. SPIE 6006, 60060L(2005).
[CrossRef]

Kawakita, M.

Kikuchi, H.

Kim, E.-S.

J.-H. Ko, D.-C. Hwang, Y.-W. Jung, and E.-S. Kim, “Intelligent mobile robot system for path planning using stereo camera-based geometry information.” Proc. SPIE 6006, 60060L(2005).
[CrossRef]

D.-H. Shin, E.-S. Kim, and B. Lee, “Computational reconstruction technique of three-dimensional object in integral imaging using a lenslet array,” Jpn. J. Appl. Phys. 44, 8016-8018(2005).
[CrossRef]

D.-H. Shin, M. Cho, K.-C. Park, and E.-S. Kim, “Computational technique of volumetric object reconstruction in integral imaging by use of real and virtual image fields,” ETRI J. 27, 208-712 (2005).
[CrossRef]

J.-H. Lee, J.-H. Ko, K.-J. Lee, J.-H. Jang, and E.-S. Kim, “Implementation of stereo camera-based automatic unmanned ground vehicle system for adaptive target detection,” Proc. SPIE . 5608, 188-197 (2004).
[CrossRef]

D.-C. Hwang, D.-H. Shin, and E.-S. Kim, “Extraction of depth cue of a 3-D object using a computational integral imaging reconstruction scheme,” in Adaptive Optics: Analysis and Methods/Computational Optical Sensing and Imaging/Information Photonics/Signal Recovery and Synthesis Topical Meetings on CD-ROM, OSA Technical Digest (CD) (Optical Society of America, 2007), paper DTuB8.

Kim, J.

Kim, Y.

Kiryati, N.

Y. Y. Schechner, N. Kiryati, and R. Basri, “Separation of transparent layers using focus,” EE-PUB-1086 (Technion-Israel Institute of Technology, 1997).

Ko, J.-H.

J.-H. Ko, D.-C. Hwang, Y.-W. Jung, and E.-S. Kim, “Intelligent mobile robot system for path planning using stereo camera-based geometry information.” Proc. SPIE 6006, 60060L(2005).
[CrossRef]

J.-H. Lee, J.-H. Ko, K.-J. Lee, J.-H. Jang, and E.-S. Kim, “Implementation of stereo camera-based automatic unmanned ground vehicle system for adaptive target detection,” Proc. SPIE . 5608, 188-197 (2004).
[CrossRef]

Lee, B.

Lee, C.-w.

Lee, J.-H.

J.-H. Lee, J.-H. Ko, K.-J. Lee, J.-H. Jang, and E.-S. Kim, “Implementation of stereo camera-based automatic unmanned ground vehicle system for adaptive target detection,” Proc. SPIE . 5608, 188-197 (2004).
[CrossRef]

Lee, K.-J.

J.-H. Lee, J.-H. Ko, K.-J. Lee, J.-H. Jang, and E.-S. Kim, “Implementation of stereo camera-based automatic unmanned ground vehicle system for adaptive target detection,” Proc. SPIE . 5608, 188-197 (2004).
[CrossRef]

Min, S.-W.

Oike, Y.

Y. Oike, M. Ikeda, and K. Asada, “A 120×110 position sensor with the capability of sensitive and selective light detection in wide dynamic range for robust active range finding,” IEEE J Solid-State Circuits 39, 246-251 (2004).
[CrossRef]

Park, J.-H.

Park, J.-I.

J.-I. Park and S. Inoue, “Acquisition of sharp depth map from multiple cameras,” Signal Process. 14, 7-19 (1998).

Park, K.-C.

D.-H. Shin, M. Cho, K.-C. Park, and E.-S. Kim, “Computational technique of volumetric object reconstruction in integral imaging by use of real and virtual image fields,” ETRI J. 27, 208-712 (2005).
[CrossRef]

Passalis, G.

Schechner, Y. Y.

Y. Y. Schechner, N. Kiryati, and R. Basri, “Separation of transparent layers using focus,” EE-PUB-1086 (Technion-Israel Institute of Technology, 1997).

Sgouros, N.

Shin, D.-H.

H. Yoo and D.-H. Shin, “Improved analysis on the signal property of computational integral imaging system,” Opt. Express 15, 14107-14114 (2007).
[CrossRef] [PubMed]

D.-H. Shin, M. Cho, K.-C. Park, and E.-S. Kim, “Computational technique of volumetric object reconstruction in integral imaging by use of real and virtual image fields,” ETRI J. 27, 208-712 (2005).
[CrossRef]

D.-H. Shin, E.-S. Kim, and B. Lee, “Computational reconstruction technique of three-dimensional object in integral imaging using a lenslet array,” Jpn. J. Appl. Phys. 44, 8016-8018(2005).
[CrossRef]

D.-C. Hwang, D.-H. Shin, and E.-S. Kim, “Extraction of depth cue of a 3-D object using a computational integral imaging reconstruction scheme,” in Adaptive Optics: Analysis and Methods/Computational Optical Sensing and Imaging/Information Photonics/Signal Recovery and Synthesis Topical Meetings on CD-ROM, OSA Technical Digest (CD) (Optical Society of America, 2007), paper DTuB8.

Takizawa, K.

Theohari, T.

Watson, E.

Yahav, G.

G. J. Iddan and G. Yahav, “Three-dimensional imaging in the studio and elsewhere,” Proc. SPIE . 4298, 48-55 (2000).
[CrossRef]

Yeom, S.

Yonai, J.

Yoo, H.

Appl. Opt. (4)

ETRI J. (1)

D.-H. Shin, M. Cho, K.-C. Park, and E.-S. Kim, “Computational technique of volumetric object reconstruction in integral imaging by use of real and virtual image fields,” ETRI J. 27, 208-712 (2005).
[CrossRef]

IEEE J Solid-State Circuits (1)

Y. Oike, M. Ikeda, and K. Asada, “A 120×110 position sensor with the capability of sensitive and selective light detection in wide dynamic range for robust active range finding,” IEEE J Solid-State Circuits 39, 246-251 (2004).
[CrossRef]

J. Disp. Technol. (1)

Y. S. Hwang, S.-H. Hong, and B. Javidi, “free view 3-D visualization of occluded objects by using computational synthetic aperture integral imaging,” J. Disp. Technol. 3, 64-70(2007).
[CrossRef]

Jpn. J. Appl. Phys. (1)

D.-H. Shin, E.-S. Kim, and B. Lee, “Computational reconstruction technique of three-dimensional object in integral imaging using a lenslet array,” Jpn. J. Appl. Phys. 44, 8016-8018(2005).
[CrossRef]

Opt. Express (5)

Opt. Lett. (2)

Proc. SPIE (3)

G. J. Iddan and G. Yahav, “Three-dimensional imaging in the studio and elsewhere,” Proc. SPIE . 4298, 48-55 (2000).
[CrossRef]

J.-H. Lee, J.-H. Ko, K.-J. Lee, J.-H. Jang, and E.-S. Kim, “Implementation of stereo camera-based automatic unmanned ground vehicle system for adaptive target detection,” Proc. SPIE . 5608, 188-197 (2004).
[CrossRef]

J.-H. Ko, D.-C. Hwang, Y.-W. Jung, and E.-S. Kim, “Intelligent mobile robot system for path planning using stereo camera-based geometry information.” Proc. SPIE 6006, 60060L(2005).
[CrossRef]

Signal Process. (1)

J.-I. Park and S. Inoue, “Acquisition of sharp depth map from multiple cameras,” Signal Process. 14, 7-19 (1998).

Other (2)

Y. Y. Schechner, N. Kiryati, and R. Basri, “Separation of transparent layers using focus,” EE-PUB-1086 (Technion-Israel Institute of Technology, 1997).

D.-C. Hwang, D.-H. Shin, and E.-S. Kim, “Extraction of depth cue of a 3-D object using a computational integral imaging reconstruction scheme,” in Adaptive Optics: Analysis and Methods/Computational Optical Sensing and Imaging/Information Photonics/Signal Recovery and Synthesis Topical Meetings on CD-ROM, OSA Technical Digest (CD) (Optical Society of America, 2007), paper DTuB8.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (14)

Fig. 1
Fig. 1

Conceptual diagram of (a) pickup and (b) display processes in the InIm system.

Fig. 2
Fig. 2

Operational principle of the CIIR method based on a pinhole array model.

Fig. 3
Fig. 3

Block diagram of the proposed CIIR-based depth extraction scheme.

Fig. 4
Fig. 4

Computational pickup process for the ( i , j ) th elemental image.

Fig. 5
Fig. 5

Operational principle of the CIIR algorithm.

Fig. 6
Fig. 6

Depth-dependent object plane images reconstructed with the CIIR algorithm.

Fig. 7
Fig. 7

Structure of the computational pickup and its synthesized elemental images: (a) 3D test object composed of three 2D alphabetical patterns; (b) computationally picked-up elemental images.

Fig. 8
Fig. 8

Reconstructed plane images of the objects along the longitudinal direction.

Fig. 9
Fig. 9

Focused images discriminated from the reconstructed plane images through the image separation process.

Fig. 10
Fig. 10

Extracted depth map of three alphabetical objects, “K” “W,” “U.”

Fig. 11
Fig. 11

Experimental setup for optical pickup of the elemental images.

Fig. 12
Fig. 12

Reconstructed plane images of the real objects along the longitudinal direction.

Fig. 13
Fig. 13

Focused images discriminated from the reconstructed plane images through the image separation process.

Fig. 14
Fig. 14

Extracted depth map of real objects.

Equations (10)

Equations on this page are rendered with MathJax. Learn more.

E i j ( x , y ) = O ( x g z o + i p , y g z o + j p , z o ) ,
P ( x , y , z l ) = i = 1 M j = 1 N E i j ( x z l g + i p , y z l g + j p ) .
r α = f α + f β * h ,
r β = f β + f α * h ,
R α = F α + F β H ,
R β = F β + F α H .
F ^ α = B ( R α R β H ) ,
F ^ β = B ( R β R α H ) ,
f ^ α = r α r β * h .
D map = q = 1 Q w ( q ) × f ^ q ,

Metrics