Abstract

We propose a three-dimensional (3D) imaging technique that can sense a 3D scene and computationally reconstruct it as a 3D volumetric image. Sensing of the 3D scene is carried out by obtaining elemental images optically using a pickup microlens array and a detector array. Reconstruction of volume pixels of the scene is accomplished by computationally simulating optical reconstruction according to ray optics. The entire pixels of the recorded elemental images contribute to volumetric reconstruction of the 3D scene. Image display planes at arbitrary distances from the display microlens array are computed and reconstructed by back propagating the elemental images through a computer synthesized pinhole array based on ray optics. We present experimental results of 3D image sensing and volume pixel reconstruction to test and verify the performance of the algorithm and the imaging system. The volume pixel values can be used for 3D image surface reconstruction.

© 2004 Optical Society of America

Full Article  |  PDF Article
OSA Recommended Articles
Improved resolution 3D object reconstruction using computational integral imaging with time multiplexing

Seung-Hyun Hong and Bahram Javidi
Opt. Express 12(19) 4579-4588 (2004)

Multidimensional optical sensor and imaging system

Bahram Javidi, Seung-Hyun Hong, and Osamu Matoba
Appl. Opt. 45(13) 2986-2994 (2006)

Distortion-tolerant 3D recognition of occluded objects using computational integral imaging

Seung-Hyun Hong and Bahram Javidi
Opt. Express 14(25) 12085-12095 (2006)

References

  • View by:
  • |
  • |
  • |

  1. D. H. McMahon and H. J. Caulfield, “A technique for producing wide-angle holographic displays,” Appl. Opt. 9, 91–96, (1970).
    [Crossref] [PubMed]
  2. P. Ambs, L. Bigue, R. Binet, J. Colineau, J.-C. Lehureau, and J.-P. Huignard, “3D Image Reconstruction using Electrooptic Holography,” in Proceedings of The 16th Annual Meeting of the IEEE Lasers and Electro-Optics Society, LEOS 20031, pp. 172–173, October (2003).
  3. I. Yamaguchi and T. Zhang, “Phase-shifting digital holography,” Opt. Lett. 22, 1268–1270 (1997).
    [Crossref] [PubMed]
  4. J. W. Goodman, Introduction to Fourier Optics (McGraw-Hill, New York, NY, 1996).
  5. G. Lippmann, “La photographic intergrale,” C. R. Acad. Sci. 146, 446–451 (1908).
  6. T. Okoshi, Three-dimensional imaging techniques (Academic Press, New York, 1976).
  7. H. E. Ives, “Optical properties of a Lipmann lenticulated sheet,” J. Opt. Soc. Am. 21, 171–176 (1931).
    [Crossref]
  8. C. B. Burckhardt, “Optimum parameters and resolution limitation of integral photography,” J. Opt. Soc. Am. 58, 71–76 (1968).
    [Crossref]
  9. H. Arimoto and B. Javidi, “Integral three-dimensional imaging with digital reconstruction,” Opt. Lett. 26, 157–159 (2001).
    [Crossref]
  10. T. Okoshi, “Optimum design and depth resolution of lens sheet and projection type three dimensional displays,” Appl. Opt. 10, 2284–2291 (1971).
    [Crossref] [PubMed]
  11. J.-S. Jang and B. Javidi, “Improved viewing resolution of three-dimensional integral imaging by use of nonstationary micro-optics,” Opt. Lett. 27, 324–326 (2002).
    [Crossref]
  12. J.-S. Jang and B. Javidi, “Formation of orthoscopic three-dimensional real images in direct pickup one-step integral imaging,” Opt. Eng. 42, 1869–1870 (2003).
    [Crossref]
  13. Y. Frauel and B. Javidi, “Digital three-dimensional image correlation by use of computer-reconstructed integral imaging,” Appl. Opt. 41, 5488–5496 (2002).
    [Crossref] [PubMed]
  14. J. Arai, F. Okano, H. Hoshino, and I. Yuyama, “Gradient-index lens-array method based on real time integral photography for three-dimensional Images,” Appl. Opt. 37, 2034–2045 (1998).
    [Crossref]
  15. H. Hoshino, F. Okano, H. Isono, and I. Yuyama, “Analysis of resolution limitation of integral photography,” J. Opt. Soc. Am. A 15, 2059–2065 (1998)
    [Crossref]
  16. F. Jin, J.-S. Jang, and B. Javidi, “Effect of device resolution on three-dimensional integral imaging,” submitted to Opt. Lett..
  17. D. L. Marks and D. J. Brady, “Three-dimensional source reconstruction with a scanned pinhole camera,” Opt. Lett. 23, 820–822 (1998)
    [Crossref]
  18. J. W. V. Gissen, M. A Viergever, and C. N. D. Graff, “Improved tomographic reconstruction in seven-pinhole imaging,” IEEE Trans. Med. Imag. MI-4, 91–103 (1985)
    [Crossref]
  19. L. T. Chang, B. Macdonald, and V. Perez-Mendez, “Axial tomography and three dimensional image reconstruction,” IEEE Trans. Nucl. Sci. NS-23, 568–572 (1976)
    [Crossref]
  20. L. I. Yin and S. M. Seltzer, “Tomographic decoding algorithm for a nonoverlapping redundant array,” Appl. Opt. 32, 3726–3735 (1993).
    [Crossref] [PubMed]

2003 (1)

J.-S. Jang and B. Javidi, “Formation of orthoscopic three-dimensional real images in direct pickup one-step integral imaging,” Opt. Eng. 42, 1869–1870 (2003).
[Crossref]

2002 (2)

2001 (1)

1998 (3)

1997 (1)

1993 (1)

1985 (1)

J. W. V. Gissen, M. A Viergever, and C. N. D. Graff, “Improved tomographic reconstruction in seven-pinhole imaging,” IEEE Trans. Med. Imag. MI-4, 91–103 (1985)
[Crossref]

1976 (1)

L. T. Chang, B. Macdonald, and V. Perez-Mendez, “Axial tomography and three dimensional image reconstruction,” IEEE Trans. Nucl. Sci. NS-23, 568–572 (1976)
[Crossref]

1971 (1)

1970 (1)

1968 (1)

1931 (1)

1908 (1)

G. Lippmann, “La photographic intergrale,” C. R. Acad. Sci. 146, 446–451 (1908).

Ambs, P.

P. Ambs, L. Bigue, R. Binet, J. Colineau, J.-C. Lehureau, and J.-P. Huignard, “3D Image Reconstruction using Electrooptic Holography,” in Proceedings of The 16th Annual Meeting of the IEEE Lasers and Electro-Optics Society, LEOS 20031, pp. 172–173, October (2003).

Arai, J.

Arimoto, H.

Bigue, L.

P. Ambs, L. Bigue, R. Binet, J. Colineau, J.-C. Lehureau, and J.-P. Huignard, “3D Image Reconstruction using Electrooptic Holography,” in Proceedings of The 16th Annual Meeting of the IEEE Lasers and Electro-Optics Society, LEOS 20031, pp. 172–173, October (2003).

Binet, R.

P. Ambs, L. Bigue, R. Binet, J. Colineau, J.-C. Lehureau, and J.-P. Huignard, “3D Image Reconstruction using Electrooptic Holography,” in Proceedings of The 16th Annual Meeting of the IEEE Lasers and Electro-Optics Society, LEOS 20031, pp. 172–173, October (2003).

Brady, D. J.

Burckhardt, C. B.

Caulfield, H. J.

Chang, L. T.

L. T. Chang, B. Macdonald, and V. Perez-Mendez, “Axial tomography and three dimensional image reconstruction,” IEEE Trans. Nucl. Sci. NS-23, 568–572 (1976)
[Crossref]

Colineau, J.

P. Ambs, L. Bigue, R. Binet, J. Colineau, J.-C. Lehureau, and J.-P. Huignard, “3D Image Reconstruction using Electrooptic Holography,” in Proceedings of The 16th Annual Meeting of the IEEE Lasers and Electro-Optics Society, LEOS 20031, pp. 172–173, October (2003).

Frauel, Y.

Gissen, J. W. V.

J. W. V. Gissen, M. A Viergever, and C. N. D. Graff, “Improved tomographic reconstruction in seven-pinhole imaging,” IEEE Trans. Med. Imag. MI-4, 91–103 (1985)
[Crossref]

Goodman, J. W.

J. W. Goodman, Introduction to Fourier Optics (McGraw-Hill, New York, NY, 1996).

Graff, C. N. D.

J. W. V. Gissen, M. A Viergever, and C. N. D. Graff, “Improved tomographic reconstruction in seven-pinhole imaging,” IEEE Trans. Med. Imag. MI-4, 91–103 (1985)
[Crossref]

Hoshino, H.

Huignard, J.-P.

P. Ambs, L. Bigue, R. Binet, J. Colineau, J.-C. Lehureau, and J.-P. Huignard, “3D Image Reconstruction using Electrooptic Holography,” in Proceedings of The 16th Annual Meeting of the IEEE Lasers and Electro-Optics Society, LEOS 20031, pp. 172–173, October (2003).

Isono, H.

Ives, H. E.

Jang, J.-S.

J.-S. Jang and B. Javidi, “Formation of orthoscopic three-dimensional real images in direct pickup one-step integral imaging,” Opt. Eng. 42, 1869–1870 (2003).
[Crossref]

J.-S. Jang and B. Javidi, “Improved viewing resolution of three-dimensional integral imaging by use of nonstationary micro-optics,” Opt. Lett. 27, 324–326 (2002).
[Crossref]

F. Jin, J.-S. Jang, and B. Javidi, “Effect of device resolution on three-dimensional integral imaging,” submitted to Opt. Lett..

Javidi, B.

Jin, F.

F. Jin, J.-S. Jang, and B. Javidi, “Effect of device resolution on three-dimensional integral imaging,” submitted to Opt. Lett..

Lehureau, J.-C.

P. Ambs, L. Bigue, R. Binet, J. Colineau, J.-C. Lehureau, and J.-P. Huignard, “3D Image Reconstruction using Electrooptic Holography,” in Proceedings of The 16th Annual Meeting of the IEEE Lasers and Electro-Optics Society, LEOS 20031, pp. 172–173, October (2003).

Lippmann, G.

G. Lippmann, “La photographic intergrale,” C. R. Acad. Sci. 146, 446–451 (1908).

Macdonald, B.

L. T. Chang, B. Macdonald, and V. Perez-Mendez, “Axial tomography and three dimensional image reconstruction,” IEEE Trans. Nucl. Sci. NS-23, 568–572 (1976)
[Crossref]

Marks, D. L.

McMahon, D. H.

Okano, F.

Okoshi, T.

Perez-Mendez, V.

L. T. Chang, B. Macdonald, and V. Perez-Mendez, “Axial tomography and three dimensional image reconstruction,” IEEE Trans. Nucl. Sci. NS-23, 568–572 (1976)
[Crossref]

Seltzer, S. M.

Viergever, M. A

J. W. V. Gissen, M. A Viergever, and C. N. D. Graff, “Improved tomographic reconstruction in seven-pinhole imaging,” IEEE Trans. Med. Imag. MI-4, 91–103 (1985)
[Crossref]

Yamaguchi, I.

Yin, L. I.

Yuyama, I.

Zhang, T.

Appl. Opt. (5)

C. R. Acad. Sci. (1)

G. Lippmann, “La photographic intergrale,” C. R. Acad. Sci. 146, 446–451 (1908).

IEEE Trans. Med. Imag. (1)

J. W. V. Gissen, M. A Viergever, and C. N. D. Graff, “Improved tomographic reconstruction in seven-pinhole imaging,” IEEE Trans. Med. Imag. MI-4, 91–103 (1985)
[Crossref]

IEEE Trans. Nucl. Sci. (1)

L. T. Chang, B. Macdonald, and V. Perez-Mendez, “Axial tomography and three dimensional image reconstruction,” IEEE Trans. Nucl. Sci. NS-23, 568–572 (1976)
[Crossref]

J. Opt. Soc. Am. (2)

J. Opt. Soc. Am. A (1)

Opt. Eng. (1)

J.-S. Jang and B. Javidi, “Formation of orthoscopic three-dimensional real images in direct pickup one-step integral imaging,” Opt. Eng. 42, 1869–1870 (2003).
[Crossref]

Opt. Lett. (4)

Other (4)

J. W. Goodman, Introduction to Fourier Optics (McGraw-Hill, New York, NY, 1996).

P. Ambs, L. Bigue, R. Binet, J. Colineau, J.-C. Lehureau, and J.-P. Huignard, “3D Image Reconstruction using Electrooptic Holography,” in Proceedings of The 16th Annual Meeting of the IEEE Lasers and Electro-Optics Society, LEOS 20031, pp. 172–173, October (2003).

T. Okoshi, Three-dimensional imaging techniques (Academic Press, New York, 1976).

F. Jin, J.-S. Jang, and B. Javidi, “Effect of device resolution on three-dimensional integral imaging,” submitted to Opt. Lett..

Supplementary Material (1)

» Media 1: AVI (1887 KB)     

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (6)

Fig. 1.
Fig. 1.

Pickup and reconstruction process of 3D scene using II technique. (a) Optical setup to pickup the elemental images of 3D objects. (b) Computational reconstruction and display II image. The full 3D volume image can be reconstructed. (c) Optical 3D display of the II image with the elemental images displayed on LCD. Pickup microlens array in Fig. 1(a) and display microlens array in Fig. 1(c) have the same specifications.

Fig. 2.
Fig. 2.

Computational reconstruction II process by inversely mapping of the recorded elemental images. Reconstructed image plane at distance z=L is obtained by linear superposition of each inversely mapped elemental image through computer synthesized pinhole array.

Fig. 3.
Fig. 3.

Formation of the images at reconstructed image plane by each computer synthesized pinhole array. This diagram shows only the lateral (x) axis coordinate of the reconstruction plane. The bottom elemental image is assumed to be the 1st elemental image. sx is the elemental image size along x direction. The magnification factor M is M=z/g.

Fig 4.
Fig 4.

Experimental results for 3D object reconstruction using the proposed computational reconstruction II technique. (a) Objects used in the experiments. The crossing sign and the hospital sign are 14 mm and 33.5 mm away from the pickup microlens array, respectively. (b) Some of the recorded elemental images. (c) Reconstructed image at distance of z=6 mm. (d) Reconstructed image at the distance of z=14 mm. (e) Reconstructed image at the distance of z=26 mm. (f) Reconstructed image at the distance of z=33.5 mm. (g) Reconstructed image by linear superposition of the road sign images within the depth of focus of lenslets. (h) Reconstructed image by linear superposition of the hospital sign images within the depth of focus of lenslets.

Fig. 5.
Fig. 5.

(.avi 1.84 MB) Movie of the reconstructed 3D volume imagery from the image display plane at z=3.16 mm to the image display plane at z=37 mm with increment of 0.09 mm.

Fig. 6.
Fig. 6.

Optical 3D object reconstruction using direct pickup one-step II technique [see reference 12]. Objects used in the experiments are the same as those used in the experiments in Fig. 4. (a) Reconstructed 3D pseudoscopic virtual image of the objects. (b) Reconstructed 3D orthoscopic real image of the objects.

Equations (3)

Equations on this page are rendered with MathJax. Learn more.

O pq ( x , y , z ) = I pq ( s x p ( x s x p ) M , s y q ( y s y q ) M ) ( z + g ) 2 + [ ( x s x p ) 2 + ( y s y q ) 2 ] ( 1 + 1 M ) 2 , for { s x ( p M 2 ) x s x ( p + M 2 ) s y ( q M 2 ) y s y ( p + M 2 )
O pq ( x , y , z ) = I pq ( x M + ( 1 + 1 M ) s x p , y M + ( 1 + 1 M ) s y q ) ( z + g ) 2 + [ ( x s x p ) 2 + ( y s y q ) 2 ] ( 1 + 1 M ) 2 , for { s x ( p M 2 ) x s x ( p + M 2 ) s y ( q M 2 ) y s y ( p + M 2 )
O ( x , y , z ) = p = 0 m 1 q = 0 n 1 O pq ( x , y , z ) ,

Metrics