Abstract

In a three-dimensional display scheme based on integral imaging, the mismatch of the system parameters between the pickup and display systems or between the display systems is an important issue from a practical point of view. In this paper, we propose a method that provides excellent flexibility to the integral imaging system parameters and display conditions. In the proposed method, elemental images obtained in the pickup process are digitally analyzed and full three-dimensional information of the object is extracted. The extracted three-dimensional information is then transmitted to each display system and modified, so as to be suitable for the display conditions and the system parameters of the display system. Finally elemental images are generated with the modified three-dimensional information for the display system and integrated in the form of a three-dimensional image.

© 2004 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |

  1. G. Lippmann, �??La photographie integrale,�?? C. R. Acad. Sci. 146, 446-451 (1908).
  2. S. Manolache, A. Aggoun, M. McCormick, N. Davies, and S. Y. Kung, �??Analytical model of a three-dimensional integral image recording system that uses circular and hexagonal-based spherical surface microlenses,�?? J. Opt. Soc. Am. A. 18, 1814-1821 (2001).
    [CrossRef]
  3. B. Lee, S. Jung, and J. -H. Park, �??Viewing-angle-enhanced integral imaging using lens switching,�?? Opt. Lett. 27, 818-820 (2002).
    [CrossRef]
  4. T. Naemura, T. Yoshida, and H. Harashima, �??3-D computer graphics based on integral photography,�?? Opt. Express 8, 255-262 (2001).
    [CrossRef] [PubMed]
  5. L. Erdmann and K. J. Gabriel, �??High-resolution digital integral photography by use of a scanning microlens array,�?? Appl. Opt. 40, 5592-5599 (2001).
    [CrossRef]
  6. H. Choi, S.-W. Min, S. Jung, J.-H. Park, and B. Lee, "Multiple-viewing-zone integral imaging using a dynamic barrier array for three-dimensional displays," Opt. Express 11, 927-932 (2003), <a href= "http://www.opticsexpress.org/abstract.cfm?URI=OPEX-11-8-927">http://www.opticsexpress.org/abstract.cfm?URI=OPEX-11-8-927</a>.
    [CrossRef] [PubMed]
  7. S. Jung, J.-H. Park, H. Choi, and B. Lee, "Viewing-angle-enhanced integral three-dimensional imaging along all directions without mechanical movement," Opt. Express 11, 1346-1356 (2003), <a href= "http://www.opticsexpress.org/abstract.cfm?URI=OPEX-11-12-1346">http://www.opticsexpress.org/abstract.cfm?URI=OPEX-11-12-1346</a>.
    [CrossRef] [PubMed]
  8. J.-H. Park, S. Jung, H. Choi, and B. Lee, "Integral imaging with multiple image planes using a uniaxial crystal plate," Opt. Express 11, 1862-1875 (2003), <a href= "http://www.opticsexpress.org/abstract.cfm?URI=OPEX-11-16-1862">http://www.opticsexpress.org/abstract.cfm?URI=OPEX-11-16-1862</a>.
    [CrossRef] [PubMed]
  9. S.-H. Shin and B. Javidi, �??Speckle reduced three-dimensional volume holographic display using integral imaging,�?? Appl. Opt. 41, 2644�??2649 (2002).
    [CrossRef] [PubMed]
  10. H. Arimoto and B. Javidi, �??Integral three-dimensional imaging with digital reconstruction,�?? Opt. Lett. 26, 157-159 (2001).
    [CrossRef]
  11. Y. Frauel and B. Javidi, �??Digital three-dimensional image correlation by use of computer-reconstructed integral imaging,�?? Appl. Opt. 41, 5488-5496 (2002).
    [CrossRef] [PubMed]
  12. S. Kishk and B. Javidi, "Improved resolution 3D object sensing and recognition using time multiplexed computational integral imaging," Opt. Express 11, 3528-3541 (2003), <a href= "http://www.opticsexpress.org/abstract.cfm?URI=OPEX-11-26-3528">http://www.opticsexpress.org/abstract.cfm?URI=OPEX-11-26-3528</a>.
    [CrossRef] [PubMed]
  13. S.-H. Hong, J.-S. Jang, and B. Javidi, "Three-dimensional volumetric object reconstruction using computational integral imaging," Opt. Express 12, 483-491 (2004), <a href= "http://www.opticsexpress.org/abstract.cfm?URI=OPEX-12-3-483">http://www.opticsexpress.org/abstract.cfm?URI=OPEX-12-3-483</a>.
    [CrossRef] [PubMed]
  14. R. Martínez-Cuenca, G. Saavedra, M. Martínez-Corral, and B. Javidi, "Enhanced depth of field integral imaging with sensor resolution constraints," Opt. Express 12, 5234-5241 (2004), <a href= "http://www.opticsexpress.org/abstract.cfm?URI=OPEX-12-21-5234">http://www.opticsexpress.org/abstract.cfm?URI=OPEX-12-21-5234</a>.
    [CrossRef]
  15. T. Okoshi, �??Optimum design and depth resolution of lens-sheet and projection-type three-dimensional displays,�?? Appl. Opt. 10, 2284-2291 (1971).
    [CrossRef] [PubMed]
  16. B. Lee, J.-H. Park, and H. Choi, "Scaling of three-dimensional integral imaging," in Optical Information Systems, B. Javidi and D. Psaltis, eds., Proc. SPIE 5202, 60-67 (2003).
  17. J.-H. Park, S.-W. Min, S. Jung, and B. Lee, "A new stereovision scheme using a camera and a lens array," in Algorithms and Systems for Optical Information Processing V, B. Javidi and D. Psaltis, eds., Proc. SPIE 4471, 73-80 (2001).
    [CrossRef]
  18. C. Wu, A. Aggoun, M. McCormick, and S.Y. Kung, �??Depth extraction from unidirectional image using a modified multi-baseline technique,�?? in Stereoscopic Displays and Virtual Reality Systems IX, A. J. Woods, J. O. Merritt, S. A. Benton and M. T. Bolas, eds., Proc. SPIE 4660, 135-145 (2002).
  19. J.-H. Park, S. Jung, H. Choi, Y. Kim, and B. Lee, "Depth extraction by use of a rectangular lens array and one-dimensional elemental image modification," Appl. Opt. 43, 4882-4895 (2004).
    [CrossRef] [PubMed]
  20. F. Okano, H. Hoshino, J. Arai, and I. Yuyama, �??Real-time pickup method for a three-dimensional image based on integral photography,�?? Appl. Opt. 36, 1598-1603 (1997).
    [CrossRef] [PubMed]
  21. S. T. Barnard and M. A. Fischler, �??Stereo vision,�?? in Encyclopedia of Artificial Intelligence, pp. 1083-1090, New York: John Wiley (1987).
  22. M. Yachida, Y. Kitamura, and M. Kimachi, �??Trinocular vision: New approach for correspondence problem,�?? Proc. ICPR, 1041-1044 (1986).
  23. M. Okutomi and T. Kanade, �??A multiple-baseline stereo,�?? IEEE Trans. Patt. Anal. Machine Intell. 15, 353-363 (1993).
    [CrossRef]
  24. J. Arai, F. Okano, H. Hoshino, and I. Yuyama, "Gradient-index lens-array method based on real-time integral photography for three-dimensional images," Appl. Opt. 37, 2034-2045 (1998).
    [CrossRef]
  25. S.-W. Min, S. Jung, J.-H. Park, and B. Lee, �??Three-dimensional display system based on computergenerated integral photography,�?? in Stereoscopic Displays and Virtual Reality Systems VIII, A. J. Woods, M. T. Bolas, J. O. Merritt and S. A. Benton, eds., Proc. SPIE 4297, 187-195 (2001).
  26. R. R. Schultz and R. L. Stevenson, �??Extraction of high-resolution frames from video sequences,�?? IEEE Trans. Image Processing 5, 996-1011 (1996).
    [CrossRef]
  27. R. C. Hardie, K. J. Barnard and E. E. Armstrong �??Joint MAP registration and high-resolution image estimation using a sequence of undersampled images,�?? IEEE Trans. Image Processing 6, 1621-1633 (1997).
    [CrossRef]
  28. J.-H. Park, Y. Kim, and B. Lee, "Elemental image generation based on integral imaging with enhanced resolution," in Information Optics and Photonics Technology, G. Mu, F. T. S. Yu and S. Jutamulia, eds., Proc. SPIE 5642, paper 5642-24 (2004).
  29. A. Stern and B. Javidi, "Three-dimensional image sensing and reconstruction with time-division multiplexed computational integral imaging," Appl. Opt. 42, 7036-7042 (2003).
    [CrossRef] [PubMed]
  30. S. B. Pollard, J. E. W. Mayhew, and J. P. Frisby, �??Pmf: A stereo correspondence algorithm using a disparity gradient limit,�?? Perception 14, 449-470 (1985).
    [CrossRef] [PubMed]

Appl. Opt.

C. R. Acad. Sci.

G. Lippmann, �??La photographie integrale,�?? C. R. Acad. Sci. 146, 446-451 (1908).

IEEE Trans. Image Processing

R. R. Schultz and R. L. Stevenson, �??Extraction of high-resolution frames from video sequences,�?? IEEE Trans. Image Processing 5, 996-1011 (1996).
[CrossRef]

R. C. Hardie, K. J. Barnard and E. E. Armstrong �??Joint MAP registration and high-resolution image estimation using a sequence of undersampled images,�?? IEEE Trans. Image Processing 6, 1621-1633 (1997).
[CrossRef]

IEEE Trans. Patt. Anal. Machine Intell.

M. Okutomi and T. Kanade, �??A multiple-baseline stereo,�?? IEEE Trans. Patt. Anal. Machine Intell. 15, 353-363 (1993).
[CrossRef]

J. Opt. Soc. Am. A

Opt. Express

H. Choi, S.-W. Min, S. Jung, J.-H. Park, and B. Lee, "Multiple-viewing-zone integral imaging using a dynamic barrier array for three-dimensional displays," Opt. Express 11, 927-932 (2003), <a href= "http://www.opticsexpress.org/abstract.cfm?URI=OPEX-11-8-927">http://www.opticsexpress.org/abstract.cfm?URI=OPEX-11-8-927</a>.
[CrossRef] [PubMed]

S. Jung, J.-H. Park, H. Choi, and B. Lee, "Viewing-angle-enhanced integral three-dimensional imaging along all directions without mechanical movement," Opt. Express 11, 1346-1356 (2003), <a href= "http://www.opticsexpress.org/abstract.cfm?URI=OPEX-11-12-1346">http://www.opticsexpress.org/abstract.cfm?URI=OPEX-11-12-1346</a>.
[CrossRef] [PubMed]

J.-H. Park, S. Jung, H. Choi, and B. Lee, "Integral imaging with multiple image planes using a uniaxial crystal plate," Opt. Express 11, 1862-1875 (2003), <a href= "http://www.opticsexpress.org/abstract.cfm?URI=OPEX-11-16-1862">http://www.opticsexpress.org/abstract.cfm?URI=OPEX-11-16-1862</a>.
[CrossRef] [PubMed]

S. Kishk and B. Javidi, "Improved resolution 3D object sensing and recognition using time multiplexed computational integral imaging," Opt. Express 11, 3528-3541 (2003), <a href= "http://www.opticsexpress.org/abstract.cfm?URI=OPEX-11-26-3528">http://www.opticsexpress.org/abstract.cfm?URI=OPEX-11-26-3528</a>.
[CrossRef] [PubMed]

S.-H. Hong, J.-S. Jang, and B. Javidi, "Three-dimensional volumetric object reconstruction using computational integral imaging," Opt. Express 12, 483-491 (2004), <a href= "http://www.opticsexpress.org/abstract.cfm?URI=OPEX-12-3-483">http://www.opticsexpress.org/abstract.cfm?URI=OPEX-12-3-483</a>.
[CrossRef] [PubMed]

R. Martínez-Cuenca, G. Saavedra, M. Martínez-Corral, and B. Javidi, "Enhanced depth of field integral imaging with sensor resolution constraints," Opt. Express 12, 5234-5241 (2004), <a href= "http://www.opticsexpress.org/abstract.cfm?URI=OPEX-12-21-5234">http://www.opticsexpress.org/abstract.cfm?URI=OPEX-12-21-5234</a>.
[CrossRef]

T. Naemura, T. Yoshida, and H. Harashima, �??3-D computer graphics based on integral photography,�?? Opt. Express 8, 255-262 (2001).
[CrossRef] [PubMed]

Opt. Lett.

Perception

S. B. Pollard, J. E. W. Mayhew, and J. P. Frisby, �??Pmf: A stereo correspondence algorithm using a disparity gradient limit,�?? Perception 14, 449-470 (1985).
[CrossRef] [PubMed]

Proc. ICPR 1986

M. Yachida, Y. Kitamura, and M. Kimachi, �??Trinocular vision: New approach for correspondence problem,�?? Proc. ICPR, 1041-1044 (1986).

Proc. SPIE

J.-H. Park, Y. Kim, and B. Lee, "Elemental image generation based on integral imaging with enhanced resolution," in Information Optics and Photonics Technology, G. Mu, F. T. S. Yu and S. Jutamulia, eds., Proc. SPIE 5642, paper 5642-24 (2004).

S.-W. Min, S. Jung, J.-H. Park, and B. Lee, �??Three-dimensional display system based on computergenerated integral photography,�?? in Stereoscopic Displays and Virtual Reality Systems VIII, A. J. Woods, M. T. Bolas, J. O. Merritt and S. A. Benton, eds., Proc. SPIE 4297, 187-195 (2001).

B. Lee, J.-H. Park, and H. Choi, "Scaling of three-dimensional integral imaging," in Optical Information Systems, B. Javidi and D. Psaltis, eds., Proc. SPIE 5202, 60-67 (2003).

J.-H. Park, S.-W. Min, S. Jung, and B. Lee, "A new stereovision scheme using a camera and a lens array," in Algorithms and Systems for Optical Information Processing V, B. Javidi and D. Psaltis, eds., Proc. SPIE 4471, 73-80 (2001).
[CrossRef]

C. Wu, A. Aggoun, M. McCormick, and S.Y. Kung, �??Depth extraction from unidirectional image using a modified multi-baseline technique,�?? in Stereoscopic Displays and Virtual Reality Systems IX, A. J. Woods, J. O. Merritt, S. A. Benton and M. T. Bolas, eds., Proc. SPIE 4660, 135-145 (2002).

Other

S. T. Barnard and M. A. Fischler, �??Stereo vision,�?? in Encyclopedia of Artificial Intelligence, pp. 1083-1090, New York: John Wiley (1987).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (13)

Fig. 1.
Fig. 1.

Concept of integral imaging and lens array mismatch (a) when the same lens arrays are used in the pickup and display (b) when different lens arrays are used in the pickup and display

Fig. 2.
Fig. 2.

Concept of the proposed method

Fig. 3.
Fig. 3.

Geometry of the integral imaging pickup part

Fig. 4.
Fig. 4.

Disparity of the elemental images

Fig. 5.
Fig. 5.

Concept of the pseudoscopic image problem

Fig. 6.
Fig. 6.

Conventional method for overcoming the pseudoscopic image problem (a) simplified pickup system (b) display as a virtual image

Fig. 7.
Fig. 7.

The portion of the elemental images captured by CCD

Fig. 8.
Fig. 8.

Detected disparity map (a) initial disparity map (b) regulated disparity map

Fig. 9.
Fig. 9.

3D information obtained (a) x position (b) y position (c) z position

Fig. 10.
Fig. 10.

Examples of the generated elemental images

Fig. 11.
Fig. 11.

Integrated image with depth inversion (a) integrated image (b) diffused image by a diffuser at 9.5 cm (c) diffused image by a diffuser at 13 cm

Fig. 12.
Fig. 12.

Integrated image with transverse magnification (a) diffused image by a diffuser at 9.5 cm (b) diffused image by a diffuser at 13 cm

Fig. 13.
Fig. 13.

Integrated image observed from different directions

Tables (1)

Tables Icon

Table 1. Specifications of the experimental setup

Equations (10)

Equations on this page are rendered with MathJax. Learn more.

y c , q = f a f c ( q φ y ) l z f c q φ l ,
d q 1 , q 2 = y c , q 1 y c , q 2 = ( q 2 q 1 ) f c f c φ l ( 1 z + 1 f a ) = ( q 2 q 1 ) d ,
ρ 2 tan 1 ( φ l 2 ( l + f a ) f a ) .
y f , q = g p ( q φ y ) ( z c + b ) + q φ ,
y f , q = g p ( q φ y ) ( z c + b ) + q φ .
y i , q = q φ ( 1 g p ( z c + b ) g d ( z c + b ) ) + y g p ( z c + b ) g d ( z c + b ) ,
1 z c ( z c + f a ) ( z c + b ) z c ( z c f a ) ( z c + b ) = 0
z c = z c 2 f a ,
b = z c 2 f a z c b ,
y i , q = y .

Metrics