Abstract

A method to compute high-resolution three-dimensional images based on integral imaging is presented. A sequence of integral images (IIs) is captured by means of time-division multiplexing with a moving lenslet array technique. For the acquisition of each II, the location of the lenslet array is shifted periodically within the lenslet pitch in a plane perpendicular to the optical axis. The II sequence obtained by the detector array is processed digitally with superresolution reconstruction algorithms to obtain a reconstructed image, appropriate to a viewing direction, which has a spatial resolution beyond the optical limitation.

© 2003 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. T. Okoshii, “Three-dimensional displays,” Proc. IEEE 68, 548–564 (1980).
    [CrossRef]
  2. H. Arimoto, B. Javidi, “Integral three-dimensional imaging with computed reconstruction,” Opt. Lett. 26, 157–159 (2001).
    [CrossRef]
  3. F. Okano, H. Hoshino, J. Arai, I. Yuyama, “Real-time pickup method for a three-dimensional image based on integral photography,” Appl. Opt. 36, 1598–1603 (1997).
    [CrossRef] [PubMed]
  4. J. Arai, F. Okano, H. Hoshino, I. Yuyama, “Gradient-index lens-array method based on real-time integral photography for three-dimensional images,” Appl. Opt. 37, 2034–2045 (1998).
    [CrossRef]
  5. T. Naemura, T. Yoshida, H. Harashima, “3-D computer graphics based on integral photography,” Opt. Exp. 8, 255–262 (2001), http://www.opticsexpress.org .
    [CrossRef]
  6. J. S. Jang, B. Javidi, “Improved viewing resolution of three-dimensional integral imaging by use of nonstationary micro-optics,” Opt. Lett. 27, 324–326 (2002).
    [CrossRef]
  7. J. S. Jang, B. Javidi, “Three-dimensional synthetic aperture integral imaging,” Opt. Lett. 27, 1144–1146 (2002).
    [CrossRef]
  8. Y. Frauel, B. Javidi, “Digital three-dimensional image correlation by use of computer-reconstructed integral imaging,” Appl. Opt. 41, 5488–5496 (2002).
    [CrossRef] [PubMed]
  9. C. B. Burckhardt, “Optimum parameters and resolution limitation of integral photography,” J. Opt. Soc. Am. A 58, 71–76 (1968).
    [CrossRef]
  10. T. Okoshi, “Optimum design and depth resolution of lens-sheet and projection-type three-dimensional displays,” Appl. Opt. 10, 2284–2291 (1971).
    [CrossRef] [PubMed]
  11. H. Hoshino, F. Okano, H. Isono, I. Yuyama, “Analysis of resolution limitation of integral photography,” J. Opt. Soc. Am. A 15, 2059–2065 (1998).
    [CrossRef]
  12. M. Elad, A. Feuer, “Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images,” IEEE Trans. Image Process. 6, 1646–1658 (1997).
    [CrossRef] [PubMed]
  13. A. Stern, E. Kempner, A. Shukrun, N. S. Kopeika, “Restoration and resolution enhancement of a single image from a vibration distorted image sequence,” Opt. Eng. 39, 2451–2457 (2000).
    [CrossRef]
  14. A. Stern, Y. Porat, A. Ben-Dor, N. S. Kopeika, “Enhanced-resolution image restoration from a sequence of low-frequency vibrated images by use of convex projections,” Appl. Opt. 40, 4706–4715 (2001).
    [CrossRef]
  15. A. Irani, S. Peleg, “Improving resolution by image registration,” CVGIP: Graph. Models Image Process 53, 231–239 (1991).
    [CrossRef]
  16. A. Irani, S. Peleg, “Motion analysis for image enhancement: resolution, occlusion, and transparency,” J. Visual Commun. Image Represent. 4, 324–335 (1993).
    [CrossRef]
  17. B. Cohen, I. Dinstein, “Polyphase back-projection filtering for image resolution enhancement,” IEE Proc. Vision Image Signal Process. 147, 318–322 (2000).
    [CrossRef]
  18. R. L. Lagendijk, Iterative Identification and Restoration of Images (Kluwer Academic, Dordrecht, The Netherlands, 1991).
    [CrossRef]
  19. B. E. A. Saleh, M. C. Teich, Fundamentals of Photonics (Wiley, New York, 1991).
    [CrossRef]
  20. Z. Zalevsky, D. Mendlovic, A. W. Lohmann, “Understanding superesolution in Wigner space,” J. Opt. Soc. Am. A 17, 2422–2430 (2000).
    [CrossRef]
  21. A. K. Jain, Fundamentals of Digital Image Processing (Prentice-Hall, Englewood Cliffs, N.J., 1989).

2002 (3)

2001 (3)

2000 (3)

A. Stern, E. Kempner, A. Shukrun, N. S. Kopeika, “Restoration and resolution enhancement of a single image from a vibration distorted image sequence,” Opt. Eng. 39, 2451–2457 (2000).
[CrossRef]

B. Cohen, I. Dinstein, “Polyphase back-projection filtering for image resolution enhancement,” IEE Proc. Vision Image Signal Process. 147, 318–322 (2000).
[CrossRef]

Z. Zalevsky, D. Mendlovic, A. W. Lohmann, “Understanding superesolution in Wigner space,” J. Opt. Soc. Am. A 17, 2422–2430 (2000).
[CrossRef]

1998 (2)

1997 (2)

F. Okano, H. Hoshino, J. Arai, I. Yuyama, “Real-time pickup method for a three-dimensional image based on integral photography,” Appl. Opt. 36, 1598–1603 (1997).
[CrossRef] [PubMed]

M. Elad, A. Feuer, “Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images,” IEEE Trans. Image Process. 6, 1646–1658 (1997).
[CrossRef] [PubMed]

1993 (1)

A. Irani, S. Peleg, “Motion analysis for image enhancement: resolution, occlusion, and transparency,” J. Visual Commun. Image Represent. 4, 324–335 (1993).
[CrossRef]

1991 (1)

A. Irani, S. Peleg, “Improving resolution by image registration,” CVGIP: Graph. Models Image Process 53, 231–239 (1991).
[CrossRef]

1980 (1)

T. Okoshii, “Three-dimensional displays,” Proc. IEEE 68, 548–564 (1980).
[CrossRef]

1971 (1)

1968 (1)

C. B. Burckhardt, “Optimum parameters and resolution limitation of integral photography,” J. Opt. Soc. Am. A 58, 71–76 (1968).
[CrossRef]

Arai, J.

Arimoto, H.

Ben-Dor, A.

Burckhardt, C. B.

C. B. Burckhardt, “Optimum parameters and resolution limitation of integral photography,” J. Opt. Soc. Am. A 58, 71–76 (1968).
[CrossRef]

Cohen, B.

B. Cohen, I. Dinstein, “Polyphase back-projection filtering for image resolution enhancement,” IEE Proc. Vision Image Signal Process. 147, 318–322 (2000).
[CrossRef]

Dinstein, I.

B. Cohen, I. Dinstein, “Polyphase back-projection filtering for image resolution enhancement,” IEE Proc. Vision Image Signal Process. 147, 318–322 (2000).
[CrossRef]

Elad, M.

M. Elad, A. Feuer, “Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images,” IEEE Trans. Image Process. 6, 1646–1658 (1997).
[CrossRef] [PubMed]

Feuer, A.

M. Elad, A. Feuer, “Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images,” IEEE Trans. Image Process. 6, 1646–1658 (1997).
[CrossRef] [PubMed]

Frauel, Y.

Harashima, H.

T. Naemura, T. Yoshida, H. Harashima, “3-D computer graphics based on integral photography,” Opt. Exp. 8, 255–262 (2001), http://www.opticsexpress.org .
[CrossRef]

Hoshino, H.

Irani, A.

A. Irani, S. Peleg, “Motion analysis for image enhancement: resolution, occlusion, and transparency,” J. Visual Commun. Image Represent. 4, 324–335 (1993).
[CrossRef]

A. Irani, S. Peleg, “Improving resolution by image registration,” CVGIP: Graph. Models Image Process 53, 231–239 (1991).
[CrossRef]

Isono, H.

Jain, A. K.

A. K. Jain, Fundamentals of Digital Image Processing (Prentice-Hall, Englewood Cliffs, N.J., 1989).

Jang, J. S.

Javidi, B.

Kempner, E.

A. Stern, E. Kempner, A. Shukrun, N. S. Kopeika, “Restoration and resolution enhancement of a single image from a vibration distorted image sequence,” Opt. Eng. 39, 2451–2457 (2000).
[CrossRef]

Kopeika, N. S.

A. Stern, Y. Porat, A. Ben-Dor, N. S. Kopeika, “Enhanced-resolution image restoration from a sequence of low-frequency vibrated images by use of convex projections,” Appl. Opt. 40, 4706–4715 (2001).
[CrossRef]

A. Stern, E. Kempner, A. Shukrun, N. S. Kopeika, “Restoration and resolution enhancement of a single image from a vibration distorted image sequence,” Opt. Eng. 39, 2451–2457 (2000).
[CrossRef]

Lagendijk, R. L.

R. L. Lagendijk, Iterative Identification and Restoration of Images (Kluwer Academic, Dordrecht, The Netherlands, 1991).
[CrossRef]

Lohmann, A. W.

Mendlovic, D.

Naemura, T.

T. Naemura, T. Yoshida, H. Harashima, “3-D computer graphics based on integral photography,” Opt. Exp. 8, 255–262 (2001), http://www.opticsexpress.org .
[CrossRef]

Okano, F.

Okoshi, T.

Okoshii, T.

T. Okoshii, “Three-dimensional displays,” Proc. IEEE 68, 548–564 (1980).
[CrossRef]

Peleg, S.

A. Irani, S. Peleg, “Motion analysis for image enhancement: resolution, occlusion, and transparency,” J. Visual Commun. Image Represent. 4, 324–335 (1993).
[CrossRef]

A. Irani, S. Peleg, “Improving resolution by image registration,” CVGIP: Graph. Models Image Process 53, 231–239 (1991).
[CrossRef]

Porat, Y.

Saleh, B. E. A.

B. E. A. Saleh, M. C. Teich, Fundamentals of Photonics (Wiley, New York, 1991).
[CrossRef]

Shukrun, A.

A. Stern, E. Kempner, A. Shukrun, N. S. Kopeika, “Restoration and resolution enhancement of a single image from a vibration distorted image sequence,” Opt. Eng. 39, 2451–2457 (2000).
[CrossRef]

Stern, A.

A. Stern, Y. Porat, A. Ben-Dor, N. S. Kopeika, “Enhanced-resolution image restoration from a sequence of low-frequency vibrated images by use of convex projections,” Appl. Opt. 40, 4706–4715 (2001).
[CrossRef]

A. Stern, E. Kempner, A. Shukrun, N. S. Kopeika, “Restoration and resolution enhancement of a single image from a vibration distorted image sequence,” Opt. Eng. 39, 2451–2457 (2000).
[CrossRef]

Teich, M. C.

B. E. A. Saleh, M. C. Teich, Fundamentals of Photonics (Wiley, New York, 1991).
[CrossRef]

Yoshida, T.

T. Naemura, T. Yoshida, H. Harashima, “3-D computer graphics based on integral photography,” Opt. Exp. 8, 255–262 (2001), http://www.opticsexpress.org .
[CrossRef]

Yuyama, I.

Zalevsky, Z.

Appl. Opt. (5)

CVGIP: Graph. Models Image Process (1)

A. Irani, S. Peleg, “Improving resolution by image registration,” CVGIP: Graph. Models Image Process 53, 231–239 (1991).
[CrossRef]

IEE Proc. Vision Image Signal Process (1)

B. Cohen, I. Dinstein, “Polyphase back-projection filtering for image resolution enhancement,” IEE Proc. Vision Image Signal Process. 147, 318–322 (2000).
[CrossRef]

IEEE Trans. Image Process (1)

M. Elad, A. Feuer, “Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images,” IEEE Trans. Image Process. 6, 1646–1658 (1997).
[CrossRef] [PubMed]

J. Opt. Soc. Am. A (3)

J. Visual Commun. Image Represent (1)

A. Irani, S. Peleg, “Motion analysis for image enhancement: resolution, occlusion, and transparency,” J. Visual Commun. Image Represent. 4, 324–335 (1993).
[CrossRef]

Opt. Eng. (1)

A. Stern, E. Kempner, A. Shukrun, N. S. Kopeika, “Restoration and resolution enhancement of a single image from a vibration distorted image sequence,” Opt. Eng. 39, 2451–2457 (2000).
[CrossRef]

Opt. Exp. (1)

T. Naemura, T. Yoshida, H. Harashima, “3-D computer graphics based on integral photography,” Opt. Exp. 8, 255–262 (2001), http://www.opticsexpress.org .
[CrossRef]

Opt. Lett. (3)

Proc. IEEE (1)

T. Okoshii, “Three-dimensional displays,” Proc. IEEE 68, 548–564 (1980).
[CrossRef]

Other (3)

R. L. Lagendijk, Iterative Identification and Restoration of Images (Kluwer Academic, Dordrecht, The Netherlands, 1991).
[CrossRef]

B. E. A. Saleh, M. C. Teich, Fundamentals of Photonics (Wiley, New York, 1991).
[CrossRef]

A. K. Jain, Fundamentals of Digital Image Processing (Prentice-Hall, Englewood Cliffs, N.J., 1989).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (10)

Fig. 1
Fig. 1

Integral imaging of a 3-D object f(x 1, y 1, z 1). The lenslet pitch and aperture size are denoted by p and w, respectively. Elementary images are denoted by arrows in the detector plane. Solid arrows represent parallel rays propagating in direction α. The lenslet array motion is performed in the (x, y) plane within one lenslet pitch.

Fig. 2
Fig. 2

Multichannel modeling of time-division multiplexing CII.

Fig. 3
Fig. 3

Sampling grid (dashed-dotted lines) used to sample the II to generate an image appropriate to viewing directions α. Circles represent elementary images.

Fig. 4
Fig. 4

Optical setup used in the experiment.

Fig. 5
Fig. 5

Enlarged part of a captured II.

Fig. 6
Fig. 6

Example of a measured PSF. The horizontal and vertical axes are in units of CCD pixels.

Fig. 7
Fig. 7

(a), (b) Low-resolution reconstructed images from only one II appropriate for two viewing directions; (c), (d) high-resolution reconstruction of (a) and (b), respectively, by use of the time-division multiplexing CII method with a sequence of four shifted IIs.

Fig. 8
Fig. 8

(a) Low-resolution image reconstructed from one II, (b) interpolation of five images of the sequence, (c) convergence rate, (d) high-resolution reconstruction with the time-division multiplexing CII method.

Fig. 9
Fig. 9

Comparison of the average spectrum in the scanning (horizontal) direction of the low-resolution (LR) image (solid curve) and a high-resolution (HR) reconstruction (dashed curve) by use of time-division multiplexing CII.

Fig. 10
Fig. 10

Image enhancement of a low-resolution image: (a) Wiener filtered image, (b) reconstruction by application of the CII method on only one image from the sequence.

Equations (11)

Equations on this page are rendered with MathJax. Learn more.

βmax=minmαc diff, βNyq cycles/rad,
βNyq  1p.
αdiff=wλcycles/rad,
d1k, d2kT=dxkp, dxypT
p2=m2p,
skα=sxα, syαT-m2dxk, dykT.
gˆkn=[Tkfˆαn*hPSF]s,
fˆαn+1=fˆαn+1Nk=1N Tk-1gk-gˆkns*p,
δ-hPSF*p2<1.
0<1-HωPω<1 ω,
Mω=H*ωHω2+λω,

Metrics