Abstract

We present the theoretical and simulation results on the analysis of Synthetic Aperture Integral Imaging (SAII) technique and its sensitivity to pickup position uncertainty. SAII is a passive three dimensional imaging technique based on multiple image acquisitions with different perspective of the scene under incoherent or natural illumination. In practical SAII applications, there is always an uncertainty associated with the position at which each sensor captures the elemental image. We present a theoretical analysis that quantifies image degradation in terms of Mean Square Error (MSE) metric. Simulation results are also presented to identify the parameters affecting the reconstruction degradation and to confirm the analysis. We show that in SAII with a given uncertainty in the sensor locations, the high spatial frequency content of the 3D reconstructed images are most degraded. We also show an inverse relationship between the reconstruction distance and degradation metric. To the best of our knowledge, this is the first time that the effects of sensor position uncertainty on 3D computational reconstruction in synthetic aperture integral imaging systems have been quantitatively analyzed.

© 2007 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. S. A. Benton, ed., Selected Papers on Three-Dimensional Displays (SPIE Optical Engineering Press, Bellingham, WA, 2001).
  2. B. Javidi and F. Okano, eds., Three Dimensional Television, Video, and Display Technologies (Springer, Berlin, 2002).
  3. T. Okoshi, Three-dimensional Imaging Techniques (Academic Press, New York, 1976).
  4. B. Javidi, S.-H. Hong, and O. Matoba, "Multi dimensional optical sensors and imaging systems," Appl. Opt. 45, 2986-2994 (2006).
    [CrossRef] [PubMed]
  5. M. G. Lippmann, "Epreuves reversibles donnant la sensation durelief," J. Phys. 7, 821-825 (1908).
  6. Y. A. Dudnikov, ‘‘On the design of a scheme for producing integral photographs by a combination method,’’Sov. J. Opt. Technol. 41, 426-429 (1974).
  7. H. E. Ives, "Optical properties of a Lippmann lenticuled sheet," J. Opt. Soc. Am. 21, 171-176 (1931).
    [CrossRef]
  8. P. Sokolov, "Autostereoscpy and Integral Photography by Professor Lippmann’s Method," (Moscow State Univ. Press, Moscow, Russia, 1911).
  9. F. Okano, H. Hoshino, J. Arai, I. Yuyama, "Real time pickup method for a three-dimensional image based on integral photography," Appl. Opt. 36, 1598-1603 (1997).
    [CrossRef] [PubMed]
  10. A. Stern and B. Javidi, "Three-dimensional image sensing, visualization, and processing using integral imaging," Proc. IEEE 94, 591-607 (2006).
    [CrossRef]
  11. B. Wilburn, N. Joshi, V. Vaish, A. Barth, A. Adams, M. Horowitz, M. Levoy, "High performance imaging using large camera arrays," Proc. of the ACM 24, 765-776 (2005).
  12. J. S. Jang and B. Javidi, "Three-dimensional synthetic aperture integral imaging," Opt. Lett. 27, 1144-1146 (2002).
    [CrossRef]
  13. B. Burckhardt, "Optimum parameters and resolution limitation of integral photography," J. Opt. Soc. Am. 58, 71-76 (1968).
    [CrossRef]
  14. H. Hoshino, F. Okano, H. Isono, and I. Yuyama, "Analysis of resolution limitation of integral photography," J. Opt. Soc. Am. A 15, 2059-2065 (1998).
    [CrossRef]
  15. S. -H. Hong, J. -S Jang, and B. Javidi, "Three-dimensional volumetric object reconstruction using computational integral imaging." Opt. Express 12, 483-491 (2004).
    [CrossRef] [PubMed]
  16. A. Stern and B. Javidi, "3-D computational synthetic aperture integral imaging (COMPSAII)," Opt. Express 11, 2446-2451 (2003).
    [CrossRef] [PubMed]
  17. Y. Igarishi, H. Murata, and M. Ueda, "3D display system using a computer-generated integral photograph," Jpn. J. Appl. Phys. 17, 1683-1684 (1978).
    [CrossRef]
  18. [REMOVED HYPERLINK FIELD]L. Erdmann and K. J. Gabriel, "High resolution digital photography by use of a scanning microlens array," Appl. Opt. 40, 5592-5599 (2001).
    [CrossRef]
  19. S. Kishk and B. Javidi, "Improved resolution 3D object sensing and recognition using time multiplexed computational integral imaging," Opt. Express 11, 3528-3541 (2003).
    [CrossRef] [PubMed]
  20. R. Martínez-Cuenca, G. Saavedra, M. Martinez-Corral and B. Javidi, "Enhanced depth of field integral imaging with sensor resolution constraints," Opt. Express 12, 5237-5242 (2004).
    [CrossRef] [PubMed]
  21. J.-S. Jang and B. Javidi, "Improved viewing resolution of three-dimensional integral imaging by use of nonstationary micro-optics," Opt. Lett. 27, 324-326 (2002).
    [CrossRef]
  22. J. Hong, J. -H. Park, S. Jung, and B. Lee, "Depth-enhanced integral imaging by use of optical path control," Opt. Lett. 29, 1790-1792 (2004)
    [CrossRef] [PubMed]
  23. S. -W. Min, J. Kim, and B. Lee, "Wide-viewing projection-type integral imaging system with an embossed screen," Opt. Lett. 29, 2420-2422 (2004)
    [CrossRef] [PubMed]
  24. M. Martínez-Corral, B. Javidi, R. Martínez-Cuenca, and G. Saavedra, "Integral imaging with improved depth of field by use of amplitude modulated microlens array," Appl. Opt. 43, 5806-5813 (2004).
    [CrossRef] [PubMed]
  25. Y. S. Hwang, S. -H. Hong, and B. Javidi, "Free View 3-D Visualization of Occluded Objects by Using Computational Synthetic Aperture Integral Imaging," J. Display Technol. 3, 64-70 (2007)
    [CrossRef]
  26. S. Yeom, B. Javidi, and E. Watson, "Photon counting passive 3D image sensing for automatic target recognition," Opt. Express 13, 9310-9330 (2005).
    [CrossRef] [PubMed]
  27. Y. Frauel and B. Javidi, "Digital three-dimensional image correlation by use of computer-reconstructed integral imaging," Appl. Opt. 41, 5488-5496 (2002).
    [CrossRef] [PubMed]
  28. J. Arai, M. Okui, M. Kobayashi, and F. Okano, "Geometrical effects of positional errors in integral photography," J. Opt. Soc. Am. A 21, 951-958 (2004)
    [CrossRef]
  29. N. Mukhopadhyay, Probability and Statistical Inference (Marcel Dekker, Inc. New York, 2000).

2007 (1)

2006 (2)

B. Javidi, S.-H. Hong, and O. Matoba, "Multi dimensional optical sensors and imaging systems," Appl. Opt. 45, 2986-2994 (2006).
[CrossRef] [PubMed]

A. Stern and B. Javidi, "Three-dimensional image sensing, visualization, and processing using integral imaging," Proc. IEEE 94, 591-607 (2006).
[CrossRef]

2005 (1)

2004 (6)

2003 (2)

2002 (3)

2001 (1)

1998 (1)

1997 (1)

1978 (1)

Y. Igarishi, H. Murata, and M. Ueda, "3D display system using a computer-generated integral photograph," Jpn. J. Appl. Phys. 17, 1683-1684 (1978).
[CrossRef]

1974 (1)

Y. A. Dudnikov, ‘‘On the design of a scheme for producing integral photographs by a combination method,’’Sov. J. Opt. Technol. 41, 426-429 (1974).

1968 (1)

1931 (1)

1908 (1)

M. G. Lippmann, "Epreuves reversibles donnant la sensation durelief," J. Phys. 7, 821-825 (1908).

Arai, J.

Burckhardt, B.

Dudnikov, Y. A.

Y. A. Dudnikov, ‘‘On the design of a scheme for producing integral photographs by a combination method,’’Sov. J. Opt. Technol. 41, 426-429 (1974).

Erdmann, L.

Frauel, Y.

Gabriel, K. J.

Hong, J.

Hong, S. -H.

Hong, S.-H.

Hoshino, H.

Hwang, Y. S.

Igarishi, Y.

Y. Igarishi, H. Murata, and M. Ueda, "3D display system using a computer-generated integral photograph," Jpn. J. Appl. Phys. 17, 1683-1684 (1978).
[CrossRef]

Isono, H.

Ives, H. E.

Jang, J. -S

Jang, J. S.

Jang, J.-S.

Javidi, B.

Y. S. Hwang, S. -H. Hong, and B. Javidi, "Free View 3-D Visualization of Occluded Objects by Using Computational Synthetic Aperture Integral Imaging," J. Display Technol. 3, 64-70 (2007)
[CrossRef]

B. Javidi, S.-H. Hong, and O. Matoba, "Multi dimensional optical sensors and imaging systems," Appl. Opt. 45, 2986-2994 (2006).
[CrossRef] [PubMed]

A. Stern and B. Javidi, "Three-dimensional image sensing, visualization, and processing using integral imaging," Proc. IEEE 94, 591-607 (2006).
[CrossRef]

S. Yeom, B. Javidi, and E. Watson, "Photon counting passive 3D image sensing for automatic target recognition," Opt. Express 13, 9310-9330 (2005).
[CrossRef] [PubMed]

M. Martínez-Corral, B. Javidi, R. Martínez-Cuenca, and G. Saavedra, "Integral imaging with improved depth of field by use of amplitude modulated microlens array," Appl. Opt. 43, 5806-5813 (2004).
[CrossRef] [PubMed]

R. Martínez-Cuenca, G. Saavedra, M. Martinez-Corral and B. Javidi, "Enhanced depth of field integral imaging with sensor resolution constraints," Opt. Express 12, 5237-5242 (2004).
[CrossRef] [PubMed]

S. -H. Hong, J. -S Jang, and B. Javidi, "Three-dimensional volumetric object reconstruction using computational integral imaging." Opt. Express 12, 483-491 (2004).
[CrossRef] [PubMed]

S. Kishk and B. Javidi, "Improved resolution 3D object sensing and recognition using time multiplexed computational integral imaging," Opt. Express 11, 3528-3541 (2003).
[CrossRef] [PubMed]

A. Stern and B. Javidi, "3-D computational synthetic aperture integral imaging (COMPSAII)," Opt. Express 11, 2446-2451 (2003).
[CrossRef] [PubMed]

Y. Frauel and B. Javidi, "Digital three-dimensional image correlation by use of computer-reconstructed integral imaging," Appl. Opt. 41, 5488-5496 (2002).
[CrossRef] [PubMed]

J. S. Jang and B. Javidi, "Three-dimensional synthetic aperture integral imaging," Opt. Lett. 27, 1144-1146 (2002).
[CrossRef]

J.-S. Jang and B. Javidi, "Improved viewing resolution of three-dimensional integral imaging by use of nonstationary micro-optics," Opt. Lett. 27, 324-326 (2002).
[CrossRef]

Jung, S.

Kim, J.

Kishk, S.

Kobayashi, M.

Lee, B.

Lippmann, M. G.

M. G. Lippmann, "Epreuves reversibles donnant la sensation durelief," J. Phys. 7, 821-825 (1908).

Martinez-Corral, M.

Martínez-Corral, M.

Martínez-Cuenca, R.

Matoba, O.

Min, S. -W.

Murata, H.

Y. Igarishi, H. Murata, and M. Ueda, "3D display system using a computer-generated integral photograph," Jpn. J. Appl. Phys. 17, 1683-1684 (1978).
[CrossRef]

Okano, F.

Okui, M.

Park, J. -H.

Saavedra, G.

Stern, A.

A. Stern and B. Javidi, "Three-dimensional image sensing, visualization, and processing using integral imaging," Proc. IEEE 94, 591-607 (2006).
[CrossRef]

A. Stern and B. Javidi, "3-D computational synthetic aperture integral imaging (COMPSAII)," Opt. Express 11, 2446-2451 (2003).
[CrossRef] [PubMed]

Ueda, M.

Y. Igarishi, H. Murata, and M. Ueda, "3D display system using a computer-generated integral photograph," Jpn. J. Appl. Phys. 17, 1683-1684 (1978).
[CrossRef]

Watson, E.

Yeom, S.

Yuyama, I.

Appl. Opt. (5)

J. Display Technol. (1)

J. Opt. Soc. Am. (2)

J. Opt. Soc. Am. A (2)

J. Phys. (1)

M. G. Lippmann, "Epreuves reversibles donnant la sensation durelief," J. Phys. 7, 821-825 (1908).

Jpn. J. Appl. Phys. (1)

Y. Igarishi, H. Murata, and M. Ueda, "3D display system using a computer-generated integral photograph," Jpn. J. Appl. Phys. 17, 1683-1684 (1978).
[CrossRef]

Opt. Express (5)

Opt. Lett. (4)

Proc. IEEE (1)

A. Stern and B. Javidi, "Three-dimensional image sensing, visualization, and processing using integral imaging," Proc. IEEE 94, 591-607 (2006).
[CrossRef]

Sov. J. Opt. Technol. (1)

Y. A. Dudnikov, ‘‘On the design of a scheme for producing integral photographs by a combination method,’’Sov. J. Opt. Technol. 41, 426-429 (1974).

Other (6)

P. Sokolov, "Autostereoscpy and Integral Photography by Professor Lippmann’s Method," (Moscow State Univ. Press, Moscow, Russia, 1911).

S. A. Benton, ed., Selected Papers on Three-Dimensional Displays (SPIE Optical Engineering Press, Bellingham, WA, 2001).

B. Javidi and F. Okano, eds., Three Dimensional Television, Video, and Display Technologies (Springer, Berlin, 2002).

T. Okoshi, Three-dimensional Imaging Techniques (Academic Press, New York, 1976).

B. Wilburn, N. Joshi, V. Vaish, A. Barth, A. Adams, M. Horowitz, M. Levoy, "High performance imaging using large camera arrays," Proc. of the ACM 24, 765-776 (2005).

N. Mukhopadhyay, Probability and Statistical Inference (Marcel Dekker, Inc. New York, 2000).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (9)

Fig. 1.
Fig. 1.

Pickup process of three dimensional synthetic aperture integral imaging. The camera at each location captures the scene form a unique perspective, which is later used for 3D computational reconstruction.

Fig. 2.
Fig. 2.

Schematic of II reconstruction process (left), Arrangement of elemental images (right)

Fig. 3.
Fig. 3.

Comparison of the MSE results of the Monte Carlo simulation and Eq. (17) for a point source located at z=24cm to z=40cm (M=12 to M=20) at g=2cm.

Fig. 4.
Fig. 4.

Result of the Monte Carlo simulation for the point source reconstruction from its dislocated elemental images. Point source is located at (a) z=40cm and (b) 24cm. For zero position errors, the plots would have been a single point.

Fig. 5.
Fig. 5.

(a). A 2D image of the 3D scene, (b). subset of elemental images for 3D scene in (a).

Fig. 6.
Fig. 6.

3D scene reconstruction at distances (a) z=24 cm, (b) z=30 cm and (c) z=36 cm.

Fig. 7.
Fig. 7.

Reconstruction at z=24 cm using (a) original camera position, and (b) using distorted camera position with 30% pitch error.

Fig. 8.
Fig. 8.

(a). box-and-whisker diagram of the MSEs for z=24cm (M=12) to z=40cm (M=20) when the pitch error is 30%. (b). Mean of MSE corresponding to 10%, 20%, 30% and 50% pitch errors for the distances from z=24cm to z=40cm i.e M=12 to M=20.

Fig. 9.
Fig. 9.

Total error for the range of distances from (a) z=35cm to38cm, (b) z=29cm to 32cm and (c) z=24cm to 27cm.

Equations (29)

Equations on this page are rendered with MathJax. Learn more.

I kl ( x , y , z 0 ) = O kl ( x + ( 1 + 1 M ) S x k , y + ( 1 + 1 M ) S y l ) ,
I ( x , y , z 0 ) = k = 1 K l = 1 L I kl ( x , y , z 0 ) R 2 ( x , y ) ,
R 2 ( x , y ) = ( z 0 + g ) 2 + [ ( Mx S x k ) 2 + ( My S y l ) 2 ] × ( 1 M + 1 ) 2 ,
I e ( x , y , z ) = i = 1 K × K I i ( x + Δ p i M , y ) R 2 ,
err ( p , z ) = I ( p , z ) I e ( p , z ) .
E err ( x , y , z ) 2 = 1 R 4 E { i K × K I i ( x , y ) I i ( x + Δ p i M , y ) 2 } ,
Err ( f , z ) = F . T { err ( p , z ) } = 1 R 2 i = 1 K × K [ I ˜ i ( f x , f y ) I ˜ i ( f x , f y ) e jf x Δ p i M ] .
= 1 R 2 i = 1 K × K [ I ˜ i I ˜ i e jf x Δ p i M ]
E Err ( f , z ) 2 = 1 R 4 [ ( 2 γ γ * ) i K × K I ˜ i 2 + 1 γ 2 i j K × K j i K × K I ˜ i I ˜ j * ] ,
E Err ( f , z ) 2 = 2 ( 1 γ ) R 4 × i K × K I ˜ i 2 + ( 1 γ ) 2 R 4 × i j K × K j i K × K I ˜ i I ˜ j * .
E Err ( f , z ) 2 = 2 ( 1 γ 2 ) R 4 × i K × K I ˜ i 2 + ( 1 γ 2 ) 2 R 4 × i j K × K j i K × K I ˜ i I ˜ j * .
err ( p , z ) 2 d p = Err ( f , z ) 2 d f ,
MSE ( z ) = E err ( p , z ) 2 d p = E Err ( f , z ) 2 d f
= 1 ( z + g ) 4 [ 2 ( 1 γ 2 ) i K × K I ˜ i 2 ] d f + [ ( 1 γ 2 ) 2 i j K × K j i K × K I ˜ i I ˜ j * ] d f
FWHM = 2 M ln ( 2 ) σ = 1.66 z g σ .
I k ( p ) = δ ( p + kS p ( 1 + g z ) ) ,
E Err ( f x , z ) 2 = 2 ( 1 γ ) ( z + g ) 4 K + 2 ( 1 γ ) 2 ( z + g ) 4 k l K l k K e jf x S p ( 1 + g z ) k e + jf x S p ( 1 + g z ) l .
E Err ( f x , z ) 2 = 2 γ ( 1 γ ) ( z + g ) 4 K + 2 ( 1 γ ) 2 ( z + g ) 4 sin 2 ( Kf x S p ( 1 + g z ) ) sin 2 ( f x S p ( 1 + g z ) ) .
E Err ( f x , z ) 2 = 2 ( 1 e f x 2 σ 2 g 2 2 z 2 ) ( z + g ) 4 [ Ke f x 2 σ 2 g 2 2 z 2 +
( 1 e f x 2 σ 2 g 2 2 z 2 ) sin 2 ( Kf x S p ( 1 + g z ) ) sin 2 ( f x S p ( 1 + g z ) ) ] .
Err ( f , z ) 2 = 1 R 4 i K × K I ˜ i ( 1 e jf x Δ p i M ) 2
= 1 R 4 [ i K × K j K × K I ˜ i I ˜ j * ( 1 e jf x Δ p i M ) ( 1 e + jf x Δ p i M ) ] .
E { Err ( f , z ) 2 } = 1 R 4 [ i K × K j K × K I ˜ i I ˜ j * E { ( 1 e jf x Δ p i M ) ( 1 e + jf x Δ p i M } ]
= 1 R 4 [ i K × K i K × K I ˜ i I ˜ i * E { 2 e jf x Δ p i M e + jf x Δ p i M } ] +
1 R 4 [ i j K × K j i K × K I ˜ i I ˜ j * E { ( 1 e jf x Δ p i M ) ( 1 e + jf x Δ p j M ) } ]
E { Err ( f , z ) 2 } = 1 R 4 [ i K × K i K × K I ˜ i I ˜ i * ( 2 γ γ * ) ] +
1 R 4 [ i j K × K j i K × K I ˜ i I ˜ j * E { ( 1 e jf x Δ p i M ) ( 1 e + jf x Δ p j M ) } ]
E = { ( 1 e jf x Δ p i M ) ( 1 e + jf x Δ p i M ) } = E { 1 e jf x Δ p i M } E { 1 e + jf x Δ p i M } .
= ( 1 γ ) ( 1 γ * ) = 1 γ 2

Metrics