Abstract

Recent works have demonstrated that three-dimensional (3D) object reconstruction is possible from integral images captured in severely photon starved conditions. In this paper we propose an iterative approach to implement a maximum likelihood expectation maximization estimator with several types of regularization for 3D reconstruction from photon counting integral images. We show that the proposed algorithms outperform the previously reported approaches for photon counting 3D integral imaging reconstruction. To the best of our knowledge, this is the first report on using iterative statistical reconstruction techniques for 3D photon counting integral imaging.

© 2011 OSA

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. A. Stern and B. Javidi, “Three dimensional image sensing, visualization, and processing using integral imaging,” Proc. IEEE 94(3), 591–607 (2006).
    [CrossRef]
  2. G. Lippmann, “La photographie integrale,” C. R. Acad. Sci. 146, 446–451 (1908).
  3. B. Javidi, R. Ponce-Díaz, and S.-H. Hong, “Three-dimensional recognition of occluded objects by using computational integral imaging,” Opt. Lett. 31(8), 1106–1108 (2006).
    [CrossRef] [PubMed]
  4. B. Javidi, F. Okano, and J.-Y. Son, eds., Three-Dimensional Imaging, Visualization, and Display (Springer, 2008).
  5. T. Okoshi, “Three-dimensional displays,” Proc. IEEE 68(5), 548–564 (1980).
    [CrossRef]
  6. H. Hoshino, F. Okano, H. Isono, and I. Yuyama, “Analysis of resolution limitation of integral photography,” J. Opt. Soc. Am. A 15(8), 2059–2065 (1998).
    [CrossRef]
  7. R. Martinez-Cuenca, G. Saavedra, M. Martinez-Corral, and B. Javidi, “Progress in 3-D multiperspective display by integral imaging,” Proc. IEEE 97(6), 1067–1077 (2009).
    [CrossRef]
  8. M. C. Forman, N. Davies, and M. McCormick, “Continuous parallax in discrete pixelated integral three-dimensional displays,” J. Opt. Soc. Am. A 20(3), 411–420 (2003).
    [CrossRef] [PubMed]
  9. F. Okano, J. Arai, K. Mitani, and M. Okui, “Real-time integral imaging based on extremely high resolution video system,” Proc. IEEE 94(3), 490–501 (2006).
    [CrossRef]
  10. B. Javidi, S.-H. Hong, and O. Matoba, “Multidimensional optical sensor and imaging system,” Appl. Opt. 45(13), 2986–2994 (2006).
    [CrossRef] [PubMed]
  11. H. Arimoto and B. Javidi, “Integral three-dimensional imaging with digital reconstruction,” Opt. Lett. 26(3), 157–159 (2001).
    [CrossRef] [PubMed]
  12. S. Yeom, B. Javidi, and E. Watson, “Photon counting passive 3D image sensing for automatic target recognition,” Opt. Express 13(23), 9310–9330 (2005).
    [CrossRef] [PubMed]
  13. B. Tavakoli, B. Javidi, and E. Watson, “Three dimensional visualization by photon counting computational Integral Imaging,” Opt. Express 16(7), 4426–4436 (2008).
    [CrossRef] [PubMed]
  14. J. Jung, M. Cho, D. K. Dey, and B. Javidi, “Three-dimensional photon counting integral imaging using Bayesian estimation,” Opt. Lett. 35(11), 1825–1827 (2010).
    [CrossRef] [PubMed]
  15. I. Moon and B. Javidi, “Three dimensional imaging and recognition using truncated photon counting model and parametric maximum likelihood estimator,” Opt. Express 17(18), 15709–15715 (2009).
    [CrossRef] [PubMed]
  16. I. Moon and B. Javidi, “Three-dimensional recognition of photon-starved events using computational integral imaging and statistical sampling,” Opt. Lett. 34(6), 731–733 (2009).
    [CrossRef] [PubMed]
  17. J. W. Goodman, Statistical Optics (Wiley, 1985).
  18. M. Bertero, P. Boccacci, G. Desidera, and G. Vicidomini, “Image deblurring with Poisson data: from cells to galaxies,” Inverse Probl. 25(12), 123006 (2009).
    [CrossRef]
  19. R. M. Lewitt and S. Matej, “Overview of methods for image reconstruction from projections in emission computed tomography,” Proc. IEEE 91(10), 1588–1611 (2003).
    [CrossRef]
  20. S. Geman and D. Geman, “Stochastic relaxation, Gibbs distributions and Baysesian restoration of images,” IEEE Trans. Pattern Anal. Mach. Intell. PAMI-6(6), 721–741 (1984).
    [CrossRef]
  21. S. Alenius and U. Ruotsalainen, “Generalization of median root prior reconstruction,” IEEE Trans. Med. Imaging 21(11), 1413–1420 (2002).
    [CrossRef] [PubMed]
  22. V. Y. Panin, G. L. Zeng, and G. T. Gullberg, “Total variation regulated EM algorithm [SPECT reconstruction],” IEEE Trans. Nucl. Sci. 46(6), 2202–2210 (1999).
    [CrossRef]
  23. N. Dey, L. Blanc-Feraud, C. Zimmer, P. Roux, Z. Kam, J. C. Olivo-Marin, and J. Zerubia, “Richardson-Lucy algorithm with total variation regularization for 3D confocal microscope deconvolution,” Microsc. Res. Tech. 69(4), 260–266 (2006).
    [CrossRef] [PubMed]

2010

2009

I. Moon and B. Javidi, “Three-dimensional recognition of photon-starved events using computational integral imaging and statistical sampling,” Opt. Lett. 34(6), 731–733 (2009).
[CrossRef] [PubMed]

I. Moon and B. Javidi, “Three dimensional imaging and recognition using truncated photon counting model and parametric maximum likelihood estimator,” Opt. Express 17(18), 15709–15715 (2009).
[CrossRef] [PubMed]

R. Martinez-Cuenca, G. Saavedra, M. Martinez-Corral, and B. Javidi, “Progress in 3-D multiperspective display by integral imaging,” Proc. IEEE 97(6), 1067–1077 (2009).
[CrossRef]

M. Bertero, P. Boccacci, G. Desidera, and G. Vicidomini, “Image deblurring with Poisson data: from cells to galaxies,” Inverse Probl. 25(12), 123006 (2009).
[CrossRef]

2008

2006

F. Okano, J. Arai, K. Mitani, and M. Okui, “Real-time integral imaging based on extremely high resolution video system,” Proc. IEEE 94(3), 490–501 (2006).
[CrossRef]

A. Stern and B. Javidi, “Three dimensional image sensing, visualization, and processing using integral imaging,” Proc. IEEE 94(3), 591–607 (2006).
[CrossRef]

N. Dey, L. Blanc-Feraud, C. Zimmer, P. Roux, Z. Kam, J. C. Olivo-Marin, and J. Zerubia, “Richardson-Lucy algorithm with total variation regularization for 3D confocal microscope deconvolution,” Microsc. Res. Tech. 69(4), 260–266 (2006).
[CrossRef] [PubMed]

B. Javidi, R. Ponce-Díaz, and S.-H. Hong, “Three-dimensional recognition of occluded objects by using computational integral imaging,” Opt. Lett. 31(8), 1106–1108 (2006).
[CrossRef] [PubMed]

B. Javidi, S.-H. Hong, and O. Matoba, “Multidimensional optical sensor and imaging system,” Appl. Opt. 45(13), 2986–2994 (2006).
[CrossRef] [PubMed]

2005

2003

M. C. Forman, N. Davies, and M. McCormick, “Continuous parallax in discrete pixelated integral three-dimensional displays,” J. Opt. Soc. Am. A 20(3), 411–420 (2003).
[CrossRef] [PubMed]

R. M. Lewitt and S. Matej, “Overview of methods for image reconstruction from projections in emission computed tomography,” Proc. IEEE 91(10), 1588–1611 (2003).
[CrossRef]

2002

S. Alenius and U. Ruotsalainen, “Generalization of median root prior reconstruction,” IEEE Trans. Med. Imaging 21(11), 1413–1420 (2002).
[CrossRef] [PubMed]

2001

1999

V. Y. Panin, G. L. Zeng, and G. T. Gullberg, “Total variation regulated EM algorithm [SPECT reconstruction],” IEEE Trans. Nucl. Sci. 46(6), 2202–2210 (1999).
[CrossRef]

1998

1984

S. Geman and D. Geman, “Stochastic relaxation, Gibbs distributions and Baysesian restoration of images,” IEEE Trans. Pattern Anal. Mach. Intell. PAMI-6(6), 721–741 (1984).
[CrossRef]

1980

T. Okoshi, “Three-dimensional displays,” Proc. IEEE 68(5), 548–564 (1980).
[CrossRef]

1908

G. Lippmann, “La photographie integrale,” C. R. Acad. Sci. 146, 446–451 (1908).

Alenius, S.

S. Alenius and U. Ruotsalainen, “Generalization of median root prior reconstruction,” IEEE Trans. Med. Imaging 21(11), 1413–1420 (2002).
[CrossRef] [PubMed]

Arai, J.

F. Okano, J. Arai, K. Mitani, and M. Okui, “Real-time integral imaging based on extremely high resolution video system,” Proc. IEEE 94(3), 490–501 (2006).
[CrossRef]

Arimoto, H.

Bertero, M.

M. Bertero, P. Boccacci, G. Desidera, and G. Vicidomini, “Image deblurring with Poisson data: from cells to galaxies,” Inverse Probl. 25(12), 123006 (2009).
[CrossRef]

Blanc-Feraud, L.

N. Dey, L. Blanc-Feraud, C. Zimmer, P. Roux, Z. Kam, J. C. Olivo-Marin, and J. Zerubia, “Richardson-Lucy algorithm with total variation regularization for 3D confocal microscope deconvolution,” Microsc. Res. Tech. 69(4), 260–266 (2006).
[CrossRef] [PubMed]

Boccacci, P.

M. Bertero, P. Boccacci, G. Desidera, and G. Vicidomini, “Image deblurring with Poisson data: from cells to galaxies,” Inverse Probl. 25(12), 123006 (2009).
[CrossRef]

Cho, M.

Davies, N.

Desidera, G.

M. Bertero, P. Boccacci, G. Desidera, and G. Vicidomini, “Image deblurring with Poisson data: from cells to galaxies,” Inverse Probl. 25(12), 123006 (2009).
[CrossRef]

Dey, D. K.

Dey, N.

N. Dey, L. Blanc-Feraud, C. Zimmer, P. Roux, Z. Kam, J. C. Olivo-Marin, and J. Zerubia, “Richardson-Lucy algorithm with total variation regularization for 3D confocal microscope deconvolution,” Microsc. Res. Tech. 69(4), 260–266 (2006).
[CrossRef] [PubMed]

Forman, M. C.

Geman, D.

S. Geman and D. Geman, “Stochastic relaxation, Gibbs distributions and Baysesian restoration of images,” IEEE Trans. Pattern Anal. Mach. Intell. PAMI-6(6), 721–741 (1984).
[CrossRef]

Geman, S.

S. Geman and D. Geman, “Stochastic relaxation, Gibbs distributions and Baysesian restoration of images,” IEEE Trans. Pattern Anal. Mach. Intell. PAMI-6(6), 721–741 (1984).
[CrossRef]

Gullberg, G. T.

V. Y. Panin, G. L. Zeng, and G. T. Gullberg, “Total variation regulated EM algorithm [SPECT reconstruction],” IEEE Trans. Nucl. Sci. 46(6), 2202–2210 (1999).
[CrossRef]

Hong, S.-H.

Hoshino, H.

Isono, H.

Javidi, B.

J. Jung, M. Cho, D. K. Dey, and B. Javidi, “Three-dimensional photon counting integral imaging using Bayesian estimation,” Opt. Lett. 35(11), 1825–1827 (2010).
[CrossRef] [PubMed]

I. Moon and B. Javidi, “Three dimensional imaging and recognition using truncated photon counting model and parametric maximum likelihood estimator,” Opt. Express 17(18), 15709–15715 (2009).
[CrossRef] [PubMed]

R. Martinez-Cuenca, G. Saavedra, M. Martinez-Corral, and B. Javidi, “Progress in 3-D multiperspective display by integral imaging,” Proc. IEEE 97(6), 1067–1077 (2009).
[CrossRef]

I. Moon and B. Javidi, “Three-dimensional recognition of photon-starved events using computational integral imaging and statistical sampling,” Opt. Lett. 34(6), 731–733 (2009).
[CrossRef] [PubMed]

B. Tavakoli, B. Javidi, and E. Watson, “Three dimensional visualization by photon counting computational Integral Imaging,” Opt. Express 16(7), 4426–4436 (2008).
[CrossRef] [PubMed]

A. Stern and B. Javidi, “Three dimensional image sensing, visualization, and processing using integral imaging,” Proc. IEEE 94(3), 591–607 (2006).
[CrossRef]

B. Javidi, R. Ponce-Díaz, and S.-H. Hong, “Three-dimensional recognition of occluded objects by using computational integral imaging,” Opt. Lett. 31(8), 1106–1108 (2006).
[CrossRef] [PubMed]

B. Javidi, S.-H. Hong, and O. Matoba, “Multidimensional optical sensor and imaging system,” Appl. Opt. 45(13), 2986–2994 (2006).
[CrossRef] [PubMed]

S. Yeom, B. Javidi, and E. Watson, “Photon counting passive 3D image sensing for automatic target recognition,” Opt. Express 13(23), 9310–9330 (2005).
[CrossRef] [PubMed]

H. Arimoto and B. Javidi, “Integral three-dimensional imaging with digital reconstruction,” Opt. Lett. 26(3), 157–159 (2001).
[CrossRef] [PubMed]

Jung, J.

Kam, Z.

N. Dey, L. Blanc-Feraud, C. Zimmer, P. Roux, Z. Kam, J. C. Olivo-Marin, and J. Zerubia, “Richardson-Lucy algorithm with total variation regularization for 3D confocal microscope deconvolution,” Microsc. Res. Tech. 69(4), 260–266 (2006).
[CrossRef] [PubMed]

Lewitt, R. M.

R. M. Lewitt and S. Matej, “Overview of methods for image reconstruction from projections in emission computed tomography,” Proc. IEEE 91(10), 1588–1611 (2003).
[CrossRef]

Lippmann, G.

G. Lippmann, “La photographie integrale,” C. R. Acad. Sci. 146, 446–451 (1908).

Martinez-Corral, M.

R. Martinez-Cuenca, G. Saavedra, M. Martinez-Corral, and B. Javidi, “Progress in 3-D multiperspective display by integral imaging,” Proc. IEEE 97(6), 1067–1077 (2009).
[CrossRef]

Martinez-Cuenca, R.

R. Martinez-Cuenca, G. Saavedra, M. Martinez-Corral, and B. Javidi, “Progress in 3-D multiperspective display by integral imaging,” Proc. IEEE 97(6), 1067–1077 (2009).
[CrossRef]

Matej, S.

R. M. Lewitt and S. Matej, “Overview of methods for image reconstruction from projections in emission computed tomography,” Proc. IEEE 91(10), 1588–1611 (2003).
[CrossRef]

Matoba, O.

McCormick, M.

Mitani, K.

F. Okano, J. Arai, K. Mitani, and M. Okui, “Real-time integral imaging based on extremely high resolution video system,” Proc. IEEE 94(3), 490–501 (2006).
[CrossRef]

Moon, I.

Okano, F.

F. Okano, J. Arai, K. Mitani, and M. Okui, “Real-time integral imaging based on extremely high resolution video system,” Proc. IEEE 94(3), 490–501 (2006).
[CrossRef]

H. Hoshino, F. Okano, H. Isono, and I. Yuyama, “Analysis of resolution limitation of integral photography,” J. Opt. Soc. Am. A 15(8), 2059–2065 (1998).
[CrossRef]

Okoshi, T.

T. Okoshi, “Three-dimensional displays,” Proc. IEEE 68(5), 548–564 (1980).
[CrossRef]

Okui, M.

F. Okano, J. Arai, K. Mitani, and M. Okui, “Real-time integral imaging based on extremely high resolution video system,” Proc. IEEE 94(3), 490–501 (2006).
[CrossRef]

Olivo-Marin, J. C.

N. Dey, L. Blanc-Feraud, C. Zimmer, P. Roux, Z. Kam, J. C. Olivo-Marin, and J. Zerubia, “Richardson-Lucy algorithm with total variation regularization for 3D confocal microscope deconvolution,” Microsc. Res. Tech. 69(4), 260–266 (2006).
[CrossRef] [PubMed]

Panin, V. Y.

V. Y. Panin, G. L. Zeng, and G. T. Gullberg, “Total variation regulated EM algorithm [SPECT reconstruction],” IEEE Trans. Nucl. Sci. 46(6), 2202–2210 (1999).
[CrossRef]

Ponce-Díaz, R.

Roux, P.

N. Dey, L. Blanc-Feraud, C. Zimmer, P. Roux, Z. Kam, J. C. Olivo-Marin, and J. Zerubia, “Richardson-Lucy algorithm with total variation regularization for 3D confocal microscope deconvolution,” Microsc. Res. Tech. 69(4), 260–266 (2006).
[CrossRef] [PubMed]

Ruotsalainen, U.

S. Alenius and U. Ruotsalainen, “Generalization of median root prior reconstruction,” IEEE Trans. Med. Imaging 21(11), 1413–1420 (2002).
[CrossRef] [PubMed]

Saavedra, G.

R. Martinez-Cuenca, G. Saavedra, M. Martinez-Corral, and B. Javidi, “Progress in 3-D multiperspective display by integral imaging,” Proc. IEEE 97(6), 1067–1077 (2009).
[CrossRef]

Stern, A.

A. Stern and B. Javidi, “Three dimensional image sensing, visualization, and processing using integral imaging,” Proc. IEEE 94(3), 591–607 (2006).
[CrossRef]

Tavakoli, B.

Vicidomini, G.

M. Bertero, P. Boccacci, G. Desidera, and G. Vicidomini, “Image deblurring with Poisson data: from cells to galaxies,” Inverse Probl. 25(12), 123006 (2009).
[CrossRef]

Watson, E.

Yeom, S.

Yuyama, I.

Zeng, G. L.

V. Y. Panin, G. L. Zeng, and G. T. Gullberg, “Total variation regulated EM algorithm [SPECT reconstruction],” IEEE Trans. Nucl. Sci. 46(6), 2202–2210 (1999).
[CrossRef]

Zerubia, J.

N. Dey, L. Blanc-Feraud, C. Zimmer, P. Roux, Z. Kam, J. C. Olivo-Marin, and J. Zerubia, “Richardson-Lucy algorithm with total variation regularization for 3D confocal microscope deconvolution,” Microsc. Res. Tech. 69(4), 260–266 (2006).
[CrossRef] [PubMed]

Zimmer, C.

N. Dey, L. Blanc-Feraud, C. Zimmer, P. Roux, Z. Kam, J. C. Olivo-Marin, and J. Zerubia, “Richardson-Lucy algorithm with total variation regularization for 3D confocal microscope deconvolution,” Microsc. Res. Tech. 69(4), 260–266 (2006).
[CrossRef] [PubMed]

Appl. Opt.

C. R. Acad. Sci.

G. Lippmann, “La photographie integrale,” C. R. Acad. Sci. 146, 446–451 (1908).

IEEE Trans. Med. Imaging

S. Alenius and U. Ruotsalainen, “Generalization of median root prior reconstruction,” IEEE Trans. Med. Imaging 21(11), 1413–1420 (2002).
[CrossRef] [PubMed]

IEEE Trans. Nucl. Sci.

V. Y. Panin, G. L. Zeng, and G. T. Gullberg, “Total variation regulated EM algorithm [SPECT reconstruction],” IEEE Trans. Nucl. Sci. 46(6), 2202–2210 (1999).
[CrossRef]

IEEE Trans. Pattern Anal. Mach. Intell.

S. Geman and D. Geman, “Stochastic relaxation, Gibbs distributions and Baysesian restoration of images,” IEEE Trans. Pattern Anal. Mach. Intell. PAMI-6(6), 721–741 (1984).
[CrossRef]

Inverse Probl.

M. Bertero, P. Boccacci, G. Desidera, and G. Vicidomini, “Image deblurring with Poisson data: from cells to galaxies,” Inverse Probl. 25(12), 123006 (2009).
[CrossRef]

J. Opt. Soc. Am. A

Microsc. Res. Tech.

N. Dey, L. Blanc-Feraud, C. Zimmer, P. Roux, Z. Kam, J. C. Olivo-Marin, and J. Zerubia, “Richardson-Lucy algorithm with total variation regularization for 3D confocal microscope deconvolution,” Microsc. Res. Tech. 69(4), 260–266 (2006).
[CrossRef] [PubMed]

Opt. Express

Opt. Lett.

Proc. IEEE

A. Stern and B. Javidi, “Three dimensional image sensing, visualization, and processing using integral imaging,” Proc. IEEE 94(3), 591–607 (2006).
[CrossRef]

R. M. Lewitt and S. Matej, “Overview of methods for image reconstruction from projections in emission computed tomography,” Proc. IEEE 91(10), 1588–1611 (2003).
[CrossRef]

T. Okoshi, “Three-dimensional displays,” Proc. IEEE 68(5), 548–564 (1980).
[CrossRef]

R. Martinez-Cuenca, G. Saavedra, M. Martinez-Corral, and B. Javidi, “Progress in 3-D multiperspective display by integral imaging,” Proc. IEEE 97(6), 1067–1077 (2009).
[CrossRef]

F. Okano, J. Arai, K. Mitani, and M. Okui, “Real-time integral imaging based on extremely high resolution video system,” Proc. IEEE 94(3), 490–501 (2006).
[CrossRef]

Other

J. W. Goodman, Statistical Optics (Wiley, 1985).

B. Javidi, F. Okano, and J.-Y. Son, eds., Three-Dimensional Imaging, Visualization, and Display (Springer, 2008).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (3)

Fig. 1
Fig. 1

(a) forward projection of a 3D object in the InI plane through microlens array. (b) Backward projection.

Fig. 2
Fig. 2

Reconstruction at Z0 = 370mm using InI with N p ~ 2 × 10 3 photon/pixel, using: (a) normal illuminations conditions, (b) one elemental image, (c) MLEM without penalty (5), (d) PMLEM with prior reflecting quadratic penalty, (e) PMLEM with median penalty, (f) PMLEM with total-variation (g) using MLE.

Fig. 3
Fig. 3

Reconstruction PSNR for very low photon counts.

Tables (1)

Tables Icon

Table 1 Reconstruction results with maximum likelihood estimation (MLE), maximum likelihood expectation maximization (MLEM) and penalized MLEM (PMLEM) with three types of penalties

Equations (12)

Equations on this page are rendered with MathJax. Learn more.

f B P ( x . y . z 0 ) = 1 K L k = 0 K 1 l = 0 L 1 g k l ( x + ( 1 M 0 ) S x k , y + ( 1 M 0 ) S y l )
f B P = A T g
g = A f
f ( k + 1 ) = f ( k ) s + β P f ( k ) A T g A f ( k )
f ^ ( k + 1 ) = f ^ ( k ) s j A T g A f ^ ( k ) ,
P f ^ i , j ( k ) = P { f ^ ( k ) [ i , j ] } = r , s S i , j w i , j ( f ^ ( k ) [ i , j ] f ^ ( k ) [ i + r , j + s ] ) 2
P f ^ i , j ( k ) = P { f ^ ( k ) [ i , j ] } = f ^ ( k ) [ i , j ] M e d { f ^ ( k ) [ i , j ] ; S i , j } M e d { f ^ ( k ) [ i , j ] ; S i , j }
P f ^ i , j = P { f ^ [ i , j ] } = f ^ ( k ) [ i , j ] f ^ ( k ) [ i 1 , j ] u [ i 1 , j ] + f ^ ( k ) [ i , j ] f ^ ( k ) [ i , j 1 ] u [ i , j 1 ] f ^ ( k ) [ i + 1 , j ] + f ^ ( k ) [ i , j + 1 ] 2 f ^ ( k ) [ i , j ] u [ i , j ] .
u ( i , j ) = ( f ^ ( k ) [ i + 1 , j ] f ^ ( k ) [ i , j ] ) 2 + ( f ^ ( k ) [ i , j + 1 ] f ^ ( k ) [ i , j ] ) 2 + ε .
1 N M i = 1 N j = 1 M | f ^ i , j ( k ) f ^ i , j ( k 1 ) | 2 ε t h
P S N R = 10 log 10 max ( f ^ i , j f u l l ) M S E
M S E = 1 i , j R O I i j | f i , j f u l l f i , j e s t | 2

Metrics