Abstract

Passive 3D sensing using integral imaging techniques has been well studied in the literature. It has been shown that a scene can be reconstructed at various depths using several 2D elemental images. This provides the ability to reconstruct objects in the presence of occlusions, and passively estimate their 3D profile. However, high resolution 2D elemental images are required for high quality 3D reconstruction. Compressive Sensing (CS) provides a way to dramatically reduce the amount of data that needs to be collected to form the elemental images, which in turn can reduce the storage and bandwidth requirements. In this paper, we explore the effects of CS in acquisition of the elemental images, and ultimately on passive 3D scene reconstruction and object recognition. Our experiments show that the performance of passive 3D sensing systems remains robust even when elemental images are recovered from very few compressive measurements.

© 2012 OSA

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. G. Lippmann, “La Photographie Integrale,” C.-R. Acad. des Sci.146, 446–451 (1908).
  2. L. Yang, M. McCormick, and N. Davies, “Discussion of the optics of a new 3-D imaging system,” Appl. Opt.27(21), 4529–4534 (1988).
    [CrossRef] [PubMed]
  3. F. Okano, J. Arai, K. Mitani, and M. Okui, “Real-time integral imaging based on extremely high resolution video system,” Proc. IEEE94(3), 490–501 (2006).
    [CrossRef]
  4. A. Stern and B. Javidi, “Three-dimensional image sensing, visualization, and processing using integral imaging,” Proc. IEEE94(3), 591–607 (2006).
    [CrossRef]
  5. R. Martinez-Cuenca, G. Saavedra, M. Martinez-Corral, and B. Javidi, “Progress in 3-D multiperspective display by integral imaging,” Proc. IEEE97(6), 1067–1077 (2009).
    [CrossRef]
  6. J.-H. Park, K. Hong, and B. Lee, “Recent progress in three-dimensional information processing based on integral imaging,” Appl. Opt.48(34), H77–H94 (2009).
    [CrossRef] [PubMed]
  7. J.-S. Jang and B. Javidi, “Three-dimensional synthetic aperture integral imaging,” Opt. Lett.27(13), 1144–1146 (2002).
    [CrossRef] [PubMed]
  8. S.-H. Hong, J.-S. Jang, and B. Javidi, “Three-dimensional volumetric object reconstruction using computational integral imaging,” Opt. Express12(3), 483–491 (2004).
    [CrossRef] [PubMed]
  9. E. Candes, J. Romberg, and T. Tao, “Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information,” IEEE Trans. Inf. Theory52(2), 489–509 (2006).
    [CrossRef]
  10. D. Donoho, “Compressed Sensing,” IEEE Trans. Inf. Theory52(4), 1289–1306 (2006).
    [CrossRef]
  11. M. Wakin, J. Laska, M. Duarte, D. Baron, S. Sarvotham, D. Takhar, K. Kelly, and R. Baraniuk, “An Architecture for Compressive Imaging,” Proc. Int. Conf. on Image Processing (2006).
  12. M. Duarte, M. Wakin, S. Sarvotham, D. Baron, and R. Baraniuk, “Distributed Compressed Sensing of Jointly Sparse Signals,” Asilomar Conf. on Signals, Systems, and Computers 1537–1541 (2005).
  13. D. L. Donoho, Y. Tsaig, I. Drori, and J. L. Starck, “Sparse Solution of Underdetermined Linear Equations by Stagewise Orthogonal Matching Pursuit,” Dep. of Stat., Stanford Univ., Technical Report 2006–2, April (2006).
  14. R. Muise, “Compressive imaging: An application,” SIAM J. Imaging Science2(4), 1255–1276 (2009).
    [CrossRef]
  15. D. Bottisti and R. Muise, “Image exploitation from encoded measurements,” Proc. SPIE 8165816518 (2011).
  16. M. Aharon, M. Elad, and A. M. Bruckstein, “The K-SVD: An algorithm for designing of overcomplete dictionaries for sparse representation,” IEEE Trans. Signal Process.54(11), 4311–4322 (2006).
    [CrossRef]
  17. J. Tropp and A. Gilbert, “Signal recovery from random measurements via orthogonal matching pursuit,” IEEE Trans. Inf. Theory53(12), 4655–4666 (2007).
    [CrossRef]
  18. Y. Rivenson and A. Stern, “Compressed imaging with a separable sensing operator,” IEEE Signal Process. Lett.16(6), 449–452 (2009).
    [CrossRef]
  19. A. Stern, “Compressed imaging system with linear sensors,” Opt. Lett.32(21), 3077–3079 (2007).
    [CrossRef] [PubMed]
  20. Y. Rivenson and A. Stern, “Conditions for practicing compressive Fresnel holography,” Opt. Lett.36(17), 3365–3367 (2011).
    [CrossRef] [PubMed]
  21. A. Van Nevel and A. Mahalanobis, “Comparative study of MACH filter variants using LADAR imagery,” Opt. Eng.42, 541–550 (2002).
    [CrossRef]
  22. E. Elhara, A. Stern, O. Hadar, and B. Javidi, “A hybrid compression method for integral images using discrete wavelet transform and discrete cosine transform,” IEEE JDT3, 321–325 (2007).

2011

D. Bottisti and R. Muise, “Image exploitation from encoded measurements,” Proc. SPIE 8165816518 (2011).

Y. Rivenson and A. Stern, “Conditions for practicing compressive Fresnel holography,” Opt. Lett.36(17), 3365–3367 (2011).
[CrossRef] [PubMed]

2009

Y. Rivenson and A. Stern, “Compressed imaging with a separable sensing operator,” IEEE Signal Process. Lett.16(6), 449–452 (2009).
[CrossRef]

R. Muise, “Compressive imaging: An application,” SIAM J. Imaging Science2(4), 1255–1276 (2009).
[CrossRef]

R. Martinez-Cuenca, G. Saavedra, M. Martinez-Corral, and B. Javidi, “Progress in 3-D multiperspective display by integral imaging,” Proc. IEEE97(6), 1067–1077 (2009).
[CrossRef]

J.-H. Park, K. Hong, and B. Lee, “Recent progress in three-dimensional information processing based on integral imaging,” Appl. Opt.48(34), H77–H94 (2009).
[CrossRef] [PubMed]

2007

A. Stern, “Compressed imaging system with linear sensors,” Opt. Lett.32(21), 3077–3079 (2007).
[CrossRef] [PubMed]

J. Tropp and A. Gilbert, “Signal recovery from random measurements via orthogonal matching pursuit,” IEEE Trans. Inf. Theory53(12), 4655–4666 (2007).
[CrossRef]

E. Elhara, A. Stern, O. Hadar, and B. Javidi, “A hybrid compression method for integral images using discrete wavelet transform and discrete cosine transform,” IEEE JDT3, 321–325 (2007).

2006

M. Aharon, M. Elad, and A. M. Bruckstein, “The K-SVD: An algorithm for designing of overcomplete dictionaries for sparse representation,” IEEE Trans. Signal Process.54(11), 4311–4322 (2006).
[CrossRef]

E. Candes, J. Romberg, and T. Tao, “Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information,” IEEE Trans. Inf. Theory52(2), 489–509 (2006).
[CrossRef]

D. Donoho, “Compressed Sensing,” IEEE Trans. Inf. Theory52(4), 1289–1306 (2006).
[CrossRef]

F. Okano, J. Arai, K. Mitani, and M. Okui, “Real-time integral imaging based on extremely high resolution video system,” Proc. IEEE94(3), 490–501 (2006).
[CrossRef]

A. Stern and B. Javidi, “Three-dimensional image sensing, visualization, and processing using integral imaging,” Proc. IEEE94(3), 591–607 (2006).
[CrossRef]

2004

2002

A. Van Nevel and A. Mahalanobis, “Comparative study of MACH filter variants using LADAR imagery,” Opt. Eng.42, 541–550 (2002).
[CrossRef]

J.-S. Jang and B. Javidi, “Three-dimensional synthetic aperture integral imaging,” Opt. Lett.27(13), 1144–1146 (2002).
[CrossRef] [PubMed]

1988

1908

G. Lippmann, “La Photographie Integrale,” C.-R. Acad. des Sci.146, 446–451 (1908).

Aharon, M.

M. Aharon, M. Elad, and A. M. Bruckstein, “The K-SVD: An algorithm for designing of overcomplete dictionaries for sparse representation,” IEEE Trans. Signal Process.54(11), 4311–4322 (2006).
[CrossRef]

Arai, J.

F. Okano, J. Arai, K. Mitani, and M. Okui, “Real-time integral imaging based on extremely high resolution video system,” Proc. IEEE94(3), 490–501 (2006).
[CrossRef]

Bottisti, D.

D. Bottisti and R. Muise, “Image exploitation from encoded measurements,” Proc. SPIE 8165816518 (2011).

Bruckstein, A. M.

M. Aharon, M. Elad, and A. M. Bruckstein, “The K-SVD: An algorithm for designing of overcomplete dictionaries for sparse representation,” IEEE Trans. Signal Process.54(11), 4311–4322 (2006).
[CrossRef]

Candes, E.

E. Candes, J. Romberg, and T. Tao, “Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information,” IEEE Trans. Inf. Theory52(2), 489–509 (2006).
[CrossRef]

Davies, N.

Donoho, D.

D. Donoho, “Compressed Sensing,” IEEE Trans. Inf. Theory52(4), 1289–1306 (2006).
[CrossRef]

Elad, M.

M. Aharon, M. Elad, and A. M. Bruckstein, “The K-SVD: An algorithm for designing of overcomplete dictionaries for sparse representation,” IEEE Trans. Signal Process.54(11), 4311–4322 (2006).
[CrossRef]

Elhara, E.

E. Elhara, A. Stern, O. Hadar, and B. Javidi, “A hybrid compression method for integral images using discrete wavelet transform and discrete cosine transform,” IEEE JDT3, 321–325 (2007).

Gilbert, A.

J. Tropp and A. Gilbert, “Signal recovery from random measurements via orthogonal matching pursuit,” IEEE Trans. Inf. Theory53(12), 4655–4666 (2007).
[CrossRef]

Hadar, O.

E. Elhara, A. Stern, O. Hadar, and B. Javidi, “A hybrid compression method for integral images using discrete wavelet transform and discrete cosine transform,” IEEE JDT3, 321–325 (2007).

Hong, K.

Hong, S.-H.

Jang, J.-S.

Javidi, B.

R. Martinez-Cuenca, G. Saavedra, M. Martinez-Corral, and B. Javidi, “Progress in 3-D multiperspective display by integral imaging,” Proc. IEEE97(6), 1067–1077 (2009).
[CrossRef]

E. Elhara, A. Stern, O. Hadar, and B. Javidi, “A hybrid compression method for integral images using discrete wavelet transform and discrete cosine transform,” IEEE JDT3, 321–325 (2007).

A. Stern and B. Javidi, “Three-dimensional image sensing, visualization, and processing using integral imaging,” Proc. IEEE94(3), 591–607 (2006).
[CrossRef]

S.-H. Hong, J.-S. Jang, and B. Javidi, “Three-dimensional volumetric object reconstruction using computational integral imaging,” Opt. Express12(3), 483–491 (2004).
[CrossRef] [PubMed]

J.-S. Jang and B. Javidi, “Three-dimensional synthetic aperture integral imaging,” Opt. Lett.27(13), 1144–1146 (2002).
[CrossRef] [PubMed]

Lee, B.

Lippmann, G.

G. Lippmann, “La Photographie Integrale,” C.-R. Acad. des Sci.146, 446–451 (1908).

Mahalanobis, A.

A. Van Nevel and A. Mahalanobis, “Comparative study of MACH filter variants using LADAR imagery,” Opt. Eng.42, 541–550 (2002).
[CrossRef]

Martinez-Corral, M.

R. Martinez-Cuenca, G. Saavedra, M. Martinez-Corral, and B. Javidi, “Progress in 3-D multiperspective display by integral imaging,” Proc. IEEE97(6), 1067–1077 (2009).
[CrossRef]

Martinez-Cuenca, R.

R. Martinez-Cuenca, G. Saavedra, M. Martinez-Corral, and B. Javidi, “Progress in 3-D multiperspective display by integral imaging,” Proc. IEEE97(6), 1067–1077 (2009).
[CrossRef]

McCormick, M.

Mitani, K.

F. Okano, J. Arai, K. Mitani, and M. Okui, “Real-time integral imaging based on extremely high resolution video system,” Proc. IEEE94(3), 490–501 (2006).
[CrossRef]

Muise, R.

D. Bottisti and R. Muise, “Image exploitation from encoded measurements,” Proc. SPIE 8165816518 (2011).

R. Muise, “Compressive imaging: An application,” SIAM J. Imaging Science2(4), 1255–1276 (2009).
[CrossRef]

Okano, F.

F. Okano, J. Arai, K. Mitani, and M. Okui, “Real-time integral imaging based on extremely high resolution video system,” Proc. IEEE94(3), 490–501 (2006).
[CrossRef]

Okui, M.

F. Okano, J. Arai, K. Mitani, and M. Okui, “Real-time integral imaging based on extremely high resolution video system,” Proc. IEEE94(3), 490–501 (2006).
[CrossRef]

Park, J.-H.

Rivenson, Y.

Y. Rivenson and A. Stern, “Conditions for practicing compressive Fresnel holography,” Opt. Lett.36(17), 3365–3367 (2011).
[CrossRef] [PubMed]

Y. Rivenson and A. Stern, “Compressed imaging with a separable sensing operator,” IEEE Signal Process. Lett.16(6), 449–452 (2009).
[CrossRef]

Romberg, J.

E. Candes, J. Romberg, and T. Tao, “Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information,” IEEE Trans. Inf. Theory52(2), 489–509 (2006).
[CrossRef]

Saavedra, G.

R. Martinez-Cuenca, G. Saavedra, M. Martinez-Corral, and B. Javidi, “Progress in 3-D multiperspective display by integral imaging,” Proc. IEEE97(6), 1067–1077 (2009).
[CrossRef]

Stern, A.

Y. Rivenson and A. Stern, “Conditions for practicing compressive Fresnel holography,” Opt. Lett.36(17), 3365–3367 (2011).
[CrossRef] [PubMed]

Y. Rivenson and A. Stern, “Compressed imaging with a separable sensing operator,” IEEE Signal Process. Lett.16(6), 449–452 (2009).
[CrossRef]

A. Stern, “Compressed imaging system with linear sensors,” Opt. Lett.32(21), 3077–3079 (2007).
[CrossRef] [PubMed]

E. Elhara, A. Stern, O. Hadar, and B. Javidi, “A hybrid compression method for integral images using discrete wavelet transform and discrete cosine transform,” IEEE JDT3, 321–325 (2007).

A. Stern and B. Javidi, “Three-dimensional image sensing, visualization, and processing using integral imaging,” Proc. IEEE94(3), 591–607 (2006).
[CrossRef]

Tao, T.

E. Candes, J. Romberg, and T. Tao, “Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information,” IEEE Trans. Inf. Theory52(2), 489–509 (2006).
[CrossRef]

Tropp, J.

J. Tropp and A. Gilbert, “Signal recovery from random measurements via orthogonal matching pursuit,” IEEE Trans. Inf. Theory53(12), 4655–4666 (2007).
[CrossRef]

Van Nevel, A.

A. Van Nevel and A. Mahalanobis, “Comparative study of MACH filter variants using LADAR imagery,” Opt. Eng.42, 541–550 (2002).
[CrossRef]

Yang, L.

Appl. Opt.

C.-R. Acad. des Sci.

G. Lippmann, “La Photographie Integrale,” C.-R. Acad. des Sci.146, 446–451 (1908).

IEEE JDT

E. Elhara, A. Stern, O. Hadar, and B. Javidi, “A hybrid compression method for integral images using discrete wavelet transform and discrete cosine transform,” IEEE JDT3, 321–325 (2007).

IEEE Signal Process. Lett.

Y. Rivenson and A. Stern, “Compressed imaging with a separable sensing operator,” IEEE Signal Process. Lett.16(6), 449–452 (2009).
[CrossRef]

IEEE Trans. Inf. Theory

E. Candes, J. Romberg, and T. Tao, “Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information,” IEEE Trans. Inf. Theory52(2), 489–509 (2006).
[CrossRef]

D. Donoho, “Compressed Sensing,” IEEE Trans. Inf. Theory52(4), 1289–1306 (2006).
[CrossRef]

J. Tropp and A. Gilbert, “Signal recovery from random measurements via orthogonal matching pursuit,” IEEE Trans. Inf. Theory53(12), 4655–4666 (2007).
[CrossRef]

IEEE Trans. Signal Process.

M. Aharon, M. Elad, and A. M. Bruckstein, “The K-SVD: An algorithm for designing of overcomplete dictionaries for sparse representation,” IEEE Trans. Signal Process.54(11), 4311–4322 (2006).
[CrossRef]

Opt. Eng.

A. Van Nevel and A. Mahalanobis, “Comparative study of MACH filter variants using LADAR imagery,” Opt. Eng.42, 541–550 (2002).
[CrossRef]

Opt. Express

Opt. Lett.

Proc. IEEE

F. Okano, J. Arai, K. Mitani, and M. Okui, “Real-time integral imaging based on extremely high resolution video system,” Proc. IEEE94(3), 490–501 (2006).
[CrossRef]

A. Stern and B. Javidi, “Three-dimensional image sensing, visualization, and processing using integral imaging,” Proc. IEEE94(3), 591–607 (2006).
[CrossRef]

R. Martinez-Cuenca, G. Saavedra, M. Martinez-Corral, and B. Javidi, “Progress in 3-D multiperspective display by integral imaging,” Proc. IEEE97(6), 1067–1077 (2009).
[CrossRef]

Proc. SPIE 8165

D. Bottisti and R. Muise, “Image exploitation from encoded measurements,” Proc. SPIE 8165816518 (2011).

SIAM J. Imaging Science

R. Muise, “Compressive imaging: An application,” SIAM J. Imaging Science2(4), 1255–1276 (2009).
[CrossRef]

Other

M. Wakin, J. Laska, M. Duarte, D. Baron, S. Sarvotham, D. Takhar, K. Kelly, and R. Baraniuk, “An Architecture for Compressive Imaging,” Proc. Int. Conf. on Image Processing (2006).

M. Duarte, M. Wakin, S. Sarvotham, D. Baron, and R. Baraniuk, “Distributed Compressed Sensing of Jointly Sparse Signals,” Asilomar Conf. on Signals, Systems, and Computers 1537–1541 (2005).

D. L. Donoho, Y. Tsaig, I. Drori, and J. L. Starck, “Sparse Solution of Underdetermined Linear Equations by Stagewise Orthogonal Matching Pursuit,” Dep. of Stat., Stanford Univ., Technical Report 2006–2, April (2006).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (10)

Fig. 1
Fig. 1

3D integral imaging. (a) pickup and (b) computational reconstruction.

Fig. 2
Fig. 2

(a): 8 × 8 Mask. (b): The sampling Matrix for Mask.

Fig. 3
Fig. 3

Truth image (left). Sensed information (middle). Reconstructed with K-SVD dictionary and OMP algorithm (right).

Fig. 4
Fig. 4

The ideal elemental image in (a) was compressively sensed and reconstructed with 1/16th fewer measurements than pixels in (b), and with 1/64th fewer measurements than pixels in (c).

Fig. 5
Fig. 5

Procedure of 3D integral imaging with compressive sensing.

Fig. 6
Fig. 6

Elemental images with and without background obtained by (a) and (b) conventional integral imaging, (c) and (d) 16x compressive sensing, (e) and (f) 64x compressive sensing.

Fig. 7
Fig. 7

Reconstructed elemental images using K-SVD dictionary and OMP algorithm; (a)-(b) original elemental images with and without background, (c)-(d) from 16x compressively sensed elemental images, (e)-(f) from 64x compressively sensed elemental images.

Fig. 8
Fig. 8

3D reconstruction results at z = 440mm by using (a) and (b) original elemental images, (c) and (d) 16x compressively sensed elemental images, (e) and (f) 64x compressively sensed elemental images.

Fig. 9
Fig. 9

Peak signal to noise ratio for 3D integral imaging with compressive sensing.

Fig. 10
Fig. 10

Peak to Sidelobe Ratio (PSR) results for each compressive sensing.

Equations (8)

Equations on this page are rendered with MathJax. Learn more.

Ai=y
Ai=y, Aϕc=y, ψc=y,
Aϕc=y
S x = f p x N x c x z , S y = f p y N y c y z
I z 16x ( x,y )= 1 O( x,y ) k=0 K1 l=0 L1 I ^ kl 16x ( x+k S x ,y+l S y )
I z 64x ( x,y )= 1 O( x,y ) k=0 K1 l=0 L1 I ^ kl 64x ( x+k S x ,y+l S y )
PSNR=10 log 10 ( I max 2 MSE ),
H( u,v )= M( u,v ) αD( u,v )+βS( u,v )+γC( u,v )

Metrics