Abstract

The compressive sensing paradigm exploits the inherent sparsity/compressibility of signals to reduce the number of measurements required for reliable reconstruction/recovery. In many applications additional prior information beyond signal sparsity, such as structure in sparsity, is available, and current efforts are mainly limited to exploiting that information exclusively in the signal reconstruction problem. In this work, we describe an information-theoretic framework that incorporates the additional prior information as well as appropriate measurement constraints in the design of compressive measurements. Using a Gaussian binomial mixture prior we design and analyze the performance of optimized projections relative to random projections under two specific design constraints and different operating measurement signal-to-noise ratio (SNR) regimes. We find that the information-optimized designs yield significant, in some cases nearly an order of magnitude, improvements in the reconstruction performance with respect to the random projections. These improvements are especially notable in the low measurement SNR regime where the energy-efficient design of optimized projections is most advantageous. In such cases, the optimized projection design departs significantly from random projections in terms of their incoherence with the representation basis. In fact, we find that the maximizing incoherence of projections with the representation basis is not necessarily optimal in the presence of additional prior information and finite measurement noise/error. We also apply the information-optimized projections to the compressive image formation problem for natural scenes, and the improved visual quality of reconstructed images with respect to random projections and other compressive measurement design affirms the overall effectiveness of the information-theoretic design framework.

© 2013 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. E. T. Whittaker, “On the functions which are represented by the expansions of the interpolation theory,” Proc. R. Soc. Edinburgh 35, 181–194 (1915).
  2. H. Nyquist, “Certain topics in telegraph transmission theory,” Trans. Am. Inst. Electr. Eng. 47, 617–644 (1928).
    [CrossRef]
  3. C. E. Shannon, “Communication in the presence of noise,” Proc. IRE 37, 10–21 (1949).
    [CrossRef]
  4. W. Chen and W. Pratt, “Scene adaptive coder,” IEEE Trans. Commun. 32, 225–232 (1984).
    [CrossRef]
  5. G. Wallace, “The JPEG still picture compression standard,” Commun. ACM 34, 30–44 (1991).
    [CrossRef]
  6. D. Taubman and M. Marcellin, JPEG2000: Image Compression Fundamentals, Standards, and Practice (Kluwer, 2001).
  7. Y. Tsaig and D. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory 52, 1289–1306 (2006).
    [CrossRef]
  8. D. Donoho and Y. Tsaig, “Extensions of compressed sensing,” Signal Process. 86, 549–571 (2006).
    [CrossRef]
  9. E. Candes and J. Romberg, “Signal recovery from random projections,” Proc. SPIE 5674, 76–86 (2005).
    [CrossRef]
  10. E. Candes and T. Tal, “Near-optimal signal recovery from random projections: universal encoding strategies?,” IEEE Trans. Inf. Theory 52, 5406–5425 (2006).
    [CrossRef]
  11. F. Krahmer and R. Ward, “New and improved Johnson–Lindenstrauss embeddings via the restricted isometry property,” SIAM J. Math. Anal.43, 1269–1281 (2010).
  12. J. Romberg, “Imaging via compressing sampling,” IEEE Signal Process. Mag. 25, (2) 14–20 (2008).
    [CrossRef]
  13. M. A. Neifeld and P. Shankar, “Feature-specific imaging,” Appl. Opt. 42, 3379–3389 (2003).
    [CrossRef]
  14. H. Pal and M. A. Neifeld, “Multispectral principal component imaging,” Opt. Express 11, 2118–2125 (2003).
    [CrossRef]
  15. M. A. Neifeld and J. Ke, “Optical architectures for compressive imaging,” Appl. Opt. 46, 5293–5303 (2007).
    [CrossRef]
  16. A. Ashok and M. A. Neifeld, “Compressive imaging: hybrid measurement basis design,” J. Opt. Soc. Am. A 28, 1041–1050 (2011).
    [CrossRef]
  17. P. B. Fellgett, “On the ultimate sensitivity and practical performance of radiation detectors,” J. Opt. Soc. Am. 39, 970–976 (1949).
    [CrossRef]
  18. E. Candes and T. Tao, “Decoding by linear programming,” IEEE Trans. Inf. Theory 51, 4203–4215 (2005).
    [CrossRef]
  19. H. Rauhut, K. Schnass, and P. Vandergheynst, “Compressed sensing and redundant dictionaries,” IEEE Trans. Inf. Theory 54, 2210–2219 (2008).
    [CrossRef]
  20. L. He and L. Carin, “Exploiting structure in wavelet-based Bayesian compressive sensing,” IEEE Trans. Signal Process. 57, 3488–3497 (2009).
    [CrossRef]
  21. Y. C. Eldar and M. Mishali, “Robust recovery of signals from a structured union of subspaces,” IEEE Trans. Inf. Theory 55, 5302–5316 (2009).
    [CrossRef]
  22. M. Chen, J. Silva, J. Paisley, C. Wang, D. Dunson, and L. Carin, “Compressive sensing on manifolds using a nonparametric mixture of factor analyzers: algorithm and performance bounds,” IEEE Trans. Signal Process. 58, 6140–6155 (2010).
    [CrossRef]
  23. R. G. Baraniuk, V. Cevher, M. F. Duarte, and C. Hegde, “Model-based compressive sensing,” IEEE Trans. Inf. Theory 56, 1982–2001 (2010).
    [CrossRef]
  24. M. Elad, “Optimized projections for compressed sensing,” IEEE Trans. Signal Process. 55, 5695–5702 (2007).
    [CrossRef]
  25. G. Puy, P. Vandergheynst, and Y. Wiaux, “On variable density compressive sampling,” IEEE Signal Process. Lett. 18, 595–598 (2011).
    [CrossRef]
  26. H. S. Chang, Y. Weiss, and W. T. Freeman, “Informative sensing,” submitted to IEEE Trans. Inf. Theory, preprint available at arXiv http://arxiv.org/abs/0901.4275 (2009).
  27. W. R. Carson, M. Chen, M. R. D. Rodrigues, R. Calderbank, and L. Carin, “Communications-inspired projection design with application to compressive sensing,” preprint available at arXiv:1206.1973 http://arxiv.org/abs/1206.1973 (2012).
  28. R. M. Willett, R. F. Marcia, and J. M. Nichols, “Compressed sensing for practical optical imaging systems: a tutorial,” Opt. Eng. 50, 072601 (2012).
    [CrossRef]
  29. J. Portilla, V. Strela, M. J. Wainwright, and E. P. Simoncelli, “Image denoising using scale mixtures of Gaussians in the wavelet domain,” IEEE Trans. Image Process. 12, 1338–1351 (2003).
    [CrossRef]
  30. E. Candes and J. Romberg, “Sparsity and incoherence in compressive sampling,” Inverse Problems 23, 969–985 (2007).
    [CrossRef]
  31. S. Ji and L. Carin, “Bayesian compressive sensing and projection optimization,” in Proceedings of the 24th International Conference on Machine Learning (ICML) (ACM, 2007), pp. 377–384.
  32. M. W. Seeger and H. Nickisch, “Compressed sensing and Bayesian experimental design,” in Proceedings of the 25th International Conference on Machine Learning (ICML) (ACM, 2008), pp. 912–919.
  33. M. A. Neifeld, A. Ashok, and P. K. Baheti, “Task-specific information for imaging system analysis,” J. Opt. Soc. Am. A 24, B25–B41 (2007).
    [CrossRef]
  34. A. Ashok, P. K. Baheti, and M. A. Neifeld, “Compressive imaging system design using task-specific information,” Appl. Opt. 47, 4457–4471 (2008).
    [CrossRef]
  35. D. Stowell and M. D. Plumbley, “Fast multidimensional entropy estimation by k-d partitioning,” IEEE Signal Process. Lett. 16, 537–540 (2009).
    [CrossRef]
  36. M. A. Tanner, Tools for Statistical Inference: Methods for the Exploration of Posterior Distributions and Likelihood Functions, 3rd ed. (Springer-Verlag, 1996).
  37. D. J. Tolhurst, Y. Tadmor, and T. Chao, “Amplitude spectra of natural images,” Ophthalmic Physiolog. Opt. 12, 229–232 (1992).
    [CrossRef]
  38. D. L. Ruderman, “Origins of scaling in natural images,” Vis. Res. 37, 3385–3398 (1997).
    [CrossRef]
  39. E. P. Simoncelli and B. A. Olshausen, “Natural image statistics and neural representation,” Annu. Rev. Neurosci. 24, 1193–1216 (2001).
    [CrossRef]
  40. USC-SIPI Image Database. Available at http://sipi.usc.edu/database .
  41. H. Lee, A. Battle, R. Raina, and A. Y. Ng, “Efficient sparse coding algorithms,” in Proceedings of Advances in Neural Information Processing Systems (NIPS), Vol. 19 (MIT, 2007), pp. 801–808.
  42. Y. Yang, J. Wright, T. S. Huang, and M. Yi, “Image super-resolution via sparse representation,” IEEE Trans. Image Process. 19, 2861–2873 (2010).
    [CrossRef]

2012 (1)

R. M. Willett, R. F. Marcia, and J. M. Nichols, “Compressed sensing for practical optical imaging systems: a tutorial,” Opt. Eng. 50, 072601 (2012).
[CrossRef]

2011 (2)

A. Ashok and M. A. Neifeld, “Compressive imaging: hybrid measurement basis design,” J. Opt. Soc. Am. A 28, 1041–1050 (2011).
[CrossRef]

G. Puy, P. Vandergheynst, and Y. Wiaux, “On variable density compressive sampling,” IEEE Signal Process. Lett. 18, 595–598 (2011).
[CrossRef]

2010 (3)

M. Chen, J. Silva, J. Paisley, C. Wang, D. Dunson, and L. Carin, “Compressive sensing on manifolds using a nonparametric mixture of factor analyzers: algorithm and performance bounds,” IEEE Trans. Signal Process. 58, 6140–6155 (2010).
[CrossRef]

R. G. Baraniuk, V. Cevher, M. F. Duarte, and C. Hegde, “Model-based compressive sensing,” IEEE Trans. Inf. Theory 56, 1982–2001 (2010).
[CrossRef]

Y. Yang, J. Wright, T. S. Huang, and M. Yi, “Image super-resolution via sparse representation,” IEEE Trans. Image Process. 19, 2861–2873 (2010).
[CrossRef]

2009 (3)

D. Stowell and M. D. Plumbley, “Fast multidimensional entropy estimation by k-d partitioning,” IEEE Signal Process. Lett. 16, 537–540 (2009).
[CrossRef]

L. He and L. Carin, “Exploiting structure in wavelet-based Bayesian compressive sensing,” IEEE Trans. Signal Process. 57, 3488–3497 (2009).
[CrossRef]

Y. C. Eldar and M. Mishali, “Robust recovery of signals from a structured union of subspaces,” IEEE Trans. Inf. Theory 55, 5302–5316 (2009).
[CrossRef]

2008 (3)

J. Romberg, “Imaging via compressing sampling,” IEEE Signal Process. Mag. 25, (2) 14–20 (2008).
[CrossRef]

H. Rauhut, K. Schnass, and P. Vandergheynst, “Compressed sensing and redundant dictionaries,” IEEE Trans. Inf. Theory 54, 2210–2219 (2008).
[CrossRef]

A. Ashok, P. K. Baheti, and M. A. Neifeld, “Compressive imaging system design using task-specific information,” Appl. Opt. 47, 4457–4471 (2008).
[CrossRef]

2007 (4)

M. A. Neifeld and J. Ke, “Optical architectures for compressive imaging,” Appl. Opt. 46, 5293–5303 (2007).
[CrossRef]

M. A. Neifeld, A. Ashok, and P. K. Baheti, “Task-specific information for imaging system analysis,” J. Opt. Soc. Am. A 24, B25–B41 (2007).
[CrossRef]

E. Candes and J. Romberg, “Sparsity and incoherence in compressive sampling,” Inverse Problems 23, 969–985 (2007).
[CrossRef]

M. Elad, “Optimized projections for compressed sensing,” IEEE Trans. Signal Process. 55, 5695–5702 (2007).
[CrossRef]

2006 (3)

Y. Tsaig and D. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory 52, 1289–1306 (2006).
[CrossRef]

D. Donoho and Y. Tsaig, “Extensions of compressed sensing,” Signal Process. 86, 549–571 (2006).
[CrossRef]

E. Candes and T. Tal, “Near-optimal signal recovery from random projections: universal encoding strategies?,” IEEE Trans. Inf. Theory 52, 5406–5425 (2006).
[CrossRef]

2005 (2)

E. Candes and J. Romberg, “Signal recovery from random projections,” Proc. SPIE 5674, 76–86 (2005).
[CrossRef]

E. Candes and T. Tao, “Decoding by linear programming,” IEEE Trans. Inf. Theory 51, 4203–4215 (2005).
[CrossRef]

2003 (3)

J. Portilla, V. Strela, M. J. Wainwright, and E. P. Simoncelli, “Image denoising using scale mixtures of Gaussians in the wavelet domain,” IEEE Trans. Image Process. 12, 1338–1351 (2003).
[CrossRef]

M. A. Neifeld and P. Shankar, “Feature-specific imaging,” Appl. Opt. 42, 3379–3389 (2003).
[CrossRef]

H. Pal and M. A. Neifeld, “Multispectral principal component imaging,” Opt. Express 11, 2118–2125 (2003).
[CrossRef]

2001 (1)

E. P. Simoncelli and B. A. Olshausen, “Natural image statistics and neural representation,” Annu. Rev. Neurosci. 24, 1193–1216 (2001).
[CrossRef]

1997 (1)

D. L. Ruderman, “Origins of scaling in natural images,” Vis. Res. 37, 3385–3398 (1997).
[CrossRef]

1992 (1)

D. J. Tolhurst, Y. Tadmor, and T. Chao, “Amplitude spectra of natural images,” Ophthalmic Physiolog. Opt. 12, 229–232 (1992).
[CrossRef]

1991 (1)

G. Wallace, “The JPEG still picture compression standard,” Commun. ACM 34, 30–44 (1991).
[CrossRef]

1984 (1)

W. Chen and W. Pratt, “Scene adaptive coder,” IEEE Trans. Commun. 32, 225–232 (1984).
[CrossRef]

1949 (2)

1928 (1)

H. Nyquist, “Certain topics in telegraph transmission theory,” Trans. Am. Inst. Electr. Eng. 47, 617–644 (1928).
[CrossRef]

1915 (1)

E. T. Whittaker, “On the functions which are represented by the expansions of the interpolation theory,” Proc. R. Soc. Edinburgh 35, 181–194 (1915).

Ashok, A.

Baheti, P. K.

Baraniuk, R. G.

R. G. Baraniuk, V. Cevher, M. F. Duarte, and C. Hegde, “Model-based compressive sensing,” IEEE Trans. Inf. Theory 56, 1982–2001 (2010).
[CrossRef]

Battle, A.

H. Lee, A. Battle, R. Raina, and A. Y. Ng, “Efficient sparse coding algorithms,” in Proceedings of Advances in Neural Information Processing Systems (NIPS), Vol. 19 (MIT, 2007), pp. 801–808.

Calderbank, R.

W. R. Carson, M. Chen, M. R. D. Rodrigues, R. Calderbank, and L. Carin, “Communications-inspired projection design with application to compressive sensing,” preprint available at arXiv:1206.1973 http://arxiv.org/abs/1206.1973 (2012).

Candes, E.

E. Candes and J. Romberg, “Sparsity and incoherence in compressive sampling,” Inverse Problems 23, 969–985 (2007).
[CrossRef]

E. Candes and T. Tal, “Near-optimal signal recovery from random projections: universal encoding strategies?,” IEEE Trans. Inf. Theory 52, 5406–5425 (2006).
[CrossRef]

E. Candes and J. Romberg, “Signal recovery from random projections,” Proc. SPIE 5674, 76–86 (2005).
[CrossRef]

E. Candes and T. Tao, “Decoding by linear programming,” IEEE Trans. Inf. Theory 51, 4203–4215 (2005).
[CrossRef]

Carin, L.

M. Chen, J. Silva, J. Paisley, C. Wang, D. Dunson, and L. Carin, “Compressive sensing on manifolds using a nonparametric mixture of factor analyzers: algorithm and performance bounds,” IEEE Trans. Signal Process. 58, 6140–6155 (2010).
[CrossRef]

L. He and L. Carin, “Exploiting structure in wavelet-based Bayesian compressive sensing,” IEEE Trans. Signal Process. 57, 3488–3497 (2009).
[CrossRef]

S. Ji and L. Carin, “Bayesian compressive sensing and projection optimization,” in Proceedings of the 24th International Conference on Machine Learning (ICML) (ACM, 2007), pp. 377–384.

W. R. Carson, M. Chen, M. R. D. Rodrigues, R. Calderbank, and L. Carin, “Communications-inspired projection design with application to compressive sensing,” preprint available at arXiv:1206.1973 http://arxiv.org/abs/1206.1973 (2012).

Carson, W. R.

W. R. Carson, M. Chen, M. R. D. Rodrigues, R. Calderbank, and L. Carin, “Communications-inspired projection design with application to compressive sensing,” preprint available at arXiv:1206.1973 http://arxiv.org/abs/1206.1973 (2012).

Cevher, V.

R. G. Baraniuk, V. Cevher, M. F. Duarte, and C. Hegde, “Model-based compressive sensing,” IEEE Trans. Inf. Theory 56, 1982–2001 (2010).
[CrossRef]

Chang, H. S.

H. S. Chang, Y. Weiss, and W. T. Freeman, “Informative sensing,” submitted to IEEE Trans. Inf. Theory, preprint available at arXiv http://arxiv.org/abs/0901.4275 (2009).

Chao, T.

D. J. Tolhurst, Y. Tadmor, and T. Chao, “Amplitude spectra of natural images,” Ophthalmic Physiolog. Opt. 12, 229–232 (1992).
[CrossRef]

Chen, M.

M. Chen, J. Silva, J. Paisley, C. Wang, D. Dunson, and L. Carin, “Compressive sensing on manifolds using a nonparametric mixture of factor analyzers: algorithm and performance bounds,” IEEE Trans. Signal Process. 58, 6140–6155 (2010).
[CrossRef]

W. R. Carson, M. Chen, M. R. D. Rodrigues, R. Calderbank, and L. Carin, “Communications-inspired projection design with application to compressive sensing,” preprint available at arXiv:1206.1973 http://arxiv.org/abs/1206.1973 (2012).

Chen, W.

W. Chen and W. Pratt, “Scene adaptive coder,” IEEE Trans. Commun. 32, 225–232 (1984).
[CrossRef]

Donoho, D.

D. Donoho and Y. Tsaig, “Extensions of compressed sensing,” Signal Process. 86, 549–571 (2006).
[CrossRef]

Y. Tsaig and D. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory 52, 1289–1306 (2006).
[CrossRef]

Duarte, M. F.

R. G. Baraniuk, V. Cevher, M. F. Duarte, and C. Hegde, “Model-based compressive sensing,” IEEE Trans. Inf. Theory 56, 1982–2001 (2010).
[CrossRef]

Dunson, D.

M. Chen, J. Silva, J. Paisley, C. Wang, D. Dunson, and L. Carin, “Compressive sensing on manifolds using a nonparametric mixture of factor analyzers: algorithm and performance bounds,” IEEE Trans. Signal Process. 58, 6140–6155 (2010).
[CrossRef]

Elad, M.

M. Elad, “Optimized projections for compressed sensing,” IEEE Trans. Signal Process. 55, 5695–5702 (2007).
[CrossRef]

Eldar, Y. C.

Y. C. Eldar and M. Mishali, “Robust recovery of signals from a structured union of subspaces,” IEEE Trans. Inf. Theory 55, 5302–5316 (2009).
[CrossRef]

Fellgett, P. B.

Freeman, W. T.

H. S. Chang, Y. Weiss, and W. T. Freeman, “Informative sensing,” submitted to IEEE Trans. Inf. Theory, preprint available at arXiv http://arxiv.org/abs/0901.4275 (2009).

He, L.

L. He and L. Carin, “Exploiting structure in wavelet-based Bayesian compressive sensing,” IEEE Trans. Signal Process. 57, 3488–3497 (2009).
[CrossRef]

Hegde, C.

R. G. Baraniuk, V. Cevher, M. F. Duarte, and C. Hegde, “Model-based compressive sensing,” IEEE Trans. Inf. Theory 56, 1982–2001 (2010).
[CrossRef]

Huang, T. S.

Y. Yang, J. Wright, T. S. Huang, and M. Yi, “Image super-resolution via sparse representation,” IEEE Trans. Image Process. 19, 2861–2873 (2010).
[CrossRef]

Ji, S.

S. Ji and L. Carin, “Bayesian compressive sensing and projection optimization,” in Proceedings of the 24th International Conference on Machine Learning (ICML) (ACM, 2007), pp. 377–384.

Ke, J.

Krahmer, F.

F. Krahmer and R. Ward, “New and improved Johnson–Lindenstrauss embeddings via the restricted isometry property,” SIAM J. Math. Anal.43, 1269–1281 (2010).

Lee, H.

H. Lee, A. Battle, R. Raina, and A. Y. Ng, “Efficient sparse coding algorithms,” in Proceedings of Advances in Neural Information Processing Systems (NIPS), Vol. 19 (MIT, 2007), pp. 801–808.

Marcellin, M.

D. Taubman and M. Marcellin, JPEG2000: Image Compression Fundamentals, Standards, and Practice (Kluwer, 2001).

Marcia, R. F.

R. M. Willett, R. F. Marcia, and J. M. Nichols, “Compressed sensing for practical optical imaging systems: a tutorial,” Opt. Eng. 50, 072601 (2012).
[CrossRef]

Mishali, M.

Y. C. Eldar and M. Mishali, “Robust recovery of signals from a structured union of subspaces,” IEEE Trans. Inf. Theory 55, 5302–5316 (2009).
[CrossRef]

Neifeld, M. A.

Ng, A. Y.

H. Lee, A. Battle, R. Raina, and A. Y. Ng, “Efficient sparse coding algorithms,” in Proceedings of Advances in Neural Information Processing Systems (NIPS), Vol. 19 (MIT, 2007), pp. 801–808.

Nichols, J. M.

R. M. Willett, R. F. Marcia, and J. M. Nichols, “Compressed sensing for practical optical imaging systems: a tutorial,” Opt. Eng. 50, 072601 (2012).
[CrossRef]

Nickisch, H.

M. W. Seeger and H. Nickisch, “Compressed sensing and Bayesian experimental design,” in Proceedings of the 25th International Conference on Machine Learning (ICML) (ACM, 2008), pp. 912–919.

Nyquist, H.

H. Nyquist, “Certain topics in telegraph transmission theory,” Trans. Am. Inst. Electr. Eng. 47, 617–644 (1928).
[CrossRef]

Olshausen, B. A.

E. P. Simoncelli and B. A. Olshausen, “Natural image statistics and neural representation,” Annu. Rev. Neurosci. 24, 1193–1216 (2001).
[CrossRef]

Paisley, J.

M. Chen, J. Silva, J. Paisley, C. Wang, D. Dunson, and L. Carin, “Compressive sensing on manifolds using a nonparametric mixture of factor analyzers: algorithm and performance bounds,” IEEE Trans. Signal Process. 58, 6140–6155 (2010).
[CrossRef]

Pal, H.

Plumbley, M. D.

D. Stowell and M. D. Plumbley, “Fast multidimensional entropy estimation by k-d partitioning,” IEEE Signal Process. Lett. 16, 537–540 (2009).
[CrossRef]

Portilla, J.

J. Portilla, V. Strela, M. J. Wainwright, and E. P. Simoncelli, “Image denoising using scale mixtures of Gaussians in the wavelet domain,” IEEE Trans. Image Process. 12, 1338–1351 (2003).
[CrossRef]

Pratt, W.

W. Chen and W. Pratt, “Scene adaptive coder,” IEEE Trans. Commun. 32, 225–232 (1984).
[CrossRef]

Puy, G.

G. Puy, P. Vandergheynst, and Y. Wiaux, “On variable density compressive sampling,” IEEE Signal Process. Lett. 18, 595–598 (2011).
[CrossRef]

Raina, R.

H. Lee, A. Battle, R. Raina, and A. Y. Ng, “Efficient sparse coding algorithms,” in Proceedings of Advances in Neural Information Processing Systems (NIPS), Vol. 19 (MIT, 2007), pp. 801–808.

Rauhut, H.

H. Rauhut, K. Schnass, and P. Vandergheynst, “Compressed sensing and redundant dictionaries,” IEEE Trans. Inf. Theory 54, 2210–2219 (2008).
[CrossRef]

Rodrigues, M. R. D.

W. R. Carson, M. Chen, M. R. D. Rodrigues, R. Calderbank, and L. Carin, “Communications-inspired projection design with application to compressive sensing,” preprint available at arXiv:1206.1973 http://arxiv.org/abs/1206.1973 (2012).

Romberg, J.

J. Romberg, “Imaging via compressing sampling,” IEEE Signal Process. Mag. 25, (2) 14–20 (2008).
[CrossRef]

E. Candes and J. Romberg, “Sparsity and incoherence in compressive sampling,” Inverse Problems 23, 969–985 (2007).
[CrossRef]

E. Candes and J. Romberg, “Signal recovery from random projections,” Proc. SPIE 5674, 76–86 (2005).
[CrossRef]

Ruderman, D. L.

D. L. Ruderman, “Origins of scaling in natural images,” Vis. Res. 37, 3385–3398 (1997).
[CrossRef]

Schnass, K.

H. Rauhut, K. Schnass, and P. Vandergheynst, “Compressed sensing and redundant dictionaries,” IEEE Trans. Inf. Theory 54, 2210–2219 (2008).
[CrossRef]

Seeger, M. W.

M. W. Seeger and H. Nickisch, “Compressed sensing and Bayesian experimental design,” in Proceedings of the 25th International Conference on Machine Learning (ICML) (ACM, 2008), pp. 912–919.

Shankar, P.

Shannon, C. E.

C. E. Shannon, “Communication in the presence of noise,” Proc. IRE 37, 10–21 (1949).
[CrossRef]

Silva, J.

M. Chen, J. Silva, J. Paisley, C. Wang, D. Dunson, and L. Carin, “Compressive sensing on manifolds using a nonparametric mixture of factor analyzers: algorithm and performance bounds,” IEEE Trans. Signal Process. 58, 6140–6155 (2010).
[CrossRef]

Simoncelli, E. P.

J. Portilla, V. Strela, M. J. Wainwright, and E. P. Simoncelli, “Image denoising using scale mixtures of Gaussians in the wavelet domain,” IEEE Trans. Image Process. 12, 1338–1351 (2003).
[CrossRef]

E. P. Simoncelli and B. A. Olshausen, “Natural image statistics and neural representation,” Annu. Rev. Neurosci. 24, 1193–1216 (2001).
[CrossRef]

Stowell, D.

D. Stowell and M. D. Plumbley, “Fast multidimensional entropy estimation by k-d partitioning,” IEEE Signal Process. Lett. 16, 537–540 (2009).
[CrossRef]

Strela, V.

J. Portilla, V. Strela, M. J. Wainwright, and E. P. Simoncelli, “Image denoising using scale mixtures of Gaussians in the wavelet domain,” IEEE Trans. Image Process. 12, 1338–1351 (2003).
[CrossRef]

Tadmor, Y.

D. J. Tolhurst, Y. Tadmor, and T. Chao, “Amplitude spectra of natural images,” Ophthalmic Physiolog. Opt. 12, 229–232 (1992).
[CrossRef]

Tal, T.

E. Candes and T. Tal, “Near-optimal signal recovery from random projections: universal encoding strategies?,” IEEE Trans. Inf. Theory 52, 5406–5425 (2006).
[CrossRef]

Tanner, M. A.

M. A. Tanner, Tools for Statistical Inference: Methods for the Exploration of Posterior Distributions and Likelihood Functions, 3rd ed. (Springer-Verlag, 1996).

Tao, T.

E. Candes and T. Tao, “Decoding by linear programming,” IEEE Trans. Inf. Theory 51, 4203–4215 (2005).
[CrossRef]

Taubman, D.

D. Taubman and M. Marcellin, JPEG2000: Image Compression Fundamentals, Standards, and Practice (Kluwer, 2001).

Tolhurst, D. J.

D. J. Tolhurst, Y. Tadmor, and T. Chao, “Amplitude spectra of natural images,” Ophthalmic Physiolog. Opt. 12, 229–232 (1992).
[CrossRef]

Tsaig, Y.

D. Donoho and Y. Tsaig, “Extensions of compressed sensing,” Signal Process. 86, 549–571 (2006).
[CrossRef]

Y. Tsaig and D. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory 52, 1289–1306 (2006).
[CrossRef]

Vandergheynst, P.

G. Puy, P. Vandergheynst, and Y. Wiaux, “On variable density compressive sampling,” IEEE Signal Process. Lett. 18, 595–598 (2011).
[CrossRef]

H. Rauhut, K. Schnass, and P. Vandergheynst, “Compressed sensing and redundant dictionaries,” IEEE Trans. Inf. Theory 54, 2210–2219 (2008).
[CrossRef]

Wainwright, M. J.

J. Portilla, V. Strela, M. J. Wainwright, and E. P. Simoncelli, “Image denoising using scale mixtures of Gaussians in the wavelet domain,” IEEE Trans. Image Process. 12, 1338–1351 (2003).
[CrossRef]

Wallace, G.

G. Wallace, “The JPEG still picture compression standard,” Commun. ACM 34, 30–44 (1991).
[CrossRef]

Wang, C.

M. Chen, J. Silva, J. Paisley, C. Wang, D. Dunson, and L. Carin, “Compressive sensing on manifolds using a nonparametric mixture of factor analyzers: algorithm and performance bounds,” IEEE Trans. Signal Process. 58, 6140–6155 (2010).
[CrossRef]

Ward, R.

F. Krahmer and R. Ward, “New and improved Johnson–Lindenstrauss embeddings via the restricted isometry property,” SIAM J. Math. Anal.43, 1269–1281 (2010).

Weiss, Y.

H. S. Chang, Y. Weiss, and W. T. Freeman, “Informative sensing,” submitted to IEEE Trans. Inf. Theory, preprint available at arXiv http://arxiv.org/abs/0901.4275 (2009).

Whittaker, E. T.

E. T. Whittaker, “On the functions which are represented by the expansions of the interpolation theory,” Proc. R. Soc. Edinburgh 35, 181–194 (1915).

Wiaux, Y.

G. Puy, P. Vandergheynst, and Y. Wiaux, “On variable density compressive sampling,” IEEE Signal Process. Lett. 18, 595–598 (2011).
[CrossRef]

Willett, R. M.

R. M. Willett, R. F. Marcia, and J. M. Nichols, “Compressed sensing for practical optical imaging systems: a tutorial,” Opt. Eng. 50, 072601 (2012).
[CrossRef]

Wright, J.

Y. Yang, J. Wright, T. S. Huang, and M. Yi, “Image super-resolution via sparse representation,” IEEE Trans. Image Process. 19, 2861–2873 (2010).
[CrossRef]

Yang, Y.

Y. Yang, J. Wright, T. S. Huang, and M. Yi, “Image super-resolution via sparse representation,” IEEE Trans. Image Process. 19, 2861–2873 (2010).
[CrossRef]

Yi, M.

Y. Yang, J. Wright, T. S. Huang, and M. Yi, “Image super-resolution via sparse representation,” IEEE Trans. Image Process. 19, 2861–2873 (2010).
[CrossRef]

Annu. Rev. Neurosci. (1)

E. P. Simoncelli and B. A. Olshausen, “Natural image statistics and neural representation,” Annu. Rev. Neurosci. 24, 1193–1216 (2001).
[CrossRef]

Appl. Opt. (3)

Commun. ACM (1)

G. Wallace, “The JPEG still picture compression standard,” Commun. ACM 34, 30–44 (1991).
[CrossRef]

IEEE Signal Process. Lett. (2)

D. Stowell and M. D. Plumbley, “Fast multidimensional entropy estimation by k-d partitioning,” IEEE Signal Process. Lett. 16, 537–540 (2009).
[CrossRef]

G. Puy, P. Vandergheynst, and Y. Wiaux, “On variable density compressive sampling,” IEEE Signal Process. Lett. 18, 595–598 (2011).
[CrossRef]

IEEE Signal Process. Mag. (1)

J. Romberg, “Imaging via compressing sampling,” IEEE Signal Process. Mag. 25, (2) 14–20 (2008).
[CrossRef]

IEEE Trans. Commun. (1)

W. Chen and W. Pratt, “Scene adaptive coder,” IEEE Trans. Commun. 32, 225–232 (1984).
[CrossRef]

IEEE Trans. Image Process. (2)

J. Portilla, V. Strela, M. J. Wainwright, and E. P. Simoncelli, “Image denoising using scale mixtures of Gaussians in the wavelet domain,” IEEE Trans. Image Process. 12, 1338–1351 (2003).
[CrossRef]

Y. Yang, J. Wright, T. S. Huang, and M. Yi, “Image super-resolution via sparse representation,” IEEE Trans. Image Process. 19, 2861–2873 (2010).
[CrossRef]

IEEE Trans. Inf. Theory (6)

R. G. Baraniuk, V. Cevher, M. F. Duarte, and C. Hegde, “Model-based compressive sensing,” IEEE Trans. Inf. Theory 56, 1982–2001 (2010).
[CrossRef]

Y. Tsaig and D. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory 52, 1289–1306 (2006).
[CrossRef]

E. Candes and T. Tao, “Decoding by linear programming,” IEEE Trans. Inf. Theory 51, 4203–4215 (2005).
[CrossRef]

H. Rauhut, K. Schnass, and P. Vandergheynst, “Compressed sensing and redundant dictionaries,” IEEE Trans. Inf. Theory 54, 2210–2219 (2008).
[CrossRef]

E. Candes and T. Tal, “Near-optimal signal recovery from random projections: universal encoding strategies?,” IEEE Trans. Inf. Theory 52, 5406–5425 (2006).
[CrossRef]

Y. C. Eldar and M. Mishali, “Robust recovery of signals from a structured union of subspaces,” IEEE Trans. Inf. Theory 55, 5302–5316 (2009).
[CrossRef]

IEEE Trans. Signal Process. (3)

M. Chen, J. Silva, J. Paisley, C. Wang, D. Dunson, and L. Carin, “Compressive sensing on manifolds using a nonparametric mixture of factor analyzers: algorithm and performance bounds,” IEEE Trans. Signal Process. 58, 6140–6155 (2010).
[CrossRef]

L. He and L. Carin, “Exploiting structure in wavelet-based Bayesian compressive sensing,” IEEE Trans. Signal Process. 57, 3488–3497 (2009).
[CrossRef]

M. Elad, “Optimized projections for compressed sensing,” IEEE Trans. Signal Process. 55, 5695–5702 (2007).
[CrossRef]

Inverse Problems (1)

E. Candes and J. Romberg, “Sparsity and incoherence in compressive sampling,” Inverse Problems 23, 969–985 (2007).
[CrossRef]

J. Opt. Soc. Am. (1)

J. Opt. Soc. Am. A (2)

Ophthalmic Physiolog. Opt. (1)

D. J. Tolhurst, Y. Tadmor, and T. Chao, “Amplitude spectra of natural images,” Ophthalmic Physiolog. Opt. 12, 229–232 (1992).
[CrossRef]

Opt. Eng. (1)

R. M. Willett, R. F. Marcia, and J. M. Nichols, “Compressed sensing for practical optical imaging systems: a tutorial,” Opt. Eng. 50, 072601 (2012).
[CrossRef]

Opt. Express (1)

Proc. IRE (1)

C. E. Shannon, “Communication in the presence of noise,” Proc. IRE 37, 10–21 (1949).
[CrossRef]

Proc. R. Soc. Edinburgh (1)

E. T. Whittaker, “On the functions which are represented by the expansions of the interpolation theory,” Proc. R. Soc. Edinburgh 35, 181–194 (1915).

Proc. SPIE (1)

E. Candes and J. Romberg, “Signal recovery from random projections,” Proc. SPIE 5674, 76–86 (2005).
[CrossRef]

Signal Process. (1)

D. Donoho and Y. Tsaig, “Extensions of compressed sensing,” Signal Process. 86, 549–571 (2006).
[CrossRef]

Trans. Am. Inst. Electr. Eng. (1)

H. Nyquist, “Certain topics in telegraph transmission theory,” Trans. Am. Inst. Electr. Eng. 47, 617–644 (1928).
[CrossRef]

Vis. Res. (1)

D. L. Ruderman, “Origins of scaling in natural images,” Vis. Res. 37, 3385–3398 (1997).
[CrossRef]

Other (9)

USC-SIPI Image Database. Available at http://sipi.usc.edu/database .

H. Lee, A. Battle, R. Raina, and A. Y. Ng, “Efficient sparse coding algorithms,” in Proceedings of Advances in Neural Information Processing Systems (NIPS), Vol. 19 (MIT, 2007), pp. 801–808.

M. A. Tanner, Tools for Statistical Inference: Methods for the Exploration of Posterior Distributions and Likelihood Functions, 3rd ed. (Springer-Verlag, 1996).

S. Ji and L. Carin, “Bayesian compressive sensing and projection optimization,” in Proceedings of the 24th International Conference on Machine Learning (ICML) (ACM, 2007), pp. 377–384.

M. W. Seeger and H. Nickisch, “Compressed sensing and Bayesian experimental design,” in Proceedings of the 25th International Conference on Machine Learning (ICML) (ACM, 2008), pp. 912–919.

H. S. Chang, Y. Weiss, and W. T. Freeman, “Informative sensing,” submitted to IEEE Trans. Inf. Theory, preprint available at arXiv http://arxiv.org/abs/0901.4275 (2009).

W. R. Carson, M. Chen, M. R. D. Rodrigues, R. Calderbank, and L. Carin, “Communications-inspired projection design with application to compressive sensing,” preprint available at arXiv:1206.1973 http://arxiv.org/abs/1206.1973 (2012).

D. Taubman and M. Marcellin, JPEG2000: Image Compression Fundamentals, Standards, and Practice (Kluwer, 2001).

F. Krahmer and R. Ward, “New and improved Johnson–Lindenstrauss embeddings via the restricted isometry property,” SIAM J. Math. Anal.43, 1269–1281 (2010).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (26)

Fig. 1.
Fig. 1.

Coefficient vectors realized using GBM prior with parameters zmax=1, q=0.1, N=16: (a) sparse coefficient vector (zmin=0) and (b) compressible coefficient vector (zmin=5×102).

Fig. 2.
Fig. 2.

Gaussian mixture priors: (a) GSM and (b) GBM. Note that the GBM prior is more concentrated around zero and is therefore more sparse/compressible relative to the GSM prior.

Fig. 3.
Fig. 3.

Example of measurement matrix with different mutual coherence for M=3 measurements and N=5 dimensional signal. The graph shows the connectivity between measurement vector elements and coefficient vector elements: (a) maximum coherence where each measurement is only connected to one transform coefficient, (b) lower coherence where each measurement is connected to more than one transform coefficient, and (c) minimal coherence where each measurement is connected to all transform coefficients.

Fig. 4.
Fig. 4.

Reconstruction error for TSI and random projections as a function of number of compressive measurements for two measurement SNR. Dense prior with (a) uniform variance model and (b) non-uniform variance model.

Fig. 5.
Fig. 5.

Plots of energy allocation for the TSI optimal projections with M=8 at (a) low SNR (20 dB) case and (b) high SNR (50 dB) case for a non-uniform GBM prior.

Fig. 6.
Fig. 6.

TSI optimal projection in the mixture direction that maximize incoherence with the representation basis for M=4 in high SNR regime (50 dB) for (a) uniform variance and (b) non-uniform variance GBM prior models.

Fig. 7.
Fig. 7.

TSI contour plot for first projection vector shown in (a) and plot of coherence (top) and TSI (bottom) as function of projection vector angle for the uniform variance sparse/compressible GBM prior.

Fig. 8.
Fig. 8.

Reconstruction error for TSI optimal and random projections as a function of number of compressive measurements for two measurement SNR. Sparse/compressible prior with (a) uniform variance model and (b) non-uniform variance model.

Fig. 9.
Fig. 9.

TSI contour plot for first projection vector shown in (a) and plot of coherence (top) and TSI (bottom) as function of projection vector angle for the non-uniform variance sparse/compressible GBM prior.

Fig. 10.
Fig. 10.

Reconstruction error for TSI and random projections, with photon-count constraint, as a function of number of compressive measurements for two measurement SNR. Dense prior with (a) uniform variance model and (b) non-uniform variance model.

Fig. 11.
Fig. 11.

TSI optimal projection vectors obtained with the photon-count constraint for (a) low SNR and (b) high SNR regimes using dense prior with non-uniform variance. Note the zero-photon allocation for projection vectors P⃗1 and P⃗6 in low SNR case.

Fig. 12.
Fig. 12.

TSI contour plot for first projection vector shown in (a) and plot of coherence (top) and TSI (bottom) as function of projection vector angle for the non-uniform variance sparse/compressible GBM prior.

Fig. 13.
Fig. 13.

Reconstruction error for TSI and random projections, with photon-count constraint, as a function of number of compressive measurements for two measurement SNR. Dense prior with (a) uniform variance model and (b) non-uniform variance model.

Fig. 14.
Fig. 14.

TSI contour plot for first projection vector shown in (a) and plot of coherence (top) and TSI (bottom) as function of projection vector angle for the non-uniform variance sparse/compressible GBM prior.

Fig. 15.
Fig. 15.

Example natural images from the training dataset used to extract GBM model parameters.

Fig. 16.
Fig. 16.

(a) Test image 1 and reconstructions at 4× compression and 1% measurement noise obtained with (b) random, (c) minimum coherence, (d) information sensing, (e) variable sampling density, (f) super-resolution, and (g) TSI optimal projection designs.

Fig. 17.
Fig. 17.

(a) Test image 2 and reconstructions at 4× compression and 1% measurement noise obtained with (b) random, (c) minimum coherence, (d) information sensing, (e) variable sampling density, (f) super-resolution, and (g) TSI-optimal projection designs.

Fig. 18.
Fig. 18.

(a) Test image 3 and reconstructions at 4× compression and 1% measurement noise obtained with (b) random, (c) minimum coherence, (d) information sensing, (e) variable sampling density, (f) super-resolution, and (g) TSI optimal projection designs.

Fig. 19.
Fig. 19.

(a) Test image 4 and reconstructions at 4× compression and 1% measurement noise obtained with (b) random, (c) minimum coherence, (d) information sensing, (e) variable sampling density, (f) super-resolution, and (g) TSI optimal projection designs.

Fig. 20.
Fig. 20.

(a) Test image 5 and reconstructions at 4× compression and 1% measurement noise obtained with (b) random, (c) minimum coherence, (d) information sensing, (e) variable sampling density, (f) super-resolution, and (g) TSI optimal projection designs.

Fig. 21.
Fig. 21.

(a)–(c) Test images 1–3 and reconstructions at 4× compression and 5% measurement noise obtained with (d),(h),(l) random, (e),(i),(m) minimum coherence, (f),(j),(n) super-resolution, and (g),(k),(o) TSI optimal projection designs.

Fig. 22.
Fig. 22.

(a),(b) Test images 4 and 5 and reconstructions at 4× compression and 5% measurement noise obtained with (c),(g) random, (d),(h) minimum coherence, (e),(i) super-resolution, and (f),(j) TSI optimal projection designs.

Fig. 23.
Fig. 23.

(a)–(c) Test images 1–3 and reconstructions at 9× compression and 1% measurement noise obtained with (d),(h),(l) random, (e),(i),(m) minimum coherence, (f),(j)(n) super-resolution, and (g),(k),(o) TSI optimal projection designs.

Fig. 24.
Fig. 24.

(a),(b) Test images 4 and 5 and reconstructions at 9× compression and 1% measurement noise obtained with (c),(g) random, (d),(h) minimum coherence, (e),(i) super-resolution, and (f),(j) TSI optimal projection designs.

Fig. 25.
Fig. 25.

(a)–(c) Test images 1–3 and reconstructions at 9× compression and 5% measurement noise obtained with (d),(h),(l) random, (e),(i),(m) minimum coherence, (f),(j)(n) super-resolution, and (g),(k),(o) TSI optimal projection designs.

Fig. 26.
Fig. 26.

(a),(b) Test images 4 and 5 and reconstructions at 9× compression and 5% measurement noise obtained with (c),(g) random, (d),(h) minimum coherence, (e),(i) super-resolution, and (f),(j) TSI optimal projection designs.

Tables (2)

Tables Icon

Table 1. Average Reconstruction PSNR (Averaged over Five Test Images and Measurement Noise) for Each Candidate Projection Design at Four Different Measurement Noise Levels and 4× Compression Ratio

Tables Icon

Table 2. Average Reconstruction PSNR (Averaged over Five Test Images and Measurement Noise) for Each Candidate Projection Design at Four Different Measurement Noise Levels and 9× Compression Ratio

Equations (16)

Equations on this page are rendered with MathJax. Learn more.

g⃗=Pf⃗+n⃗equivalentlyg⃗=PWθ⃗+n⃗,
p(θ⃗|z⃗)=i=1Np(θi|zi),
p(θi|zi)=N(0,zi2),
Pr(zi=zmaxi)=q,Pr(zi=zmini)=1q,
Pr(S=K)=(Nk)qK(1q)(NK),
μPW=maxij|P⃗iT·W⃗j|P⃗i2W⃗j2,
J(g⃗;θ⃗)=h(g⃗)h(g⃗|θ⃗)=h(θ⃗)h(θ⃗|g⃗),
argmaxPJ(g⃗;θ⃗)s.t.f(P)C,
P(n+1)=P(n)+γ·J(θ⃗;g⃗)P|P=P(n),s.t.f(Pn+1)C.
J(θ⃗;g⃗)P=h(g⃗)Ph^(g⃗)P,
RMSE=Eθ⃗,P,n⃗(θ⃗θ⃗^mmse22),
θ⃗^mmse=θ⃗·p(θ⃗|g⃗)dNθ.
p(θi|g⃗,θ⃗[N]i,zi,z⃗[N]i)p(zi|g⃗,θi,θ⃗[N]i,z⃗[N]i)p(θi+1|g⃗,θ⃗[N]i+1,zi+1,z⃗[N]i+1)p(zi+1|g⃗,θi+1,θ⃗[N]i+1,z⃗[N]i+1),
i=1MP⃗i22=E.
Σθ=(σθ120σθ220σθN2,),
i=1M|Pij|1,j.

Metrics