Abstract

Hyperspectral video acquisition is a trade-off between spectral and temporal resolution. We present an algorithm for recovering dense hyperspectral video of dynamic scenes from a few measured multispectral bands per frame using optical flow and sparse coding. Different set of bands are measured in each video frame and optical flow is used to register them. Optical flow errors are corrected by exploiting sparsity in the spectra and the spatial correlation between images of a scene at different wavelengths. A redundant dictionary of atoms is learned that can sparsely approximate training spectra. The restoration of correct spectra is formulated as an ℓ1 convex optimization problem that minimizes a Mahalanobis-like weighted distance between the restored and corrupt signals as well as the restored signal and the median of the eight connected neighbours of the corrupt signal such that the restored signal is a sparse linear combination of the dictionary atoms. Spectral restoration is followed by spatial restoration using a guided dictionary approach where one dictionary is learned for measured bands and another for a band that is to be spatially restored. By constraining the sparse coding coefficients of both dictionaries to be the same, the restoration of corrupt band is guided by the more reliable measured bands. Experiments on real data and comparison with an existing volumetric image denoising technique shows the superiority of our algorithm.

© 2012 OSA

PDF Article

References

  • View by:
  • |
  • |
  • |

  1. D. Kittle, K. Choi, A. Wagadarikar, and D. Brady, “Multiframe image estimation for coded aperture snapshot spectral imagers,” Appl. Opt. 49, 6824–6833 (2010).
    [CrossRef] [PubMed]
  2. M. Shankar, N. Pitsianis, and D. Brady, “Compressive video sensors using multichannel imagers,” Appl. Opt. 49, B9–B17 (2010).
    [CrossRef] [PubMed]
  3. A. Wagadarikar, N. Pitsianis, X. Sun, and D. Brady, “Video rate spectral imaging using a coded aperture snapshot spectral imager,” Opt. Express 17, 6368–6388 (2009).
    [CrossRef] [PubMed]
  4. Y. Pati, R. Rexaiifar, and P. Krishnaprasad, “Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition,” in Proceedings of the 27th Asilomar Conference on Signals, Systems, and Computers (IEEE, 1993), 40–44.
  5. R. Tibshirani, “Regression shrinkage and selection via the Lasso,” J. R. Stat. Soc. Ser. B 58, 267–288 (1996).
  6. M. Aharon, M. Elad, and A. Bruckstein, “K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation,” IEEE Trans. Signal Process. 54, 4311–4322 (2006).
    [CrossRef]
  7. M. Elad and M. Aharon, “Image denoising via sparse and redundant representations over learned dictionaries,” IEEE Trans. Image Process. 15, 3736–3745 (2006).
    [CrossRef] [PubMed]
  8. M. Elad and G. Sapiro, “Sparse representation for color image restoration,” IEEE Trans. Image Process. 17, 53–69 (2008).
    [CrossRef] [PubMed]
  9. H. Othman and S. Qian, “Noise reduction of hyperspectral imagery using hybrid spatial-spectral derivative-domain wavelet shrinkage,” IEEE Trans. Geosci. Remote Sens. 44, 397–408 (2006).
    [CrossRef]
  10. S. Bourguignon, D. Mary, and E. Slezak, “Sparsity-based denoising of hyperspectral astrophysical data with colored noise: Application to the MUSE instrument,” in 2nd Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (IEEE, 2010), 1–4.
  11. G. Farnebäck, “Two-frame motion estimation based on polynomial expansion,” in Proceedings of the 13th Scandinavian Conference on Image Analysis (Springer, 2003), 363–370.
  12. J. Parkkinen, J. Hallikainen, and T. Jaaskelainen, “Characteristic spectra of munsell colors,” J. Opt. Soc. Am. A 6, 318–322 (1989).
    [CrossRef]
  13. J. Mairal, J. Ponce, and G. Sapiro, “Online learning for matrix factorization and sparse coding,” J. Mach. Learn. Res. 11, 19–60 (2010).
  14. B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani, “Least angle regression,” Ann. Stat. 32, 407–499 (2004).
    [CrossRef]
  15. J. Yang, J. Wright, T. Huang, and Y. Ma, “Image super-resolution via sparse representation,” IEEE Trans. Image Process. 19, 2861–2873 (2010).
    [CrossRef]
  16. P. Ndajah, H. Kikuchi, M. Yukawa, H. Watanabe, and S. Muramatsu, “An investigation on the quality of denoised images,” Int. J. Circuits, Systems and Signal Process. 5, 423–434 (2011).

2011 (1)

P. Ndajah, H. Kikuchi, M. Yukawa, H. Watanabe, and S. Muramatsu, “An investigation on the quality of denoised images,” Int. J. Circuits, Systems and Signal Process. 5, 423–434 (2011).

2010 (4)

M. Shankar, N. Pitsianis, and D. Brady, “Compressive video sensors using multichannel imagers,” Appl. Opt. 49, B9–B17 (2010).
[CrossRef] [PubMed]

D. Kittle, K. Choi, A. Wagadarikar, and D. Brady, “Multiframe image estimation for coded aperture snapshot spectral imagers,” Appl. Opt. 49, 6824–6833 (2010).
[CrossRef] [PubMed]

J. Mairal, J. Ponce, and G. Sapiro, “Online learning for matrix factorization and sparse coding,” J. Mach. Learn. Res. 11, 19–60 (2010).

J. Yang, J. Wright, T. Huang, and Y. Ma, “Image super-resolution via sparse representation,” IEEE Trans. Image Process. 19, 2861–2873 (2010).
[CrossRef]

2009 (1)

2008 (1)

M. Elad and G. Sapiro, “Sparse representation for color image restoration,” IEEE Trans. Image Process. 17, 53–69 (2008).
[CrossRef] [PubMed]

2006 (3)

H. Othman and S. Qian, “Noise reduction of hyperspectral imagery using hybrid spatial-spectral derivative-domain wavelet shrinkage,” IEEE Trans. Geosci. Remote Sens. 44, 397–408 (2006).
[CrossRef]

M. Aharon, M. Elad, and A. Bruckstein, “K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation,” IEEE Trans. Signal Process. 54, 4311–4322 (2006).
[CrossRef]

M. Elad and M. Aharon, “Image denoising via sparse and redundant representations over learned dictionaries,” IEEE Trans. Image Process. 15, 3736–3745 (2006).
[CrossRef] [PubMed]

2004 (1)

B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani, “Least angle regression,” Ann. Stat. 32, 407–499 (2004).
[CrossRef]

1996 (1)

R. Tibshirani, “Regression shrinkage and selection via the Lasso,” J. R. Stat. Soc. Ser. B 58, 267–288 (1996).

1989 (1)

Aharon, M.

M. Elad and M. Aharon, “Image denoising via sparse and redundant representations over learned dictionaries,” IEEE Trans. Image Process. 15, 3736–3745 (2006).
[CrossRef] [PubMed]

M. Aharon, M. Elad, and A. Bruckstein, “K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation,” IEEE Trans. Signal Process. 54, 4311–4322 (2006).
[CrossRef]

Bourguignon, S.

S. Bourguignon, D. Mary, and E. Slezak, “Sparsity-based denoising of hyperspectral astrophysical data with colored noise: Application to the MUSE instrument,” in 2nd Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (IEEE, 2010), 1–4.

Brady, D.

Bruckstein, A.

M. Aharon, M. Elad, and A. Bruckstein, “K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation,” IEEE Trans. Signal Process. 54, 4311–4322 (2006).
[CrossRef]

Choi, K.

Efron, B.

B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani, “Least angle regression,” Ann. Stat. 32, 407–499 (2004).
[CrossRef]

Elad, M.

M. Elad and G. Sapiro, “Sparse representation for color image restoration,” IEEE Trans. Image Process. 17, 53–69 (2008).
[CrossRef] [PubMed]

M. Aharon, M. Elad, and A. Bruckstein, “K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation,” IEEE Trans. Signal Process. 54, 4311–4322 (2006).
[CrossRef]

M. Elad and M. Aharon, “Image denoising via sparse and redundant representations over learned dictionaries,” IEEE Trans. Image Process. 15, 3736–3745 (2006).
[CrossRef] [PubMed]

Farnebäck, G.

G. Farnebäck, “Two-frame motion estimation based on polynomial expansion,” in Proceedings of the 13th Scandinavian Conference on Image Analysis (Springer, 2003), 363–370.

Hallikainen, J.

Hastie, T.

B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani, “Least angle regression,” Ann. Stat. 32, 407–499 (2004).
[CrossRef]

Huang, T.

J. Yang, J. Wright, T. Huang, and Y. Ma, “Image super-resolution via sparse representation,” IEEE Trans. Image Process. 19, 2861–2873 (2010).
[CrossRef]

Jaaskelainen, T.

Johnstone, I.

B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani, “Least angle regression,” Ann. Stat. 32, 407–499 (2004).
[CrossRef]

Kikuchi, H.

P. Ndajah, H. Kikuchi, M. Yukawa, H. Watanabe, and S. Muramatsu, “An investigation on the quality of denoised images,” Int. J. Circuits, Systems and Signal Process. 5, 423–434 (2011).

Kittle, D.

Krishnaprasad, P.

Y. Pati, R. Rexaiifar, and P. Krishnaprasad, “Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition,” in Proceedings of the 27th Asilomar Conference on Signals, Systems, and Computers (IEEE, 1993), 40–44.

Ma, Y.

J. Yang, J. Wright, T. Huang, and Y. Ma, “Image super-resolution via sparse representation,” IEEE Trans. Image Process. 19, 2861–2873 (2010).
[CrossRef]

Mairal, J.

J. Mairal, J. Ponce, and G. Sapiro, “Online learning for matrix factorization and sparse coding,” J. Mach. Learn. Res. 11, 19–60 (2010).

Mary, D.

S. Bourguignon, D. Mary, and E. Slezak, “Sparsity-based denoising of hyperspectral astrophysical data with colored noise: Application to the MUSE instrument,” in 2nd Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (IEEE, 2010), 1–4.

Muramatsu, S.

P. Ndajah, H. Kikuchi, M. Yukawa, H. Watanabe, and S. Muramatsu, “An investigation on the quality of denoised images,” Int. J. Circuits, Systems and Signal Process. 5, 423–434 (2011).

Ndajah, P.

P. Ndajah, H. Kikuchi, M. Yukawa, H. Watanabe, and S. Muramatsu, “An investigation on the quality of denoised images,” Int. J. Circuits, Systems and Signal Process. 5, 423–434 (2011).

Othman, H.

H. Othman and S. Qian, “Noise reduction of hyperspectral imagery using hybrid spatial-spectral derivative-domain wavelet shrinkage,” IEEE Trans. Geosci. Remote Sens. 44, 397–408 (2006).
[CrossRef]

Parkkinen, J.

Pati, Y.

Y. Pati, R. Rexaiifar, and P. Krishnaprasad, “Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition,” in Proceedings of the 27th Asilomar Conference on Signals, Systems, and Computers (IEEE, 1993), 40–44.

Pitsianis, N.

Ponce, J.

J. Mairal, J. Ponce, and G. Sapiro, “Online learning for matrix factorization and sparse coding,” J. Mach. Learn. Res. 11, 19–60 (2010).

Qian, S.

H. Othman and S. Qian, “Noise reduction of hyperspectral imagery using hybrid spatial-spectral derivative-domain wavelet shrinkage,” IEEE Trans. Geosci. Remote Sens. 44, 397–408 (2006).
[CrossRef]

Rexaiifar, R.

Y. Pati, R. Rexaiifar, and P. Krishnaprasad, “Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition,” in Proceedings of the 27th Asilomar Conference on Signals, Systems, and Computers (IEEE, 1993), 40–44.

Sapiro, G.

J. Mairal, J. Ponce, and G. Sapiro, “Online learning for matrix factorization and sparse coding,” J. Mach. Learn. Res. 11, 19–60 (2010).

M. Elad and G. Sapiro, “Sparse representation for color image restoration,” IEEE Trans. Image Process. 17, 53–69 (2008).
[CrossRef] [PubMed]

Shankar, M.

Slezak, E.

S. Bourguignon, D. Mary, and E. Slezak, “Sparsity-based denoising of hyperspectral astrophysical data with colored noise: Application to the MUSE instrument,” in 2nd Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (IEEE, 2010), 1–4.

Sun, X.

Tibshirani, R.

B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani, “Least angle regression,” Ann. Stat. 32, 407–499 (2004).
[CrossRef]

R. Tibshirani, “Regression shrinkage and selection via the Lasso,” J. R. Stat. Soc. Ser. B 58, 267–288 (1996).

Wagadarikar, A.

Watanabe, H.

P. Ndajah, H. Kikuchi, M. Yukawa, H. Watanabe, and S. Muramatsu, “An investigation on the quality of denoised images,” Int. J. Circuits, Systems and Signal Process. 5, 423–434 (2011).

Wright, J.

J. Yang, J. Wright, T. Huang, and Y. Ma, “Image super-resolution via sparse representation,” IEEE Trans. Image Process. 19, 2861–2873 (2010).
[CrossRef]

Yang, J.

J. Yang, J. Wright, T. Huang, and Y. Ma, “Image super-resolution via sparse representation,” IEEE Trans. Image Process. 19, 2861–2873 (2010).
[CrossRef]

Yukawa, M.

P. Ndajah, H. Kikuchi, M. Yukawa, H. Watanabe, and S. Muramatsu, “An investigation on the quality of denoised images,” Int. J. Circuits, Systems and Signal Process. 5, 423–434 (2011).

Ann. Stat. (1)

B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani, “Least angle regression,” Ann. Stat. 32, 407–499 (2004).
[CrossRef]

Appl. Opt. (2)

IEEE Trans. Geosci. Remote Sens. (1)

H. Othman and S. Qian, “Noise reduction of hyperspectral imagery using hybrid spatial-spectral derivative-domain wavelet shrinkage,” IEEE Trans. Geosci. Remote Sens. 44, 397–408 (2006).
[CrossRef]

IEEE Trans. Image Process. (3)

M. Elad and M. Aharon, “Image denoising via sparse and redundant representations over learned dictionaries,” IEEE Trans. Image Process. 15, 3736–3745 (2006).
[CrossRef] [PubMed]

M. Elad and G. Sapiro, “Sparse representation for color image restoration,” IEEE Trans. Image Process. 17, 53–69 (2008).
[CrossRef] [PubMed]

J. Yang, J. Wright, T. Huang, and Y. Ma, “Image super-resolution via sparse representation,” IEEE Trans. Image Process. 19, 2861–2873 (2010).
[CrossRef]

IEEE Trans. Signal Process. (1)

M. Aharon, M. Elad, and A. Bruckstein, “K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation,” IEEE Trans. Signal Process. 54, 4311–4322 (2006).
[CrossRef]

Int. J. Circuits, Systems and Signal Process. (1)

P. Ndajah, H. Kikuchi, M. Yukawa, H. Watanabe, and S. Muramatsu, “An investigation on the quality of denoised images,” Int. J. Circuits, Systems and Signal Process. 5, 423–434 (2011).

J. Mach. Learn. Res. (1)

J. Mairal, J. Ponce, and G. Sapiro, “Online learning for matrix factorization and sparse coding,” J. Mach. Learn. Res. 11, 19–60 (2010).

J. Opt. Soc. Am. A (1)

J. R. Stat. Soc. Ser. B (1)

R. Tibshirani, “Regression shrinkage and selection via the Lasso,” J. R. Stat. Soc. Ser. B 58, 267–288 (1996).

Opt. Express (1)

Other (3)

Y. Pati, R. Rexaiifar, and P. Krishnaprasad, “Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition,” in Proceedings of the 27th Asilomar Conference on Signals, Systems, and Computers (IEEE, 1993), 40–44.

S. Bourguignon, D. Mary, and E. Slezak, “Sparsity-based denoising of hyperspectral astrophysical data with colored noise: Application to the MUSE instrument,” in 2nd Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (IEEE, 2010), 1–4.

G. Farnebäck, “Two-frame motion estimation based on polynomial expansion,” in Proceedings of the 13th Scandinavian Conference on Image Analysis (Springer, 2003), 363–370.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Metrics