Abstract

The technique of reconstructing a higher-resolution (HR) image of size ML×ML by digitally processing L×L subpixel-shifted lower-resolution (LR) copies of it, each of size M×M, has now become well established. This particular digital superresolution problem is analyzed from the standpoint of the generalized sampling theorem. It is shown both theoretically and by computer simulation that the choice of regularly spaced subpixel shifts for the LR images tends to maximize the robustness and minimize the error of reconstruction of the HR image. In practice, since subpixel-level control of LR image shifts may be nearly impossible to achieve, however, a more likely scenario, which is also discussed, is one involving random subpixel shifts. It is shown that without reasonably tight bounds on the range of random shifts, the reconstruction is likely to fail in the presence of even small amounts of noise unless either reliable prior information or additional data are available.

© 2007 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. R. Tsai and T. Huang, "Multiframe image restoration and registration," in Advances in Computer Vision and Image Processing, T.Huang, ed. (JAI, 1984), Vol. 1, pp. 317-339.
  2. S. Kim, N. Bose, and H. Valenzuela, "Recursive reconstruction of high-resolution image from noisy undersampled multiframes," IEEE Trans. Acoust., Speech, Signal Process. 38, 1013-1027 (1990).
    [CrossRef]
  3. H. Ur and D. Gross, "Improved resolution from sub-pixel shifted images," CVGIP: Graph. Models Image Process. 54, 181-186 (1992).
    [CrossRef]
  4. A. Papoulis, "Generalized sampling expansion," IEEE Trans. Circuits Syst. 24, 652-654 (1977).
    [CrossRef]
  5. J. L. Brown, Jr., "Sampling of bandlimited signals," Handbook of Statistics, N.K.Bose and C.R.Rao, eds. (North-Holland, 1993), Vol. 10, pp. 59-101.
    [CrossRef]
  6. R. Schulz and R. Stevenson, "Extraction of high-resolution frames from video sequences," IEEE Trans. Image Process. 5, 996-1011 (1996).
    [CrossRef]
  7. L. Poletto and P. Nicolosi, "Enhancing the spatial resolution of a two-dimensional array detector," Opt. Eng. (Bellingham) 38, 748-757 (1999).
    [CrossRef]
  8. S. Farisu, D. Robinson, M. Elad, and P. Milanfar, "Fast and robust multiframe superresolution," IEEE Trans. Image Process. 13, 1327-1344 (2004).
    [CrossRef]
  9. R. Hardie, K. Barnard, and E. Armstrong, "Joint map registration and high-resolution image estimation using a sequence of undersampled images," IEEE Trans. Image Process. 6, 1621-1633 (1997).
    [CrossRef] [PubMed]
  10. P. Vandewalle, L. Sbaiz, M. Vetterli, and S. Süsstrunk, "Superresolution from highly undersampled images," in Proceedings IEEE Conference on Image Processing, (IEEE, 2005), Vol. 1, pp. 889-892.
  11. M. Elad and A. Feuer, "Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images," IEEE Trans. Image Process. 6, 1646-1658 (1997).
    [CrossRef] [PubMed]
  12. D. Rajan and S. Chaudhuri, "Simultaneous estimation of superresolved scene and depth maps from low-resolution defocused observations," IEEE Trans. Pattern Anal. Mach. Intell. 25, 1102-1117 (2003).
    [CrossRef]
  13. S. Park, M. Park, and M. Gang, "Superresolution image reconstruction: a technical overview," IEEE Signal Process. Mag. 20, 21-36 (2003).
    [CrossRef]
  14. S. Farisu, D. Robinson, M. Elad, and P. Milanfar, "Advances and challenges in superresolution," Int. J. Imaging Syst. Technol. 14, 47-57 (2004).
    [CrossRef]
  15. J. Tanida, T. Kumugai, K. Yamada, S. Miyatake, K. Ishida, T. Morimoto, N. Kondou, D. Miyazaki, and Y. Ichioka, "Thin observation module by bound optics (TOMBO): Concept and experimental verification," Appl. Opt. 40, 1806-1813 (2001).
    [CrossRef]
  16. Y. Kitamura, R. Shogenji, K. Yamada, S. Miyatake, M. Miyamoto, T. Morimoto, Y. Masaki, N. Kondou, D. Miyazaki, J. Tanida, and Y. Ichioka, "Reconstruction of a high-resolution image on a compound-eye image-capturing system," Appl. Opt. 43, 1719-1727 (2004).
    [CrossRef] [PubMed]
  17. M. Elad and Y. Hel-Or, "A fast superresolution reconstruction algorithm for pure translational motion and common space-invariant blur," IEEE Trans. Image Process. 10, 1187-1193 (2001).
    [CrossRef]
  18. R. Lagendijk and J. Biemond, Iterative Identification and Restoration of Images (Kluwer, 1991).
    [CrossRef]
  19. If the HR signal itself is an undersampled, aliased version of an original higher-resolution signal, then it is only the aliased HR signal that can be reconstructed in this way. Indeed, since the undersampled HR signal, but not the original higher-resolution signal, may always be regarded as being critically sampled by the HR array and thus bandlimited to (−B,B), there is no scope for confusion in this statement. Consistent with this picture, the use of the qualifiers LR and HR is favored over aliased and critically sampled throughout this paper.
  20. C. Cullen, Matrices and Linear Transformations, 2nd ed. (Dover, 1972), p. 117.
  21. D. Robinson and P. Milanfar, "Statistical performance analysis of superresolution," IEEE Trans. Image Process. 15, 1413-1428 (2006).
    [CrossRef] [PubMed]
  22. R. Barnard, V. Pauca, T. Torgersen, R. Plemmons, S. Prasad, J. van der Gracht, J. Nagy, J. Chung, G. Behrmann, S. Mathews, and M. Mirotznik, "High-resolution iris image reconstruction from low-resolution imagery," in Advanced Signal Processing Algorithms, Architectures, and Implementations XVI, Proc. SPIE 6313, 63130D (2006).
  23. This statement is mathematically accurate only when M » L because the (L−1) vanishing columns of S′ modify (L−1) of the M blocks from their otherwise identical M(x̱) form. However, since the inequality M » L is generally true in a practical situation, one need not question the accuracy of the statement in practice.
  24. D. Brady, M. Fiddy, U. Shahid, and T. Suleski, "Compressive optical MONTAGE photography initiative: noise and error analysis," presented at the Computational Optical Sensing and Imaging (COSI) Conference, Charlotte, N.C., June 6-8, 2005 (conference proceedings on CD-ROM), paper CMB3.
  25. M. Neifeld and A. Ashok, "Imaging using alternate point spread function: lenslets with pseudo-random phase diversity," presented at the Computational Optical Sensing and Imaging (COSI) Conference, Charlotte, N.C., June 6-8, 2005 (conference proceedings on CD-ROM), paper CMB1.
  26. P. Morse and H. Feshbach, Methods of Theoretical Physics, Part I (McGraw-Hill, 1953), pp. 466-467.
  27. M. Krook, G. Carrier, and C. Pearson, Functions of a Complex Variable (McGraw-Hill, 1966), p. 57.

2006 (1)

D. Robinson and P. Milanfar, "Statistical performance analysis of superresolution," IEEE Trans. Image Process. 15, 1413-1428 (2006).
[CrossRef] [PubMed]

2004 (3)

S. Farisu, D. Robinson, M. Elad, and P. Milanfar, "Advances and challenges in superresolution," Int. J. Imaging Syst. Technol. 14, 47-57 (2004).
[CrossRef]

Y. Kitamura, R. Shogenji, K. Yamada, S. Miyatake, M. Miyamoto, T. Morimoto, Y. Masaki, N. Kondou, D. Miyazaki, J. Tanida, and Y. Ichioka, "Reconstruction of a high-resolution image on a compound-eye image-capturing system," Appl. Opt. 43, 1719-1727 (2004).
[CrossRef] [PubMed]

S. Farisu, D. Robinson, M. Elad, and P. Milanfar, "Fast and robust multiframe superresolution," IEEE Trans. Image Process. 13, 1327-1344 (2004).
[CrossRef]

2003 (2)

D. Rajan and S. Chaudhuri, "Simultaneous estimation of superresolved scene and depth maps from low-resolution defocused observations," IEEE Trans. Pattern Anal. Mach. Intell. 25, 1102-1117 (2003).
[CrossRef]

S. Park, M. Park, and M. Gang, "Superresolution image reconstruction: a technical overview," IEEE Signal Process. Mag. 20, 21-36 (2003).
[CrossRef]

2001 (2)

M. Elad and Y. Hel-Or, "A fast superresolution reconstruction algorithm for pure translational motion and common space-invariant blur," IEEE Trans. Image Process. 10, 1187-1193 (2001).
[CrossRef]

J. Tanida, T. Kumugai, K. Yamada, S. Miyatake, K. Ishida, T. Morimoto, N. Kondou, D. Miyazaki, and Y. Ichioka, "Thin observation module by bound optics (TOMBO): Concept and experimental verification," Appl. Opt. 40, 1806-1813 (2001).
[CrossRef]

1999 (1)

L. Poletto and P. Nicolosi, "Enhancing the spatial resolution of a two-dimensional array detector," Opt. Eng. (Bellingham) 38, 748-757 (1999).
[CrossRef]

1997 (2)

R. Hardie, K. Barnard, and E. Armstrong, "Joint map registration and high-resolution image estimation using a sequence of undersampled images," IEEE Trans. Image Process. 6, 1621-1633 (1997).
[CrossRef] [PubMed]

M. Elad and A. Feuer, "Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images," IEEE Trans. Image Process. 6, 1646-1658 (1997).
[CrossRef] [PubMed]

1996 (1)

R. Schulz and R. Stevenson, "Extraction of high-resolution frames from video sequences," IEEE Trans. Image Process. 5, 996-1011 (1996).
[CrossRef]

1992 (1)

H. Ur and D. Gross, "Improved resolution from sub-pixel shifted images," CVGIP: Graph. Models Image Process. 54, 181-186 (1992).
[CrossRef]

1990 (1)

S. Kim, N. Bose, and H. Valenzuela, "Recursive reconstruction of high-resolution image from noisy undersampled multiframes," IEEE Trans. Acoust., Speech, Signal Process. 38, 1013-1027 (1990).
[CrossRef]

1977 (1)

A. Papoulis, "Generalized sampling expansion," IEEE Trans. Circuits Syst. 24, 652-654 (1977).
[CrossRef]

Armstrong, E.

R. Hardie, K. Barnard, and E. Armstrong, "Joint map registration and high-resolution image estimation using a sequence of undersampled images," IEEE Trans. Image Process. 6, 1621-1633 (1997).
[CrossRef] [PubMed]

Ashok, A.

M. Neifeld and A. Ashok, "Imaging using alternate point spread function: lenslets with pseudo-random phase diversity," presented at the Computational Optical Sensing and Imaging (COSI) Conference, Charlotte, N.C., June 6-8, 2005 (conference proceedings on CD-ROM), paper CMB1.

Barnard, K.

R. Hardie, K. Barnard, and E. Armstrong, "Joint map registration and high-resolution image estimation using a sequence of undersampled images," IEEE Trans. Image Process. 6, 1621-1633 (1997).
[CrossRef] [PubMed]

Barnard, R.

R. Barnard, V. Pauca, T. Torgersen, R. Plemmons, S. Prasad, J. van der Gracht, J. Nagy, J. Chung, G. Behrmann, S. Mathews, and M. Mirotznik, "High-resolution iris image reconstruction from low-resolution imagery," in Advanced Signal Processing Algorithms, Architectures, and Implementations XVI, Proc. SPIE 6313, 63130D (2006).

Behrmann, G.

R. Barnard, V. Pauca, T. Torgersen, R. Plemmons, S. Prasad, J. van der Gracht, J. Nagy, J. Chung, G. Behrmann, S. Mathews, and M. Mirotznik, "High-resolution iris image reconstruction from low-resolution imagery," in Advanced Signal Processing Algorithms, Architectures, and Implementations XVI, Proc. SPIE 6313, 63130D (2006).

Biemond, J.

R. Lagendijk and J. Biemond, Iterative Identification and Restoration of Images (Kluwer, 1991).
[CrossRef]

Bose, N.

S. Kim, N. Bose, and H. Valenzuela, "Recursive reconstruction of high-resolution image from noisy undersampled multiframes," IEEE Trans. Acoust., Speech, Signal Process. 38, 1013-1027 (1990).
[CrossRef]

Brady, D.

D. Brady, M. Fiddy, U. Shahid, and T. Suleski, "Compressive optical MONTAGE photography initiative: noise and error analysis," presented at the Computational Optical Sensing and Imaging (COSI) Conference, Charlotte, N.C., June 6-8, 2005 (conference proceedings on CD-ROM), paper CMB3.

Brown, J. L.

J. L. Brown, Jr., "Sampling of bandlimited signals," Handbook of Statistics, N.K.Bose and C.R.Rao, eds. (North-Holland, 1993), Vol. 10, pp. 59-101.
[CrossRef]

Carrier, G.

M. Krook, G. Carrier, and C. Pearson, Functions of a Complex Variable (McGraw-Hill, 1966), p. 57.

Chaudhuri, S.

D. Rajan and S. Chaudhuri, "Simultaneous estimation of superresolved scene and depth maps from low-resolution defocused observations," IEEE Trans. Pattern Anal. Mach. Intell. 25, 1102-1117 (2003).
[CrossRef]

Chung, J.

R. Barnard, V. Pauca, T. Torgersen, R. Plemmons, S. Prasad, J. van der Gracht, J. Nagy, J. Chung, G. Behrmann, S. Mathews, and M. Mirotznik, "High-resolution iris image reconstruction from low-resolution imagery," in Advanced Signal Processing Algorithms, Architectures, and Implementations XVI, Proc. SPIE 6313, 63130D (2006).

Cullen, C.

C. Cullen, Matrices and Linear Transformations, 2nd ed. (Dover, 1972), p. 117.

Elad, M.

S. Farisu, D. Robinson, M. Elad, and P. Milanfar, "Advances and challenges in superresolution," Int. J. Imaging Syst. Technol. 14, 47-57 (2004).
[CrossRef]

S. Farisu, D. Robinson, M. Elad, and P. Milanfar, "Fast and robust multiframe superresolution," IEEE Trans. Image Process. 13, 1327-1344 (2004).
[CrossRef]

M. Elad and Y. Hel-Or, "A fast superresolution reconstruction algorithm for pure translational motion and common space-invariant blur," IEEE Trans. Image Process. 10, 1187-1193 (2001).
[CrossRef]

M. Elad and A. Feuer, "Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images," IEEE Trans. Image Process. 6, 1646-1658 (1997).
[CrossRef] [PubMed]

Farisu, S.

S. Farisu, D. Robinson, M. Elad, and P. Milanfar, "Advances and challenges in superresolution," Int. J. Imaging Syst. Technol. 14, 47-57 (2004).
[CrossRef]

S. Farisu, D. Robinson, M. Elad, and P. Milanfar, "Fast and robust multiframe superresolution," IEEE Trans. Image Process. 13, 1327-1344 (2004).
[CrossRef]

Feshbach, H.

P. Morse and H. Feshbach, Methods of Theoretical Physics, Part I (McGraw-Hill, 1953), pp. 466-467.

Feuer, A.

M. Elad and A. Feuer, "Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images," IEEE Trans. Image Process. 6, 1646-1658 (1997).
[CrossRef] [PubMed]

Fiddy, M.

D. Brady, M. Fiddy, U. Shahid, and T. Suleski, "Compressive optical MONTAGE photography initiative: noise and error analysis," presented at the Computational Optical Sensing and Imaging (COSI) Conference, Charlotte, N.C., June 6-8, 2005 (conference proceedings on CD-ROM), paper CMB3.

Gang, M.

S. Park, M. Park, and M. Gang, "Superresolution image reconstruction: a technical overview," IEEE Signal Process. Mag. 20, 21-36 (2003).
[CrossRef]

Gross, D.

H. Ur and D. Gross, "Improved resolution from sub-pixel shifted images," CVGIP: Graph. Models Image Process. 54, 181-186 (1992).
[CrossRef]

Hardie, R.

R. Hardie, K. Barnard, and E. Armstrong, "Joint map registration and high-resolution image estimation using a sequence of undersampled images," IEEE Trans. Image Process. 6, 1621-1633 (1997).
[CrossRef] [PubMed]

Hel-Or, Y.

M. Elad and Y. Hel-Or, "A fast superresolution reconstruction algorithm for pure translational motion and common space-invariant blur," IEEE Trans. Image Process. 10, 1187-1193 (2001).
[CrossRef]

Huang, T.

R. Tsai and T. Huang, "Multiframe image restoration and registration," in Advances in Computer Vision and Image Processing, T.Huang, ed. (JAI, 1984), Vol. 1, pp. 317-339.

Ichioka, Y.

Ishida, K.

Kim, S.

S. Kim, N. Bose, and H. Valenzuela, "Recursive reconstruction of high-resolution image from noisy undersampled multiframes," IEEE Trans. Acoust., Speech, Signal Process. 38, 1013-1027 (1990).
[CrossRef]

Kitamura, Y.

Kondou, N.

Krook, M.

M. Krook, G. Carrier, and C. Pearson, Functions of a Complex Variable (McGraw-Hill, 1966), p. 57.

Kumugai, T.

Lagendijk, R.

R. Lagendijk and J. Biemond, Iterative Identification and Restoration of Images (Kluwer, 1991).
[CrossRef]

Masaki, Y.

Mathews, S.

R. Barnard, V. Pauca, T. Torgersen, R. Plemmons, S. Prasad, J. van der Gracht, J. Nagy, J. Chung, G. Behrmann, S. Mathews, and M. Mirotznik, "High-resolution iris image reconstruction from low-resolution imagery," in Advanced Signal Processing Algorithms, Architectures, and Implementations XVI, Proc. SPIE 6313, 63130D (2006).

Milanfar, P.

D. Robinson and P. Milanfar, "Statistical performance analysis of superresolution," IEEE Trans. Image Process. 15, 1413-1428 (2006).
[CrossRef] [PubMed]

S. Farisu, D. Robinson, M. Elad, and P. Milanfar, "Fast and robust multiframe superresolution," IEEE Trans. Image Process. 13, 1327-1344 (2004).
[CrossRef]

S. Farisu, D. Robinson, M. Elad, and P. Milanfar, "Advances and challenges in superresolution," Int. J. Imaging Syst. Technol. 14, 47-57 (2004).
[CrossRef]

Mirotznik, M.

R. Barnard, V. Pauca, T. Torgersen, R. Plemmons, S. Prasad, J. van der Gracht, J. Nagy, J. Chung, G. Behrmann, S. Mathews, and M. Mirotznik, "High-resolution iris image reconstruction from low-resolution imagery," in Advanced Signal Processing Algorithms, Architectures, and Implementations XVI, Proc. SPIE 6313, 63130D (2006).

Miyamoto, M.

Miyatake, S.

Miyazaki, D.

Morimoto, T.

Morse, P.

P. Morse and H. Feshbach, Methods of Theoretical Physics, Part I (McGraw-Hill, 1953), pp. 466-467.

Nagy, J.

R. Barnard, V. Pauca, T. Torgersen, R. Plemmons, S. Prasad, J. van der Gracht, J. Nagy, J. Chung, G. Behrmann, S. Mathews, and M. Mirotznik, "High-resolution iris image reconstruction from low-resolution imagery," in Advanced Signal Processing Algorithms, Architectures, and Implementations XVI, Proc. SPIE 6313, 63130D (2006).

Neifeld, M.

M. Neifeld and A. Ashok, "Imaging using alternate point spread function: lenslets with pseudo-random phase diversity," presented at the Computational Optical Sensing and Imaging (COSI) Conference, Charlotte, N.C., June 6-8, 2005 (conference proceedings on CD-ROM), paper CMB1.

Nicolosi, P.

L. Poletto and P. Nicolosi, "Enhancing the spatial resolution of a two-dimensional array detector," Opt. Eng. (Bellingham) 38, 748-757 (1999).
[CrossRef]

Papoulis, A.

A. Papoulis, "Generalized sampling expansion," IEEE Trans. Circuits Syst. 24, 652-654 (1977).
[CrossRef]

Park, M.

S. Park, M. Park, and M. Gang, "Superresolution image reconstruction: a technical overview," IEEE Signal Process. Mag. 20, 21-36 (2003).
[CrossRef]

Park, S.

S. Park, M. Park, and M. Gang, "Superresolution image reconstruction: a technical overview," IEEE Signal Process. Mag. 20, 21-36 (2003).
[CrossRef]

Pauca, V.

R. Barnard, V. Pauca, T. Torgersen, R. Plemmons, S. Prasad, J. van der Gracht, J. Nagy, J. Chung, G. Behrmann, S. Mathews, and M. Mirotznik, "High-resolution iris image reconstruction from low-resolution imagery," in Advanced Signal Processing Algorithms, Architectures, and Implementations XVI, Proc. SPIE 6313, 63130D (2006).

Pearson, C.

M. Krook, G. Carrier, and C. Pearson, Functions of a Complex Variable (McGraw-Hill, 1966), p. 57.

Plemmons, R.

R. Barnard, V. Pauca, T. Torgersen, R. Plemmons, S. Prasad, J. van der Gracht, J. Nagy, J. Chung, G. Behrmann, S. Mathews, and M. Mirotznik, "High-resolution iris image reconstruction from low-resolution imagery," in Advanced Signal Processing Algorithms, Architectures, and Implementations XVI, Proc. SPIE 6313, 63130D (2006).

Poletto, L.

L. Poletto and P. Nicolosi, "Enhancing the spatial resolution of a two-dimensional array detector," Opt. Eng. (Bellingham) 38, 748-757 (1999).
[CrossRef]

Prasad, S.

R. Barnard, V. Pauca, T. Torgersen, R. Plemmons, S. Prasad, J. van der Gracht, J. Nagy, J. Chung, G. Behrmann, S. Mathews, and M. Mirotznik, "High-resolution iris image reconstruction from low-resolution imagery," in Advanced Signal Processing Algorithms, Architectures, and Implementations XVI, Proc. SPIE 6313, 63130D (2006).

Rajan, D.

D. Rajan and S. Chaudhuri, "Simultaneous estimation of superresolved scene and depth maps from low-resolution defocused observations," IEEE Trans. Pattern Anal. Mach. Intell. 25, 1102-1117 (2003).
[CrossRef]

Robinson, D.

D. Robinson and P. Milanfar, "Statistical performance analysis of superresolution," IEEE Trans. Image Process. 15, 1413-1428 (2006).
[CrossRef] [PubMed]

S. Farisu, D. Robinson, M. Elad, and P. Milanfar, "Advances and challenges in superresolution," Int. J. Imaging Syst. Technol. 14, 47-57 (2004).
[CrossRef]

S. Farisu, D. Robinson, M. Elad, and P. Milanfar, "Fast and robust multiframe superresolution," IEEE Trans. Image Process. 13, 1327-1344 (2004).
[CrossRef]

Sbaiz, L.

P. Vandewalle, L. Sbaiz, M. Vetterli, and S. Süsstrunk, "Superresolution from highly undersampled images," in Proceedings IEEE Conference on Image Processing, (IEEE, 2005), Vol. 1, pp. 889-892.

Schulz, R.

R. Schulz and R. Stevenson, "Extraction of high-resolution frames from video sequences," IEEE Trans. Image Process. 5, 996-1011 (1996).
[CrossRef]

Shahid, U.

D. Brady, M. Fiddy, U. Shahid, and T. Suleski, "Compressive optical MONTAGE photography initiative: noise and error analysis," presented at the Computational Optical Sensing and Imaging (COSI) Conference, Charlotte, N.C., June 6-8, 2005 (conference proceedings on CD-ROM), paper CMB3.

Shogenji, R.

Stevenson, R.

R. Schulz and R. Stevenson, "Extraction of high-resolution frames from video sequences," IEEE Trans. Image Process. 5, 996-1011 (1996).
[CrossRef]

Suleski, T.

D. Brady, M. Fiddy, U. Shahid, and T. Suleski, "Compressive optical MONTAGE photography initiative: noise and error analysis," presented at the Computational Optical Sensing and Imaging (COSI) Conference, Charlotte, N.C., June 6-8, 2005 (conference proceedings on CD-ROM), paper CMB3.

Süsstrunk, S.

P. Vandewalle, L. Sbaiz, M. Vetterli, and S. Süsstrunk, "Superresolution from highly undersampled images," in Proceedings IEEE Conference on Image Processing, (IEEE, 2005), Vol. 1, pp. 889-892.

Tanida, J.

Torgersen, T.

R. Barnard, V. Pauca, T. Torgersen, R. Plemmons, S. Prasad, J. van der Gracht, J. Nagy, J. Chung, G. Behrmann, S. Mathews, and M. Mirotznik, "High-resolution iris image reconstruction from low-resolution imagery," in Advanced Signal Processing Algorithms, Architectures, and Implementations XVI, Proc. SPIE 6313, 63130D (2006).

Tsai, R.

R. Tsai and T. Huang, "Multiframe image restoration and registration," in Advances in Computer Vision and Image Processing, T.Huang, ed. (JAI, 1984), Vol. 1, pp. 317-339.

Ur, H.

H. Ur and D. Gross, "Improved resolution from sub-pixel shifted images," CVGIP: Graph. Models Image Process. 54, 181-186 (1992).
[CrossRef]

Valenzuela, H.

S. Kim, N. Bose, and H. Valenzuela, "Recursive reconstruction of high-resolution image from noisy undersampled multiframes," IEEE Trans. Acoust., Speech, Signal Process. 38, 1013-1027 (1990).
[CrossRef]

van der Gracht, J.

R. Barnard, V. Pauca, T. Torgersen, R. Plemmons, S. Prasad, J. van der Gracht, J. Nagy, J. Chung, G. Behrmann, S. Mathews, and M. Mirotznik, "High-resolution iris image reconstruction from low-resolution imagery," in Advanced Signal Processing Algorithms, Architectures, and Implementations XVI, Proc. SPIE 6313, 63130D (2006).

Vandewalle, P.

P. Vandewalle, L. Sbaiz, M. Vetterli, and S. Süsstrunk, "Superresolution from highly undersampled images," in Proceedings IEEE Conference on Image Processing, (IEEE, 2005), Vol. 1, pp. 889-892.

Vetterli, M.

P. Vandewalle, L. Sbaiz, M. Vetterli, and S. Süsstrunk, "Superresolution from highly undersampled images," in Proceedings IEEE Conference on Image Processing, (IEEE, 2005), Vol. 1, pp. 889-892.

Yamada, K.

Appl. Opt. (2)

CVGIP: Graph. Models Image Process. (1)

H. Ur and D. Gross, "Improved resolution from sub-pixel shifted images," CVGIP: Graph. Models Image Process. 54, 181-186 (1992).
[CrossRef]

IEEE Signal Process. Mag. (1)

S. Park, M. Park, and M. Gang, "Superresolution image reconstruction: a technical overview," IEEE Signal Process. Mag. 20, 21-36 (2003).
[CrossRef]

IEEE Trans. Acoust., Speech, Signal Process. (1)

S. Kim, N. Bose, and H. Valenzuela, "Recursive reconstruction of high-resolution image from noisy undersampled multiframes," IEEE Trans. Acoust., Speech, Signal Process. 38, 1013-1027 (1990).
[CrossRef]

IEEE Trans. Circuits Syst. (1)

A. Papoulis, "Generalized sampling expansion," IEEE Trans. Circuits Syst. 24, 652-654 (1977).
[CrossRef]

IEEE Trans. Image Process. (6)

R. Schulz and R. Stevenson, "Extraction of high-resolution frames from video sequences," IEEE Trans. Image Process. 5, 996-1011 (1996).
[CrossRef]

S. Farisu, D. Robinson, M. Elad, and P. Milanfar, "Fast and robust multiframe superresolution," IEEE Trans. Image Process. 13, 1327-1344 (2004).
[CrossRef]

R. Hardie, K. Barnard, and E. Armstrong, "Joint map registration and high-resolution image estimation using a sequence of undersampled images," IEEE Trans. Image Process. 6, 1621-1633 (1997).
[CrossRef] [PubMed]

M. Elad and A. Feuer, "Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images," IEEE Trans. Image Process. 6, 1646-1658 (1997).
[CrossRef] [PubMed]

M. Elad and Y. Hel-Or, "A fast superresolution reconstruction algorithm for pure translational motion and common space-invariant blur," IEEE Trans. Image Process. 10, 1187-1193 (2001).
[CrossRef]

D. Robinson and P. Milanfar, "Statistical performance analysis of superresolution," IEEE Trans. Image Process. 15, 1413-1428 (2006).
[CrossRef] [PubMed]

IEEE Trans. Pattern Anal. Mach. Intell. (1)

D. Rajan and S. Chaudhuri, "Simultaneous estimation of superresolved scene and depth maps from low-resolution defocused observations," IEEE Trans. Pattern Anal. Mach. Intell. 25, 1102-1117 (2003).
[CrossRef]

Int. J. Imaging Syst. Technol. (1)

S. Farisu, D. Robinson, M. Elad, and P. Milanfar, "Advances and challenges in superresolution," Int. J. Imaging Syst. Technol. 14, 47-57 (2004).
[CrossRef]

Opt. Eng. (Bellingham) (1)

L. Poletto and P. Nicolosi, "Enhancing the spatial resolution of a two-dimensional array detector," Opt. Eng. (Bellingham) 38, 748-757 (1999).
[CrossRef]

Other (12)

P. Vandewalle, L. Sbaiz, M. Vetterli, and S. Süsstrunk, "Superresolution from highly undersampled images," in Proceedings IEEE Conference on Image Processing, (IEEE, 2005), Vol. 1, pp. 889-892.

J. L. Brown, Jr., "Sampling of bandlimited signals," Handbook of Statistics, N.K.Bose and C.R.Rao, eds. (North-Holland, 1993), Vol. 10, pp. 59-101.
[CrossRef]

R. Tsai and T. Huang, "Multiframe image restoration and registration," in Advances in Computer Vision and Image Processing, T.Huang, ed. (JAI, 1984), Vol. 1, pp. 317-339.

R. Lagendijk and J. Biemond, Iterative Identification and Restoration of Images (Kluwer, 1991).
[CrossRef]

If the HR signal itself is an undersampled, aliased version of an original higher-resolution signal, then it is only the aliased HR signal that can be reconstructed in this way. Indeed, since the undersampled HR signal, but not the original higher-resolution signal, may always be regarded as being critically sampled by the HR array and thus bandlimited to (−B,B), there is no scope for confusion in this statement. Consistent with this picture, the use of the qualifiers LR and HR is favored over aliased and critically sampled throughout this paper.

C. Cullen, Matrices and Linear Transformations, 2nd ed. (Dover, 1972), p. 117.

R. Barnard, V. Pauca, T. Torgersen, R. Plemmons, S. Prasad, J. van der Gracht, J. Nagy, J. Chung, G. Behrmann, S. Mathews, and M. Mirotznik, "High-resolution iris image reconstruction from low-resolution imagery," in Advanced Signal Processing Algorithms, Architectures, and Implementations XVI, Proc. SPIE 6313, 63130D (2006).

This statement is mathematically accurate only when M » L because the (L−1) vanishing columns of S′ modify (L−1) of the M blocks from their otherwise identical M(x̱) form. However, since the inequality M » L is generally true in a practical situation, one need not question the accuracy of the statement in practice.

D. Brady, M. Fiddy, U. Shahid, and T. Suleski, "Compressive optical MONTAGE photography initiative: noise and error analysis," presented at the Computational Optical Sensing and Imaging (COSI) Conference, Charlotte, N.C., June 6-8, 2005 (conference proceedings on CD-ROM), paper CMB3.

M. Neifeld and A. Ashok, "Imaging using alternate point spread function: lenslets with pseudo-random phase diversity," presented at the Computational Optical Sensing and Imaging (COSI) Conference, Charlotte, N.C., June 6-8, 2005 (conference proceedings on CD-ROM), paper CMB1.

P. Morse and H. Feshbach, Methods of Theoretical Physics, Part I (McGraw-Hill, 1953), pp. 466-467.

M. Krook, G. Carrier, and C. Pearson, Functions of a Complex Variable (McGraw-Hill, 1966), p. 57.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (7)

Fig. 1
Fig. 1

Noise-free digital SR. (a) Real parts of the true (solid curve) and reconstructed (dashed curves) traces of a 1D radial section of an iris, for λ = 0.64 . (b) Normalized MSE of reconstruction for the various nonuniform distributions of subpixel shifts for λ equal to 0.64 (crosses), 0.064 (circles), 0.0064 (squares), and 0.00064 (diamonds).

Fig. 2
Fig. 2

Noisy digital SR for n f = 0.01 . (a) Same as Fig. 1a, except λ = 0.40 . (b) Same as Fig. 1b, except that the four values of λ are 0.40 (crosses), 0.040 (circles), 0.0040 (squares), and 0.00040 (diamonds).

Fig. 3
Fig. 3

Same as Fig. 2, except n f = 1 .

Fig. 4
Fig. 4

(a) Typical LR signal subsampled on M = 32   pixels , with (solid curve) and without (dashed curve) noise. (b) Real parts of the true (solid curve) and reconstructed (dashed curve) spatial spectra of the 1D iris section, for the case considered in Fig. 3a.

Fig. 5
Fig. 5

Asymptotically evaluated mean value and standard deviation of P ( χ ̱ ) for the case of a uniformly random distribution of the shifts of the LR images over a full LR-pixel width.

Fig. 6
Fig. 6

Asymptotically evaluated mean value and standard deviation of P ( χ ̱ ) for the case of a uniformly random distribution of the shifts of the LR images over a half LR-pixel width, f = 0.5 .

Fig. 7
Fig. 7

Representation of the angles, θ j = 2 π χ j ( 2 L ) , and their extensions by π on the unit circle, for the case L = 5 .

Equations (63)

Equations on this page are rendered with MathJax. Learn more.

g l ( x ) = f h l ( x ) = B B H l ( ν ) F ( ν ) exp ( i 2 π ν x ) d ν , l = 1 , , L .
h l ( x ) = h ( x χ l ( 2 B ) ) , l = 1 , , L ,
h ( x ) = { 2 B L , if x < L ( 4 B ) 0 , otherwise } .
H l ( ν ) = H 0 ( ν ) exp [ 2 π i ν χ l ( 2 B ) ] , l = 1 , , L ,
H 0 ( ν ) = sin ( π ν L ( 2 B ) ) π ν L ( 2 B ) .
D ( ν ; χ ̱ ) = det [ H 1 ( ν ) H 1 ( ν + δ ν ) H 1 ( ν + ( L 1 ) δ ν ) H 2 ( ν ) H 2 ( ν + δ ν ) H 2 ( ν + ( L 1 ) δ ν ) H L ( ν ) H L ( ν + δ ν ) H L ( ν + ( L 1 ) δ ν ) ] ,
D ( ν ; χ ̱ ) = [ l = 0 L 1 H 0 ( ν + l δ ν ) exp ( i 2 π ν χ l ( 2 B ) ) ] P ( χ ̱ ) ,
P ( χ ̱ ) = det [ 1 exp ( i 2 π χ 1 L ) exp [ i 2 π ( L 1 ) χ 1 L ] 1 exp ( i 2 π χ 2 L ) exp [ i 2 π ( L 1 ) χ 2 L ] 1 exp ( i 2 π χ L L ) exp [ i 2 π ( L 1 ) χ L L ] ] .
P ( χ ̱ ) = j = 1 L 1 l = j + 1 L [ exp ( i 2 π χ l L ) exp ( i 2 π χ j L ) ] .
P ( χ ̱ ) = 2 L ( L 1 ) 2 j = 1 L 1 l = j + 1 L sin [ π ( χ l χ j ) L ] .
H 0 ( ν ) j = 1 L 1 H 0 ( ν + j δ ν ) l = j + 1 L sin [ π ( χ l χ j ) L ] 0 .
F j k ( l ) = δ j , k χ l , D m n = l = 1 L δ ( m 1 ) L + l , n .
g ̱ ( l ) = D F ( l ) f ̱ + n ̱ ( l ) ,
g ̱ = ( D F ( 1 ) D F ( 2 ) D F ( L ) ) f ̱ + n ̱ S f ̱ + n ̱ ,
( T P ) p q = 1 P exp [ i 2 π ( p 1 ) ( q 1 ) P ] .
g ̱ ( l ) = D F ( l ) f ̱ + n ̱ ( l ) ,
D = T M D T N 1 , F ( l ) = T N F ( l ) T N 1 .
F j k ( l ) = δ j k exp [ i 2 π ( k 1 ) χ l N ] .
D m n = 1 L { p = 1 L exp [ i 2 π ( n 1 ) ( p 1 ) N ] } q = 1 L δ m + ( q 1 ) M , n .
( D F ( l ) ) m n = 1 L { p = 1 L exp [ i 2 π ( n 1 ) ( p 1 + χ l ) N ] } q = 1 L δ m + ( q 1 ) M , n .
S = [ a 1 0 a 3 α 1 2 0 a 5 α 1 4 0 0 a 2 α 1 0 a 4 α 1 3 0 a 6 α 1 5 a 1 0 a 3 α 2 2 0 a 5 α 2 4 0 0 a 2 α 2 0 a 4 α 2 3 0 a 6 α 2 5 a 1 0 a 3 α 3 2 0 a 5 α 3 4 0 0 a 2 α 3 0 a 4 α 3 3 0 a 6 α 3 5 ] ,
S ̃ = [ a 1 a 3 α 1 2 a 5 α 1 4 0 0 0 a 1 a 3 α 2 2 a 5 α 2 4 0 0 0 a 1 a 3 α 3 2 a 5 α 3 4 0 0 0 0 0 0 a 2 α 1 a 4 α 1 3 a 6 α 1 5 0 0 0 a 2 α 2 a 4 α 2 3 a 6 α 2 5 0 0 0 a 2 α 3 a 4 α 3 3 a 6 α 3 5 ] .
S ̃ S = [ M ( χ ̱ ) 0 0 M ( χ ̱ ) ] ,
M ( χ ̱ ) = [ 1 exp ( i 2 π χ 1 3 ) exp ( i 4 π χ 1 3 ) 1 exp ( i 2 π χ 2 3 ) exp ( i 4 π χ 2 3 ) 1 exp ( i 2 π χ 3 3 ) exp ( i 4 π χ 3 3 ) ] .
f ̱ ̂ = arg min f ̱ ( g ̱ S f ̱ ̂ 2 + λ L f ̱ 2 ) ,
f ̱ ̂ = ( S S + λ L L ) 1 M g ̱ .
* ln P ( χ ̱ * ) = 0 .
j = 1 m 1 cot [ π ( χ m * χ j * ) L ] l = m + 1 L cot [ π ( χ l * χ m * ) L ] = 0 .
j m L cot [ π ( χ m * χ j * ) L ] = 0 , m = 1 , , L .
h j l 2 ln P ( χ ̱ * ) χ j * χ l * = π 2 L 2 { 1 sin 2 ( π ( l j ) L ) for j l k l 1 sin 2 ( π ( l k ) L ) for j = l } .
λ l = m = 0 L 1 h n , n + m exp ( i 2 π m l L ) .
λ l = m = 1 L 1 cos ( 2 π m l L ) 1 sin 2 ( m π L ) , l = 1 , , L .
λ l = 2 l ( L l ) , l = 0 , 1 , , L 1 .
S χ = { ( χ l 1 ( 1 ) , χ l 2 ( 2 ) ) χ l i ( i ) [ 0 , L ) , l i = 1 , , L ; i = 1 , 2 } .
ln P ( χ ̱ ) ln P ( χ ̱ * ) + 1 2 δ χ ̱ T h δ χ ̱ .
δ η l δ χ ̱ T u ̱ ( l )
P ( χ ̱ ) P ( χ ̱ * ) exp [ ( 1 2 ) l = 1 L 1 β l δ η l 2 ] = P ( χ ̱ * ) l = 1 L 1 exp [ ( 1 2 ) β l δ η l 2 ] ,
p ( χ ) = { 1 L for 0 < χ < L 0 , otherwise } ,
P ( χ ̱ ) P ( χ ̱ * ) L ( L 3 2 ) l = 1 L 1 [ π l ( L l ) ] 1 2 ,
P ( χ ̱ ) 2 P ( χ ̱ * ) 2 L ( L 3 2 ) l = 1 L 1 [ π 2 l ( L l ) ] 1 2 ,
P ( χ ̱ ) P ( χ ̱ * ) L ( L 3 2 ) π ( L 1 ) 2 ( L 1 ) ! ;
P ( χ ̱ ) 2 P ( χ ̱ * ) 2 L ( L 3 2 ) ( π 2 ) ( L 1 ) 2 ( L 1 ) ! .
P ( χ ̱ * ) = 2 L ( L 1 ) 2 j = 1 L 1 l = j + 1 L sin [ π ( l j ) L ] = 2 L ( L 1 ) 2 ( sin π L ) L 1 ( sin 2 π L ) L 2 ( sin ( L 1 ) π L ) .
D m n = 1 M N q = 1 M r = 1 N exp { i 2 π [ ( m 1 ) ( q 1 ) M ( n 1 ) ( r 1 ) N ] } p = 1 L δ r , ( q 1 ) L + p = 1 M N p = 1 L exp [ i 2 π ( n 1 ) ( p 1 ) N ] q = 1 M exp [ i 2 π ( m n ) ( q 1 ) M ] ,
[ σ 1 2 + O ( λ ) O ( λ ) O ( λ ) O ( λ ) σ 2 2 + O ( λ ) O ( λ ) O ( λ ) O ( λ ) σ r 2 + O ( λ ) O ( λ ) O ( λ ) O ( λ ) O ( λ ) ] .
j m , L + m 2 L cot [ 2 π ( χ m * χ j * ) ( 2 L ) ] = 0 , m = 1 , , 2 L ,
j m , L + m 2 L cot ( θ m * θ j * ) = 0 , m = 1 , , 2 L , 0 θ 1 * θ 2 L * 2 π .
m = δ ( x m ) = p = exp ( i 2 π p x ) ,
m = 0 L 1 f ( m ) = p = 0 L exp ( i 2 π p x ) f ( x ) d x .
f ( x ) = 1 cos ( 2 π l L ) x 1 cos ( 2 π L ) x
S ( l ) m = 0 L 1 1 cos ( 2 π l L ) m 1 cos ( 2 π L ) m = p = 0 L exp ( i 2 π p x ) 1 cos ( 2 π l L ) x 1 cos ( 2 π L ) x d x .
L 2 π 0 2 π d θ exp ( i p L θ ) 1 cos l θ 1 cos θ .
z = 1 d z i z [ 1 ( z l + z l ) 2 1 ( z + z 1 ) 2 ] z p L = i z = 1 d z ( z l 1 ) 2 ( z 1 ) 2 z p L l .
( i ) 2 π i 1 ( l 1 ) ! d l 1 d z l 1 ( 1 + z + + z l 1 ) 2 z = 0 ,
S ( l ) m = 0 L 1 1 cos ( 2 π l L ) m 1 cos ( 2 π L ) m = l L .
λ l = 2 l ( L l ) , l = 1 , , L .
H = [ H 11 ( ν 1 , ν 2 ) H 11 ( ν 1 + ( L 1 ) δ ν , ν 2 ) H 11 ( ν 1 , ν 2 + δ ν ) H L 1 ( ν 1 , ν 2 ) H L 1 ( ν 1 + ( L 1 ) δ ν , ν 2 ) H L 1 ( ν 1 , ν 2 + δ ν ) H 12 ( ν 1 , ν 2 ) H 12 ( ν 1 + ( L 1 ) δ ν , ν 2 ) H 12 ( ν 1 , ν 2 + δ ν ) H L L ( ν 1 , ν 2 ) H L L ( ν 1 + ( L 1 ) δ ν , ν 2 ) H L L ( ν 1 , ν 2 + δ ν ) ] ,
H l 1 , l 2 ( ν 1 , ν 2 ) = H 11 ( ν 1 , ν 2 ) exp { i 2 π [ ν 1 χ l 1 ( 1 ) + ν 2 χ l 2 ( 2 ) ] ( 2 B ) } .
D ( ν 1 , ν 2 ; χ ̱ ( 1 ) , χ ̱ ( 2 ) ) = det H = [ k 1 = 1 L k 2 = 1 L H 11 ( ν 1 + ( k 1 1 ) δ ν , ν 2 + ( k 2 1 ) δ ν ) ] det G ,
G l 1 l 2 ( ν 1 + ( k 1 1 ) δ ν , ν 2 + ( k 2 1 ) δ ν ) = G l 1 , k 1 ( 1 ) G l 2 , k 2 ( 2 ) ,
G l α , k α ( α ) = exp { i 2 π [ ν α + ( k α 1 ) δ ν ] χ l α ( α ) ( 2 B ) } , α = 1 , 2 .
D ( ν 1 , ν 2 ; χ ̱ ( 1 ) , χ ̱ ( 2 ) ) = [ k 1 = 1 L k 2 = 1 L H 11 ( ν 1 + ( k 1 1 ) δ ν , ν 2 + ( k 2 1 ) δ ν ) ] ( det G ( 1 ) ) L ( det G ( 2 ) ) L .
ln D ( ν 1 , ν 2 ; χ ̱ ( 1 ) , χ ̱ ( 2 ) ) = C ( ν 1 , ν 2 ) + L ln P ( χ ̱ ( 1 ) ) + L ln P ( χ ̱ ( 2 ) ) ,

Metrics