Abstract

Algorithms that use optical system diversity to improve multiplexed image reconstruction from multiple low-resolution images are analyzed and demonstrated. Compared with systems using identical imagers, systems using additional lower-resolution imagers can have improved accuracy and computation. The diverse system is not sensitive to boundary conditions and can take full advantage of improvements that decrease noise and allow an increased number of bits per pixel to represent spatial information in a scene.

© 2006 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. S. Chaudhuri, ed. Super-Resolution Imaging (Kluwer Academic, 2001).
  2. M. Elad and A. Feuer, "Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images," IEEE Trans. Image Process. 6, 1646-1658 (1997).
    [CrossRef] [PubMed]
  3. S. Baker and T. Kanade, "Limits on super-resolution and how to break them," IEEE Trans. Pattern Anal. Mach. Intell. 24, 1167-1183 (2002).
    [CrossRef]
  4. N. Nguyen, P. Milanfar, and G. H. Golub, "A computationally efficient image superresolution algorithm," IEEE Trans. Image Process. 10, 573-583 (2001).
    [CrossRef]
  5. M. K. Ng and N. K. Bose, "Mathematical analysis of super-resolution methodology," IEEE Signal Process. Mag. 20, 62-74 (2003).
    [CrossRef]
  6. S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, "Fast and robust multi-frame super-resolution," IEEE Trans. Image Process. 13, 1327-1344 (2004).
    [CrossRef] [PubMed]
  7. J. Tanida, T. Kumagai, K. Yamada, S. Miyatake, K. Ishida, T. Morimoto, N. Kondou, D. Miyazaki, and Y. Ichioka, "Thin observation module by bound optics (TOMBO): concept and experimental verification," Appl. Opt. 40, 1806-1813 (2001).
    [CrossRef]
  8. Y. Kitamura, R. Shogenji, K. Yamada, S. Miyatake, M. Miyamoto, T. Morimoto, Y. Masaki, N. Kondou, D. Miyazaki, J. Tanida, and Y. Ichioka, "Reconstruction of a high resolution image on a compound-eye image capturing system," Appl. Opt. 43, 1719-1727 (2004).
    [CrossRef] [PubMed]
  9. M. P. Christensen, M. W. Haney, D. Rajan, S. L. Wood, and S. C. Douglas, "PANOPTES: a thin agile multi-resolution imaging sensor," Presented at the Government Microcircuit Applications and Critical Technology Conference (GOMACTech-05), Las Vegas, Nev., 4-7 April 2005 paper 21.5.
  10. M. P. Christensen, V. Bhakta, D. Rajan, S. C. Douglas, S. L. Wood, and M. W. Haney, "Adaptive flat multiresolution multiplexed computational imaging architecture utilizing micromirror arrays to steer subimager field of views," Appl. Opt. 45, 2884-2892 (2006).
    [CrossRef] [PubMed]
  11. S. L. Wood, B. J. Smithson, M. P. Christensen, and D. Rajan, Rerformance of a MVE algorithm for compound eye image reconstruction using lens diversity," in Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005 (ICASSP' 05) (IEEE, 2005), Vol. II. pp. 593-596.
  12. R. N. Bracewell, Two-Dimensional Imaging (Prentice-Hall, 1995).
  13. A. Macovski, Medical Imaging Systems (Prentice-Hall, 1983).
  14. T. Kailath, Linear Systems (Prentice-Hall, 1980).
  15. A. K. Jain, Fundamentals of Digital Image Processing (Prentice-Hall, 1989), Chap. 2.
  16. R. M. Gray, "On the asympototic eigenvalue distribution of Toeplitz matrices," IEEE Trans. Inf. Theory IT-18, 725-730 (1972).
    [CrossRef]
  17. S. L. Wood, D. Rajan, M. P. Christensen, S. C.Douglas, and B. J. Smithson, "Resolution improvement for compound eye images through lens diversity," in Digital Signal Processing Workshop 2004 and the Third IEEE Signal Processing Education Workshop (IEEE, 2004), pp. 151-155, doi: .
    [CrossRef]
  18. M. Born and E. Wolf, Principles of Optics (Cambridge U. Press, 1959).
  19. X. Ying and Z. Hu, "Distortion correction of fisheye lens using a nonparametric imaging model," in Proceedings of Asian Conference on Computer Vision (Asian Federation of Computer Vision Societies 2004), pp. 527-532.
  20. C. Brauer-Burchardt and K. Voss, "A new algorithm to correct fish-eye-and strong wide-angle-lens-distortion from single images," in Proceedings of IEEE International Conference on Image Processing (IEEE, 2001), pp. 225-228.

2006

2004

2003

M. K. Ng and N. K. Bose, "Mathematical analysis of super-resolution methodology," IEEE Signal Process. Mag. 20, 62-74 (2003).
[CrossRef]

2002

S. Baker and T. Kanade, "Limits on super-resolution and how to break them," IEEE Trans. Pattern Anal. Mach. Intell. 24, 1167-1183 (2002).
[CrossRef]

2001

1997

M. Elad and A. Feuer, "Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images," IEEE Trans. Image Process. 6, 1646-1658 (1997).
[CrossRef] [PubMed]

1972

R. M. Gray, "On the asympototic eigenvalue distribution of Toeplitz matrices," IEEE Trans. Inf. Theory IT-18, 725-730 (1972).
[CrossRef]

Baker, S.

S. Baker and T. Kanade, "Limits on super-resolution and how to break them," IEEE Trans. Pattern Anal. Mach. Intell. 24, 1167-1183 (2002).
[CrossRef]

Bhakta, V.

Born, M.

M. Born and E. Wolf, Principles of Optics (Cambridge U. Press, 1959).

Bose, N. K.

M. K. Ng and N. K. Bose, "Mathematical analysis of super-resolution methodology," IEEE Signal Process. Mag. 20, 62-74 (2003).
[CrossRef]

Bracewell, R. N.

R. N. Bracewell, Two-Dimensional Imaging (Prentice-Hall, 1995).

Brauer-Burchardt, C.

C. Brauer-Burchardt and K. Voss, "A new algorithm to correct fish-eye-and strong wide-angle-lens-distortion from single images," in Proceedings of IEEE International Conference on Image Processing (IEEE, 2001), pp. 225-228.

Chaudhuri, S.

S. Chaudhuri, ed. Super-Resolution Imaging (Kluwer Academic, 2001).

Christensen, M. P.

M. P. Christensen, V. Bhakta, D. Rajan, S. C. Douglas, S. L. Wood, and M. W. Haney, "Adaptive flat multiresolution multiplexed computational imaging architecture utilizing micromirror arrays to steer subimager field of views," Appl. Opt. 45, 2884-2892 (2006).
[CrossRef] [PubMed]

S. L. Wood, B. J. Smithson, M. P. Christensen, and D. Rajan, Rerformance of a MVE algorithm for compound eye image reconstruction using lens diversity," in Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005 (ICASSP' 05) (IEEE, 2005), Vol. II. pp. 593-596.

S. L. Wood, D. Rajan, M. P. Christensen, S. C.Douglas, and B. J. Smithson, "Resolution improvement for compound eye images through lens diversity," in Digital Signal Processing Workshop 2004 and the Third IEEE Signal Processing Education Workshop (IEEE, 2004), pp. 151-155, doi: .
[CrossRef]

M. P. Christensen, M. W. Haney, D. Rajan, S. L. Wood, and S. C. Douglas, "PANOPTES: a thin agile multi-resolution imaging sensor," Presented at the Government Microcircuit Applications and Critical Technology Conference (GOMACTech-05), Las Vegas, Nev., 4-7 April 2005 paper 21.5.

Douglas, S. C.

M. P. Christensen, V. Bhakta, D. Rajan, S. C. Douglas, S. L. Wood, and M. W. Haney, "Adaptive flat multiresolution multiplexed computational imaging architecture utilizing micromirror arrays to steer subimager field of views," Appl. Opt. 45, 2884-2892 (2006).
[CrossRef] [PubMed]

M. P. Christensen, M. W. Haney, D. Rajan, S. L. Wood, and S. C. Douglas, "PANOPTES: a thin agile multi-resolution imaging sensor," Presented at the Government Microcircuit Applications and Critical Technology Conference (GOMACTech-05), Las Vegas, Nev., 4-7 April 2005 paper 21.5.

S. L. Wood, D. Rajan, M. P. Christensen, S. C.Douglas, and B. J. Smithson, "Resolution improvement for compound eye images through lens diversity," in Digital Signal Processing Workshop 2004 and the Third IEEE Signal Processing Education Workshop (IEEE, 2004), pp. 151-155, doi: .
[CrossRef]

Elad, M.

S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, "Fast and robust multi-frame super-resolution," IEEE Trans. Image Process. 13, 1327-1344 (2004).
[CrossRef] [PubMed]

M. Elad and A. Feuer, "Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images," IEEE Trans. Image Process. 6, 1646-1658 (1997).
[CrossRef] [PubMed]

Farsiu, S.

S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, "Fast and robust multi-frame super-resolution," IEEE Trans. Image Process. 13, 1327-1344 (2004).
[CrossRef] [PubMed]

Feuer, A.

M. Elad and A. Feuer, "Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images," IEEE Trans. Image Process. 6, 1646-1658 (1997).
[CrossRef] [PubMed]

Golub, G. H.

N. Nguyen, P. Milanfar, and G. H. Golub, "A computationally efficient image superresolution algorithm," IEEE Trans. Image Process. 10, 573-583 (2001).
[CrossRef]

Gray, R. M.

R. M. Gray, "On the asympototic eigenvalue distribution of Toeplitz matrices," IEEE Trans. Inf. Theory IT-18, 725-730 (1972).
[CrossRef]

Haney, M. W.

M. P. Christensen, V. Bhakta, D. Rajan, S. C. Douglas, S. L. Wood, and M. W. Haney, "Adaptive flat multiresolution multiplexed computational imaging architecture utilizing micromirror arrays to steer subimager field of views," Appl. Opt. 45, 2884-2892 (2006).
[CrossRef] [PubMed]

M. P. Christensen, M. W. Haney, D. Rajan, S. L. Wood, and S. C. Douglas, "PANOPTES: a thin agile multi-resolution imaging sensor," Presented at the Government Microcircuit Applications and Critical Technology Conference (GOMACTech-05), Las Vegas, Nev., 4-7 April 2005 paper 21.5.

Hu, Z.

X. Ying and Z. Hu, "Distortion correction of fisheye lens using a nonparametric imaging model," in Proceedings of Asian Conference on Computer Vision (Asian Federation of Computer Vision Societies 2004), pp. 527-532.

Ichioka, Y.

Ishida, K.

Jain, A. K.

A. K. Jain, Fundamentals of Digital Image Processing (Prentice-Hall, 1989), Chap. 2.

Kailath, T.

T. Kailath, Linear Systems (Prentice-Hall, 1980).

Kanade, T.

S. Baker and T. Kanade, "Limits on super-resolution and how to break them," IEEE Trans. Pattern Anal. Mach. Intell. 24, 1167-1183 (2002).
[CrossRef]

Kitamura, Y.

Kondou, N.

Kumagai, T.

Macovski, A.

A. Macovski, Medical Imaging Systems (Prentice-Hall, 1983).

Masaki, Y.

Milanfar, P.

S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, "Fast and robust multi-frame super-resolution," IEEE Trans. Image Process. 13, 1327-1344 (2004).
[CrossRef] [PubMed]

N. Nguyen, P. Milanfar, and G. H. Golub, "A computationally efficient image superresolution algorithm," IEEE Trans. Image Process. 10, 573-583 (2001).
[CrossRef]

Miyamoto, M.

Miyatake, S.

Miyazaki, D.

Morimoto, T.

Ng, M. K.

M. K. Ng and N. K. Bose, "Mathematical analysis of super-resolution methodology," IEEE Signal Process. Mag. 20, 62-74 (2003).
[CrossRef]

Nguyen, N.

N. Nguyen, P. Milanfar, and G. H. Golub, "A computationally efficient image superresolution algorithm," IEEE Trans. Image Process. 10, 573-583 (2001).
[CrossRef]

Rajan, D.

M. P. Christensen, V. Bhakta, D. Rajan, S. C. Douglas, S. L. Wood, and M. W. Haney, "Adaptive flat multiresolution multiplexed computational imaging architecture utilizing micromirror arrays to steer subimager field of views," Appl. Opt. 45, 2884-2892 (2006).
[CrossRef] [PubMed]

S. L. Wood, B. J. Smithson, M. P. Christensen, and D. Rajan, Rerformance of a MVE algorithm for compound eye image reconstruction using lens diversity," in Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005 (ICASSP' 05) (IEEE, 2005), Vol. II. pp. 593-596.

S. L. Wood, D. Rajan, M. P. Christensen, S. C.Douglas, and B. J. Smithson, "Resolution improvement for compound eye images through lens diversity," in Digital Signal Processing Workshop 2004 and the Third IEEE Signal Processing Education Workshop (IEEE, 2004), pp. 151-155, doi: .
[CrossRef]

M. P. Christensen, M. W. Haney, D. Rajan, S. L. Wood, and S. C. Douglas, "PANOPTES: a thin agile multi-resolution imaging sensor," Presented at the Government Microcircuit Applications and Critical Technology Conference (GOMACTech-05), Las Vegas, Nev., 4-7 April 2005 paper 21.5.

Robinson, D.

S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, "Fast and robust multi-frame super-resolution," IEEE Trans. Image Process. 13, 1327-1344 (2004).
[CrossRef] [PubMed]

Shogenji, R.

Smithson, B. J.

S. L. Wood, D. Rajan, M. P. Christensen, S. C.Douglas, and B. J. Smithson, "Resolution improvement for compound eye images through lens diversity," in Digital Signal Processing Workshop 2004 and the Third IEEE Signal Processing Education Workshop (IEEE, 2004), pp. 151-155, doi: .
[CrossRef]

S. L. Wood, B. J. Smithson, M. P. Christensen, and D. Rajan, Rerformance of a MVE algorithm for compound eye image reconstruction using lens diversity," in Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005 (ICASSP' 05) (IEEE, 2005), Vol. II. pp. 593-596.

Tanida, J.

Voss, K.

C. Brauer-Burchardt and K. Voss, "A new algorithm to correct fish-eye-and strong wide-angle-lens-distortion from single images," in Proceedings of IEEE International Conference on Image Processing (IEEE, 2001), pp. 225-228.

Wolf, E.

M. Born and E. Wolf, Principles of Optics (Cambridge U. Press, 1959).

Wood, S. L.

M. P. Christensen, V. Bhakta, D. Rajan, S. C. Douglas, S. L. Wood, and M. W. Haney, "Adaptive flat multiresolution multiplexed computational imaging architecture utilizing micromirror arrays to steer subimager field of views," Appl. Opt. 45, 2884-2892 (2006).
[CrossRef] [PubMed]

S. L. Wood, B. J. Smithson, M. P. Christensen, and D. Rajan, Rerformance of a MVE algorithm for compound eye image reconstruction using lens diversity," in Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005 (ICASSP' 05) (IEEE, 2005), Vol. II. pp. 593-596.

S. L. Wood, D. Rajan, M. P. Christensen, S. C.Douglas, and B. J. Smithson, "Resolution improvement for compound eye images through lens diversity," in Digital Signal Processing Workshop 2004 and the Third IEEE Signal Processing Education Workshop (IEEE, 2004), pp. 151-155, doi: .
[CrossRef]

M. P. Christensen, M. W. Haney, D. Rajan, S. L. Wood, and S. C. Douglas, "PANOPTES: a thin agile multi-resolution imaging sensor," Presented at the Government Microcircuit Applications and Critical Technology Conference (GOMACTech-05), Las Vegas, Nev., 4-7 April 2005 paper 21.5.

Yamada, K.

Ying, X.

X. Ying and Z. Hu, "Distortion correction of fisheye lens using a nonparametric imaging model," in Proceedings of Asian Conference on Computer Vision (Asian Federation of Computer Vision Societies 2004), pp. 527-532.

Appl. Opt.

IEEE Signal Process. Mag.

M. K. Ng and N. K. Bose, "Mathematical analysis of super-resolution methodology," IEEE Signal Process. Mag. 20, 62-74 (2003).
[CrossRef]

IEEE Trans. Image Process.

S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, "Fast and robust multi-frame super-resolution," IEEE Trans. Image Process. 13, 1327-1344 (2004).
[CrossRef] [PubMed]

M. Elad and A. Feuer, "Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images," IEEE Trans. Image Process. 6, 1646-1658 (1997).
[CrossRef] [PubMed]

N. Nguyen, P. Milanfar, and G. H. Golub, "A computationally efficient image superresolution algorithm," IEEE Trans. Image Process. 10, 573-583 (2001).
[CrossRef]

IEEE Trans. Inf. Theory

R. M. Gray, "On the asympototic eigenvalue distribution of Toeplitz matrices," IEEE Trans. Inf. Theory IT-18, 725-730 (1972).
[CrossRef]

IEEE Trans. Pattern Anal. Mach. Intell.

S. Baker and T. Kanade, "Limits on super-resolution and how to break them," IEEE Trans. Pattern Anal. Mach. Intell. 24, 1167-1183 (2002).
[CrossRef]

Other

S. Chaudhuri, ed. Super-Resolution Imaging (Kluwer Academic, 2001).

M. P. Christensen, M. W. Haney, D. Rajan, S. L. Wood, and S. C. Douglas, "PANOPTES: a thin agile multi-resolution imaging sensor," Presented at the Government Microcircuit Applications and Critical Technology Conference (GOMACTech-05), Las Vegas, Nev., 4-7 April 2005 paper 21.5.

S. L. Wood, D. Rajan, M. P. Christensen, S. C.Douglas, and B. J. Smithson, "Resolution improvement for compound eye images through lens diversity," in Digital Signal Processing Workshop 2004 and the Third IEEE Signal Processing Education Workshop (IEEE, 2004), pp. 151-155, doi: .
[CrossRef]

M. Born and E. Wolf, Principles of Optics (Cambridge U. Press, 1959).

X. Ying and Z. Hu, "Distortion correction of fisheye lens using a nonparametric imaging model," in Proceedings of Asian Conference on Computer Vision (Asian Federation of Computer Vision Societies 2004), pp. 527-532.

C. Brauer-Burchardt and K. Voss, "A new algorithm to correct fish-eye-and strong wide-angle-lens-distortion from single images," in Proceedings of IEEE International Conference on Image Processing (IEEE, 2001), pp. 225-228.

S. L. Wood, B. J. Smithson, M. P. Christensen, and D. Rajan, Rerformance of a MVE algorithm for compound eye image reconstruction using lens diversity," in Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005 (ICASSP' 05) (IEEE, 2005), Vol. II. pp. 593-596.

R. N. Bracewell, Two-Dimensional Imaging (Prentice-Hall, 1995).

A. Macovski, Medical Imaging Systems (Prentice-Hall, 1983).

T. Kailath, Linear Systems (Prentice-Hall, 1980).

A. K. Jain, Fundamentals of Digital Image Processing (Prentice-Hall, 1989), Chap. 2.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (9)

Fig. 1
Fig. 1

Basic block diagrams for (a) the physical optical system and (b) the corresponding mathematical model.

Fig. 2
Fig. 2

Projections of source-image pixels onto detector arrays.

Fig. 3
Fig. 3

Projections of source-image pixels onto three different SI detector arrays.

Fig. 4
Fig. 4

High-resolution 400 × 400 pixel image of (a) an airport (from the University of Southern California Signal and Image Processing Institute Image Database), (b) the image tile 76 × 76 pixels, (c) the SI outputs for q = 5.

Fig. 5
Fig. 5

Expected squared error for (a) 19 × 19 pixel tile from detectors with q = 3 (dashed curves) and detectors with q = 3, 4, and 5 (solid curves); (b) 19 × 19 (dashed curves) and 35 × 35 pixel tiles (solid curves); (c) detectors with q = 5, 6, and 7 (dashed curves) and q = 3, 4, and 5 (solid curves).

Fig. 6
Fig. 6

(a) Image of Bay Bridge (from the University of Southern California Signal and Image Processing Institute Image Database) and (b) 35 × 35 pixel tile. Reconstructions using a noise variance of 0.01 for (c) q = 3 and (d) q = 3, 4, 5.

Fig. 7
Fig. 7

(a) Projection of a grid through a lens with distortion; (b) grid at the center of the lens; (c) grid near the edge of the lens.

Fig. 8
Fig. 8

Performance of a lens with distortion when the tiles are viewed through different parts of the lens (solid curves) versus only near the center (dashed curves).

Fig. 9
Fig. 9

Simulated performance of a lens with fish-eye-style distortion compared with SI arrays with q = 3 only and q = 3, 4, 5 on an airplane image (solid curves) and the Bay Bridge image (dashed curves) for noise variances of (a) 1.0 and (b) 0.01.

Equations (21)

Equations on this page are rendered with MathJax. Learn more.

g = H f + v .
f ^ = f 0 + K ( g H f 0 ) .
g ̃ 0 = g g ^ 0 = g H f 0 ,
K = P ^ 0 H T ( H P ^ 0 H T + R ^ v ) 1 .
ξ = E ( f ˜ f ˜ T ) = ( I K H ) P 0 ( I K H ) T + K R v K T .
ξ = P 0 P 0 H T ( H P 0 H T + R v ) 1 H P 0 .
K = P ^ 0 ( P ^ 0 + R ^ v ) 1 .
K = p 0 I N H T q 2 p 0 + q 2 σ 2 I M = p 0 q 2 p 0 + q 2 σ 2 H T .
g 0 , 0 = H Z 0 f + v 00 = H 0 , 0 f + v 0 , 0 ,
g 0 , 1 = H Z 1 f + v 01 = H 0 , 1 f + v 0 , 1 ,
g 1 , 0 = H Z N xE f + v 10 = H 1 , 0 f + v 1 , 0 ,
g q 1 , q 1 = H Z ( q 1 ) ( N xE + 1 ) f + v q 1 , q 1 = H q 1 , q 1 f + v q 1 , q 1 .
g = H q f + v ,
g T = [ g 0 , 0           T , g 0 , 1           T , , g q 1 , q 1                               T ] T ,
H q     T = [ H 0 , 0             T , H 0 , 1             T , , H q 1 , q 1                               T ] T ,
v T = [ v 0 , 0             T , v 0 , 1             T , , v q 1 , q 1                                 T ] T .
λ i = { p 0 σ 2 / ( p 0 s i     2 + σ 2 ) for   1 i N p 0 for   N + 1 i N E .
g = [ g q 1 g q 2 g q 3 ] = [ H q 1 H q 2 H q 3 ] f + v = H f + v .
f ^ = ( H T R ^ v     1 H + P ^ 0     1 ) 1 ×   ( H T R ^ v     1 g + P ^ 0     1 f 0 ) ,
H T R ^ v     1 H = H q 1         T R ^ v 1     1 H q 1 + H q 2           T R ^ v 2         1 H q 2 + H q 3           T R ^ v 3         1 H q 3 .
r = r / q 0 [ 1 C ( r / q 0 ) 2 ] .

Metrics