Abstract

In this paper the influence of the number of lenslets on the performance of image restoration algorithms for the thin observation module by bound optics (TOMBO) imaging system was investigated, and the lenslet number was optimized to achieve thin system and high imaging performance. Subimages with different numbers of lenslets were generated following the TOMBO observation model, and image restoration algorithms were applied to evaluate the imaging performance of the TOMBO system. The optimal lenslet number was determined via theoretical performance optimization and verified via experimental comparisons of angular resolutions of two TOMBO systems and a conventional single-lens system.

© 2014 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. J. Tanida, T. Kumagai, K. Yamada, S. Miyatake, K. Ishida, T. Morimoto, N. Kondou, D. Miyazaki, Y. Ichioka, “Thin observation module by bound optics (TOMBO): concept and experimental verification,” Appl. Opt. 40(11), 1806–1813 (2001).
    [CrossRef] [PubMed]
  2. J. W. Duparré, F. C. Wippermann, “Micro-optical artificial compound eyes,” Bioinspir. Biomim. 1(1), R1–R16 (2006).
    [CrossRef] [PubMed]
  3. D. Mendlovic, “Toward a super imaging system,” Appl. Opt. 52(4), 561–566 (2013).
    [CrossRef] [PubMed]
  4. K. Choi, T. J. Schulz, “Signal-processing approaches for image-resolution restoration for TOMBO imagery,” Appl. Opt. 47(10), B104–B116 (2008).
    [CrossRef] [PubMed]
  5. M. Shankar, R. Willett, N. Pitsianis, T. Schulz, R. Gibbons, R. Te Kolste, J. Carriere, C. Chen, D. Prather, D. Brady, “Thin infrared imaging systems through multichannel sampling,” Appl. Opt. 47(10), B1–B10 (2008).
    [CrossRef] [PubMed]
  6. A. V. Kanaev, D. A. Scribner, J. R. Ackerman, E. F. Fleet, “Analysis and application of multiframe superresolution processing for conventional imaging systems and lenslet arrays,” Appl. Opt. 46(20), 4320–4328 (2007).
    [CrossRef] [PubMed]
  7. Y. Kitamura, R. Shogenji, K. Yamada, S. Miyatake, M. Miyamoto, T. Morimoto, Y. Masaki, N. Kondou, D. Miyazaki, J. Tanida, Y. Ichioka, “Reconstruction of a high-resolution image on a compound-eye image-capturing system,” Appl. Opt. 43(8), 1719–1727 (2004).
    [CrossRef] [PubMed]
  8. A. Stern, B. Javidi, “Three-dimensional image sensing and reconstruction with time-division multiplexed computational integral imaging,” Appl. Opt. 42(35), 7036–7042 (2003).
    [CrossRef] [PubMed]
  9. R. Horisaki, S. Irie, Y. Ogura, J. Tanida, “Three-dimensional information acqusition using a compound imaging system,” Opt. Rev. 14(5), 347–350 (2007).
    [CrossRef]
  10. A. V. Kanaev, J. R. Ackerman, E. F. Fleet, D. A. Scribner, “TOMBO sensor with scene-independent superresolution processing,” Opt. Lett. 32(19), 2855–2857 (2007).
    [CrossRef] [PubMed]
  11. A. A. El-Sallam, F. Boussaid, “Spectral-based blind image restoration method for thin TOMBO imagers,” Sensors (Basel Switzerland) 8(9), 6108–6124 (2008).
    [CrossRef]
  12. S. Mendelowitz, I. Klapp, D. Mendlovic, “Design of an image restoration algorithm for the TOMBO imaging system,” J. Opt. Soc. Am. A 30(6), 1193–1204 (2013).
    [CrossRef] [PubMed]
  13. Z. Lin, H. Y. Shum, “Fundamental limits of reconstruction-based superresolution algorithms under local translation,” IEEE Trans. Pattern Anal. Mach. Intell. 26(1), 83–97 (2004).
    [CrossRef] [PubMed]
  14. S. Baker, T. Kanade, “Limits on super-resolution and how to break them,” IEEE Trans. Pattern Anal. Mach. Intell. 24(9), 1167–1183 (2002).
    [CrossRef]
  15. D. Robinson, P. Milanfar, “Statistical performance analysis of super-resolution,” IEEE Trans. Image Process. 15(6), 1413–1428 (2006).
    [CrossRef] [PubMed]
  16. S. Farsiu, M. D. Robinson, M. Elad, P. Milanfar, “Fast and robust multiframe super resolution,” IEEE Trans. Image Process. 13(10), 1327–1344 (2004).
    [CrossRef] [PubMed]
  17. S. Villena, M. Vega, S. D. Babaccan, R. Molina, A. K. Katsaggelos, “Bayesian combination of sparse and non-sparse priors in image super resolution,” Digit. Signal Process. 23(2), 530–541 (2013).
    [CrossRef]
  18. D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60(2), 91–110 (2004).
    [CrossRef]
  19. M. A. Fischler, R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24(6), 381–395 (1981).
    [CrossRef]
  20. M. W. Haney, “Performance scaling in flat imagers,” Appl. Opt. 45(13), 2901–2910 (2006).
    [CrossRef] [PubMed]

2013 (3)

S. Villena, M. Vega, S. D. Babaccan, R. Molina, A. K. Katsaggelos, “Bayesian combination of sparse and non-sparse priors in image super resolution,” Digit. Signal Process. 23(2), 530–541 (2013).
[CrossRef]

D. Mendlovic, “Toward a super imaging system,” Appl. Opt. 52(4), 561–566 (2013).
[CrossRef] [PubMed]

S. Mendelowitz, I. Klapp, D. Mendlovic, “Design of an image restoration algorithm for the TOMBO imaging system,” J. Opt. Soc. Am. A 30(6), 1193–1204 (2013).
[CrossRef] [PubMed]

2008 (3)

2007 (3)

2006 (3)

M. W. Haney, “Performance scaling in flat imagers,” Appl. Opt. 45(13), 2901–2910 (2006).
[CrossRef] [PubMed]

J. W. Duparré, F. C. Wippermann, “Micro-optical artificial compound eyes,” Bioinspir. Biomim. 1(1), R1–R16 (2006).
[CrossRef] [PubMed]

D. Robinson, P. Milanfar, “Statistical performance analysis of super-resolution,” IEEE Trans. Image Process. 15(6), 1413–1428 (2006).
[CrossRef] [PubMed]

2004 (4)

S. Farsiu, M. D. Robinson, M. Elad, P. Milanfar, “Fast and robust multiframe super resolution,” IEEE Trans. Image Process. 13(10), 1327–1344 (2004).
[CrossRef] [PubMed]

Z. Lin, H. Y. Shum, “Fundamental limits of reconstruction-based superresolution algorithms under local translation,” IEEE Trans. Pattern Anal. Mach. Intell. 26(1), 83–97 (2004).
[CrossRef] [PubMed]

D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60(2), 91–110 (2004).
[CrossRef]

Y. Kitamura, R. Shogenji, K. Yamada, S. Miyatake, M. Miyamoto, T. Morimoto, Y. Masaki, N. Kondou, D. Miyazaki, J. Tanida, Y. Ichioka, “Reconstruction of a high-resolution image on a compound-eye image-capturing system,” Appl. Opt. 43(8), 1719–1727 (2004).
[CrossRef] [PubMed]

2003 (1)

2002 (1)

S. Baker, T. Kanade, “Limits on super-resolution and how to break them,” IEEE Trans. Pattern Anal. Mach. Intell. 24(9), 1167–1183 (2002).
[CrossRef]

2001 (1)

1981 (1)

M. A. Fischler, R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24(6), 381–395 (1981).
[CrossRef]

Ackerman, J. R.

Babaccan, S. D.

S. Villena, M. Vega, S. D. Babaccan, R. Molina, A. K. Katsaggelos, “Bayesian combination of sparse and non-sparse priors in image super resolution,” Digit. Signal Process. 23(2), 530–541 (2013).
[CrossRef]

Baker, S.

S. Baker, T. Kanade, “Limits on super-resolution and how to break them,” IEEE Trans. Pattern Anal. Mach. Intell. 24(9), 1167–1183 (2002).
[CrossRef]

Bolles, R. C.

M. A. Fischler, R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24(6), 381–395 (1981).
[CrossRef]

Boussaid, F.

A. A. El-Sallam, F. Boussaid, “Spectral-based blind image restoration method for thin TOMBO imagers,” Sensors (Basel Switzerland) 8(9), 6108–6124 (2008).
[CrossRef]

Brady, D.

Carriere, J.

Chen, C.

Choi, K.

Duparré, J. W.

J. W. Duparré, F. C. Wippermann, “Micro-optical artificial compound eyes,” Bioinspir. Biomim. 1(1), R1–R16 (2006).
[CrossRef] [PubMed]

Elad, M.

S. Farsiu, M. D. Robinson, M. Elad, P. Milanfar, “Fast and robust multiframe super resolution,” IEEE Trans. Image Process. 13(10), 1327–1344 (2004).
[CrossRef] [PubMed]

El-Sallam, A. A.

A. A. El-Sallam, F. Boussaid, “Spectral-based blind image restoration method for thin TOMBO imagers,” Sensors (Basel Switzerland) 8(9), 6108–6124 (2008).
[CrossRef]

Farsiu, S.

S. Farsiu, M. D. Robinson, M. Elad, P. Milanfar, “Fast and robust multiframe super resolution,” IEEE Trans. Image Process. 13(10), 1327–1344 (2004).
[CrossRef] [PubMed]

Fischler, M. A.

M. A. Fischler, R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24(6), 381–395 (1981).
[CrossRef]

Fleet, E. F.

Gibbons, R.

Haney, M. W.

Horisaki, R.

R. Horisaki, S. Irie, Y. Ogura, J. Tanida, “Three-dimensional information acqusition using a compound imaging system,” Opt. Rev. 14(5), 347–350 (2007).
[CrossRef]

Ichioka, Y.

Irie, S.

R. Horisaki, S. Irie, Y. Ogura, J. Tanida, “Three-dimensional information acqusition using a compound imaging system,” Opt. Rev. 14(5), 347–350 (2007).
[CrossRef]

Ishida, K.

Javidi, B.

Kanade, T.

S. Baker, T. Kanade, “Limits on super-resolution and how to break them,” IEEE Trans. Pattern Anal. Mach. Intell. 24(9), 1167–1183 (2002).
[CrossRef]

Kanaev, A. V.

Katsaggelos, A. K.

S. Villena, M. Vega, S. D. Babaccan, R. Molina, A. K. Katsaggelos, “Bayesian combination of sparse and non-sparse priors in image super resolution,” Digit. Signal Process. 23(2), 530–541 (2013).
[CrossRef]

Kitamura, Y.

Klapp, I.

Kondou, N.

Kumagai, T.

Lin, Z.

Z. Lin, H. Y. Shum, “Fundamental limits of reconstruction-based superresolution algorithms under local translation,” IEEE Trans. Pattern Anal. Mach. Intell. 26(1), 83–97 (2004).
[CrossRef] [PubMed]

Lowe, D. G.

D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60(2), 91–110 (2004).
[CrossRef]

Masaki, Y.

Mendelowitz, S.

Mendlovic, D.

Milanfar, P.

D. Robinson, P. Milanfar, “Statistical performance analysis of super-resolution,” IEEE Trans. Image Process. 15(6), 1413–1428 (2006).
[CrossRef] [PubMed]

S. Farsiu, M. D. Robinson, M. Elad, P. Milanfar, “Fast and robust multiframe super resolution,” IEEE Trans. Image Process. 13(10), 1327–1344 (2004).
[CrossRef] [PubMed]

Miyamoto, M.

Miyatake, S.

Miyazaki, D.

Molina, R.

S. Villena, M. Vega, S. D. Babaccan, R. Molina, A. K. Katsaggelos, “Bayesian combination of sparse and non-sparse priors in image super resolution,” Digit. Signal Process. 23(2), 530–541 (2013).
[CrossRef]

Morimoto, T.

Ogura, Y.

R. Horisaki, S. Irie, Y. Ogura, J. Tanida, “Three-dimensional information acqusition using a compound imaging system,” Opt. Rev. 14(5), 347–350 (2007).
[CrossRef]

Pitsianis, N.

Prather, D.

Robinson, D.

D. Robinson, P. Milanfar, “Statistical performance analysis of super-resolution,” IEEE Trans. Image Process. 15(6), 1413–1428 (2006).
[CrossRef] [PubMed]

Robinson, M. D.

S. Farsiu, M. D. Robinson, M. Elad, P. Milanfar, “Fast and robust multiframe super resolution,” IEEE Trans. Image Process. 13(10), 1327–1344 (2004).
[CrossRef] [PubMed]

Schulz, T.

Schulz, T. J.

Scribner, D. A.

Shankar, M.

Shogenji, R.

Shum, H. Y.

Z. Lin, H. Y. Shum, “Fundamental limits of reconstruction-based superresolution algorithms under local translation,” IEEE Trans. Pattern Anal. Mach. Intell. 26(1), 83–97 (2004).
[CrossRef] [PubMed]

Stern, A.

Tanida, J.

Te Kolste, R.

Vega, M.

S. Villena, M. Vega, S. D. Babaccan, R. Molina, A. K. Katsaggelos, “Bayesian combination of sparse and non-sparse priors in image super resolution,” Digit. Signal Process. 23(2), 530–541 (2013).
[CrossRef]

Villena, S.

S. Villena, M. Vega, S. D. Babaccan, R. Molina, A. K. Katsaggelos, “Bayesian combination of sparse and non-sparse priors in image super resolution,” Digit. Signal Process. 23(2), 530–541 (2013).
[CrossRef]

Willett, R.

Wippermann, F. C.

J. W. Duparré, F. C. Wippermann, “Micro-optical artificial compound eyes,” Bioinspir. Biomim. 1(1), R1–R16 (2006).
[CrossRef] [PubMed]

Yamada, K.

Appl. Opt. (8)

J. Tanida, T. Kumagai, K. Yamada, S. Miyatake, K. Ishida, T. Morimoto, N. Kondou, D. Miyazaki, Y. Ichioka, “Thin observation module by bound optics (TOMBO): concept and experimental verification,” Appl. Opt. 40(11), 1806–1813 (2001).
[CrossRef] [PubMed]

A. Stern, B. Javidi, “Three-dimensional image sensing and reconstruction with time-division multiplexed computational integral imaging,” Appl. Opt. 42(35), 7036–7042 (2003).
[CrossRef] [PubMed]

Y. Kitamura, R. Shogenji, K. Yamada, S. Miyatake, M. Miyamoto, T. Morimoto, Y. Masaki, N. Kondou, D. Miyazaki, J. Tanida, Y. Ichioka, “Reconstruction of a high-resolution image on a compound-eye image-capturing system,” Appl. Opt. 43(8), 1719–1727 (2004).
[CrossRef] [PubMed]

M. W. Haney, “Performance scaling in flat imagers,” Appl. Opt. 45(13), 2901–2910 (2006).
[CrossRef] [PubMed]

A. V. Kanaev, D. A. Scribner, J. R. Ackerman, E. F. Fleet, “Analysis and application of multiframe superresolution processing for conventional imaging systems and lenslet arrays,” Appl. Opt. 46(20), 4320–4328 (2007).
[CrossRef] [PubMed]

M. Shankar, R. Willett, N. Pitsianis, T. Schulz, R. Gibbons, R. Te Kolste, J. Carriere, C. Chen, D. Prather, D. Brady, “Thin infrared imaging systems through multichannel sampling,” Appl. Opt. 47(10), B1–B10 (2008).
[CrossRef] [PubMed]

K. Choi, T. J. Schulz, “Signal-processing approaches for image-resolution restoration for TOMBO imagery,” Appl. Opt. 47(10), B104–B116 (2008).
[CrossRef] [PubMed]

D. Mendlovic, “Toward a super imaging system,” Appl. Opt. 52(4), 561–566 (2013).
[CrossRef] [PubMed]

Bioinspir. Biomim. (1)

J. W. Duparré, F. C. Wippermann, “Micro-optical artificial compound eyes,” Bioinspir. Biomim. 1(1), R1–R16 (2006).
[CrossRef] [PubMed]

Commun. ACM (1)

M. A. Fischler, R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24(6), 381–395 (1981).
[CrossRef]

Digit. Signal Process. (1)

S. Villena, M. Vega, S. D. Babaccan, R. Molina, A. K. Katsaggelos, “Bayesian combination of sparse and non-sparse priors in image super resolution,” Digit. Signal Process. 23(2), 530–541 (2013).
[CrossRef]

IEEE Trans. Image Process. (2)

D. Robinson, P. Milanfar, “Statistical performance analysis of super-resolution,” IEEE Trans. Image Process. 15(6), 1413–1428 (2006).
[CrossRef] [PubMed]

S. Farsiu, M. D. Robinson, M. Elad, P. Milanfar, “Fast and robust multiframe super resolution,” IEEE Trans. Image Process. 13(10), 1327–1344 (2004).
[CrossRef] [PubMed]

IEEE Trans. Pattern Anal. Mach. Intell. (2)

Z. Lin, H. Y. Shum, “Fundamental limits of reconstruction-based superresolution algorithms under local translation,” IEEE Trans. Pattern Anal. Mach. Intell. 26(1), 83–97 (2004).
[CrossRef] [PubMed]

S. Baker, T. Kanade, “Limits on super-resolution and how to break them,” IEEE Trans. Pattern Anal. Mach. Intell. 24(9), 1167–1183 (2002).
[CrossRef]

Int. J. Comput. Vis. (1)

D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60(2), 91–110 (2004).
[CrossRef]

J. Opt. Soc. Am. A (1)

Opt. Lett. (1)

Opt. Rev. (1)

R. Horisaki, S. Irie, Y. Ogura, J. Tanida, “Three-dimensional information acqusition using a compound imaging system,” Opt. Rev. 14(5), 347–350 (2007).
[CrossRef]

Sensors (Basel Switzerland) (1)

A. A. El-Sallam, F. Boussaid, “Spectral-based blind image restoration method for thin TOMBO imagers,” Sensors (Basel Switzerland) 8(9), 6108–6124 (2008).
[CrossRef]

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (15)

Fig. 1
Fig. 1

The optical arrangement of the TOMBO imaging system.

Fig. 2
Fig. 2

The 1/3 translation between two low-resolution pixels of subimages when downsampling factor d is 3. The dotted squares represent high-resolution pixels from one input image, whereas the red square represents a low-resolution pixel from the top left corner of subimages L2,2, and the green square represents a low-resolution pixel from the top left corner of subimages L3,3.

Fig. 3
Fig. 3

(a) Input image. (b) Simulated image captured by the single-lens system. Simulated subimages with lenslet numbers (c) 2 × 2, (d) 3 × 3, (e) 4 × 4, (f) 5 × 5, (g) 6 × 6, (h) 7 × 7, (i) 8 × 8, (j) 9 × 9, and (k) 10 × 10.

Fig. 4
Fig. 4

(a) Input image. (b) Simulated image captured by the single-lens system. Restored images with actual translation parameters and lenslet numbers (c) 2 × 2, (d) 3 × 3, (e) 4 × 4, (f) 5 × 5, (g) 6 × 6, (h) 7 × 7, (i) 8 × 8, (j) 9 × 9, and (k) 10 × 10.

Fig. 5
Fig. 5

(a) Input image. (b) Simulated image captured by the single-lens system. Restored images with estimated translation parameters and lenslet numbers (c) 2 × 2, (d) 3 × 3, (e) 4 × 4, (f) 5 × 5, (g) 6 × 6, (h) 7 × 7, (i) 8 × 8, (j) 9 × 9, and (k) 10 × 10.

Fig. 6
Fig. 6

Averaged MSE of restored images with the actual translation parameters versus lenslet number.

Fig. 7
Fig. 7

Averaged PSNR of restored images with the actual translation parameters versus lenslet number.

Fig. 8
Fig. 8

Averaged MSE of restored images with the estimated translation parameters by SIFT-RANSAC versus lenslet number.

Fig. 9
Fig. 9

Averaged PSNR of restored images with the estimated translation parameters by SIFT-RANSAC versus lenslet number.

Fig. 10
Fig. 10

The averaged horizontal registration errors versus lenslet number.

Fig. 11
Fig. 11

The averaged vertical registration errors versus lenslet number.

Fig. 12
Fig. 12

The optical arrangements of experimental setups for (a) the TOMBO system and (b) the single-lens system.

Fig. 13
Fig. 13

Photo of the experimental setup for a TOMBO system.

Fig. 14
Fig. 14

The data captured by (a) the 4 × 4 lenslet TOMBO system and (b) the 5 × 5 lenslet TOMBO system.

Fig. 15
Fig. 15

The restored images in the (a) 4 × 4 and (b) 5 × 5 lenslet TOMBO systems and (c) the image captured by the single-lens system.

Equations (3)

Equations on this page are rendered with MathJax. Learn more.

L i,j =[ b i,j t i,j ( r i,j )H]D+ v i,j ,
PSNR=20 log 10 MAX MSE ,
α= 2b× 10 3 f c ×206,265,

Metrics