Abstract

In this paper the influence of the number of lenslets on the performance of image restoration algorithms for the thin observation module by bound optics (TOMBO) imaging system was investigated, and the lenslet number was optimized to achieve thin system and high imaging performance. Subimages with different numbers of lenslets were generated following the TOMBO observation model, and image restoration algorithms were applied to evaluate the imaging performance of the TOMBO system. The optimal lenslet number was determined via theoretical performance optimization and verified via experimental comparisons of angular resolutions of two TOMBO systems and a conventional single-lens system.

© 2014 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. J. Tanida, T. Kumagai, K. Yamada, S. Miyatake, K. Ishida, T. Morimoto, N. Kondou, D. Miyazaki, and Y. Ichioka, “Thin observation module by bound optics (TOMBO): concept and experimental verification,” Appl. Opt. 40(11), 1806–1813 (2001).
    [CrossRef] [PubMed]
  2. J. W. Duparré and F. C. Wippermann, “Micro-optical artificial compound eyes,” Bioinspir. Biomim. 1(1), R1–R16 (2006).
    [CrossRef] [PubMed]
  3. D. Mendlovic, “Toward a super imaging system,” Appl. Opt. 52(4), 561–566 (2013).
    [CrossRef] [PubMed]
  4. K. Choi and T. J. Schulz, “Signal-processing approaches for image-resolution restoration for TOMBO imagery,” Appl. Opt. 47(10), B104–B116 (2008).
    [CrossRef] [PubMed]
  5. M. Shankar, R. Willett, N. Pitsianis, T. Schulz, R. Gibbons, R. Te Kolste, J. Carriere, C. Chen, D. Prather, and D. Brady, “Thin infrared imaging systems through multichannel sampling,” Appl. Opt. 47(10), B1–B10 (2008).
    [CrossRef] [PubMed]
  6. A. V. Kanaev, D. A. Scribner, J. R. Ackerman, and E. F. Fleet, “Analysis and application of multiframe superresolution processing for conventional imaging systems and lenslet arrays,” Appl. Opt. 46(20), 4320–4328 (2007).
    [CrossRef] [PubMed]
  7. Y. Kitamura, R. Shogenji, K. Yamada, S. Miyatake, M. Miyamoto, T. Morimoto, Y. Masaki, N. Kondou, D. Miyazaki, J. Tanida, and Y. Ichioka, “Reconstruction of a high-resolution image on a compound-eye image-capturing system,” Appl. Opt. 43(8), 1719–1727 (2004).
    [CrossRef] [PubMed]
  8. A. Stern and B. Javidi, “Three-dimensional image sensing and reconstruction with time-division multiplexed computational integral imaging,” Appl. Opt. 42(35), 7036–7042 (2003).
    [CrossRef] [PubMed]
  9. R. Horisaki, S. Irie, Y. Ogura, and J. Tanida, “Three-dimensional information acqusition using a compound imaging system,” Opt. Rev. 14(5), 347–350 (2007).
    [CrossRef]
  10. A. V. Kanaev, J. R. Ackerman, E. F. Fleet, and D. A. Scribner, “TOMBO sensor with scene-independent superresolution processing,” Opt. Lett. 32(19), 2855–2857 (2007).
    [CrossRef] [PubMed]
  11. A. A. El-Sallam and F. Boussaid, “Spectral-based blind image restoration method for thin TOMBO imagers,” Sensors (Basel Switzerland) 8(9), 6108–6124 (2008).
    [CrossRef]
  12. S. Mendelowitz, I. Klapp, and D. Mendlovic, “Design of an image restoration algorithm for the TOMBO imaging system,” J. Opt. Soc. Am. A 30(6), 1193–1204 (2013).
    [CrossRef] [PubMed]
  13. Z. Lin and H. Y. Shum, “Fundamental limits of reconstruction-based superresolution algorithms under local translation,” IEEE Trans. Pattern Anal. Mach. Intell. 26(1), 83–97 (2004).
    [CrossRef] [PubMed]
  14. S. Baker and T. Kanade, “Limits on super-resolution and how to break them,” IEEE Trans. Pattern Anal. Mach. Intell. 24(9), 1167–1183 (2002).
    [CrossRef]
  15. D. Robinson and P. Milanfar, “Statistical performance analysis of super-resolution,” IEEE Trans. Image Process. 15(6), 1413–1428 (2006).
    [CrossRef] [PubMed]
  16. S. Farsiu, M. D. Robinson, M. Elad, and P. Milanfar, “Fast and robust multiframe super resolution,” IEEE Trans. Image Process. 13(10), 1327–1344 (2004).
    [CrossRef] [PubMed]
  17. S. Villena, M. Vega, S. D. Babaccan, R. Molina, and A. K. Katsaggelos, “Bayesian combination of sparse and non-sparse priors in image super resolution,” Digit. Signal Process. 23(2), 530–541 (2013).
    [CrossRef]
  18. D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60(2), 91–110 (2004).
    [CrossRef]
  19. M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24(6), 381–395 (1981).
    [CrossRef]
  20. M. W. Haney, “Performance scaling in flat imagers,” Appl. Opt. 45(13), 2901–2910 (2006).
    [CrossRef] [PubMed]

2013 (3)

S. Villena, M. Vega, S. D. Babaccan, R. Molina, and A. K. Katsaggelos, “Bayesian combination of sparse and non-sparse priors in image super resolution,” Digit. Signal Process. 23(2), 530–541 (2013).
[CrossRef]

D. Mendlovic, “Toward a super imaging system,” Appl. Opt. 52(4), 561–566 (2013).
[CrossRef] [PubMed]

S. Mendelowitz, I. Klapp, and D. Mendlovic, “Design of an image restoration algorithm for the TOMBO imaging system,” J. Opt. Soc. Am. A 30(6), 1193–1204 (2013).
[CrossRef] [PubMed]

2008 (3)

2007 (3)

2006 (3)

M. W. Haney, “Performance scaling in flat imagers,” Appl. Opt. 45(13), 2901–2910 (2006).
[CrossRef] [PubMed]

J. W. Duparré and F. C. Wippermann, “Micro-optical artificial compound eyes,” Bioinspir. Biomim. 1(1), R1–R16 (2006).
[CrossRef] [PubMed]

D. Robinson and P. Milanfar, “Statistical performance analysis of super-resolution,” IEEE Trans. Image Process. 15(6), 1413–1428 (2006).
[CrossRef] [PubMed]

2004 (4)

S. Farsiu, M. D. Robinson, M. Elad, and P. Milanfar, “Fast and robust multiframe super resolution,” IEEE Trans. Image Process. 13(10), 1327–1344 (2004).
[CrossRef] [PubMed]

Z. Lin and H. Y. Shum, “Fundamental limits of reconstruction-based superresolution algorithms under local translation,” IEEE Trans. Pattern Anal. Mach. Intell. 26(1), 83–97 (2004).
[CrossRef] [PubMed]

D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60(2), 91–110 (2004).
[CrossRef]

Y. Kitamura, R. Shogenji, K. Yamada, S. Miyatake, M. Miyamoto, T. Morimoto, Y. Masaki, N. Kondou, D. Miyazaki, J. Tanida, and Y. Ichioka, “Reconstruction of a high-resolution image on a compound-eye image-capturing system,” Appl. Opt. 43(8), 1719–1727 (2004).
[CrossRef] [PubMed]

2003 (1)

2002 (1)

S. Baker and T. Kanade, “Limits on super-resolution and how to break them,” IEEE Trans. Pattern Anal. Mach. Intell. 24(9), 1167–1183 (2002).
[CrossRef]

2001 (1)

1981 (1)

M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24(6), 381–395 (1981).
[CrossRef]

Ackerman, J. R.

Babaccan, S. D.

S. Villena, M. Vega, S. D. Babaccan, R. Molina, and A. K. Katsaggelos, “Bayesian combination of sparse and non-sparse priors in image super resolution,” Digit. Signal Process. 23(2), 530–541 (2013).
[CrossRef]

Baker, S.

S. Baker and T. Kanade, “Limits on super-resolution and how to break them,” IEEE Trans. Pattern Anal. Mach. Intell. 24(9), 1167–1183 (2002).
[CrossRef]

Bolles, R. C.

M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24(6), 381–395 (1981).
[CrossRef]

Boussaid, F.

A. A. El-Sallam and F. Boussaid, “Spectral-based blind image restoration method for thin TOMBO imagers,” Sensors (Basel Switzerland) 8(9), 6108–6124 (2008).
[CrossRef]

Brady, D.

Carriere, J.

Chen, C.

Choi, K.

Duparré, J. W.

J. W. Duparré and F. C. Wippermann, “Micro-optical artificial compound eyes,” Bioinspir. Biomim. 1(1), R1–R16 (2006).
[CrossRef] [PubMed]

Elad, M.

S. Farsiu, M. D. Robinson, M. Elad, and P. Milanfar, “Fast and robust multiframe super resolution,” IEEE Trans. Image Process. 13(10), 1327–1344 (2004).
[CrossRef] [PubMed]

El-Sallam, A. A.

A. A. El-Sallam and F. Boussaid, “Spectral-based blind image restoration method for thin TOMBO imagers,” Sensors (Basel Switzerland) 8(9), 6108–6124 (2008).
[CrossRef]

Farsiu, S.

S. Farsiu, M. D. Robinson, M. Elad, and P. Milanfar, “Fast and robust multiframe super resolution,” IEEE Trans. Image Process. 13(10), 1327–1344 (2004).
[CrossRef] [PubMed]

Fischler, M. A.

M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24(6), 381–395 (1981).
[CrossRef]

Fleet, E. F.

Gibbons, R.

Haney, M. W.

Horisaki, R.

R. Horisaki, S. Irie, Y. Ogura, and J. Tanida, “Three-dimensional information acqusition using a compound imaging system,” Opt. Rev. 14(5), 347–350 (2007).
[CrossRef]

Ichioka, Y.

Irie, S.

R. Horisaki, S. Irie, Y. Ogura, and J. Tanida, “Three-dimensional information acqusition using a compound imaging system,” Opt. Rev. 14(5), 347–350 (2007).
[CrossRef]

Ishida, K.

Javidi, B.

Kanade, T.

S. Baker and T. Kanade, “Limits on super-resolution and how to break them,” IEEE Trans. Pattern Anal. Mach. Intell. 24(9), 1167–1183 (2002).
[CrossRef]

Kanaev, A. V.

Katsaggelos, A. K.

S. Villena, M. Vega, S. D. Babaccan, R. Molina, and A. K. Katsaggelos, “Bayesian combination of sparse and non-sparse priors in image super resolution,” Digit. Signal Process. 23(2), 530–541 (2013).
[CrossRef]

Kitamura, Y.

Klapp, I.

Kondou, N.

Kumagai, T.

Lin, Z.

Z. Lin and H. Y. Shum, “Fundamental limits of reconstruction-based superresolution algorithms under local translation,” IEEE Trans. Pattern Anal. Mach. Intell. 26(1), 83–97 (2004).
[CrossRef] [PubMed]

Lowe, D. G.

D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60(2), 91–110 (2004).
[CrossRef]

Masaki, Y.

Mendelowitz, S.

Mendlovic, D.

Milanfar, P.

D. Robinson and P. Milanfar, “Statistical performance analysis of super-resolution,” IEEE Trans. Image Process. 15(6), 1413–1428 (2006).
[CrossRef] [PubMed]

S. Farsiu, M. D. Robinson, M. Elad, and P. Milanfar, “Fast and robust multiframe super resolution,” IEEE Trans. Image Process. 13(10), 1327–1344 (2004).
[CrossRef] [PubMed]

Miyamoto, M.

Miyatake, S.

Miyazaki, D.

Molina, R.

S. Villena, M. Vega, S. D. Babaccan, R. Molina, and A. K. Katsaggelos, “Bayesian combination of sparse and non-sparse priors in image super resolution,” Digit. Signal Process. 23(2), 530–541 (2013).
[CrossRef]

Morimoto, T.

Ogura, Y.

R. Horisaki, S. Irie, Y. Ogura, and J. Tanida, “Three-dimensional information acqusition using a compound imaging system,” Opt. Rev. 14(5), 347–350 (2007).
[CrossRef]

Pitsianis, N.

Prather, D.

Robinson, D.

D. Robinson and P. Milanfar, “Statistical performance analysis of super-resolution,” IEEE Trans. Image Process. 15(6), 1413–1428 (2006).
[CrossRef] [PubMed]

Robinson, M. D.

S. Farsiu, M. D. Robinson, M. Elad, and P. Milanfar, “Fast and robust multiframe super resolution,” IEEE Trans. Image Process. 13(10), 1327–1344 (2004).
[CrossRef] [PubMed]

Schulz, T.

Schulz, T. J.

Scribner, D. A.

Shankar, M.

Shogenji, R.

Shum, H. Y.

Z. Lin and H. Y. Shum, “Fundamental limits of reconstruction-based superresolution algorithms under local translation,” IEEE Trans. Pattern Anal. Mach. Intell. 26(1), 83–97 (2004).
[CrossRef] [PubMed]

Stern, A.

Tanida, J.

Te Kolste, R.

Vega, M.

S. Villena, M. Vega, S. D. Babaccan, R. Molina, and A. K. Katsaggelos, “Bayesian combination of sparse and non-sparse priors in image super resolution,” Digit. Signal Process. 23(2), 530–541 (2013).
[CrossRef]

Villena, S.

S. Villena, M. Vega, S. D. Babaccan, R. Molina, and A. K. Katsaggelos, “Bayesian combination of sparse and non-sparse priors in image super resolution,” Digit. Signal Process. 23(2), 530–541 (2013).
[CrossRef]

Willett, R.

Wippermann, F. C.

J. W. Duparré and F. C. Wippermann, “Micro-optical artificial compound eyes,” Bioinspir. Biomim. 1(1), R1–R16 (2006).
[CrossRef] [PubMed]

Yamada, K.

Appl. Opt. (8)

D. Mendlovic, “Toward a super imaging system,” Appl. Opt. 52(4), 561–566 (2013).
[CrossRef] [PubMed]

K. Choi and T. J. Schulz, “Signal-processing approaches for image-resolution restoration for TOMBO imagery,” Appl. Opt. 47(10), B104–B116 (2008).
[CrossRef] [PubMed]

M. Shankar, R. Willett, N. Pitsianis, T. Schulz, R. Gibbons, R. Te Kolste, J. Carriere, C. Chen, D. Prather, and D. Brady, “Thin infrared imaging systems through multichannel sampling,” Appl. Opt. 47(10), B1–B10 (2008).
[CrossRef] [PubMed]

A. V. Kanaev, D. A. Scribner, J. R. Ackerman, and E. F. Fleet, “Analysis and application of multiframe superresolution processing for conventional imaging systems and lenslet arrays,” Appl. Opt. 46(20), 4320–4328 (2007).
[CrossRef] [PubMed]

Y. Kitamura, R. Shogenji, K. Yamada, S. Miyatake, M. Miyamoto, T. Morimoto, Y. Masaki, N. Kondou, D. Miyazaki, J. Tanida, and Y. Ichioka, “Reconstruction of a high-resolution image on a compound-eye image-capturing system,” Appl. Opt. 43(8), 1719–1727 (2004).
[CrossRef] [PubMed]

A. Stern and B. Javidi, “Three-dimensional image sensing and reconstruction with time-division multiplexed computational integral imaging,” Appl. Opt. 42(35), 7036–7042 (2003).
[CrossRef] [PubMed]

J. Tanida, T. Kumagai, K. Yamada, S. Miyatake, K. Ishida, T. Morimoto, N. Kondou, D. Miyazaki, and Y. Ichioka, “Thin observation module by bound optics (TOMBO): concept and experimental verification,” Appl. Opt. 40(11), 1806–1813 (2001).
[CrossRef] [PubMed]

M. W. Haney, “Performance scaling in flat imagers,” Appl. Opt. 45(13), 2901–2910 (2006).
[CrossRef] [PubMed]

Bioinspir. Biomim. (1)

J. W. Duparré and F. C. Wippermann, “Micro-optical artificial compound eyes,” Bioinspir. Biomim. 1(1), R1–R16 (2006).
[CrossRef] [PubMed]

Commun. ACM (1)

M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24(6), 381–395 (1981).
[CrossRef]

Digit. Signal Process. (1)

S. Villena, M. Vega, S. D. Babaccan, R. Molina, and A. K. Katsaggelos, “Bayesian combination of sparse and non-sparse priors in image super resolution,” Digit. Signal Process. 23(2), 530–541 (2013).
[CrossRef]

IEEE Trans. Image Process. (2)

D. Robinson and P. Milanfar, “Statistical performance analysis of super-resolution,” IEEE Trans. Image Process. 15(6), 1413–1428 (2006).
[CrossRef] [PubMed]

S. Farsiu, M. D. Robinson, M. Elad, and P. Milanfar, “Fast and robust multiframe super resolution,” IEEE Trans. Image Process. 13(10), 1327–1344 (2004).
[CrossRef] [PubMed]

IEEE Trans. Pattern Anal. Mach. Intell. (2)

Z. Lin and H. Y. Shum, “Fundamental limits of reconstruction-based superresolution algorithms under local translation,” IEEE Trans. Pattern Anal. Mach. Intell. 26(1), 83–97 (2004).
[CrossRef] [PubMed]

S. Baker and T. Kanade, “Limits on super-resolution and how to break them,” IEEE Trans. Pattern Anal. Mach. Intell. 24(9), 1167–1183 (2002).
[CrossRef]

Int. J. Comput. Vis. (1)

D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60(2), 91–110 (2004).
[CrossRef]

J. Opt. Soc. Am. A (1)

Opt. Lett. (1)

Opt. Rev. (1)

R. Horisaki, S. Irie, Y. Ogura, and J. Tanida, “Three-dimensional information acqusition using a compound imaging system,” Opt. Rev. 14(5), 347–350 (2007).
[CrossRef]

Sensors (Basel Switzerland) (1)

A. A. El-Sallam and F. Boussaid, “Spectral-based blind image restoration method for thin TOMBO imagers,” Sensors (Basel Switzerland) 8(9), 6108–6124 (2008).
[CrossRef]

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (15)

Fig. 1
Fig. 1

The optical arrangement of the TOMBO imaging system.

Fig. 2
Fig. 2

The 1/3 translation between two low-resolution pixels of subimages when downsampling factor d is 3. The dotted squares represent high-resolution pixels from one input image, whereas the red square represents a low-resolution pixel from the top left corner of subimages L2,2, and the green square represents a low-resolution pixel from the top left corner of subimages L3,3.

Fig. 3
Fig. 3

(a) Input image. (b) Simulated image captured by the single-lens system. Simulated subimages with lenslet numbers (c) 2 × 2, (d) 3 × 3, (e) 4 × 4, (f) 5 × 5, (g) 6 × 6, (h) 7 × 7, (i) 8 × 8, (j) 9 × 9, and (k) 10 × 10.

Fig. 4
Fig. 4

(a) Input image. (b) Simulated image captured by the single-lens system. Restored images with actual translation parameters and lenslet numbers (c) 2 × 2, (d) 3 × 3, (e) 4 × 4, (f) 5 × 5, (g) 6 × 6, (h) 7 × 7, (i) 8 × 8, (j) 9 × 9, and (k) 10 × 10.

Fig. 5
Fig. 5

(a) Input image. (b) Simulated image captured by the single-lens system. Restored images with estimated translation parameters and lenslet numbers (c) 2 × 2, (d) 3 × 3, (e) 4 × 4, (f) 5 × 5, (g) 6 × 6, (h) 7 × 7, (i) 8 × 8, (j) 9 × 9, and (k) 10 × 10.

Fig. 6
Fig. 6

Averaged MSE of restored images with the actual translation parameters versus lenslet number.

Fig. 7
Fig. 7

Averaged PSNR of restored images with the actual translation parameters versus lenslet number.

Fig. 8
Fig. 8

Averaged MSE of restored images with the estimated translation parameters by SIFT-RANSAC versus lenslet number.

Fig. 9
Fig. 9

Averaged PSNR of restored images with the estimated translation parameters by SIFT-RANSAC versus lenslet number.

Fig. 10
Fig. 10

The averaged horizontal registration errors versus lenslet number.

Fig. 11
Fig. 11

The averaged vertical registration errors versus lenslet number.

Fig. 12
Fig. 12

The optical arrangements of experimental setups for (a) the TOMBO system and (b) the single-lens system.

Fig. 13
Fig. 13

Photo of the experimental setup for a TOMBO system.

Fig. 14
Fig. 14

The data captured by (a) the 4 × 4 lenslet TOMBO system and (b) the 5 × 5 lenslet TOMBO system.

Fig. 15
Fig. 15

The restored images in the (a) 4 × 4 and (b) 5 × 5 lenslet TOMBO systems and (c) the image captured by the single-lens system.

Equations (3)

Equations on this page are rendered with MathJax. Learn more.

L i,j =[ b i,j t i,j ( r i,j )H]D+ v i,j ,
PSNR=20 log 10 MAX MSE ,
α= 2b× 10 3 f c ×206,265,

Metrics