Abstract

In this paper the influence of the number of lenslets on the performance of image restoration algorithms for the thin observation module by bound optics (TOMBO) imaging system was investigated, and the lenslet number was optimized to achieve thin system and high imaging performance. Subimages with different numbers of lenslets were generated following the TOMBO observation model, and image restoration algorithms were applied to evaluate the imaging performance of the TOMBO system. The optimal lenslet number was determined via theoretical performance optimization and verified via experimental comparisons of angular resolutions of two TOMBO systems and a conventional single-lens system.

© 2014 Optical Society of America

1. Introduction

Thin observation module by bound optics (TOMBO) based on a combination of imaging techniques and efficient image processing algorithms is a fast-developing area of computational photography [13]. The TOMBO imaging system is an optical system that achieves thinness and high-resolution imaging by replacing a conventional full-aperture lens with a lenslet array [4]. The optical arrangement of the TOMBO imaging system is shown in Fig. 1. Regarding thinness, the focal length of each lenslet can be 1/n of the conventional single-lens system with equal f number, with n being the number of subimages of the scene in one direction across the image sensor. The reduction in the focal length of the lenslet (keeping the f number the same) would keep the focal resolution unaffected with the compromise of reduction in angular resolution [5]. In the view of high resolution, recovering the lost angular resolution by collecting subimages can be achieved. The lenslet array accumulates a series of subimages from the same scene which are undersampled. Each subimage resolution is determined by pixel size rather than the optical performance of the lenslet, and these subimages can be processed in a manner similar to multiframe superresolution processing to obtain a fully upsampled, high-resolution image [6]. A number of TOMBO image restoration algorithms have been proposed and proven to be effective in the literature [712].

 

Fig. 1 The optical arrangement of the TOMBO imaging system.

Download Full Size | PPT Slide | PDF

In this paper we focus on the optimization of the number of lenslets to achieve a thin system with high image restoration performance for the TOMBO system. For one TOMBO system with a fixed image sensor pixel number, a larger lenslet number leads to more subimages and fewer pixels for each subimage. More subimages only improve the resolution marginally if a special set of low-resolution pixels has been captured. On the other hand, subimages with fewer pixels demand a larger magnification factor, which leads to the performance deterioration of existing algorithms [13]. Therefore, optimizing the lenslet number is an important practical problem regarding the design of TOMBO. However, due to the specific architecture of TOMBO, to the best of our knowledge, this problem has not been adequately discussed.

In the past, the TOMBO system has been analyzed when image restoration is based on Moire magnification [2]. The performance limits of reconstruction-based superresolution algorithms and the fundamental requirements of low-resolution images with unlimited numbers have been discussed in great detail [1315]. Unlike previous work, this paper bases the image restoration on multiframe superresolution algorithms which could increase the high-frequency components and remove the degradations caused by the imaging process. The number of low-resolution images is constrained by the pixel number of each low-resolution image in the TOMBO system, and this constraint determines the maximum magnification factor of superresolution algorithms and translation requirements between low-resolution images. Moreover, we focus on balancing the low-resolution image numbers and image restoration performance for practical TOMBO systems (more low-resolution images correspond to thinner system dimensions, and higher image restoration performance leads to angular resolution closer to that of the single-lens system).

In this paper, the optimization of the number of lenslets for the TOMBO system is performed by analyzing the influence of lenslet number on the image restoration performance. The imaging process (comprising image global translation, blur, downsampling, and noise) of each subimaging system with different lenslet numbers is presented. Then iterative backprojection (IBP) [8], bilateral total variance (BTV) [16], and l1 norm combined with simultaneous auto regressive (l1-SAR) [17] are applied to restore high-resolution images with actual translation parameters as well as translation parameters estimated by scale invariant feature transform [18] and random sample consensus [19] (SIFT-RANSAC), respectively. Mean square error (MSE) and peak signal-to-noise ratio (PSNR) of restored images are calculated for image quality assessment. The optimal lenslet number is obtained by analyzing the influence of lenslet number on the restored images with 100 synthetic and real images as input. Experimental comparison of angular resolutions among two TOMBO systems and a single-lens system is performed to validate the optimal lenslet number.

2. Performance simulation

Consider a TOMBO system with n × n subimaging units. Each captured subimage can be modeled as [12]

Li,j=[bi,jti,j(ri,j)H]D+vi,j,
where Li,j represents the blurred, noisy, and low-resolution output image captured by subimaging system in the i row of the j column of TOMBO (i, j = 1, 2, …, n); H is the input high-resolution image; bi,j is a two-dimensional PSF representing the channel blur for each imaging unit; ti,j(ri,j) is a global translation shift operator which has ri,j = [Δxi, Δyj] translation with respect to the input image; ↓D is the downsampling operator; and vi,j is the noise, such as fixed-detector thermal noise, signal-dependent shot noise, and background noise [20].

In our simulation, a rotationally symmetric Gaussian low-pass filter of size [5] with standard deviation σ2 = 1 is used as channel blur bi,j, variance σ2 of two-dimensional zero mean white Gaussian noise is set to 2, and downsampling factor d equals the lenslet number n.

Because the sufficient number of low-resolution subimages is M2 when the translation between adjacent subimages is 1/d (M is an integer magnification factor of superresolution algorithms) [13], global translation for input image is set to ri,j = [(i – 1), (j – 1)] (i.e., Δxi = i – 1 and Δyj = j – 1), and magnification factor M equals the lenslet number n in our simulation. For instance, translation between two low-resolution pixels is 1/3 in horizontal and vertical as shown in Fig. 2 when downsampling factor d is 3. Take the system in [5]. as an example: the translation between adjacent subimages is manufactured as 70 μm, and this corresponds to 2 1/3 pixels of the sensor. Figure 3 shows simulated subimages with different lenslet numbers. All these subimages are resized to resolution of the input image with nearest interpolation.

 

Fig. 2 The 1/3 translation between two low-resolution pixels of subimages when downsampling factor d is 3. The dotted squares represent high-resolution pixels from one input image, whereas the red square represents a low-resolution pixel from the top left corner of subimages L2,2, and the green square represents a low-resolution pixel from the top left corner of subimages L3,3.

Download Full Size | PPT Slide | PDF

 

Fig. 3 (a) Input image. (b) Simulated image captured by the single-lens system. Simulated subimages with lenslet numbers (c) 2 × 2, (d) 3 × 3, (e) 4 × 4, (f) 5 × 5, (g) 6 × 6, (h) 7 × 7, (i) 8 × 8, (j) 9 × 9, and (k) 10 × 10.

Download Full Size | PPT Slide | PDF

Figure 4 presents the restored images using the l1-SAR algorithm with actual translation parameters. In the l1-SAR algorithm, the magnification factor M equals the lenslet number n, and the number of iterations is 20. Furthermore, for consideration of practical applications, restored images using the same algorithm with estimated translation parameters by SIFT-RANSAC are shown in Fig. 5. Matched pixel pairs between any two subimages are generated by SIFT, and then RANSAC is used to remove incorrect matches.

 

Fig. 4 (a) Input image. (b) Simulated image captured by the single-lens system. Restored images with actual translation parameters and lenslet numbers (c) 2 × 2, (d) 3 × 3, (e) 4 × 4, (f) 5 × 5, (g) 6 × 6, (h) 7 × 7, (i) 8 × 8, (j) 9 × 9, and (k) 10 × 10.

Download Full Size | PPT Slide | PDF

 

Fig. 5 (a) Input image. (b) Simulated image captured by the single-lens system. Restored images with estimated translation parameters and lenslet numbers (c) 2 × 2, (d) 3 × 3, (e) 4 × 4, (f) 5 × 5, (g) 6 × 6, (h) 7 × 7, (i) 8 × 8, (j) 9 × 9, and (k) 10 × 10.

Download Full Size | PPT Slide | PDF

Restored images with 100 synthetic and real images as input are tested for assessment of the influence of the lenslet number. IBP, BTV, and l1-SAR are utilized as restored algorithms. Moreover, MSE and PSNR of restored images are averaged for quantitative evaluation of the influence. PSNR is defined as

PSNR=20log10MAXMSE,
where MAX is the maximum pixel intensity of an input image and MSE is the mean square error between the input image and the restored image. For the sake of simplicity, temporal noise is not taken into account in Eq. (2).

The influence of the lenslet number on the averaged image quality (MSE and PSNR) of restored images is present in Figs. 6, 7, 8, and 9. As shown in these figures, l1-SAR has higher restoration performance than BTV, whereas BTV is better than IBP. Moreover, under the actual translation conditions, the restoration performance remains almost unchanged with the increasing lenslet number. On the other hand, under the estimated translation conditions the performance deteriorates with the increasing lenslet number. The registration error is a key factor that leads to the deterioration of restoration performances.

 

Fig. 6 Averaged MSE of restored images with the actual translation parameters versus lenslet number.

Download Full Size | PPT Slide | PDF

 

Fig. 7 Averaged PSNR of restored images with the actual translation parameters versus lenslet number.

Download Full Size | PPT Slide | PDF

 

Fig. 8 Averaged MSE of restored images with the estimated translation parameters by SIFT-RANSAC versus lenslet number.

Download Full Size | PPT Slide | PDF

 

Fig. 9 Averaged PSNR of restored images with the estimated translation parameters by SIFT-RANSAC versus lenslet number.

Download Full Size | PPT Slide | PDF

Figures 10 and 11 present the averaged horizontal and vertical registration errors estimated using maximum likelihood [7] and SIFT-RANSAC. As shown in these two figures, in general SIFT-RANSAC has smaller registration errors than maximum likelihood. However, the registration errors of both registration algorithms increase as the lenslet number increases. On the contrary, in the image restoration procedure, as actual translation parameter is equal to 1/n, higher registration accuracy is required as the lenslet number increases. Under this condition, larger registration errors lead to restoration performance deterioration.

 

Fig. 10 The averaged horizontal registration errors versus lenslet number.

Download Full Size | PPT Slide | PDF

 

Fig. 11 The averaged vertical registration errors versus lenslet number.

Download Full Size | PPT Slide | PDF

As a larger lenslet number leads to thinner system dimensions, and the simulation results show that MSE or PNSR with 4 × 4 lenslets is almost the same as that in the single-lens system, the optimal lenslet number for TOMBO system is determined to be 4 × 4.

3. Experiment

In this section, the optimal lenslet number for TOMBO systems is verified by experimental comparison of angular resolutions among a 4 × 4 lenslet TOMBO system, a 5 × 5 lenslet TOMBO system, and a conventional single-lens system using the same monochrome image sensor. The sensor size is 10.9 mm × 10.9 mm with 1024 × 1024 pixels. For the 4 × 4 lenslet TOMBO system, lenses are aligned with a pitch of 2.6 mm. The focal length fl and the diameter of each lens are 20 mm and 2.6 mm, respectively. For the 5 × 5 lenslet TOMBO system, lenses are aligned with a pitch of 2 mm. The focal length fl and the diameter of each lens are 16 mm and 2 mm, respectively. For the single-lens system, we utilize Nikon Nikkor lens whose focal lens fs is 85 mm and f number ranges from 1.8 to 16.

The angular resolutions of the three systems with approximately the same f number ( = 8) are measured by a collimator. In the collimator, an illuminated USAF 1951 resolution target is positioned at the front focal plane of the objective lens as shown in Fig. 12. With this configuration, all light beams passing a point in the resolution target plane form a collimated light bundle behind the objective lens. The focal length fc and clear aperture of the collimator are 1000 mm and 100 mm, respectively. Figure 13 presents a photo of the experimental setup for a TOMBO system.

 

Fig. 12 The optical arrangements of experimental setups for (a) the TOMBO system and (b) the single-lens system.

Download Full Size | PPT Slide | PDF

 

Fig. 13 Photo of the experimental setup for a TOMBO system.

Download Full Size | PPT Slide | PDF

The data captured by the 4 × 4 and 5 × 5 lenslet TOMBO systems are shown in Fig. 14. Figure 15 shows the restored images and the image captured by the single-lens system. The restored images are acquired by SIFT-RANSAC and l1-SAR with magnification factors of 4 for Fig. 15(a) and 5 for Fig. 15(b). In contrast to that in Fig. 15(a), the images shown in Fig. 15(b) suffer deterioration of restoration performance as demonstrated in the simulation. Compared with Fig. 15(c), the contrast between white bar and black section is lower in Fig. 15(a), but according to Eq. (3) the angular resolutions of these two images are the same.

α=2b×103fc×206,265,
where b is the minimal interval of white bars that can be distinguished, fc is the focal length of the collimator, and the value 206,265 is the arcseconds for one radian. As shown in Figs. 15(a) and 15(c), b in yellow squares corresponds to the resolution target’s group 1 element 3, which indicates 2.52 line pairs/mm. So the angular resolution for the 4 × 4 lenslet TOMBO system and the single-lens system is 81.85″, whereas the 5 × 5 lenslet TOMBO system has a poorer angular resolution performance. These results confirm that 4 × 4 is the optimal lenslet number for TOMBO systems, as the simulations showed.

 

Fig. 14 The data captured by (a) the 4 × 4 lenslet TOMBO system and (b) the 5 × 5 lenslet TOMBO system.

Download Full Size | PPT Slide | PDF

 

Fig. 15 The restored images in the (a) 4 × 4 and (b) 5 × 5 lenslet TOMBO systems and (c) the image captured by the single-lens system.

Download Full Size | PPT Slide | PDF

4. Conclusion and future work

The subimage capturing processes of the TOMBO imaging system with different lenslet numbers have been simulated, and restoration algorithms have been applied to restore high-resolution images. MAE and PSNR of the restored images have been calculated for quantitative image quality assessment to optimize the lenslet number. The optimal lenslet number has been determined to be 4 × 4 by the simulations and confirmed by imaging experiments with 4 × 4 and 5 × 5 TOMBO imaging systems and a single-lens imaging system.

It is worth mentioning that the optimal lenslet number discussed in this paper is based on the applications in which imaging resolution and system thinness are of primary importance. However, for cases in which other performance parameters, such as sensitivity, are of primary concern, related analyses could still be explored by employing other performance metric functions (such as the capacity metric proposed in [20].) with parameters such as sensitivity, object scene field of view, and resolution.

Future studies for an observation model of the TOMBO imaging system and more accurate registration between subimages should be performed. An ideal imaging process was assumed in the observation models introduced in Section 2. The consideration of the optical aberration in a real imaging system could improve the model. Furthermore, larger optimal lenslet numbers could be expected for thinner system dimensions with more accurate registration.

Acknowledgments

This work was supported by the Preeminent Youth Fund of Sichuan Province under grant 2012JQ0012 and the National Natural Science Foundation of China under grant 11173008.

References and links

1. J. Tanida, T. Kumagai, K. Yamada, S. Miyatake, K. Ishida, T. Morimoto, N. Kondou, D. Miyazaki, and Y. Ichioka, “Thin observation module by bound optics (TOMBO): concept and experimental verification,” Appl. Opt. 40(11), 1806–1813 (2001). [CrossRef]   [PubMed]  

2. J. W. Duparré and F. C. Wippermann, “Micro-optical artificial compound eyes,” Bioinspir. Biomim. 1(1), R1–R16 (2006). [CrossRef]   [PubMed]  

3. D. Mendlovic, “Toward a super imaging system,” Appl. Opt. 52(4), 561–566 (2013). [CrossRef]   [PubMed]  

4. K. Choi and T. J. Schulz, “Signal-processing approaches for image-resolution restoration for TOMBO imagery,” Appl. Opt. 47(10), B104–B116 (2008). [CrossRef]   [PubMed]  

5. M. Shankar, R. Willett, N. Pitsianis, T. Schulz, R. Gibbons, R. Te Kolste, J. Carriere, C. Chen, D. Prather, and D. Brady, “Thin infrared imaging systems through multichannel sampling,” Appl. Opt. 47(10), B1–B10 (2008). [CrossRef]   [PubMed]  

6. A. V. Kanaev, D. A. Scribner, J. R. Ackerman, and E. F. Fleet, “Analysis and application of multiframe superresolution processing for conventional imaging systems and lenslet arrays,” Appl. Opt. 46(20), 4320–4328 (2007). [CrossRef]   [PubMed]  

7. Y. Kitamura, R. Shogenji, K. Yamada, S. Miyatake, M. Miyamoto, T. Morimoto, Y. Masaki, N. Kondou, D. Miyazaki, J. Tanida, and Y. Ichioka, “Reconstruction of a high-resolution image on a compound-eye image-capturing system,” Appl. Opt. 43(8), 1719–1727 (2004). [CrossRef]   [PubMed]  

8. A. Stern and B. Javidi, “Three-dimensional image sensing and reconstruction with time-division multiplexed computational integral imaging,” Appl. Opt. 42(35), 7036–7042 (2003). [CrossRef]   [PubMed]  

9. R. Horisaki, S. Irie, Y. Ogura, and J. Tanida, “Three-dimensional information acqusition using a compound imaging system,” Opt. Rev. 14(5), 347–350 (2007). [CrossRef]  

10. A. V. Kanaev, J. R. Ackerman, E. F. Fleet, and D. A. Scribner, “TOMBO sensor with scene-independent superresolution processing,” Opt. Lett. 32(19), 2855–2857 (2007). [CrossRef]   [PubMed]  

11. A. A. El-Sallam and F. Boussaid, “Spectral-based blind image restoration method for thin TOMBO imagers,” Sensors (Basel Switzerland) 8(9), 6108–6124 (2008). [CrossRef]  

12. S. Mendelowitz, I. Klapp, and D. Mendlovic, “Design of an image restoration algorithm for the TOMBO imaging system,” J. Opt. Soc. Am. A 30(6), 1193–1204 (2013). [CrossRef]   [PubMed]  

13. Z. Lin and H. Y. Shum, “Fundamental limits of reconstruction-based superresolution algorithms under local translation,” IEEE Trans. Pattern Anal. Mach. Intell. 26(1), 83–97 (2004). [CrossRef]   [PubMed]  

14. S. Baker and T. Kanade, “Limits on super-resolution and how to break them,” IEEE Trans. Pattern Anal. Mach. Intell. 24(9), 1167–1183 (2002). [CrossRef]  

15. D. Robinson and P. Milanfar, “Statistical performance analysis of super-resolution,” IEEE Trans. Image Process. 15(6), 1413–1428 (2006). [CrossRef]   [PubMed]  

16. S. Farsiu, M. D. Robinson, M. Elad, and P. Milanfar, “Fast and robust multiframe super resolution,” IEEE Trans. Image Process. 13(10), 1327–1344 (2004). [CrossRef]   [PubMed]  

17. S. Villena, M. Vega, S. D. Babaccan, R. Molina, and A. K. Katsaggelos, “Bayesian combination of sparse and non-sparse priors in image super resolution,” Digit. Signal Process. 23(2), 530–541 (2013). [CrossRef]  

18. D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60(2), 91–110 (2004). [CrossRef]  

19. M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24(6), 381–395 (1981). [CrossRef]  

20. M. W. Haney, “Performance scaling in flat imagers,” Appl. Opt. 45(13), 2901–2910 (2006). [CrossRef]   [PubMed]  

References

  • View by:
  • |
  • |
  • |

  1. J. Tanida, T. Kumagai, K. Yamada, S. Miyatake, K. Ishida, T. Morimoto, N. Kondou, D. Miyazaki, Y. Ichioka, “Thin observation module by bound optics (TOMBO): concept and experimental verification,” Appl. Opt. 40(11), 1806–1813 (2001).
    [CrossRef] [PubMed]
  2. J. W. Duparré, F. C. Wippermann, “Micro-optical artificial compound eyes,” Bioinspir. Biomim. 1(1), R1–R16 (2006).
    [CrossRef] [PubMed]
  3. D. Mendlovic, “Toward a super imaging system,” Appl. Opt. 52(4), 561–566 (2013).
    [CrossRef] [PubMed]
  4. K. Choi, T. J. Schulz, “Signal-processing approaches for image-resolution restoration for TOMBO imagery,” Appl. Opt. 47(10), B104–B116 (2008).
    [CrossRef] [PubMed]
  5. M. Shankar, R. Willett, N. Pitsianis, T. Schulz, R. Gibbons, R. Te Kolste, J. Carriere, C. Chen, D. Prather, D. Brady, “Thin infrared imaging systems through multichannel sampling,” Appl. Opt. 47(10), B1–B10 (2008).
    [CrossRef] [PubMed]
  6. A. V. Kanaev, D. A. Scribner, J. R. Ackerman, E. F. Fleet, “Analysis and application of multiframe superresolution processing for conventional imaging systems and lenslet arrays,” Appl. Opt. 46(20), 4320–4328 (2007).
    [CrossRef] [PubMed]
  7. Y. Kitamura, R. Shogenji, K. Yamada, S. Miyatake, M. Miyamoto, T. Morimoto, Y. Masaki, N. Kondou, D. Miyazaki, J. Tanida, Y. Ichioka, “Reconstruction of a high-resolution image on a compound-eye image-capturing system,” Appl. Opt. 43(8), 1719–1727 (2004).
    [CrossRef] [PubMed]
  8. A. Stern, B. Javidi, “Three-dimensional image sensing and reconstruction with time-division multiplexed computational integral imaging,” Appl. Opt. 42(35), 7036–7042 (2003).
    [CrossRef] [PubMed]
  9. R. Horisaki, S. Irie, Y. Ogura, J. Tanida, “Three-dimensional information acqusition using a compound imaging system,” Opt. Rev. 14(5), 347–350 (2007).
    [CrossRef]
  10. A. V. Kanaev, J. R. Ackerman, E. F. Fleet, D. A. Scribner, “TOMBO sensor with scene-independent superresolution processing,” Opt. Lett. 32(19), 2855–2857 (2007).
    [CrossRef] [PubMed]
  11. A. A. El-Sallam, F. Boussaid, “Spectral-based blind image restoration method for thin TOMBO imagers,” Sensors (Basel Switzerland) 8(9), 6108–6124 (2008).
    [CrossRef]
  12. S. Mendelowitz, I. Klapp, D. Mendlovic, “Design of an image restoration algorithm for the TOMBO imaging system,” J. Opt. Soc. Am. A 30(6), 1193–1204 (2013).
    [CrossRef] [PubMed]
  13. Z. Lin, H. Y. Shum, “Fundamental limits of reconstruction-based superresolution algorithms under local translation,” IEEE Trans. Pattern Anal. Mach. Intell. 26(1), 83–97 (2004).
    [CrossRef] [PubMed]
  14. S. Baker, T. Kanade, “Limits on super-resolution and how to break them,” IEEE Trans. Pattern Anal. Mach. Intell. 24(9), 1167–1183 (2002).
    [CrossRef]
  15. D. Robinson, P. Milanfar, “Statistical performance analysis of super-resolution,” IEEE Trans. Image Process. 15(6), 1413–1428 (2006).
    [CrossRef] [PubMed]
  16. S. Farsiu, M. D. Robinson, M. Elad, P. Milanfar, “Fast and robust multiframe super resolution,” IEEE Trans. Image Process. 13(10), 1327–1344 (2004).
    [CrossRef] [PubMed]
  17. S. Villena, M. Vega, S. D. Babaccan, R. Molina, A. K. Katsaggelos, “Bayesian combination of sparse and non-sparse priors in image super resolution,” Digit. Signal Process. 23(2), 530–541 (2013).
    [CrossRef]
  18. D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60(2), 91–110 (2004).
    [CrossRef]
  19. M. A. Fischler, R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24(6), 381–395 (1981).
    [CrossRef]
  20. M. W. Haney, “Performance scaling in flat imagers,” Appl. Opt. 45(13), 2901–2910 (2006).
    [CrossRef] [PubMed]

2013 (3)

S. Villena, M. Vega, S. D. Babaccan, R. Molina, A. K. Katsaggelos, “Bayesian combination of sparse and non-sparse priors in image super resolution,” Digit. Signal Process. 23(2), 530–541 (2013).
[CrossRef]

D. Mendlovic, “Toward a super imaging system,” Appl. Opt. 52(4), 561–566 (2013).
[CrossRef] [PubMed]

S. Mendelowitz, I. Klapp, D. Mendlovic, “Design of an image restoration algorithm for the TOMBO imaging system,” J. Opt. Soc. Am. A 30(6), 1193–1204 (2013).
[CrossRef] [PubMed]

2008 (3)

2007 (3)

2006 (3)

M. W. Haney, “Performance scaling in flat imagers,” Appl. Opt. 45(13), 2901–2910 (2006).
[CrossRef] [PubMed]

J. W. Duparré, F. C. Wippermann, “Micro-optical artificial compound eyes,” Bioinspir. Biomim. 1(1), R1–R16 (2006).
[CrossRef] [PubMed]

D. Robinson, P. Milanfar, “Statistical performance analysis of super-resolution,” IEEE Trans. Image Process. 15(6), 1413–1428 (2006).
[CrossRef] [PubMed]

2004 (4)

S. Farsiu, M. D. Robinson, M. Elad, P. Milanfar, “Fast and robust multiframe super resolution,” IEEE Trans. Image Process. 13(10), 1327–1344 (2004).
[CrossRef] [PubMed]

Z. Lin, H. Y. Shum, “Fundamental limits of reconstruction-based superresolution algorithms under local translation,” IEEE Trans. Pattern Anal. Mach. Intell. 26(1), 83–97 (2004).
[CrossRef] [PubMed]

D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60(2), 91–110 (2004).
[CrossRef]

Y. Kitamura, R. Shogenji, K. Yamada, S. Miyatake, M. Miyamoto, T. Morimoto, Y. Masaki, N. Kondou, D. Miyazaki, J. Tanida, Y. Ichioka, “Reconstruction of a high-resolution image on a compound-eye image-capturing system,” Appl. Opt. 43(8), 1719–1727 (2004).
[CrossRef] [PubMed]

2003 (1)

2002 (1)

S. Baker, T. Kanade, “Limits on super-resolution and how to break them,” IEEE Trans. Pattern Anal. Mach. Intell. 24(9), 1167–1183 (2002).
[CrossRef]

2001 (1)

1981 (1)

M. A. Fischler, R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24(6), 381–395 (1981).
[CrossRef]

Ackerman, J. R.

Babaccan, S. D.

S. Villena, M. Vega, S. D. Babaccan, R. Molina, A. K. Katsaggelos, “Bayesian combination of sparse and non-sparse priors in image super resolution,” Digit. Signal Process. 23(2), 530–541 (2013).
[CrossRef]

Baker, S.

S. Baker, T. Kanade, “Limits on super-resolution and how to break them,” IEEE Trans. Pattern Anal. Mach. Intell. 24(9), 1167–1183 (2002).
[CrossRef]

Bolles, R. C.

M. A. Fischler, R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24(6), 381–395 (1981).
[CrossRef]

Boussaid, F.

A. A. El-Sallam, F. Boussaid, “Spectral-based blind image restoration method for thin TOMBO imagers,” Sensors (Basel Switzerland) 8(9), 6108–6124 (2008).
[CrossRef]

Brady, D.

Carriere, J.

Chen, C.

Choi, K.

Duparré, J. W.

J. W. Duparré, F. C. Wippermann, “Micro-optical artificial compound eyes,” Bioinspir. Biomim. 1(1), R1–R16 (2006).
[CrossRef] [PubMed]

Elad, M.

S. Farsiu, M. D. Robinson, M. Elad, P. Milanfar, “Fast and robust multiframe super resolution,” IEEE Trans. Image Process. 13(10), 1327–1344 (2004).
[CrossRef] [PubMed]

El-Sallam, A. A.

A. A. El-Sallam, F. Boussaid, “Spectral-based blind image restoration method for thin TOMBO imagers,” Sensors (Basel Switzerland) 8(9), 6108–6124 (2008).
[CrossRef]

Farsiu, S.

S. Farsiu, M. D. Robinson, M. Elad, P. Milanfar, “Fast and robust multiframe super resolution,” IEEE Trans. Image Process. 13(10), 1327–1344 (2004).
[CrossRef] [PubMed]

Fischler, M. A.

M. A. Fischler, R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24(6), 381–395 (1981).
[CrossRef]

Fleet, E. F.

Gibbons, R.

Haney, M. W.

Horisaki, R.

R. Horisaki, S. Irie, Y. Ogura, J. Tanida, “Three-dimensional information acqusition using a compound imaging system,” Opt. Rev. 14(5), 347–350 (2007).
[CrossRef]

Ichioka, Y.

Irie, S.

R. Horisaki, S. Irie, Y. Ogura, J. Tanida, “Three-dimensional information acqusition using a compound imaging system,” Opt. Rev. 14(5), 347–350 (2007).
[CrossRef]

Ishida, K.

Javidi, B.

Kanade, T.

S. Baker, T. Kanade, “Limits on super-resolution and how to break them,” IEEE Trans. Pattern Anal. Mach. Intell. 24(9), 1167–1183 (2002).
[CrossRef]

Kanaev, A. V.

Katsaggelos, A. K.

S. Villena, M. Vega, S. D. Babaccan, R. Molina, A. K. Katsaggelos, “Bayesian combination of sparse and non-sparse priors in image super resolution,” Digit. Signal Process. 23(2), 530–541 (2013).
[CrossRef]

Kitamura, Y.

Klapp, I.

Kondou, N.

Kumagai, T.

Lin, Z.

Z. Lin, H. Y. Shum, “Fundamental limits of reconstruction-based superresolution algorithms under local translation,” IEEE Trans. Pattern Anal. Mach. Intell. 26(1), 83–97 (2004).
[CrossRef] [PubMed]

Lowe, D. G.

D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60(2), 91–110 (2004).
[CrossRef]

Masaki, Y.

Mendelowitz, S.

Mendlovic, D.

Milanfar, P.

D. Robinson, P. Milanfar, “Statistical performance analysis of super-resolution,” IEEE Trans. Image Process. 15(6), 1413–1428 (2006).
[CrossRef] [PubMed]

S. Farsiu, M. D. Robinson, M. Elad, P. Milanfar, “Fast and robust multiframe super resolution,” IEEE Trans. Image Process. 13(10), 1327–1344 (2004).
[CrossRef] [PubMed]

Miyamoto, M.

Miyatake, S.

Miyazaki, D.

Molina, R.

S. Villena, M. Vega, S. D. Babaccan, R. Molina, A. K. Katsaggelos, “Bayesian combination of sparse and non-sparse priors in image super resolution,” Digit. Signal Process. 23(2), 530–541 (2013).
[CrossRef]

Morimoto, T.

Ogura, Y.

R. Horisaki, S. Irie, Y. Ogura, J. Tanida, “Three-dimensional information acqusition using a compound imaging system,” Opt. Rev. 14(5), 347–350 (2007).
[CrossRef]

Pitsianis, N.

Prather, D.

Robinson, D.

D. Robinson, P. Milanfar, “Statistical performance analysis of super-resolution,” IEEE Trans. Image Process. 15(6), 1413–1428 (2006).
[CrossRef] [PubMed]

Robinson, M. D.

S. Farsiu, M. D. Robinson, M. Elad, P. Milanfar, “Fast and robust multiframe super resolution,” IEEE Trans. Image Process. 13(10), 1327–1344 (2004).
[CrossRef] [PubMed]

Schulz, T.

Schulz, T. J.

Scribner, D. A.

Shankar, M.

Shogenji, R.

Shum, H. Y.

Z. Lin, H. Y. Shum, “Fundamental limits of reconstruction-based superresolution algorithms under local translation,” IEEE Trans. Pattern Anal. Mach. Intell. 26(1), 83–97 (2004).
[CrossRef] [PubMed]

Stern, A.

Tanida, J.

Te Kolste, R.

Vega, M.

S. Villena, M. Vega, S. D. Babaccan, R. Molina, A. K. Katsaggelos, “Bayesian combination of sparse and non-sparse priors in image super resolution,” Digit. Signal Process. 23(2), 530–541 (2013).
[CrossRef]

Villena, S.

S. Villena, M. Vega, S. D. Babaccan, R. Molina, A. K. Katsaggelos, “Bayesian combination of sparse and non-sparse priors in image super resolution,” Digit. Signal Process. 23(2), 530–541 (2013).
[CrossRef]

Willett, R.

Wippermann, F. C.

J. W. Duparré, F. C. Wippermann, “Micro-optical artificial compound eyes,” Bioinspir. Biomim. 1(1), R1–R16 (2006).
[CrossRef] [PubMed]

Yamada, K.

Appl. Opt. (8)

D. Mendlovic, “Toward a super imaging system,” Appl. Opt. 52(4), 561–566 (2013).
[CrossRef] [PubMed]

K. Choi, T. J. Schulz, “Signal-processing approaches for image-resolution restoration for TOMBO imagery,” Appl. Opt. 47(10), B104–B116 (2008).
[CrossRef] [PubMed]

M. Shankar, R. Willett, N. Pitsianis, T. Schulz, R. Gibbons, R. Te Kolste, J. Carriere, C. Chen, D. Prather, D. Brady, “Thin infrared imaging systems through multichannel sampling,” Appl. Opt. 47(10), B1–B10 (2008).
[CrossRef] [PubMed]

A. V. Kanaev, D. A. Scribner, J. R. Ackerman, E. F. Fleet, “Analysis and application of multiframe superresolution processing for conventional imaging systems and lenslet arrays,” Appl. Opt. 46(20), 4320–4328 (2007).
[CrossRef] [PubMed]

Y. Kitamura, R. Shogenji, K. Yamada, S. Miyatake, M. Miyamoto, T. Morimoto, Y. Masaki, N. Kondou, D. Miyazaki, J. Tanida, Y. Ichioka, “Reconstruction of a high-resolution image on a compound-eye image-capturing system,” Appl. Opt. 43(8), 1719–1727 (2004).
[CrossRef] [PubMed]

A. Stern, B. Javidi, “Three-dimensional image sensing and reconstruction with time-division multiplexed computational integral imaging,” Appl. Opt. 42(35), 7036–7042 (2003).
[CrossRef] [PubMed]

J. Tanida, T. Kumagai, K. Yamada, S. Miyatake, K. Ishida, T. Morimoto, N. Kondou, D. Miyazaki, Y. Ichioka, “Thin observation module by bound optics (TOMBO): concept and experimental verification,” Appl. Opt. 40(11), 1806–1813 (2001).
[CrossRef] [PubMed]

M. W. Haney, “Performance scaling in flat imagers,” Appl. Opt. 45(13), 2901–2910 (2006).
[CrossRef] [PubMed]

Bioinspir. Biomim. (1)

J. W. Duparré, F. C. Wippermann, “Micro-optical artificial compound eyes,” Bioinspir. Biomim. 1(1), R1–R16 (2006).
[CrossRef] [PubMed]

Commun. ACM (1)

M. A. Fischler, R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24(6), 381–395 (1981).
[CrossRef]

Digit. Signal Process. (1)

S. Villena, M. Vega, S. D. Babaccan, R. Molina, A. K. Katsaggelos, “Bayesian combination of sparse and non-sparse priors in image super resolution,” Digit. Signal Process. 23(2), 530–541 (2013).
[CrossRef]

IEEE Trans. Image Process. (2)

D. Robinson, P. Milanfar, “Statistical performance analysis of super-resolution,” IEEE Trans. Image Process. 15(6), 1413–1428 (2006).
[CrossRef] [PubMed]

S. Farsiu, M. D. Robinson, M. Elad, P. Milanfar, “Fast and robust multiframe super resolution,” IEEE Trans. Image Process. 13(10), 1327–1344 (2004).
[CrossRef] [PubMed]

IEEE Trans. Pattern Anal. Mach. Intell. (2)

Z. Lin, H. Y. Shum, “Fundamental limits of reconstruction-based superresolution algorithms under local translation,” IEEE Trans. Pattern Anal. Mach. Intell. 26(1), 83–97 (2004).
[CrossRef] [PubMed]

S. Baker, T. Kanade, “Limits on super-resolution and how to break them,” IEEE Trans. Pattern Anal. Mach. Intell. 24(9), 1167–1183 (2002).
[CrossRef]

Int. J. Comput. Vis. (1)

D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60(2), 91–110 (2004).
[CrossRef]

J. Opt. Soc. Am. A (1)

Opt. Lett. (1)

Opt. Rev. (1)

R. Horisaki, S. Irie, Y. Ogura, J. Tanida, “Three-dimensional information acqusition using a compound imaging system,” Opt. Rev. 14(5), 347–350 (2007).
[CrossRef]

Sensors (Basel Switzerland) (1)

A. A. El-Sallam, F. Boussaid, “Spectral-based blind image restoration method for thin TOMBO imagers,” Sensors (Basel Switzerland) 8(9), 6108–6124 (2008).
[CrossRef]

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (15)

Fig. 1
Fig. 1

The optical arrangement of the TOMBO imaging system.

Fig. 2
Fig. 2

The 1/3 translation between two low-resolution pixels of subimages when downsampling factor d is 3. The dotted squares represent high-resolution pixels from one input image, whereas the red square represents a low-resolution pixel from the top left corner of subimages L2,2, and the green square represents a low-resolution pixel from the top left corner of subimages L3,3.

Fig. 3
Fig. 3

(a) Input image. (b) Simulated image captured by the single-lens system. Simulated subimages with lenslet numbers (c) 2 × 2, (d) 3 × 3, (e) 4 × 4, (f) 5 × 5, (g) 6 × 6, (h) 7 × 7, (i) 8 × 8, (j) 9 × 9, and (k) 10 × 10.

Fig. 4
Fig. 4

(a) Input image. (b) Simulated image captured by the single-lens system. Restored images with actual translation parameters and lenslet numbers (c) 2 × 2, (d) 3 × 3, (e) 4 × 4, (f) 5 × 5, (g) 6 × 6, (h) 7 × 7, (i) 8 × 8, (j) 9 × 9, and (k) 10 × 10.

Fig. 5
Fig. 5

(a) Input image. (b) Simulated image captured by the single-lens system. Restored images with estimated translation parameters and lenslet numbers (c) 2 × 2, (d) 3 × 3, (e) 4 × 4, (f) 5 × 5, (g) 6 × 6, (h) 7 × 7, (i) 8 × 8, (j) 9 × 9, and (k) 10 × 10.

Fig. 6
Fig. 6

Averaged MSE of restored images with the actual translation parameters versus lenslet number.

Fig. 7
Fig. 7

Averaged PSNR of restored images with the actual translation parameters versus lenslet number.

Fig. 8
Fig. 8

Averaged MSE of restored images with the estimated translation parameters by SIFT-RANSAC versus lenslet number.

Fig. 9
Fig. 9

Averaged PSNR of restored images with the estimated translation parameters by SIFT-RANSAC versus lenslet number.

Fig. 10
Fig. 10

The averaged horizontal registration errors versus lenslet number.

Fig. 11
Fig. 11

The averaged vertical registration errors versus lenslet number.

Fig. 12
Fig. 12

The optical arrangements of experimental setups for (a) the TOMBO system and (b) the single-lens system.

Fig. 13
Fig. 13

Photo of the experimental setup for a TOMBO system.

Fig. 14
Fig. 14

The data captured by (a) the 4 × 4 lenslet TOMBO system and (b) the 5 × 5 lenslet TOMBO system.

Fig. 15
Fig. 15

The restored images in the (a) 4 × 4 and (b) 5 × 5 lenslet TOMBO systems and (c) the image captured by the single-lens system.

Equations (3)

Equations on this page are rendered with MathJax. Learn more.

L i,j =[ b i,j t i,j ( r i,j )H]D+ v i,j ,
PSNR=20 log 10 MAX MSE ,
α= 2b× 10 3 f c ×206,265,

Metrics