Abstract

One way to extend depth of field of any optical system is to take several images with different focus positions and combine them into a single composite image, which contains all regions fully focused. The challenge then becomes to select from each image the pixels that are in focus. We describe a new focus measure based on the shapelet decomposition. Results using simulated images with high noise content show that the shapelet-based focus measure is the most performance that traditional neighborhood-based focus approaches. Similarly shapelet focus measure provides significant improvement compared to traditional methods when objects have nontextured or homogeneous regions.

© 2008 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |

  1. S. Nayar, M. Watanabe, and M. Noguchi, "Real-time focus range sensor," in Proceedings of the Fifth International Conference on Computer Vision (IEEE Computer Society, 1995), pp. 995-1002.
  2. I. Blayvas, R. Kimmel, and E. Rivlin, "Role of optics in the accuracy of depth-from-defocus systems," J. Opt. Soc. Am. A 24, 967-972 (2007).
    [CrossRef]
  3. M. Noguchi and S. Nayar, "Microscopic shape from focus using active illumination," in Proceedings of International Conference on Pattern Recognition (IEEE Computer Society, 1994), pp. 147-152.
    [CrossRef]
  4. J. Goodman, Introduction to Fourier Optics (McGraw-Hill, 1996).
  5. V. Aslantas and D. T. Pham, "Depth of automatic defocusing," Opt. Express 15, 1011-1023 (2007).
    [CrossRef] [PubMed]
  6. M. Subbarao, "Efficient depth recovery through inverse optics," in Machine Vision for Inspection and Measurement, H. Freeman, ed. (Academic, 1989), pp. 120-146.
  7. O. Ghita and P. Whelan, "Real-time 3D estimation using depth from defocus," Vision, MVA (SME) 16(3), 1-6 (2000).
  8. M. Subbarao, "The optical transfer function of a diffraction limited system for polychromatic illumination," Appl. Opt. 29, 554-558 (1990).
    [CrossRef] [PubMed]
  9. E. Krotkov, "Focusing (video camera automatic)," Int. J. Comput. Vis. 1, 223-237 (1987).
    [CrossRef]
  10. E. Ben-Eliezer, Z. Zalevsky, E. Marom, and N. Konforti, "All-optical extended depth of field imaging system," J. Opt. A, Pure Appl. Opt. 5, 164-169 (2003).
    [CrossRef]
  11. Q. Yang, L. Liu, and J. Sun, "Optimized phase pupil masks for extended depth of field," Opt. Comm. 272, 56-66 (2007).
    [CrossRef]
  12. M. Cohen, G. Cauwenberghs, and M. A. Vorontsov, "Image sharpness and beam focus VLSI sensors for adaptive optics," IEEE Sens. J. 2, 680-690 (2002).
    [CrossRef]
  13. S. C. Tucker, W. T. Cathey, and E. R. Dowski, Jr., "Extended depth of field and aberration control for inexpensive digital microscope systems," Opt. Express 4, 467-474 (1999).
    [CrossRef] [PubMed]
  14. J. van der Gracht, V. P. Pauca, H. Setty, R. Narayanswamy, R. J. Plemmons, S. Prasad, and T. Torgersen, "Iris recognition with enhanced depth-of-field image acquisition," Proc. SPIE 5438, 120-129 (2004).
    [CrossRef]
  15. Q. Yang, L. Liu, J. Sun, Y. Zhu, and W. Lu, "Analysis of optical systems with extended depth of field using the Wigner distribution function," Appl. Opt. 45, 8586-8595 (2006).
    [CrossRef] [PubMed]
  16. P. Potuluri, M. R. Fetterman, and D. J. Brady, "High depth of field microscopic imaging using an interferometric camera," Opt. Express 8, 624-630 (2001).
    [CrossRef] [PubMed]
  17. A. Castro and J. Ojeda-Castañeda, "Asymmetric phase masks for extended depth of field," Appl. Opt. 43, 3474-3479 (2004).
    [CrossRef] [PubMed]
  18. Q. Li, S. Liao, H. Wei, and M. Shen, "Restoration of solar and star images with phase diversity-based blind deconvolution," Chin. Opt. Lett. 5, 201-203 (2007).
  19. M. Van Noort, L. R. Van Der Voort, and M. G. Lofdahl, "Solar image restoration by use of multi-frame blind deconvolution with multiple objects and phase diversity," Sol. Phys. 228, 191-215 (2005).
    [CrossRef]
  20. A. G. Valdecasas, D. Marshall, J. M. Becerra, and J. J. Terrero, "On the extended depth of focus algorithms for bright field microscopy," Micron 32, 559-569 (2001).
    [CrossRef] [PubMed]
  21. B. Forster, D. van de Ville, J. Berent, D. Sage, and M. Unser, "Complex wavelet for extended depth-of-field: a new method for the fusion of multichannel microscopy images," Microsc. Res. and Tech. 65, 33-42 (2004).
    [CrossRef]
  22. L. Ghouti, A. Bouridane, and M. Ibrahim, "Improved image fusion using balanced multiwavelets," in Proceedings of 12th European Signal Processing Conference (EUSIPCO, 2004), pp. 57-60.
  23. I. De and B. Chanda, "A simple and efficient algorithm for multifocus image fusion using morphological wavelets," Signal Process. 86, 924-936 (2006).
    [CrossRef]
  24. R. J. Pieper and A. Korpel, "Image processing for extended depth of field," Appl. Opt. 22, 1449-1453 (1983).
    [CrossRef] [PubMed]
  25. S. A. Sugimoto and Y. Ichioka, "Digital composition of images with increased depth of focus considering depth information," Appl. Opt. 24, 2076-2080 (1985).
    [CrossRef] [PubMed]
  26. J. L. Pech-Pacheco, G. Cristobal, J. Chamorro-Martinez, and J. Fernandez-Valdivia, "Diatom autofocusing in brightfield microscopy: a comparative study," in Proceedings of 15th International Conference on Pattern Recognition (ICPR, 2000), pp. 314-317.
    [CrossRef]
  27. N. Goldsmith, "Deep focus: a digital image processing technique to produce improved focal depth in light microscopy," Image Anal. Stereol. 19, 163-167 (2000).
    [CrossRef]
  28. H. Eltoukhy and S. Kavusi, "A computationally efficient algorithm for multi-focus image reconstruction," Proc. SPIE 5017, 332-341 (2003).
    [CrossRef]
  29. H. Xie, W. Rong, and L. Sun, "A flexible experimental system for complex microassembly under microscale force and vision-based control," Int. J. Optomech. 1, 81-102 (2007).
    [CrossRef]
  30. H. J. Heijmans and J. Goutsias, "Multiresolution signal decomposition schemes, part 2: morphological wavelets," IEEE Trans. Image Process. 9, 1897-1913 (2000).
    [CrossRef]
  31. H. Li, B. S. Manjunath, and S. K. Mitra, "Multisensor image fusion using the wavelet transform," Graph. Models Image Process. 57, 235-245 (1995).
    [CrossRef]
  32. A. Bradley and P. Bamford, "A one-pass extended depth of field algorithm based on the over-complete discrete wavelet transform," in Proceedings of Image and Vision Computing (IVCNZ, 2004), pp. 279-284.
  33. P. Hill, N. Canagarajah, and D. Bull, "Image fusion using complex wavelets," in Proceeding of the 13th British Machine Vision Conference (Cardiff, 2002), pp. 487-496.
  34. M. González-Audícana, X. Otazu, O. Fors, and A. Seco, "Comparison between Mallat's and the 'à trous' discrete wavelet transform based algorithms for the fusion of multispectral and panchromatic images," Int. J. Remote Sens. 26, 595-614 (2005).
    [CrossRef]
  35. A. Refregier, "Shapelets--I. A method for image analysis," Mon. Not. R. Astron. Soc. 338, 35-47 (2003).
    [CrossRef]
  36. P. Kovesi, "Shapelets correlated with surface normals produce surfaces," in Proceedings of Australia-Japan Advanced Workshop on Computer Vision (Adelaide, 2003), pp. 101-108.
  37. Z. Wang and A. C. Bovik, "A universal image quality index," IEEE Signal Process Lett. 9, 81-84 (2002).
    [CrossRef]

2007 (5)

2006 (2)

Q. Yang, L. Liu, J. Sun, Y. Zhu, and W. Lu, "Analysis of optical systems with extended depth of field using the Wigner distribution function," Appl. Opt. 45, 8586-8595 (2006).
[CrossRef] [PubMed]

I. De and B. Chanda, "A simple and efficient algorithm for multifocus image fusion using morphological wavelets," Signal Process. 86, 924-936 (2006).
[CrossRef]

2005 (2)

M. Van Noort, L. R. Van Der Voort, and M. G. Lofdahl, "Solar image restoration by use of multi-frame blind deconvolution with multiple objects and phase diversity," Sol. Phys. 228, 191-215 (2005).
[CrossRef]

M. González-Audícana, X. Otazu, O. Fors, and A. Seco, "Comparison between Mallat's and the 'à trous' discrete wavelet transform based algorithms for the fusion of multispectral and panchromatic images," Int. J. Remote Sens. 26, 595-614 (2005).
[CrossRef]

2004 (3)

A. Castro and J. Ojeda-Castañeda, "Asymmetric phase masks for extended depth of field," Appl. Opt. 43, 3474-3479 (2004).
[CrossRef] [PubMed]

B. Forster, D. van de Ville, J. Berent, D. Sage, and M. Unser, "Complex wavelet for extended depth-of-field: a new method for the fusion of multichannel microscopy images," Microsc. Res. and Tech. 65, 33-42 (2004).
[CrossRef]

J. van der Gracht, V. P. Pauca, H. Setty, R. Narayanswamy, R. J. Plemmons, S. Prasad, and T. Torgersen, "Iris recognition with enhanced depth-of-field image acquisition," Proc. SPIE 5438, 120-129 (2004).
[CrossRef]

2003 (3)

E. Ben-Eliezer, Z. Zalevsky, E. Marom, and N. Konforti, "All-optical extended depth of field imaging system," J. Opt. A, Pure Appl. Opt. 5, 164-169 (2003).
[CrossRef]

A. Refregier, "Shapelets--I. A method for image analysis," Mon. Not. R. Astron. Soc. 338, 35-47 (2003).
[CrossRef]

H. Eltoukhy and S. Kavusi, "A computationally efficient algorithm for multi-focus image reconstruction," Proc. SPIE 5017, 332-341 (2003).
[CrossRef]

2002 (2)

Z. Wang and A. C. Bovik, "A universal image quality index," IEEE Signal Process Lett. 9, 81-84 (2002).
[CrossRef]

M. Cohen, G. Cauwenberghs, and M. A. Vorontsov, "Image sharpness and beam focus VLSI sensors for adaptive optics," IEEE Sens. J. 2, 680-690 (2002).
[CrossRef]

2001 (2)

A. G. Valdecasas, D. Marshall, J. M. Becerra, and J. J. Terrero, "On the extended depth of focus algorithms for bright field microscopy," Micron 32, 559-569 (2001).
[CrossRef] [PubMed]

P. Potuluri, M. R. Fetterman, and D. J. Brady, "High depth of field microscopic imaging using an interferometric camera," Opt. Express 8, 624-630 (2001).
[CrossRef] [PubMed]

2000 (3)

O. Ghita and P. Whelan, "Real-time 3D estimation using depth from defocus," Vision, MVA (SME) 16(3), 1-6 (2000).

H. J. Heijmans and J. Goutsias, "Multiresolution signal decomposition schemes, part 2: morphological wavelets," IEEE Trans. Image Process. 9, 1897-1913 (2000).
[CrossRef]

N. Goldsmith, "Deep focus: a digital image processing technique to produce improved focal depth in light microscopy," Image Anal. Stereol. 19, 163-167 (2000).
[CrossRef]

1999 (1)

1995 (1)

H. Li, B. S. Manjunath, and S. K. Mitra, "Multisensor image fusion using the wavelet transform," Graph. Models Image Process. 57, 235-245 (1995).
[CrossRef]

1990 (1)

1987 (1)

E. Krotkov, "Focusing (video camera automatic)," Int. J. Comput. Vis. 1, 223-237 (1987).
[CrossRef]

1985 (1)

1983 (1)

Appl. Opt. (5)

Chin. Opt. Lett. (1)

Graph. Models Image Process. (1)

H. Li, B. S. Manjunath, and S. K. Mitra, "Multisensor image fusion using the wavelet transform," Graph. Models Image Process. 57, 235-245 (1995).
[CrossRef]

IEEE Sens. J. (1)

M. Cohen, G. Cauwenberghs, and M. A. Vorontsov, "Image sharpness and beam focus VLSI sensors for adaptive optics," IEEE Sens. J. 2, 680-690 (2002).
[CrossRef]

IEEE Signal Process Lett. (1)

Z. Wang and A. C. Bovik, "A universal image quality index," IEEE Signal Process Lett. 9, 81-84 (2002).
[CrossRef]

IEEE Trans. Image Process. (1)

H. J. Heijmans and J. Goutsias, "Multiresolution signal decomposition schemes, part 2: morphological wavelets," IEEE Trans. Image Process. 9, 1897-1913 (2000).
[CrossRef]

Image Anal. Stereol. (1)

N. Goldsmith, "Deep focus: a digital image processing technique to produce improved focal depth in light microscopy," Image Anal. Stereol. 19, 163-167 (2000).
[CrossRef]

Int. J. Comput. Vis. (1)

E. Krotkov, "Focusing (video camera automatic)," Int. J. Comput. Vis. 1, 223-237 (1987).
[CrossRef]

Int. J. Optomech. (1)

H. Xie, W. Rong, and L. Sun, "A flexible experimental system for complex microassembly under microscale force and vision-based control," Int. J. Optomech. 1, 81-102 (2007).
[CrossRef]

Int. J. Remote Sens. (1)

M. González-Audícana, X. Otazu, O. Fors, and A. Seco, "Comparison between Mallat's and the 'à trous' discrete wavelet transform based algorithms for the fusion of multispectral and panchromatic images," Int. J. Remote Sens. 26, 595-614 (2005).
[CrossRef]

J. Opt. A, Pure Appl. Opt. (1)

E. Ben-Eliezer, Z. Zalevsky, E. Marom, and N. Konforti, "All-optical extended depth of field imaging system," J. Opt. A, Pure Appl. Opt. 5, 164-169 (2003).
[CrossRef]

J. Opt. Soc. Am. A (1)

Micron (1)

A. G. Valdecasas, D. Marshall, J. M. Becerra, and J. J. Terrero, "On the extended depth of focus algorithms for bright field microscopy," Micron 32, 559-569 (2001).
[CrossRef] [PubMed]

Microsc. Res. and Tech. (1)

B. Forster, D. van de Ville, J. Berent, D. Sage, and M. Unser, "Complex wavelet for extended depth-of-field: a new method for the fusion of multichannel microscopy images," Microsc. Res. and Tech. 65, 33-42 (2004).
[CrossRef]

Mon. Not. R. Astron. Soc. (1)

A. Refregier, "Shapelets--I. A method for image analysis," Mon. Not. R. Astron. Soc. 338, 35-47 (2003).
[CrossRef]

Opt. Comm. (1)

Q. Yang, L. Liu, and J. Sun, "Optimized phase pupil masks for extended depth of field," Opt. Comm. 272, 56-66 (2007).
[CrossRef]

Opt. Express (3)

Proc. SPIE (2)

J. van der Gracht, V. P. Pauca, H. Setty, R. Narayanswamy, R. J. Plemmons, S. Prasad, and T. Torgersen, "Iris recognition with enhanced depth-of-field image acquisition," Proc. SPIE 5438, 120-129 (2004).
[CrossRef]

H. Eltoukhy and S. Kavusi, "A computationally efficient algorithm for multi-focus image reconstruction," Proc. SPIE 5017, 332-341 (2003).
[CrossRef]

Signal Process. (1)

I. De and B. Chanda, "A simple and efficient algorithm for multifocus image fusion using morphological wavelets," Signal Process. 86, 924-936 (2006).
[CrossRef]

Sol. Phys. (1)

M. Van Noort, L. R. Van Der Voort, and M. G. Lofdahl, "Solar image restoration by use of multi-frame blind deconvolution with multiple objects and phase diversity," Sol. Phys. 228, 191-215 (2005).
[CrossRef]

Vision, MVA (SME) (1)

O. Ghita and P. Whelan, "Real-time 3D estimation using depth from defocus," Vision, MVA (SME) 16(3), 1-6 (2000).

Other (9)

S. Nayar, M. Watanabe, and M. Noguchi, "Real-time focus range sensor," in Proceedings of the Fifth International Conference on Computer Vision (IEEE Computer Society, 1995), pp. 995-1002.

M. Subbarao, "Efficient depth recovery through inverse optics," in Machine Vision for Inspection and Measurement, H. Freeman, ed. (Academic, 1989), pp. 120-146.

M. Noguchi and S. Nayar, "Microscopic shape from focus using active illumination," in Proceedings of International Conference on Pattern Recognition (IEEE Computer Society, 1994), pp. 147-152.
[CrossRef]

J. Goodman, Introduction to Fourier Optics (McGraw-Hill, 1996).

L. Ghouti, A. Bouridane, and M. Ibrahim, "Improved image fusion using balanced multiwavelets," in Proceedings of 12th European Signal Processing Conference (EUSIPCO, 2004), pp. 57-60.

J. L. Pech-Pacheco, G. Cristobal, J. Chamorro-Martinez, and J. Fernandez-Valdivia, "Diatom autofocusing in brightfield microscopy: a comparative study," in Proceedings of 15th International Conference on Pattern Recognition (ICPR, 2000), pp. 314-317.
[CrossRef]

P. Kovesi, "Shapelets correlated with surface normals produce surfaces," in Proceedings of Australia-Japan Advanced Workshop on Computer Vision (Adelaide, 2003), pp. 101-108.

A. Bradley and P. Bamford, "A one-pass extended depth of field algorithm based on the over-complete discrete wavelet transform," in Proceedings of Image and Vision Computing (IVCNZ, 2004), pp. 279-284.

P. Hill, N. Canagarajah, and D. Bull, "Image fusion using complex wavelets," in Proceeding of the 13th British Machine Vision Conference (Cardiff, 2002), pp. 487-496.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (12)

Fig. 1
Fig. 1

(a) Shapelet basis functions formed by five Gaussians with height proportional to scale. (b) Functions in square brackets of Eq. (6) for shapelet basis functions shown in (a).

Fig. 2
Fig. 2

Three shapelet coefficients of scales (b) 2, (c) 8, and (d) 32 applied to experimental image shown in (a).

Fig. 3
Fig. 3

Reconstruction of image shown in Fig. 2(a) using shapelet decomposition for scales 2 i , i = 0 , 1 ,   …   ,   6 .

Fig. 4
Fig. 4

Normalized shapelet coefficients versus image sequence index (a) without and (b) with ambiguity of π in the tilt data.

Fig. 5
Fig. 5

(a) Extended depth of field image using shapelet focus measure with scale 8, (b) blurred image using a geometrical defocus blur of diameter d = 36 pixels, and (c) comparison with Tenengrad and variance focus measures.

Fig. 6
Fig. 6

(a) MSE and (b) incorrect focus position percentage for composite images from shapelet, Tenengrad, and variance focus measures using Gaussian white noise.

Fig. 7
Fig. 7

(a) Experimental image slice and (b) extended depth of field image obtained using shapelet focus measure with scale 16.

Fig. 8
Fig. 8

Topological maps obtained using (a) shapelet, (b) Tenengrad, (c) variance, and (d) complex wavelet algorithms.

Fig. 9
Fig. 9

Image quality index proposed by Wand and Bovik for (a) shapelet and Tenengrad composite images, (b) shapelet and variance composite images, and (c) shapelet and complex wavelet composite images.

Fig. 10
Fig. 10

Image slices of an experimental flat surface with homogeneous or nontextured regions (a) in focus and (b) blurred images.

Fig. 11
Fig. 11

Topological maps obtained using (a) shapelet, (b) Tenengrad, (c) variance, and (d) complex wavelet algorithms.

Fig. 12
Fig. 12

EDOF images obtained using (a) variance and (b) complex wavelet approaches.

Tables (1)

Tables Icon

Table 1 MSE Using 1: Shapelet Scale of 64 and Window Size of 35 for Variance and Tenengrad Algorithms and 2: Shapelet Scale of 128 and Window Size of 55 for Variance and Tenengrad Algorithms

Equations (22)

Equations on this page are rendered with MathJax. Learn more.

F p v ( x , y ) = 1 w x w y i w x j w y [ I p ( x + i , y + j ) I ¯ p ( x , y ) ] 2 ,
F p T ( x , y ) = i w x j w y [ S p ( x + i , y + j ) ] 2 ,
S x = [ 1 0 1 2 0 2 1 0 1 ] , S y = [ 1 2 1 0 0 0 1 2 1 ] .
F p L ( x , y , p ) = i w x j w y | L p ( x + i , y + j ) | ,
L = [ 0 1 0 1 4 1 0 1 0 ] .
FT { s g i } S σ f 2 G ,
C i = C σ i C τ i ,
C σ i = tan   σ I tan   σ s i ,
C τ i = cos ( τ I ) cos ( τ s i ) + sin ( τ I ) sin ( τ s i ) ,
d = D δ d i + δ
δ = f ( d o z ) d o z f d i ,
C τ i = 1 2 [ cos ( 2 τ f ) cos   2 ( τ s i ) + sin ( 2 τ f ) sin   2 ( τ s i ) + 1 ] .
ST { I p } i o C p , i o ,
I c ( x , y ) = I p   max ( x , y ) ,
p max = max p { C p , i o 2 } ,
MSE = x M y N [ I ref ( x , y ) I o ( x , y ) ] 2 M N ,
I F P = 100 N e N T ,
Q = 4 σ I J I J ¯ ( σ I 2 + σ J 2 ) [ I ¯ 2 + J ¯ 2 ] ,
I ¯   =   1 M N x M y N I ( x , y ) , J ¯ = 1 M N x M y N J ( x , y ) ,
σ I 2   =   1 M N 1 x M y N ( I ( x , y ) I ¯ ) 2 ,
σ J 2   =   1 M N 1 x M y N ( J ( x , y ) J ¯ ) 2 ,
σ I J   =   1 M N 1 x M y N ( I ( x , y ) I ¯ ) ( J ( x , y ) J ¯ ) ,

Metrics