Abstract

Many image processing applications are confounded by both sensor noise and cast shadows. All image sensors add noise to a captured image that can reduce algorithm sensitivity and performance, and global filters or fixed thresholds are often applied to limit their effects. Cast shadows can appear as scene changes and are difficult to adequately detect and remove from images and image sequences. We couple image-noise statistics with a dual-illumination shadow-detection algorithm to provide a novel color-based method for shadow-free scene-change detection whose performance is bound by metamerism and image noise, and has only one variable—the desired confidence interval for noise separation.

© 2010 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. T. Horprasert, D. Harwood, and L. S. Davis, “A statistical approach for real-time robust background subtraction and shadow detection,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE, 1999), pp. 1-19.
  2. R. Cucchiara, C. Grana, M. Piccardi, A. Prati, and S. Sirotti, “Improving shadow suppression in moving object detection with HSV color information,” in Proceedings of 2001 IEEE Intelligent Transportation Systems (IEEE, 2001), pp. 334-339.
  3. J. Nakamura, Image Sensors and Signal Processing for Digital Still Cameras (CRC Press, 2006), p. 336.
  4. A. El Gamal and H. Eltoukhy, “CMOS image sensors,” IEEE Circuits Devices Mag. 21, 6-20 (2005).
    [CrossRef]
  5. K. Irie, A. E. McKinnon, K. Unsworth, and I. M. Woodhead, “A model for evaluation of noise in CCD digital video cameras,” Meas. Sci. Technol. 19, 045207 (2008).
    [CrossRef]
  6. M. Xiao, C.-Z. Han, and L. Zhang, “Moving shadow detection and removal for traffic sequences,” Int. J. Autom. Comput. 4, 38-46 (2007).
    [CrossRef]
  7. J. Stauder, R. Mech, and J. Ostermann, “Detection of moving cast shadows for object segmentation,” IEEE Trans. Multimedia 1, 65-76 (1999).
    [CrossRef]
  8. D. Xu, J. Liu, X. Li, Z. Liu, and X. Tang, “Insignificant shadow detection for video segmentation,” IEEE Trans. Circuits Syst. Video Technol. 15, 1058-1064 (2005).
    [CrossRef]
  9. G. D. Finlayson, S. D. Hordley, C. Lu, and M. S. Drew, “On the removal of shadows from images,” IEEE Trans. Pattern Anal. Mach. Intell. 28, 59-68 (2006).
    [CrossRef] [PubMed]
  10. K. Irie, A. E. McKinnon, K. Unsworth, and I. M. Woodhead, “An investigation into noise-bound shadow detection and removal,” in 23rd International Conference Image and Vision Computing New Zealand (IVCNZ 2008), pp. 1-6.
    [CrossRef]
  11. K. Irie, A. E. McKinnon, K. Unsworth, and I. M. Woodhead, “A technique for evaluation of CCD video-camera noise,” IEEE Trans. Circuits Syst. Video Technol. 18, 280-284 (2008).
    [CrossRef]
  12. K. Irie, A. E. McKinnon, K. Unsworth, and I. M. Woodhead, “Measurement of digital camera image noise for imaging applications,” Sens. Transducers 90, 185-194 (2008).
  13. S. T. Henderson, Daylight and Its Spectrum, 2nd ed. (Adam Hilger, 1977).
  14. D. K. Lynch and W. Livingston, Color and Light in Nature (Press Syndicate of the Univ. of Cambridge, 1995).
  15. R. Gershon, A. D. Jepson, and J. K. Tsotsos, “From [R, G, B] to surface reflectance: computing color constant descriptors in images,” Proceedings of the 10th International Joint Conference on Artificial Intelligence (Morgan Kaufmann, 1987), Vol. 2, pp. 755-758.
  16. B. Delaunay, “Sur la sphere vide,” Izv. Akad. Nauk SSSR Otd. Mat. Estest. Nauk 7, 793-800 (1934).

2008 (3)

K. Irie, A. E. McKinnon, K. Unsworth, and I. M. Woodhead, “A model for evaluation of noise in CCD digital video cameras,” Meas. Sci. Technol. 19, 045207 (2008).
[CrossRef]

K. Irie, A. E. McKinnon, K. Unsworth, and I. M. Woodhead, “A technique for evaluation of CCD video-camera noise,” IEEE Trans. Circuits Syst. Video Technol. 18, 280-284 (2008).
[CrossRef]

K. Irie, A. E. McKinnon, K. Unsworth, and I. M. Woodhead, “Measurement of digital camera image noise for imaging applications,” Sens. Transducers 90, 185-194 (2008).

2007 (1)

M. Xiao, C.-Z. Han, and L. Zhang, “Moving shadow detection and removal for traffic sequences,” Int. J. Autom. Comput. 4, 38-46 (2007).
[CrossRef]

2006 (1)

G. D. Finlayson, S. D. Hordley, C. Lu, and M. S. Drew, “On the removal of shadows from images,” IEEE Trans. Pattern Anal. Mach. Intell. 28, 59-68 (2006).
[CrossRef] [PubMed]

2005 (2)

A. El Gamal and H. Eltoukhy, “CMOS image sensors,” IEEE Circuits Devices Mag. 21, 6-20 (2005).
[CrossRef]

D. Xu, J. Liu, X. Li, Z. Liu, and X. Tang, “Insignificant shadow detection for video segmentation,” IEEE Trans. Circuits Syst. Video Technol. 15, 1058-1064 (2005).
[CrossRef]

1999 (1)

J. Stauder, R. Mech, and J. Ostermann, “Detection of moving cast shadows for object segmentation,” IEEE Trans. Multimedia 1, 65-76 (1999).
[CrossRef]

1934 (1)

B. Delaunay, “Sur la sphere vide,” Izv. Akad. Nauk SSSR Otd. Mat. Estest. Nauk 7, 793-800 (1934).

Cucchiara, R.

R. Cucchiara, C. Grana, M. Piccardi, A. Prati, and S. Sirotti, “Improving shadow suppression in moving object detection with HSV color information,” in Proceedings of 2001 IEEE Intelligent Transportation Systems (IEEE, 2001), pp. 334-339.

Davis, L. S.

T. Horprasert, D. Harwood, and L. S. Davis, “A statistical approach for real-time robust background subtraction and shadow detection,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE, 1999), pp. 1-19.

Delaunay, B.

B. Delaunay, “Sur la sphere vide,” Izv. Akad. Nauk SSSR Otd. Mat. Estest. Nauk 7, 793-800 (1934).

Drew, M. S.

G. D. Finlayson, S. D. Hordley, C. Lu, and M. S. Drew, “On the removal of shadows from images,” IEEE Trans. Pattern Anal. Mach. Intell. 28, 59-68 (2006).
[CrossRef] [PubMed]

El Gamal, A.

A. El Gamal and H. Eltoukhy, “CMOS image sensors,” IEEE Circuits Devices Mag. 21, 6-20 (2005).
[CrossRef]

Eltoukhy, H.

A. El Gamal and H. Eltoukhy, “CMOS image sensors,” IEEE Circuits Devices Mag. 21, 6-20 (2005).
[CrossRef]

Finlayson, G. D.

G. D. Finlayson, S. D. Hordley, C. Lu, and M. S. Drew, “On the removal of shadows from images,” IEEE Trans. Pattern Anal. Mach. Intell. 28, 59-68 (2006).
[CrossRef] [PubMed]

Gershon, R.

R. Gershon, A. D. Jepson, and J. K. Tsotsos, “From [R, G, B] to surface reflectance: computing color constant descriptors in images,” Proceedings of the 10th International Joint Conference on Artificial Intelligence (Morgan Kaufmann, 1987), Vol. 2, pp. 755-758.

Grana, C.

R. Cucchiara, C. Grana, M. Piccardi, A. Prati, and S. Sirotti, “Improving shadow suppression in moving object detection with HSV color information,” in Proceedings of 2001 IEEE Intelligent Transportation Systems (IEEE, 2001), pp. 334-339.

Han, C.-Z.

M. Xiao, C.-Z. Han, and L. Zhang, “Moving shadow detection and removal for traffic sequences,” Int. J. Autom. Comput. 4, 38-46 (2007).
[CrossRef]

Harwood, D.

T. Horprasert, D. Harwood, and L. S. Davis, “A statistical approach for real-time robust background subtraction and shadow detection,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE, 1999), pp. 1-19.

Henderson, S. T.

S. T. Henderson, Daylight and Its Spectrum, 2nd ed. (Adam Hilger, 1977).

Hordley, S. D.

G. D. Finlayson, S. D. Hordley, C. Lu, and M. S. Drew, “On the removal of shadows from images,” IEEE Trans. Pattern Anal. Mach. Intell. 28, 59-68 (2006).
[CrossRef] [PubMed]

Horprasert, T.

T. Horprasert, D. Harwood, and L. S. Davis, “A statistical approach for real-time robust background subtraction and shadow detection,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE, 1999), pp. 1-19.

Irie, K.

K. Irie, A. E. McKinnon, K. Unsworth, and I. M. Woodhead, “A technique for evaluation of CCD video-camera noise,” IEEE Trans. Circuits Syst. Video Technol. 18, 280-284 (2008).
[CrossRef]

K. Irie, A. E. McKinnon, K. Unsworth, and I. M. Woodhead, “A model for evaluation of noise in CCD digital video cameras,” Meas. Sci. Technol. 19, 045207 (2008).
[CrossRef]

K. Irie, A. E. McKinnon, K. Unsworth, and I. M. Woodhead, “Measurement of digital camera image noise for imaging applications,” Sens. Transducers 90, 185-194 (2008).

K. Irie, A. E. McKinnon, K. Unsworth, and I. M. Woodhead, “An investigation into noise-bound shadow detection and removal,” in 23rd International Conference Image and Vision Computing New Zealand (IVCNZ 2008), pp. 1-6.
[CrossRef]

Jepson, A. D.

R. Gershon, A. D. Jepson, and J. K. Tsotsos, “From [R, G, B] to surface reflectance: computing color constant descriptors in images,” Proceedings of the 10th International Joint Conference on Artificial Intelligence (Morgan Kaufmann, 1987), Vol. 2, pp. 755-758.

Li, X.

D. Xu, J. Liu, X. Li, Z. Liu, and X. Tang, “Insignificant shadow detection for video segmentation,” IEEE Trans. Circuits Syst. Video Technol. 15, 1058-1064 (2005).
[CrossRef]

Liu, J.

D. Xu, J. Liu, X. Li, Z. Liu, and X. Tang, “Insignificant shadow detection for video segmentation,” IEEE Trans. Circuits Syst. Video Technol. 15, 1058-1064 (2005).
[CrossRef]

Liu, Z.

D. Xu, J. Liu, X. Li, Z. Liu, and X. Tang, “Insignificant shadow detection for video segmentation,” IEEE Trans. Circuits Syst. Video Technol. 15, 1058-1064 (2005).
[CrossRef]

Livingston, W.

D. K. Lynch and W. Livingston, Color and Light in Nature (Press Syndicate of the Univ. of Cambridge, 1995).

Lu, C.

G. D. Finlayson, S. D. Hordley, C. Lu, and M. S. Drew, “On the removal of shadows from images,” IEEE Trans. Pattern Anal. Mach. Intell. 28, 59-68 (2006).
[CrossRef] [PubMed]

Lynch, D. K.

D. K. Lynch and W. Livingston, Color and Light in Nature (Press Syndicate of the Univ. of Cambridge, 1995).

McKinnon, A. E.

K. Irie, A. E. McKinnon, K. Unsworth, and I. M. Woodhead, “A technique for evaluation of CCD video-camera noise,” IEEE Trans. Circuits Syst. Video Technol. 18, 280-284 (2008).
[CrossRef]

K. Irie, A. E. McKinnon, K. Unsworth, and I. M. Woodhead, “A model for evaluation of noise in CCD digital video cameras,” Meas. Sci. Technol. 19, 045207 (2008).
[CrossRef]

K. Irie, A. E. McKinnon, K. Unsworth, and I. M. Woodhead, “Measurement of digital camera image noise for imaging applications,” Sens. Transducers 90, 185-194 (2008).

K. Irie, A. E. McKinnon, K. Unsworth, and I. M. Woodhead, “An investigation into noise-bound shadow detection and removal,” in 23rd International Conference Image and Vision Computing New Zealand (IVCNZ 2008), pp. 1-6.
[CrossRef]

Mech, R.

J. Stauder, R. Mech, and J. Ostermann, “Detection of moving cast shadows for object segmentation,” IEEE Trans. Multimedia 1, 65-76 (1999).
[CrossRef]

Nakamura, J.

J. Nakamura, Image Sensors and Signal Processing for Digital Still Cameras (CRC Press, 2006), p. 336.

Ostermann, J.

J. Stauder, R. Mech, and J. Ostermann, “Detection of moving cast shadows for object segmentation,” IEEE Trans. Multimedia 1, 65-76 (1999).
[CrossRef]

Piccardi, M.

R. Cucchiara, C. Grana, M. Piccardi, A. Prati, and S. Sirotti, “Improving shadow suppression in moving object detection with HSV color information,” in Proceedings of 2001 IEEE Intelligent Transportation Systems (IEEE, 2001), pp. 334-339.

Prati, A.

R. Cucchiara, C. Grana, M. Piccardi, A. Prati, and S. Sirotti, “Improving shadow suppression in moving object detection with HSV color information,” in Proceedings of 2001 IEEE Intelligent Transportation Systems (IEEE, 2001), pp. 334-339.

Sirotti, S.

R. Cucchiara, C. Grana, M. Piccardi, A. Prati, and S. Sirotti, “Improving shadow suppression in moving object detection with HSV color information,” in Proceedings of 2001 IEEE Intelligent Transportation Systems (IEEE, 2001), pp. 334-339.

Stauder, J.

J. Stauder, R. Mech, and J. Ostermann, “Detection of moving cast shadows for object segmentation,” IEEE Trans. Multimedia 1, 65-76 (1999).
[CrossRef]

Tang, X.

D. Xu, J. Liu, X. Li, Z. Liu, and X. Tang, “Insignificant shadow detection for video segmentation,” IEEE Trans. Circuits Syst. Video Technol. 15, 1058-1064 (2005).
[CrossRef]

Tsotsos, J. K.

R. Gershon, A. D. Jepson, and J. K. Tsotsos, “From [R, G, B] to surface reflectance: computing color constant descriptors in images,” Proceedings of the 10th International Joint Conference on Artificial Intelligence (Morgan Kaufmann, 1987), Vol. 2, pp. 755-758.

Unsworth, K.

K. Irie, A. E. McKinnon, K. Unsworth, and I. M. Woodhead, “Measurement of digital camera image noise for imaging applications,” Sens. Transducers 90, 185-194 (2008).

K. Irie, A. E. McKinnon, K. Unsworth, and I. M. Woodhead, “A model for evaluation of noise in CCD digital video cameras,” Meas. Sci. Technol. 19, 045207 (2008).
[CrossRef]

K. Irie, A. E. McKinnon, K. Unsworth, and I. M. Woodhead, “A technique for evaluation of CCD video-camera noise,” IEEE Trans. Circuits Syst. Video Technol. 18, 280-284 (2008).
[CrossRef]

K. Irie, A. E. McKinnon, K. Unsworth, and I. M. Woodhead, “An investigation into noise-bound shadow detection and removal,” in 23rd International Conference Image and Vision Computing New Zealand (IVCNZ 2008), pp. 1-6.
[CrossRef]

Woodhead, I. M.

K. Irie, A. E. McKinnon, K. Unsworth, and I. M. Woodhead, “A technique for evaluation of CCD video-camera noise,” IEEE Trans. Circuits Syst. Video Technol. 18, 280-284 (2008).
[CrossRef]

K. Irie, A. E. McKinnon, K. Unsworth, and I. M. Woodhead, “A model for evaluation of noise in CCD digital video cameras,” Meas. Sci. Technol. 19, 045207 (2008).
[CrossRef]

K. Irie, A. E. McKinnon, K. Unsworth, and I. M. Woodhead, “Measurement of digital camera image noise for imaging applications,” Sens. Transducers 90, 185-194 (2008).

K. Irie, A. E. McKinnon, K. Unsworth, and I. M. Woodhead, “An investigation into noise-bound shadow detection and removal,” in 23rd International Conference Image and Vision Computing New Zealand (IVCNZ 2008), pp. 1-6.
[CrossRef]

Xiao, M.

M. Xiao, C.-Z. Han, and L. Zhang, “Moving shadow detection and removal for traffic sequences,” Int. J. Autom. Comput. 4, 38-46 (2007).
[CrossRef]

Xu, D.

D. Xu, J. Liu, X. Li, Z. Liu, and X. Tang, “Insignificant shadow detection for video segmentation,” IEEE Trans. Circuits Syst. Video Technol. 15, 1058-1064 (2005).
[CrossRef]

Zhang, L.

M. Xiao, C.-Z. Han, and L. Zhang, “Moving shadow detection and removal for traffic sequences,” Int. J. Autom. Comput. 4, 38-46 (2007).
[CrossRef]

IEEE Circuits Devices Mag. (1)

A. El Gamal and H. Eltoukhy, “CMOS image sensors,” IEEE Circuits Devices Mag. 21, 6-20 (2005).
[CrossRef]

IEEE Trans. Circuits Syst. Video Technol. (2)

D. Xu, J. Liu, X. Li, Z. Liu, and X. Tang, “Insignificant shadow detection for video segmentation,” IEEE Trans. Circuits Syst. Video Technol. 15, 1058-1064 (2005).
[CrossRef]

K. Irie, A. E. McKinnon, K. Unsworth, and I. M. Woodhead, “A technique for evaluation of CCD video-camera noise,” IEEE Trans. Circuits Syst. Video Technol. 18, 280-284 (2008).
[CrossRef]

IEEE Trans. Multimedia (1)

J. Stauder, R. Mech, and J. Ostermann, “Detection of moving cast shadows for object segmentation,” IEEE Trans. Multimedia 1, 65-76 (1999).
[CrossRef]

IEEE Trans. Pattern Anal. Mach. Intell. (1)

G. D. Finlayson, S. D. Hordley, C. Lu, and M. S. Drew, “On the removal of shadows from images,” IEEE Trans. Pattern Anal. Mach. Intell. 28, 59-68 (2006).
[CrossRef] [PubMed]

Int. J. Autom. Comput. (1)

M. Xiao, C.-Z. Han, and L. Zhang, “Moving shadow detection and removal for traffic sequences,” Int. J. Autom. Comput. 4, 38-46 (2007).
[CrossRef]

Izv. Akad. Nauk SSSR Otd. Mat. Estest. Nauk (1)

B. Delaunay, “Sur la sphere vide,” Izv. Akad. Nauk SSSR Otd. Mat. Estest. Nauk 7, 793-800 (1934).

Meas. Sci. Technol. (1)

K. Irie, A. E. McKinnon, K. Unsworth, and I. M. Woodhead, “A model for evaluation of noise in CCD digital video cameras,” Meas. Sci. Technol. 19, 045207 (2008).
[CrossRef]

Sens. Transducers (1)

K. Irie, A. E. McKinnon, K. Unsworth, and I. M. Woodhead, “Measurement of digital camera image noise for imaging applications,” Sens. Transducers 90, 185-194 (2008).

Other (7)

S. T. Henderson, Daylight and Its Spectrum, 2nd ed. (Adam Hilger, 1977).

D. K. Lynch and W. Livingston, Color and Light in Nature (Press Syndicate of the Univ. of Cambridge, 1995).

R. Gershon, A. D. Jepson, and J. K. Tsotsos, “From [R, G, B] to surface reflectance: computing color constant descriptors in images,” Proceedings of the 10th International Joint Conference on Artificial Intelligence (Morgan Kaufmann, 1987), Vol. 2, pp. 755-758.

T. Horprasert, D. Harwood, and L. S. Davis, “A statistical approach for real-time robust background subtraction and shadow detection,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE, 1999), pp. 1-19.

R. Cucchiara, C. Grana, M. Piccardi, A. Prati, and S. Sirotti, “Improving shadow suppression in moving object detection with HSV color information,” in Proceedings of 2001 IEEE Intelligent Transportation Systems (IEEE, 2001), pp. 334-339.

J. Nakamura, Image Sensors and Signal Processing for Digital Still Cameras (CRC Press, 2006), p. 336.

K. Irie, A. E. McKinnon, K. Unsworth, and I. M. Woodhead, “An investigation into noise-bound shadow detection and removal,” in 23rd International Conference Image and Vision Computing New Zealand (IVCNZ 2008), pp. 1-6.
[CrossRef]

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (8)

Fig. 1
Fig. 1

Measured temporal noise for the CMOS camera described in Table 1 (measured at 22 ° C ambient temperature).

Fig. 2
Fig. 2

Measured and modeled changes in band ratios (characteristic lines) for changing intensities of halogen lighting on panels of a color chart in an environment with ambient fluorescent lighting (14 of the 18 panels shown for clarity).

Fig. 3
Fig. 3

Interpolated characteristic-line data set generated from Delaunay triangulation of the 18 measured lines.

Fig. 4
Fig. 4

Scene-change detection regions for a line in x y space (direct halogen and fluorescent ambient lighting) for given values of P G , on a calibration line from Fig. 3. Regions for larger values of P G (brighter intensity) are smaller, showing that temporal image noise has a smaller effect on bright pixels and a greater effect on darker pixels.

Fig. 5
Fig. 5

Images from image set “Castle.” (a) Background image—a large printed photograph on a desk; (b) comparison image with the added objects (pencil sharpener and cup) and their respective cast shadows. (c) Pixel-level differences between (a) and (b).

Fig. 6
Fig. 6

Shadow-detection masks for image set “Castle.” (a) Result of the noise-bound scene-change algorithm. (b) Filtered mask of (a) with arbitrary-sized small holes filled and small objects removed ( < 9  pixels ) , demonstrating an example final mask that could be used for further processing in an object detection process. (c) Shadow-free image reconstructed by combing Figs. 5a, 5b using the mask in (b).

Fig. 7
Fig. 7

Images from image set “Hawaii.” (a) Background image—a large printed photograph on a desk; (b) comparison image with the added objects (arm and color chart) and a large cast shadow from the chart. (c) Pixel-level differences between (a) and (b).

Fig. 8
Fig. 8

Shadow detection for image set “Hawaii.” See Fig. 6 captions for descriptions of (a)–(c).

Tables (2)

Tables Icon

Table 1 uEye UI1210-C Camera Details

Tables Icon

Table 2 Results of the Scene-Change Detection Algorithm: Percentage Areas of Scene Change Correctly Classified as Objects or Shadows

Equations (10)

Equations on this page are rendered with MathJax. Learn more.

P k = E ( λ ) S ( λ ) C k ( λ ) d λ , k { R , G , B } ,
E ( λ ) = E a ( λ ) + n E d ( λ ) , n [ 0 , 1 ] ,
P k = ( E a ( λ ) + n E d ( λ ) ) S ( λ ) C k ( λ ) d λ , k { R , G , B } ,
n [ 0 , 1 ] .
x = ( E a ( λ ) + n E d ( λ ) ) S ( λ ) C R ( λ ) d λ ( E a ( λ ) + n E d ( λ ) ) S ( λ ) C G ( λ ) d λ , n [ 0 , 1 ] ,
y = ( E a ( λ ) + n E d ( λ ) ) S ( λ ) C B ( λ ) d λ ( E a ( λ ) + n E d ( λ ) ) S ( λ ) C G ( λ ) d λ , n [ 0 , 1 ] .
σ x = ( σ R P R ) 2 + ( σ G P G ) 2 x ,
σ y = ( σ B P B ) 2 + ( σ G P G ) 2 y .
α G = n β n , n { 0 , , 1 } ,
P free = { P ref if ρ ( P comp ) α G AND V ( P comp ) < V ( P ref ) P comp otherwise } .

Metrics