Abstract

Intrinsic image decomposition aims to factorize an image into albedo (reflectance) and shading (illumination) sub-components. Being ill posed and under-constrained, it is a very challenging computer vision problem. There are infinite pairs of reflectance and shading images that can reconstruct the same input. To address the problem, Intrinsic Images in the Wild by Bell et al. provides an optimization framework based on a dense conditional random field (CRF) formulation that considers long-range material relations. We improve upon their model by introducing illumination invariant image descriptors: color ratios. The color ratios and the intrinsic reflectance are both invariant to illumination and thus are highly correlated. Through detailed experiments, we provide ways to inject the color ratios into the dense CRF optimization. Our approach is physics based and learning free and leads to more accurate and robust reflectance decompositions.

© 2021 Optical Society of America

Full Article  |  PDF Article
More Like This
Deep intrinsic decomposition trained on surreal scenes yet with realistic light effects

Hassan A. Sial, Ramon Baldrich, and Maria Vanrell
J. Opt. Soc. Am. A 37(1) 1-15 (2020)

Improved single-illumination estimation accuracy via redefining the illuminant-invariant descriptor and the grey pixels

Xiang Yang, Xing Jin, and Jingjing Zhang
Opt. Express 26(22) 29055-29067 (2018)

Deep image enhancement for ill light imaging

Rizwan Khan, You Yang, Qiong Liu, Jialie Shen, and Bing Li
J. Opt. Soc. Am. A 38(6) 827-839 (2021)

References

  • View by:
  • |
  • |
  • |

  1. H. G. Barrow and J. M. Tenenbaum, “Recovering intrinsic scene characteristics from images,” in Computer Vision Systems (1978), pp. 3–26.
  2. A. S. Baslamisli, T. T. Groenestege, P. Das, H. A. Le, S. Karaoglu, and T. Gevers, “Joint learning of intrinsic images and semantic segmentation,” in European Conference on Computer Vision (2018).
  3. C. Xu, Y. Han, G. Baciu, and M. Li, “Fabric image recolorization based on intrinsic image decomposition,” Text. Res. J. 89, 3617–3631 (2019).
    [Crossref]
  4. S. Beigpour and J. van de Weijer, “Object recoloring based on intrinsic image estimation,” in IEEE International Conference on Computer Vision (2011).
  5. E. H. Land and J. J. McCann, “Lightness and retinex theory,” J. Opt. Soc. Am. 61, 1–11 (1971).
    [Crossref]
  6. S. Bell, K. Bala, and N. Snavely, “Intrinsic images in the wild,” ACM Trans. Graph. 33, 159 (2014).
    [Crossref]
  7. Z. Cheng, Y. Zheng, S. You, and I. Sato, “Non-local intrinsic decomposition with near-infrared priors,” in IEEE International Conference on Computer Vision (2019).
  8. T. Narihira, M. Maire, and S. X. Yu, “Direct intrinsics: learning albedo-shading decomposition by convolutional regression,” in IEEE International Conference on Computer Vision (2015).
  9. A. S. Baslamisli, H. A. Le, and T. Gevers, “CNN based learning using reflection and retinex models for intrinsic image decomposition,” in IEEE Conference on Computer Vision and Pattern Recognition (2018).
  10. P. V. Gehler, C. Rother, M. Kiefel, L. Zhang, and B. Schölkopf, “Recovering intrinsic images with a global sparsity prior on reflectance,” in Advances in Neural Information Processing Systems (2011).
  11. L. Shen and C. Yeo, “Intrinsic images decomposition using a local and global sparse representation of reflectance,” in IEEE Conference on Computer Vision and Pattern Recognition (2011).
  12. J. T. Barron and J. Malik, “Shape, illumination, and reflectance from shading,” IEEE Trans. Pattern Anal. Mach. Intell. 37, 1670–1687 (2015).
    [Crossref]
  13. G. D. Finlayson, “Colour object recognition,” master’s thesis (Simon Fraser University, 1992).
  14. L. Shen, P. Tan, and S. Lin, “Intrinsic image decomposition with non-local texture cues,” in IEEE Conference on Computer Vision and Pattern Recognition (2008).
  15. Q. Zhao, P. Tan, Q. Dai, L. Shen, E. Wu, and S. Lin, “A closed-form solution to retinex with non-local texture constraints,” IEEE Trans. Pattern Anal. Mach. Intell. 34, 1437–1444 (2012).
    [Crossref]
  16. J. Shen, X. Yang, X. Li, and Y. Jia, “Intrinsic image decomposition using optimization and user scribbles,” IEEE Trans. Cyber. 43, 425–436 (2013).
    [Crossref]
  17. E. Garces, A. Munoz, J. Lopez-Moreno, and D. Gutierrez, “Intrinsic images by clustering,” in Computer Graphics Forum (2012).
  18. X. Jiang, A. J. Schofield, and J. L. Wyatt, “Correlation-based intrinsic image extraction from a single image,” in European Conference on Computer Vision (2010).
  19. M. F. Tappen, W. T. Freeman, and E. H. Adelson, “Recovering intrinsic images from a single image,” IEEE Trans. Pattern Anal. Mach. Intell. 27, 1459–1472 (2005).
    [Crossref]
  20. S. Ding, B. Sheng, Z. Xie, and L. Ma, “Intrinsic image estimation using near-l0 sparse optimization,” Visual Comput. 33, 355–369 (2017).
    [Crossref]
  21. Y. Li and M. S. Brown, “Single image layer separation using relative smoothness,” in IEEE Conference on Computer Vision and Pattern Recognition (2014).
  22. B. Sheng, P. Li, Y. Jin, P. Tan, and T. Y. Lee, “Intrinsic image decomposition with step and drift shading separation,” IEEE Trans. Vis. Comput. Graph. 26, 1332–1346 (2020).
    [Crossref]
  23. A. Bousseau, S. Paris, and F. Durand, “User-assisted intrinsic images,” in ACM SIGGRAPH Asia 2009 (2009), paper 130.
  24. Q. Chen and V. Koltun, “A simple model for intrinsic image decomposition with depth cues,” in IEEE International Conference on Computer Vision (2013).
  25. J. Jeon, S. Cho, X. Tong, and S. Lee, “Intrinsic image decomposition using structure-texture separation and surface normals,” in European Conference on Computer Vision (2016).
  26. K. J. Lee, Q. Zhao, X. Tong, M. Gong, S. Izadi, S. U. Lee, P. Tan, and S. Lin, “Estimation of intrinsic image sequences from image+depth video,” in European Conference on Computer Vision (2012).
  27. Y. Weiss, “Deriving intrinsic images from image sequences,” in IEEE International Conference on Computer Vision (2001).
  28. W. Gong, W. Xu, L. Wu, X. Xie, and Z. Cheng, “Intrinsic image sequence decomposition using low-rank sparse model,” IEEE Access 7, 4024–4030 (2019).
    [Crossref]
  29. P. Y. Laffont and J. C. Bazin, “Intrinsic decomposition of image sequences from local temporal variations,” in IEEE International Conference on Computer Vision (2015).
  30. J. Matas, R. Marik, and J. Kittler, “On representation and matching of multi-coloured objects,” in IEEE International Conference on Computer Vision (1995).
  31. S. K. Nayar and R. M. Bolle, “Reflectance based object recognition,” Int. J. Comput. Vis. 17, 219–240 (1996).
    [Crossref]
  32. K. Barnard and G. D. Finlayson, “Shadow identification using colour ratios,” in Color and Imaging Conference (2000).
  33. T. Gevers and A. Smeulders, “Color constant ratio gradients for image segmentation and similarity of texture objects,” in IEEE Conference on Computer Vision and Pattern Recognition (2001).
  34. T. Gevers and A. Smeulders, “Object recognition based on photometric color invariants,” in Scandinavian Conference on Image Analysis (1997).
  35. J. Shi, Y. Dong, H. Su, and S. X. Yu, “Learning non-Lambertian object intrinsics across shapenet categories,” in IEEE Conference on Computer Vision and Pattern Recognition (2017).
  36. Z. Li and N. Snavely, “Learning intrinsic image decomposition from watching the world,” in IEEE Conference on Computer Vision and Pattern Recognition (2018).
  37. L. Lettry, K. Vanhoey, and L. van Gool, “DARN: a deep adversarial residual network for intrinsic image decomposition,” in IEEE Winter Conference on Applications of Computer Vision (2018).
  38. A. S. Baslamisli, P. Das, H. A. Le, S. Karaoglu, and T. Gevers, “ShadingNet: image intrinsics by fine-grained shading decomposition,” arXiv:1912.04023 (2019).
  39. H. A. Sial, R. Baldrich, and M. Vanrell, “Deep intrinsic decomposition trained on surreal scenes yet with realistic light effects,” J. Opt. Soc. Am. A 37, 1–15 (2020).
    [Crossref]
  40. S. Shafer, “Using color to separate reflection components,” Color Res. Appl. 10, 210–218 (1985).
    [Crossref]
  41. G. D. Finlayson, M. S. Drew, and B. V. Funt, “Color constancy: generalized diagonal transforms suffice,” J. Opt. Soc. Am. A 11, 3011–3019 (1994).
    [Crossref]
  42. R. Grosse, M. K. Johnson, E. H. Adelson, and W. T. Freeman, “Ground truth dataset and baseline evaluations for intrinsic image algorithms,” in IEEE International Conference on Computer Vision (2009).
  43. J. Wang, X. Li, L. Hui, and J. Yang, “Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal,” in IEEE Conference on Computer Vision and Pattern Recognition (2018).
  44. L. Qu, J. Tian, S. He, Y. Tang, and R. W. H. Lau, “Deshadownet: a multi-context embedding deep network for shadow removal,” in IEEE Conference on Computer Vision and Pattern Recognition (2017).
  45. S. Bi, X. Han, and Y. Yu, “An L1 image transform for edge-preserving smoothing and scene-level intrinsic decomposition,” ACM Trans. Graph. 34, 1 (2015).
    [Crossref]
  46. M. Yan, “Methods of determining the number of clusters in a data set and a new clustering criterion,” Ph.D. thesis (Virginia Tech, 2005).
  47. X. Li, W. Liang, X. Zhang, S. Qing, and P. C. Chang, “A cluster validity evaluation method for dynamically determining the near-optimal number of clusters,” Soft Comput. 24, 9227–9241 (2020).
    [Crossref]
  48. J. Xu, Y. Hou, D. Ren, L. Liu, F. Zhu, M. Yu, H. Wang, and L. Shao, “Star: a structure and texture aware retinex model,” IEEE Trans. Image Process. 29, 5022–5037 (2020).
    [Crossref]
  49. T. Nestmeyer and P. V. Gehler, “Reflectance adaptive filtering improves intrinsic image estimation,” in IEEE Conference on Computer Vision and Pattern Recognition (2017).
  50. A. Krebs, Y. Benezeth, and F. Marzani, “Intrinsic RGB and multispectral images recovery by independent quadratic programming,” PeerJ Comput. Sci. 6, e256 (2020).
    [Crossref]
  51. Y. Liu, Y. Li, S. You, and F. Lu, “Unsupervised learning for intrinsic image decomposition from a single image,” in IEEE Conference on Computer Vision and Pattern Recognition (2020).
  52. Z. Li and N. Snavely, “CGIntrinsics: better intrinsic image decomposition through physically-based rendering,” in European Conference on Computer Vision (2018).

2020 (5)

B. Sheng, P. Li, Y. Jin, P. Tan, and T. Y. Lee, “Intrinsic image decomposition with step and drift shading separation,” IEEE Trans. Vis. Comput. Graph. 26, 1332–1346 (2020).
[Crossref]

H. A. Sial, R. Baldrich, and M. Vanrell, “Deep intrinsic decomposition trained on surreal scenes yet with realistic light effects,” J. Opt. Soc. Am. A 37, 1–15 (2020).
[Crossref]

X. Li, W. Liang, X. Zhang, S. Qing, and P. C. Chang, “A cluster validity evaluation method for dynamically determining the near-optimal number of clusters,” Soft Comput. 24, 9227–9241 (2020).
[Crossref]

J. Xu, Y. Hou, D. Ren, L. Liu, F. Zhu, M. Yu, H. Wang, and L. Shao, “Star: a structure and texture aware retinex model,” IEEE Trans. Image Process. 29, 5022–5037 (2020).
[Crossref]

A. Krebs, Y. Benezeth, and F. Marzani, “Intrinsic RGB and multispectral images recovery by independent quadratic programming,” PeerJ Comput. Sci. 6, e256 (2020).
[Crossref]

2019 (2)

W. Gong, W. Xu, L. Wu, X. Xie, and Z. Cheng, “Intrinsic image sequence decomposition using low-rank sparse model,” IEEE Access 7, 4024–4030 (2019).
[Crossref]

C. Xu, Y. Han, G. Baciu, and M. Li, “Fabric image recolorization based on intrinsic image decomposition,” Text. Res. J. 89, 3617–3631 (2019).
[Crossref]

2017 (1)

S. Ding, B. Sheng, Z. Xie, and L. Ma, “Intrinsic image estimation using near-l0 sparse optimization,” Visual Comput. 33, 355–369 (2017).
[Crossref]

2015 (2)

J. T. Barron and J. Malik, “Shape, illumination, and reflectance from shading,” IEEE Trans. Pattern Anal. Mach. Intell. 37, 1670–1687 (2015).
[Crossref]

S. Bi, X. Han, and Y. Yu, “An L1 image transform for edge-preserving smoothing and scene-level intrinsic decomposition,” ACM Trans. Graph. 34, 1 (2015).
[Crossref]

2014 (1)

S. Bell, K. Bala, and N. Snavely, “Intrinsic images in the wild,” ACM Trans. Graph. 33, 159 (2014).
[Crossref]

2013 (1)

J. Shen, X. Yang, X. Li, and Y. Jia, “Intrinsic image decomposition using optimization and user scribbles,” IEEE Trans. Cyber. 43, 425–436 (2013).
[Crossref]

2012 (1)

Q. Zhao, P. Tan, Q. Dai, L. Shen, E. Wu, and S. Lin, “A closed-form solution to retinex with non-local texture constraints,” IEEE Trans. Pattern Anal. Mach. Intell. 34, 1437–1444 (2012).
[Crossref]

2005 (1)

M. F. Tappen, W. T. Freeman, and E. H. Adelson, “Recovering intrinsic images from a single image,” IEEE Trans. Pattern Anal. Mach. Intell. 27, 1459–1472 (2005).
[Crossref]

1996 (1)

S. K. Nayar and R. M. Bolle, “Reflectance based object recognition,” Int. J. Comput. Vis. 17, 219–240 (1996).
[Crossref]

1994 (1)

1985 (1)

S. Shafer, “Using color to separate reflection components,” Color Res. Appl. 10, 210–218 (1985).
[Crossref]

1971 (1)

Adelson, E. H.

M. F. Tappen, W. T. Freeman, and E. H. Adelson, “Recovering intrinsic images from a single image,” IEEE Trans. Pattern Anal. Mach. Intell. 27, 1459–1472 (2005).
[Crossref]

R. Grosse, M. K. Johnson, E. H. Adelson, and W. T. Freeman, “Ground truth dataset and baseline evaluations for intrinsic image algorithms,” in IEEE International Conference on Computer Vision (2009).

Baciu, G.

C. Xu, Y. Han, G. Baciu, and M. Li, “Fabric image recolorization based on intrinsic image decomposition,” Text. Res. J. 89, 3617–3631 (2019).
[Crossref]

Bala, K.

S. Bell, K. Bala, and N. Snavely, “Intrinsic images in the wild,” ACM Trans. Graph. 33, 159 (2014).
[Crossref]

Baldrich, R.

Barnard, K.

K. Barnard and G. D. Finlayson, “Shadow identification using colour ratios,” in Color and Imaging Conference (2000).

Barron, J. T.

J. T. Barron and J. Malik, “Shape, illumination, and reflectance from shading,” IEEE Trans. Pattern Anal. Mach. Intell. 37, 1670–1687 (2015).
[Crossref]

Barrow, H. G.

H. G. Barrow and J. M. Tenenbaum, “Recovering intrinsic scene characteristics from images,” in Computer Vision Systems (1978), pp. 3–26.

Baslamisli, A. S.

A. S. Baslamisli, T. T. Groenestege, P. Das, H. A. Le, S. Karaoglu, and T. Gevers, “Joint learning of intrinsic images and semantic segmentation,” in European Conference on Computer Vision (2018).

A. S. Baslamisli, H. A. Le, and T. Gevers, “CNN based learning using reflection and retinex models for intrinsic image decomposition,” in IEEE Conference on Computer Vision and Pattern Recognition (2018).

A. S. Baslamisli, P. Das, H. A. Le, S. Karaoglu, and T. Gevers, “ShadingNet: image intrinsics by fine-grained shading decomposition,” arXiv:1912.04023 (2019).

Bazin, J. C.

P. Y. Laffont and J. C. Bazin, “Intrinsic decomposition of image sequences from local temporal variations,” in IEEE International Conference on Computer Vision (2015).

Beigpour, S.

S. Beigpour and J. van de Weijer, “Object recoloring based on intrinsic image estimation,” in IEEE International Conference on Computer Vision (2011).

Bell, S.

S. Bell, K. Bala, and N. Snavely, “Intrinsic images in the wild,” ACM Trans. Graph. 33, 159 (2014).
[Crossref]

Benezeth, Y.

A. Krebs, Y. Benezeth, and F. Marzani, “Intrinsic RGB and multispectral images recovery by independent quadratic programming,” PeerJ Comput. Sci. 6, e256 (2020).
[Crossref]

Bi, S.

S. Bi, X. Han, and Y. Yu, “An L1 image transform for edge-preserving smoothing and scene-level intrinsic decomposition,” ACM Trans. Graph. 34, 1 (2015).
[Crossref]

Bolle, R. M.

S. K. Nayar and R. M. Bolle, “Reflectance based object recognition,” Int. J. Comput. Vis. 17, 219–240 (1996).
[Crossref]

Bousseau, A.

A. Bousseau, S. Paris, and F. Durand, “User-assisted intrinsic images,” in ACM SIGGRAPH Asia 2009 (2009), paper 130.

Brown, M. S.

Y. Li and M. S. Brown, “Single image layer separation using relative smoothness,” in IEEE Conference on Computer Vision and Pattern Recognition (2014).

Chang, P. C.

X. Li, W. Liang, X. Zhang, S. Qing, and P. C. Chang, “A cluster validity evaluation method for dynamically determining the near-optimal number of clusters,” Soft Comput. 24, 9227–9241 (2020).
[Crossref]

Chen, Q.

Q. Chen and V. Koltun, “A simple model for intrinsic image decomposition with depth cues,” in IEEE International Conference on Computer Vision (2013).

Cheng, Z.

W. Gong, W. Xu, L. Wu, X. Xie, and Z. Cheng, “Intrinsic image sequence decomposition using low-rank sparse model,” IEEE Access 7, 4024–4030 (2019).
[Crossref]

Z. Cheng, Y. Zheng, S. You, and I. Sato, “Non-local intrinsic decomposition with near-infrared priors,” in IEEE International Conference on Computer Vision (2019).

Cho, S.

J. Jeon, S. Cho, X. Tong, and S. Lee, “Intrinsic image decomposition using structure-texture separation and surface normals,” in European Conference on Computer Vision (2016).

Dai, Q.

Q. Zhao, P. Tan, Q. Dai, L. Shen, E. Wu, and S. Lin, “A closed-form solution to retinex with non-local texture constraints,” IEEE Trans. Pattern Anal. Mach. Intell. 34, 1437–1444 (2012).
[Crossref]

Das, P.

A. S. Baslamisli, T. T. Groenestege, P. Das, H. A. Le, S. Karaoglu, and T. Gevers, “Joint learning of intrinsic images and semantic segmentation,” in European Conference on Computer Vision (2018).

A. S. Baslamisli, P. Das, H. A. Le, S. Karaoglu, and T. Gevers, “ShadingNet: image intrinsics by fine-grained shading decomposition,” arXiv:1912.04023 (2019).

Ding, S.

S. Ding, B. Sheng, Z. Xie, and L. Ma, “Intrinsic image estimation using near-l0 sparse optimization,” Visual Comput. 33, 355–369 (2017).
[Crossref]

Dong, Y.

J. Shi, Y. Dong, H. Su, and S. X. Yu, “Learning non-Lambertian object intrinsics across shapenet categories,” in IEEE Conference on Computer Vision and Pattern Recognition (2017).

Drew, M. S.

Durand, F.

A. Bousseau, S. Paris, and F. Durand, “User-assisted intrinsic images,” in ACM SIGGRAPH Asia 2009 (2009), paper 130.

Finlayson, G. D.

G. D. Finlayson, M. S. Drew, and B. V. Funt, “Color constancy: generalized diagonal transforms suffice,” J. Opt. Soc. Am. A 11, 3011–3019 (1994).
[Crossref]

K. Barnard and G. D. Finlayson, “Shadow identification using colour ratios,” in Color and Imaging Conference (2000).

G. D. Finlayson, “Colour object recognition,” master’s thesis (Simon Fraser University, 1992).

Freeman, W. T.

M. F. Tappen, W. T. Freeman, and E. H. Adelson, “Recovering intrinsic images from a single image,” IEEE Trans. Pattern Anal. Mach. Intell. 27, 1459–1472 (2005).
[Crossref]

R. Grosse, M. K. Johnson, E. H. Adelson, and W. T. Freeman, “Ground truth dataset and baseline evaluations for intrinsic image algorithms,” in IEEE International Conference on Computer Vision (2009).

Funt, B. V.

Garces, E.

E. Garces, A. Munoz, J. Lopez-Moreno, and D. Gutierrez, “Intrinsic images by clustering,” in Computer Graphics Forum (2012).

Gehler, P. V.

P. V. Gehler, C. Rother, M. Kiefel, L. Zhang, and B. Schölkopf, “Recovering intrinsic images with a global sparsity prior on reflectance,” in Advances in Neural Information Processing Systems (2011).

T. Nestmeyer and P. V. Gehler, “Reflectance adaptive filtering improves intrinsic image estimation,” in IEEE Conference on Computer Vision and Pattern Recognition (2017).

Gevers, T.

A. S. Baslamisli, T. T. Groenestege, P. Das, H. A. Le, S. Karaoglu, and T. Gevers, “Joint learning of intrinsic images and semantic segmentation,” in European Conference on Computer Vision (2018).

A. S. Baslamisli, H. A. Le, and T. Gevers, “CNN based learning using reflection and retinex models for intrinsic image decomposition,” in IEEE Conference on Computer Vision and Pattern Recognition (2018).

A. S. Baslamisli, P. Das, H. A. Le, S. Karaoglu, and T. Gevers, “ShadingNet: image intrinsics by fine-grained shading decomposition,” arXiv:1912.04023 (2019).

T. Gevers and A. Smeulders, “Color constant ratio gradients for image segmentation and similarity of texture objects,” in IEEE Conference on Computer Vision and Pattern Recognition (2001).

T. Gevers and A. Smeulders, “Object recognition based on photometric color invariants,” in Scandinavian Conference on Image Analysis (1997).

Gong, M.

K. J. Lee, Q. Zhao, X. Tong, M. Gong, S. Izadi, S. U. Lee, P. Tan, and S. Lin, “Estimation of intrinsic image sequences from image+depth video,” in European Conference on Computer Vision (2012).

Gong, W.

W. Gong, W. Xu, L. Wu, X. Xie, and Z. Cheng, “Intrinsic image sequence decomposition using low-rank sparse model,” IEEE Access 7, 4024–4030 (2019).
[Crossref]

Groenestege, T. T.

A. S. Baslamisli, T. T. Groenestege, P. Das, H. A. Le, S. Karaoglu, and T. Gevers, “Joint learning of intrinsic images and semantic segmentation,” in European Conference on Computer Vision (2018).

Grosse, R.

R. Grosse, M. K. Johnson, E. H. Adelson, and W. T. Freeman, “Ground truth dataset and baseline evaluations for intrinsic image algorithms,” in IEEE International Conference on Computer Vision (2009).

Gutierrez, D.

E. Garces, A. Munoz, J. Lopez-Moreno, and D. Gutierrez, “Intrinsic images by clustering,” in Computer Graphics Forum (2012).

Han, X.

S. Bi, X. Han, and Y. Yu, “An L1 image transform for edge-preserving smoothing and scene-level intrinsic decomposition,” ACM Trans. Graph. 34, 1 (2015).
[Crossref]

Han, Y.

C. Xu, Y. Han, G. Baciu, and M. Li, “Fabric image recolorization based on intrinsic image decomposition,” Text. Res. J. 89, 3617–3631 (2019).
[Crossref]

He, S.

L. Qu, J. Tian, S. He, Y. Tang, and R. W. H. Lau, “Deshadownet: a multi-context embedding deep network for shadow removal,” in IEEE Conference on Computer Vision and Pattern Recognition (2017).

Hou, Y.

J. Xu, Y. Hou, D. Ren, L. Liu, F. Zhu, M. Yu, H. Wang, and L. Shao, “Star: a structure and texture aware retinex model,” IEEE Trans. Image Process. 29, 5022–5037 (2020).
[Crossref]

Hui, L.

J. Wang, X. Li, L. Hui, and J. Yang, “Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal,” in IEEE Conference on Computer Vision and Pattern Recognition (2018).

Izadi, S.

K. J. Lee, Q. Zhao, X. Tong, M. Gong, S. Izadi, S. U. Lee, P. Tan, and S. Lin, “Estimation of intrinsic image sequences from image+depth video,” in European Conference on Computer Vision (2012).

Jeon, J.

J. Jeon, S. Cho, X. Tong, and S. Lee, “Intrinsic image decomposition using structure-texture separation and surface normals,” in European Conference on Computer Vision (2016).

Jia, Y.

J. Shen, X. Yang, X. Li, and Y. Jia, “Intrinsic image decomposition using optimization and user scribbles,” IEEE Trans. Cyber. 43, 425–436 (2013).
[Crossref]

Jiang, X.

X. Jiang, A. J. Schofield, and J. L. Wyatt, “Correlation-based intrinsic image extraction from a single image,” in European Conference on Computer Vision (2010).

Jin, Y.

B. Sheng, P. Li, Y. Jin, P. Tan, and T. Y. Lee, “Intrinsic image decomposition with step and drift shading separation,” IEEE Trans. Vis. Comput. Graph. 26, 1332–1346 (2020).
[Crossref]

Johnson, M. K.

R. Grosse, M. K. Johnson, E. H. Adelson, and W. T. Freeman, “Ground truth dataset and baseline evaluations for intrinsic image algorithms,” in IEEE International Conference on Computer Vision (2009).

Karaoglu, S.

A. S. Baslamisli, P. Das, H. A. Le, S. Karaoglu, and T. Gevers, “ShadingNet: image intrinsics by fine-grained shading decomposition,” arXiv:1912.04023 (2019).

A. S. Baslamisli, T. T. Groenestege, P. Das, H. A. Le, S. Karaoglu, and T. Gevers, “Joint learning of intrinsic images and semantic segmentation,” in European Conference on Computer Vision (2018).

Kiefel, M.

P. V. Gehler, C. Rother, M. Kiefel, L. Zhang, and B. Schölkopf, “Recovering intrinsic images with a global sparsity prior on reflectance,” in Advances in Neural Information Processing Systems (2011).

Kittler, J.

J. Matas, R. Marik, and J. Kittler, “On representation and matching of multi-coloured objects,” in IEEE International Conference on Computer Vision (1995).

Koltun, V.

Q. Chen and V. Koltun, “A simple model for intrinsic image decomposition with depth cues,” in IEEE International Conference on Computer Vision (2013).

Krebs, A.

A. Krebs, Y. Benezeth, and F. Marzani, “Intrinsic RGB and multispectral images recovery by independent quadratic programming,” PeerJ Comput. Sci. 6, e256 (2020).
[Crossref]

Laffont, P. Y.

P. Y. Laffont and J. C. Bazin, “Intrinsic decomposition of image sequences from local temporal variations,” in IEEE International Conference on Computer Vision (2015).

Land, E. H.

Lau, R. W. H.

L. Qu, J. Tian, S. He, Y. Tang, and R. W. H. Lau, “Deshadownet: a multi-context embedding deep network for shadow removal,” in IEEE Conference on Computer Vision and Pattern Recognition (2017).

Le, H. A.

A. S. Baslamisli, T. T. Groenestege, P. Das, H. A. Le, S. Karaoglu, and T. Gevers, “Joint learning of intrinsic images and semantic segmentation,” in European Conference on Computer Vision (2018).

A. S. Baslamisli, H. A. Le, and T. Gevers, “CNN based learning using reflection and retinex models for intrinsic image decomposition,” in IEEE Conference on Computer Vision and Pattern Recognition (2018).

A. S. Baslamisli, P. Das, H. A. Le, S. Karaoglu, and T. Gevers, “ShadingNet: image intrinsics by fine-grained shading decomposition,” arXiv:1912.04023 (2019).

Lee, K. J.

K. J. Lee, Q. Zhao, X. Tong, M. Gong, S. Izadi, S. U. Lee, P. Tan, and S. Lin, “Estimation of intrinsic image sequences from image+depth video,” in European Conference on Computer Vision (2012).

Lee, S.

J. Jeon, S. Cho, X. Tong, and S. Lee, “Intrinsic image decomposition using structure-texture separation and surface normals,” in European Conference on Computer Vision (2016).

Lee, S. U.

K. J. Lee, Q. Zhao, X. Tong, M. Gong, S. Izadi, S. U. Lee, P. Tan, and S. Lin, “Estimation of intrinsic image sequences from image+depth video,” in European Conference on Computer Vision (2012).

Lee, T. Y.

B. Sheng, P. Li, Y. Jin, P. Tan, and T. Y. Lee, “Intrinsic image decomposition with step and drift shading separation,” IEEE Trans. Vis. Comput. Graph. 26, 1332–1346 (2020).
[Crossref]

Lettry, L.

L. Lettry, K. Vanhoey, and L. van Gool, “DARN: a deep adversarial residual network for intrinsic image decomposition,” in IEEE Winter Conference on Applications of Computer Vision (2018).

Li, M.

C. Xu, Y. Han, G. Baciu, and M. Li, “Fabric image recolorization based on intrinsic image decomposition,” Text. Res. J. 89, 3617–3631 (2019).
[Crossref]

Li, P.

B. Sheng, P. Li, Y. Jin, P. Tan, and T. Y. Lee, “Intrinsic image decomposition with step and drift shading separation,” IEEE Trans. Vis. Comput. Graph. 26, 1332–1346 (2020).
[Crossref]

Li, X.

X. Li, W. Liang, X. Zhang, S. Qing, and P. C. Chang, “A cluster validity evaluation method for dynamically determining the near-optimal number of clusters,” Soft Comput. 24, 9227–9241 (2020).
[Crossref]

J. Shen, X. Yang, X. Li, and Y. Jia, “Intrinsic image decomposition using optimization and user scribbles,” IEEE Trans. Cyber. 43, 425–436 (2013).
[Crossref]

J. Wang, X. Li, L. Hui, and J. Yang, “Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal,” in IEEE Conference on Computer Vision and Pattern Recognition (2018).

Li, Y.

Y. Liu, Y. Li, S. You, and F. Lu, “Unsupervised learning for intrinsic image decomposition from a single image,” in IEEE Conference on Computer Vision and Pattern Recognition (2020).

Y. Li and M. S. Brown, “Single image layer separation using relative smoothness,” in IEEE Conference on Computer Vision and Pattern Recognition (2014).

Li, Z.

Z. Li and N. Snavely, “Learning intrinsic image decomposition from watching the world,” in IEEE Conference on Computer Vision and Pattern Recognition (2018).

Z. Li and N. Snavely, “CGIntrinsics: better intrinsic image decomposition through physically-based rendering,” in European Conference on Computer Vision (2018).

Liang, W.

X. Li, W. Liang, X. Zhang, S. Qing, and P. C. Chang, “A cluster validity evaluation method for dynamically determining the near-optimal number of clusters,” Soft Comput. 24, 9227–9241 (2020).
[Crossref]

Lin, S.

Q. Zhao, P. Tan, Q. Dai, L. Shen, E. Wu, and S. Lin, “A closed-form solution to retinex with non-local texture constraints,” IEEE Trans. Pattern Anal. Mach. Intell. 34, 1437–1444 (2012).
[Crossref]

L. Shen, P. Tan, and S. Lin, “Intrinsic image decomposition with non-local texture cues,” in IEEE Conference on Computer Vision and Pattern Recognition (2008).

K. J. Lee, Q. Zhao, X. Tong, M. Gong, S. Izadi, S. U. Lee, P. Tan, and S. Lin, “Estimation of intrinsic image sequences from image+depth video,” in European Conference on Computer Vision (2012).

Liu, L.

J. Xu, Y. Hou, D. Ren, L. Liu, F. Zhu, M. Yu, H. Wang, and L. Shao, “Star: a structure and texture aware retinex model,” IEEE Trans. Image Process. 29, 5022–5037 (2020).
[Crossref]

Liu, Y.

Y. Liu, Y. Li, S. You, and F. Lu, “Unsupervised learning for intrinsic image decomposition from a single image,” in IEEE Conference on Computer Vision and Pattern Recognition (2020).

Lopez-Moreno, J.

E. Garces, A. Munoz, J. Lopez-Moreno, and D. Gutierrez, “Intrinsic images by clustering,” in Computer Graphics Forum (2012).

Lu, F.

Y. Liu, Y. Li, S. You, and F. Lu, “Unsupervised learning for intrinsic image decomposition from a single image,” in IEEE Conference on Computer Vision and Pattern Recognition (2020).

Ma, L.

S. Ding, B. Sheng, Z. Xie, and L. Ma, “Intrinsic image estimation using near-l0 sparse optimization,” Visual Comput. 33, 355–369 (2017).
[Crossref]

Maire, M.

T. Narihira, M. Maire, and S. X. Yu, “Direct intrinsics: learning albedo-shading decomposition by convolutional regression,” in IEEE International Conference on Computer Vision (2015).

Malik, J.

J. T. Barron and J. Malik, “Shape, illumination, and reflectance from shading,” IEEE Trans. Pattern Anal. Mach. Intell. 37, 1670–1687 (2015).
[Crossref]

Marik, R.

J. Matas, R. Marik, and J. Kittler, “On representation and matching of multi-coloured objects,” in IEEE International Conference on Computer Vision (1995).

Marzani, F.

A. Krebs, Y. Benezeth, and F. Marzani, “Intrinsic RGB and multispectral images recovery by independent quadratic programming,” PeerJ Comput. Sci. 6, e256 (2020).
[Crossref]

Matas, J.

J. Matas, R. Marik, and J. Kittler, “On representation and matching of multi-coloured objects,” in IEEE International Conference on Computer Vision (1995).

McCann, J. J.

Munoz, A.

E. Garces, A. Munoz, J. Lopez-Moreno, and D. Gutierrez, “Intrinsic images by clustering,” in Computer Graphics Forum (2012).

Narihira, T.

T. Narihira, M. Maire, and S. X. Yu, “Direct intrinsics: learning albedo-shading decomposition by convolutional regression,” in IEEE International Conference on Computer Vision (2015).

Nayar, S. K.

S. K. Nayar and R. M. Bolle, “Reflectance based object recognition,” Int. J. Comput. Vis. 17, 219–240 (1996).
[Crossref]

Nestmeyer, T.

T. Nestmeyer and P. V. Gehler, “Reflectance adaptive filtering improves intrinsic image estimation,” in IEEE Conference on Computer Vision and Pattern Recognition (2017).

Paris, S.

A. Bousseau, S. Paris, and F. Durand, “User-assisted intrinsic images,” in ACM SIGGRAPH Asia 2009 (2009), paper 130.

Qing, S.

X. Li, W. Liang, X. Zhang, S. Qing, and P. C. Chang, “A cluster validity evaluation method for dynamically determining the near-optimal number of clusters,” Soft Comput. 24, 9227–9241 (2020).
[Crossref]

Qu, L.

L. Qu, J. Tian, S. He, Y. Tang, and R. W. H. Lau, “Deshadownet: a multi-context embedding deep network for shadow removal,” in IEEE Conference on Computer Vision and Pattern Recognition (2017).

Ren, D.

J. Xu, Y. Hou, D. Ren, L. Liu, F. Zhu, M. Yu, H. Wang, and L. Shao, “Star: a structure and texture aware retinex model,” IEEE Trans. Image Process. 29, 5022–5037 (2020).
[Crossref]

Rother, C.

P. V. Gehler, C. Rother, M. Kiefel, L. Zhang, and B. Schölkopf, “Recovering intrinsic images with a global sparsity prior on reflectance,” in Advances in Neural Information Processing Systems (2011).

Sato, I.

Z. Cheng, Y. Zheng, S. You, and I. Sato, “Non-local intrinsic decomposition with near-infrared priors,” in IEEE International Conference on Computer Vision (2019).

Schofield, A. J.

X. Jiang, A. J. Schofield, and J. L. Wyatt, “Correlation-based intrinsic image extraction from a single image,” in European Conference on Computer Vision (2010).

Schölkopf, B.

P. V. Gehler, C. Rother, M. Kiefel, L. Zhang, and B. Schölkopf, “Recovering intrinsic images with a global sparsity prior on reflectance,” in Advances in Neural Information Processing Systems (2011).

Shafer, S.

S. Shafer, “Using color to separate reflection components,” Color Res. Appl. 10, 210–218 (1985).
[Crossref]

Shao, L.

J. Xu, Y. Hou, D. Ren, L. Liu, F. Zhu, M. Yu, H. Wang, and L. Shao, “Star: a structure and texture aware retinex model,” IEEE Trans. Image Process. 29, 5022–5037 (2020).
[Crossref]

Shen, J.

J. Shen, X. Yang, X. Li, and Y. Jia, “Intrinsic image decomposition using optimization and user scribbles,” IEEE Trans. Cyber. 43, 425–436 (2013).
[Crossref]

Shen, L.

Q. Zhao, P. Tan, Q. Dai, L. Shen, E. Wu, and S. Lin, “A closed-form solution to retinex with non-local texture constraints,” IEEE Trans. Pattern Anal. Mach. Intell. 34, 1437–1444 (2012).
[Crossref]

L. Shen and C. Yeo, “Intrinsic images decomposition using a local and global sparse representation of reflectance,” in IEEE Conference on Computer Vision and Pattern Recognition (2011).

L. Shen, P. Tan, and S. Lin, “Intrinsic image decomposition with non-local texture cues,” in IEEE Conference on Computer Vision and Pattern Recognition (2008).

Sheng, B.

B. Sheng, P. Li, Y. Jin, P. Tan, and T. Y. Lee, “Intrinsic image decomposition with step and drift shading separation,” IEEE Trans. Vis. Comput. Graph. 26, 1332–1346 (2020).
[Crossref]

S. Ding, B. Sheng, Z. Xie, and L. Ma, “Intrinsic image estimation using near-l0 sparse optimization,” Visual Comput. 33, 355–369 (2017).
[Crossref]

Shi, J.

J. Shi, Y. Dong, H. Su, and S. X. Yu, “Learning non-Lambertian object intrinsics across shapenet categories,” in IEEE Conference on Computer Vision and Pattern Recognition (2017).

Sial, H. A.

Smeulders, A.

T. Gevers and A. Smeulders, “Object recognition based on photometric color invariants,” in Scandinavian Conference on Image Analysis (1997).

T. Gevers and A. Smeulders, “Color constant ratio gradients for image segmentation and similarity of texture objects,” in IEEE Conference on Computer Vision and Pattern Recognition (2001).

Snavely, N.

S. Bell, K. Bala, and N. Snavely, “Intrinsic images in the wild,” ACM Trans. Graph. 33, 159 (2014).
[Crossref]

Z. Li and N. Snavely, “Learning intrinsic image decomposition from watching the world,” in IEEE Conference on Computer Vision and Pattern Recognition (2018).

Z. Li and N. Snavely, “CGIntrinsics: better intrinsic image decomposition through physically-based rendering,” in European Conference on Computer Vision (2018).

Su, H.

J. Shi, Y. Dong, H. Su, and S. X. Yu, “Learning non-Lambertian object intrinsics across shapenet categories,” in IEEE Conference on Computer Vision and Pattern Recognition (2017).

Tan, P.

B. Sheng, P. Li, Y. Jin, P. Tan, and T. Y. Lee, “Intrinsic image decomposition with step and drift shading separation,” IEEE Trans. Vis. Comput. Graph. 26, 1332–1346 (2020).
[Crossref]

Q. Zhao, P. Tan, Q. Dai, L. Shen, E. Wu, and S. Lin, “A closed-form solution to retinex with non-local texture constraints,” IEEE Trans. Pattern Anal. Mach. Intell. 34, 1437–1444 (2012).
[Crossref]

L. Shen, P. Tan, and S. Lin, “Intrinsic image decomposition with non-local texture cues,” in IEEE Conference on Computer Vision and Pattern Recognition (2008).

K. J. Lee, Q. Zhao, X. Tong, M. Gong, S. Izadi, S. U. Lee, P. Tan, and S. Lin, “Estimation of intrinsic image sequences from image+depth video,” in European Conference on Computer Vision (2012).

Tang, Y.

L. Qu, J. Tian, S. He, Y. Tang, and R. W. H. Lau, “Deshadownet: a multi-context embedding deep network for shadow removal,” in IEEE Conference on Computer Vision and Pattern Recognition (2017).

Tappen, M. F.

M. F. Tappen, W. T. Freeman, and E. H. Adelson, “Recovering intrinsic images from a single image,” IEEE Trans. Pattern Anal. Mach. Intell. 27, 1459–1472 (2005).
[Crossref]

Tenenbaum, J. M.

H. G. Barrow and J. M. Tenenbaum, “Recovering intrinsic scene characteristics from images,” in Computer Vision Systems (1978), pp. 3–26.

Tian, J.

L. Qu, J. Tian, S. He, Y. Tang, and R. W. H. Lau, “Deshadownet: a multi-context embedding deep network for shadow removal,” in IEEE Conference on Computer Vision and Pattern Recognition (2017).

Tong, X.

K. J. Lee, Q. Zhao, X. Tong, M. Gong, S. Izadi, S. U. Lee, P. Tan, and S. Lin, “Estimation of intrinsic image sequences from image+depth video,” in European Conference on Computer Vision (2012).

J. Jeon, S. Cho, X. Tong, and S. Lee, “Intrinsic image decomposition using structure-texture separation and surface normals,” in European Conference on Computer Vision (2016).

van de Weijer, J.

S. Beigpour and J. van de Weijer, “Object recoloring based on intrinsic image estimation,” in IEEE International Conference on Computer Vision (2011).

van Gool, L.

L. Lettry, K. Vanhoey, and L. van Gool, “DARN: a deep adversarial residual network for intrinsic image decomposition,” in IEEE Winter Conference on Applications of Computer Vision (2018).

Vanhoey, K.

L. Lettry, K. Vanhoey, and L. van Gool, “DARN: a deep adversarial residual network for intrinsic image decomposition,” in IEEE Winter Conference on Applications of Computer Vision (2018).

Vanrell, M.

Wang, H.

J. Xu, Y. Hou, D. Ren, L. Liu, F. Zhu, M. Yu, H. Wang, and L. Shao, “Star: a structure and texture aware retinex model,” IEEE Trans. Image Process. 29, 5022–5037 (2020).
[Crossref]

Wang, J.

J. Wang, X. Li, L. Hui, and J. Yang, “Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal,” in IEEE Conference on Computer Vision and Pattern Recognition (2018).

Weiss, Y.

Y. Weiss, “Deriving intrinsic images from image sequences,” in IEEE International Conference on Computer Vision (2001).

Wu, E.

Q. Zhao, P. Tan, Q. Dai, L. Shen, E. Wu, and S. Lin, “A closed-form solution to retinex with non-local texture constraints,” IEEE Trans. Pattern Anal. Mach. Intell. 34, 1437–1444 (2012).
[Crossref]

Wu, L.

W. Gong, W. Xu, L. Wu, X. Xie, and Z. Cheng, “Intrinsic image sequence decomposition using low-rank sparse model,” IEEE Access 7, 4024–4030 (2019).
[Crossref]

Wyatt, J. L.

X. Jiang, A. J. Schofield, and J. L. Wyatt, “Correlation-based intrinsic image extraction from a single image,” in European Conference on Computer Vision (2010).

Xie, X.

W. Gong, W. Xu, L. Wu, X. Xie, and Z. Cheng, “Intrinsic image sequence decomposition using low-rank sparse model,” IEEE Access 7, 4024–4030 (2019).
[Crossref]

Xie, Z.

S. Ding, B. Sheng, Z. Xie, and L. Ma, “Intrinsic image estimation using near-l0 sparse optimization,” Visual Comput. 33, 355–369 (2017).
[Crossref]

Xu, C.

C. Xu, Y. Han, G. Baciu, and M. Li, “Fabric image recolorization based on intrinsic image decomposition,” Text. Res. J. 89, 3617–3631 (2019).
[Crossref]

Xu, J.

J. Xu, Y. Hou, D. Ren, L. Liu, F. Zhu, M. Yu, H. Wang, and L. Shao, “Star: a structure and texture aware retinex model,” IEEE Trans. Image Process. 29, 5022–5037 (2020).
[Crossref]

Xu, W.

W. Gong, W. Xu, L. Wu, X. Xie, and Z. Cheng, “Intrinsic image sequence decomposition using low-rank sparse model,” IEEE Access 7, 4024–4030 (2019).
[Crossref]

Yan, M.

M. Yan, “Methods of determining the number of clusters in a data set and a new clustering criterion,” Ph.D. thesis (Virginia Tech, 2005).

Yang, J.

J. Wang, X. Li, L. Hui, and J. Yang, “Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal,” in IEEE Conference on Computer Vision and Pattern Recognition (2018).

Yang, X.

J. Shen, X. Yang, X. Li, and Y. Jia, “Intrinsic image decomposition using optimization and user scribbles,” IEEE Trans. Cyber. 43, 425–436 (2013).
[Crossref]

Yeo, C.

L. Shen and C. Yeo, “Intrinsic images decomposition using a local and global sparse representation of reflectance,” in IEEE Conference on Computer Vision and Pattern Recognition (2011).

You, S.

Z. Cheng, Y. Zheng, S. You, and I. Sato, “Non-local intrinsic decomposition with near-infrared priors,” in IEEE International Conference on Computer Vision (2019).

Y. Liu, Y. Li, S. You, and F. Lu, “Unsupervised learning for intrinsic image decomposition from a single image,” in IEEE Conference on Computer Vision and Pattern Recognition (2020).

Yu, M.

J. Xu, Y. Hou, D. Ren, L. Liu, F. Zhu, M. Yu, H. Wang, and L. Shao, “Star: a structure and texture aware retinex model,” IEEE Trans. Image Process. 29, 5022–5037 (2020).
[Crossref]

Yu, S. X.

T. Narihira, M. Maire, and S. X. Yu, “Direct intrinsics: learning albedo-shading decomposition by convolutional regression,” in IEEE International Conference on Computer Vision (2015).

J. Shi, Y. Dong, H. Su, and S. X. Yu, “Learning non-Lambertian object intrinsics across shapenet categories,” in IEEE Conference on Computer Vision and Pattern Recognition (2017).

Yu, Y.

S. Bi, X. Han, and Y. Yu, “An L1 image transform for edge-preserving smoothing and scene-level intrinsic decomposition,” ACM Trans. Graph. 34, 1 (2015).
[Crossref]

Zhang, L.

P. V. Gehler, C. Rother, M. Kiefel, L. Zhang, and B. Schölkopf, “Recovering intrinsic images with a global sparsity prior on reflectance,” in Advances in Neural Information Processing Systems (2011).

Zhang, X.

X. Li, W. Liang, X. Zhang, S. Qing, and P. C. Chang, “A cluster validity evaluation method for dynamically determining the near-optimal number of clusters,” Soft Comput. 24, 9227–9241 (2020).
[Crossref]

Zhao, Q.

Q. Zhao, P. Tan, Q. Dai, L. Shen, E. Wu, and S. Lin, “A closed-form solution to retinex with non-local texture constraints,” IEEE Trans. Pattern Anal. Mach. Intell. 34, 1437–1444 (2012).
[Crossref]

K. J. Lee, Q. Zhao, X. Tong, M. Gong, S. Izadi, S. U. Lee, P. Tan, and S. Lin, “Estimation of intrinsic image sequences from image+depth video,” in European Conference on Computer Vision (2012).

Zheng, Y.

Z. Cheng, Y. Zheng, S. You, and I. Sato, “Non-local intrinsic decomposition with near-infrared priors,” in IEEE International Conference on Computer Vision (2019).

Zhu, F.

J. Xu, Y. Hou, D. Ren, L. Liu, F. Zhu, M. Yu, H. Wang, and L. Shao, “Star: a structure and texture aware retinex model,” IEEE Trans. Image Process. 29, 5022–5037 (2020).
[Crossref]

ACM Trans. Graph. (2)

S. Bell, K. Bala, and N. Snavely, “Intrinsic images in the wild,” ACM Trans. Graph. 33, 159 (2014).
[Crossref]

S. Bi, X. Han, and Y. Yu, “An L1 image transform for edge-preserving smoothing and scene-level intrinsic decomposition,” ACM Trans. Graph. 34, 1 (2015).
[Crossref]

Color Res. Appl. (1)

S. Shafer, “Using color to separate reflection components,” Color Res. Appl. 10, 210–218 (1985).
[Crossref]

IEEE Access (1)

W. Gong, W. Xu, L. Wu, X. Xie, and Z. Cheng, “Intrinsic image sequence decomposition using low-rank sparse model,” IEEE Access 7, 4024–4030 (2019).
[Crossref]

IEEE Trans. Cyber. (1)

J. Shen, X. Yang, X. Li, and Y. Jia, “Intrinsic image decomposition using optimization and user scribbles,” IEEE Trans. Cyber. 43, 425–436 (2013).
[Crossref]

IEEE Trans. Image Process. (1)

J. Xu, Y. Hou, D. Ren, L. Liu, F. Zhu, M. Yu, H. Wang, and L. Shao, “Star: a structure and texture aware retinex model,” IEEE Trans. Image Process. 29, 5022–5037 (2020).
[Crossref]

IEEE Trans. Pattern Anal. Mach. Intell. (3)

M. F. Tappen, W. T. Freeman, and E. H. Adelson, “Recovering intrinsic images from a single image,” IEEE Trans. Pattern Anal. Mach. Intell. 27, 1459–1472 (2005).
[Crossref]

J. T. Barron and J. Malik, “Shape, illumination, and reflectance from shading,” IEEE Trans. Pattern Anal. Mach. Intell. 37, 1670–1687 (2015).
[Crossref]

Q. Zhao, P. Tan, Q. Dai, L. Shen, E. Wu, and S. Lin, “A closed-form solution to retinex with non-local texture constraints,” IEEE Trans. Pattern Anal. Mach. Intell. 34, 1437–1444 (2012).
[Crossref]

IEEE Trans. Vis. Comput. Graph. (1)

B. Sheng, P. Li, Y. Jin, P. Tan, and T. Y. Lee, “Intrinsic image decomposition with step and drift shading separation,” IEEE Trans. Vis. Comput. Graph. 26, 1332–1346 (2020).
[Crossref]

Int. J. Comput. Vis. (1)

S. K. Nayar and R. M. Bolle, “Reflectance based object recognition,” Int. J. Comput. Vis. 17, 219–240 (1996).
[Crossref]

J. Opt. Soc. Am. (1)

J. Opt. Soc. Am. A (2)

PeerJ Comput. Sci. (1)

A. Krebs, Y. Benezeth, and F. Marzani, “Intrinsic RGB and multispectral images recovery by independent quadratic programming,” PeerJ Comput. Sci. 6, e256 (2020).
[Crossref]

Soft Comput. (1)

X. Li, W. Liang, X. Zhang, S. Qing, and P. C. Chang, “A cluster validity evaluation method for dynamically determining the near-optimal number of clusters,” Soft Comput. 24, 9227–9241 (2020).
[Crossref]

Text. Res. J. (1)

C. Xu, Y. Han, G. Baciu, and M. Li, “Fabric image recolorization based on intrinsic image decomposition,” Text. Res. J. 89, 3617–3631 (2019).
[Crossref]

Visual Comput. (1)

S. Ding, B. Sheng, Z. Xie, and L. Ma, “Intrinsic image estimation using near-l0 sparse optimization,” Visual Comput. 33, 355–369 (2017).
[Crossref]

Other (34)

Y. Li and M. S. Brown, “Single image layer separation using relative smoothness,” in IEEE Conference on Computer Vision and Pattern Recognition (2014).

E. Garces, A. Munoz, J. Lopez-Moreno, and D. Gutierrez, “Intrinsic images by clustering,” in Computer Graphics Forum (2012).

X. Jiang, A. J. Schofield, and J. L. Wyatt, “Correlation-based intrinsic image extraction from a single image,” in European Conference on Computer Vision (2010).

G. D. Finlayson, “Colour object recognition,” master’s thesis (Simon Fraser University, 1992).

L. Shen, P. Tan, and S. Lin, “Intrinsic image decomposition with non-local texture cues,” in IEEE Conference on Computer Vision and Pattern Recognition (2008).

S. Beigpour and J. van de Weijer, “Object recoloring based on intrinsic image estimation,” in IEEE International Conference on Computer Vision (2011).

H. G. Barrow and J. M. Tenenbaum, “Recovering intrinsic scene characteristics from images,” in Computer Vision Systems (1978), pp. 3–26.

A. S. Baslamisli, T. T. Groenestege, P. Das, H. A. Le, S. Karaoglu, and T. Gevers, “Joint learning of intrinsic images and semantic segmentation,” in European Conference on Computer Vision (2018).

Z. Cheng, Y. Zheng, S. You, and I. Sato, “Non-local intrinsic decomposition with near-infrared priors,” in IEEE International Conference on Computer Vision (2019).

T. Narihira, M. Maire, and S. X. Yu, “Direct intrinsics: learning albedo-shading decomposition by convolutional regression,” in IEEE International Conference on Computer Vision (2015).

A. S. Baslamisli, H. A. Le, and T. Gevers, “CNN based learning using reflection and retinex models for intrinsic image decomposition,” in IEEE Conference on Computer Vision and Pattern Recognition (2018).

P. V. Gehler, C. Rother, M. Kiefel, L. Zhang, and B. Schölkopf, “Recovering intrinsic images with a global sparsity prior on reflectance,” in Advances in Neural Information Processing Systems (2011).

L. Shen and C. Yeo, “Intrinsic images decomposition using a local and global sparse representation of reflectance,” in IEEE Conference on Computer Vision and Pattern Recognition (2011).

R. Grosse, M. K. Johnson, E. H. Adelson, and W. T. Freeman, “Ground truth dataset and baseline evaluations for intrinsic image algorithms,” in IEEE International Conference on Computer Vision (2009).

J. Wang, X. Li, L. Hui, and J. Yang, “Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal,” in IEEE Conference on Computer Vision and Pattern Recognition (2018).

L. Qu, J. Tian, S. He, Y. Tang, and R. W. H. Lau, “Deshadownet: a multi-context embedding deep network for shadow removal,” in IEEE Conference on Computer Vision and Pattern Recognition (2017).

A. Bousseau, S. Paris, and F. Durand, “User-assisted intrinsic images,” in ACM SIGGRAPH Asia 2009 (2009), paper 130.

Q. Chen and V. Koltun, “A simple model for intrinsic image decomposition with depth cues,” in IEEE International Conference on Computer Vision (2013).

J. Jeon, S. Cho, X. Tong, and S. Lee, “Intrinsic image decomposition using structure-texture separation and surface normals,” in European Conference on Computer Vision (2016).

K. J. Lee, Q. Zhao, X. Tong, M. Gong, S. Izadi, S. U. Lee, P. Tan, and S. Lin, “Estimation of intrinsic image sequences from image+depth video,” in European Conference on Computer Vision (2012).

Y. Weiss, “Deriving intrinsic images from image sequences,” in IEEE International Conference on Computer Vision (2001).

K. Barnard and G. D. Finlayson, “Shadow identification using colour ratios,” in Color and Imaging Conference (2000).

T. Gevers and A. Smeulders, “Color constant ratio gradients for image segmentation and similarity of texture objects,” in IEEE Conference on Computer Vision and Pattern Recognition (2001).

T. Gevers and A. Smeulders, “Object recognition based on photometric color invariants,” in Scandinavian Conference on Image Analysis (1997).

J. Shi, Y. Dong, H. Su, and S. X. Yu, “Learning non-Lambertian object intrinsics across shapenet categories,” in IEEE Conference on Computer Vision and Pattern Recognition (2017).

Z. Li and N. Snavely, “Learning intrinsic image decomposition from watching the world,” in IEEE Conference on Computer Vision and Pattern Recognition (2018).

L. Lettry, K. Vanhoey, and L. van Gool, “DARN: a deep adversarial residual network for intrinsic image decomposition,” in IEEE Winter Conference on Applications of Computer Vision (2018).

A. S. Baslamisli, P. Das, H. A. Le, S. Karaoglu, and T. Gevers, “ShadingNet: image intrinsics by fine-grained shading decomposition,” arXiv:1912.04023 (2019).

T. Nestmeyer and P. V. Gehler, “Reflectance adaptive filtering improves intrinsic image estimation,” in IEEE Conference on Computer Vision and Pattern Recognition (2017).

P. Y. Laffont and J. C. Bazin, “Intrinsic decomposition of image sequences from local temporal variations,” in IEEE International Conference on Computer Vision (2015).

J. Matas, R. Marik, and J. Kittler, “On representation and matching of multi-coloured objects,” in IEEE International Conference on Computer Vision (1995).

M. Yan, “Methods of determining the number of clusters in a data set and a new clustering criterion,” Ph.D. thesis (Virginia Tech, 2005).

Y. Liu, Y. Li, S. You, and F. Lu, “Unsupervised learning for intrinsic image decomposition from a single image,” in IEEE Conference on Computer Vision and Pattern Recognition (2020).

Z. Li and N. Snavely, “CGIntrinsics: better intrinsic image decomposition through physically-based rendering,” in European Conference on Computer Vision (2018).

Data Availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (7)

Fig. 1.
Fig. 1. Ill-posed nature of the problem. The left part is an incorrect intrinsic image decomposition, whereas the right part presents the ground-truth one. Both achieve the same input image through ${{R}} \times {{S}} = {{I}}$ .
Fig. 2.
Fig. 2. Effect of the ratio driven clustering. Default model extra clusters shadows and strong shadings as reflectance. Adaptive setting $k$ by the ratios makes the model more accurate and robust to photometric effects such as strong shading and shadows.
Fig. 3.
Fig. 3. Reflectance evaluations for IIW dataset. Problematic parts are marked with green bounding boxes. The final model further handles discontinuities in the reflectance. It becomes more robust to direct light effects and also to specular highlights.
Fig. 4.
Fig. 4. Additional effect of the cross color ratios as a pairwise term. The full model has a significant degree of photometric invariance, capable of handling strong shadows.
Fig. 5.
Fig. 5. Default model completely fails handling shadow casts. The final proposed model driven by the photometric invariant color ratios is more robust to natural outdoor real world shadow handling. It can now differentiate drastic changes in pixel values and attribute them to the related intrinsics.
Fig. 6.
Fig. 6. State-of-the-art comparisons on shadow cast handling on ISTD [43]. Both unsupervised and supervised models fail to handle shadows. Our method driven by the photometric invariant color ratios is more robust, containing significantly fewer shadow cues and fewer shadow artifacts; the colors are more vivid and realistic, and the structures are well preserved. Images are best viewed in color and on the electronic version.
Fig. 7.
Fig. 7. State-of-the-art comparisons on shadow cast handling on SRD [44]. Compared with others, our albedo estimations contain very little or almost no shadow artifacts, the colors are more vivid and realistic, and the structures are well preserved. Images are best viewed in color and on the electronic version.

Tables (4)

Tables Icon

Table 1. Combination of Color Retinex and Cross Color Ratios (CCR) a

Tables Icon

Table 2. Effect of k for the K-Means Algorithm for the MIT Dataset a

Tables Icon

Table 3. Effect of k for the K-Means Algorithm for the IIW Dataset a , b

Tables Icon

Table 4. Effect of the Additional Pairwise Term for the MIT Dataset a

Equations (10)

Equations on this page are rendered with MathJax. Learn more.

I = R × S .
I c = m ( n , l ) ω f c ( λ ) e ( λ ) s ( λ ) d λ .
I c = m ( n , l ) e ( λ c ) s ( λ c ) = m ( n , l ) e c s c .
I c = S c × R c , S c = m ( n , l ) e c , R c = s c ,
F 1 = R x 1 R x 2 , F 2 = G x 1 G x 2 , F 3 = B x 1 B x 2 .
F 1 = ( m ( n , l ) ) x 1 e R x 1 s R x 1 ( m ( n , l ) ) x 2 e R x 2 s R x 2 .
M 1 = R x 1 G x 2 R x 2 G x 1 , M 2 = R x 1 B x 2 R x 2 B x 1 , M 3 = G x 1 B x 2 G x 2 B x 1 .
M 1 = ( m ( n , l ) ) x 1 e R x 1 s R x 1 ( m ( n , l ) ) x 2 e G x 2 s G x 2 ( m ( n , l ) ) x 2 e R x 2 s R x 2 ( m ( n , l ) ) x 1 e G x 1 s G x 1 = s R x 1 s G x 2 s R x 2 s G x 1 .
R , S = arg max R , S p ( R , S | I ) .
E ( x ) = ω p E p ( x ) + ω s E s ( x ) + ω l E l ( x ) ,