Abstract

Digital projectors have been increasingly utilized in various commercial and scientific applications. However, they are prone to the out-of-focus blurring problem since their depth-of-fields are typically limited. In this paper, we explore the feasibility of utilizing a deep learning-based approach to analyze the spatially-varying and depth-dependent defocus properties of digital projectors. A multimodal displaying/imaging system is built for capturing images projected at various depths. Based on the constructed dataset containing well-aligned in-focus, out-of-focus, and depth images, we propose a novel multi-channel residual deep network model to learn the end-to-end mapping function between the in-focus and out-of-focus image patches captured at different spatial locations and depths. To the best of our knowledge, it is the first research work revealing that the complex spatially-varying and depth-dependent blurring effects can be accurately learned from a number of real-captured image pairs instead of being hand-crafted as before. Experimental results demonstrate that our proposed deep learning-based method significantly outperforms the state-of-the-art defocus kernel estimation techniques and thus leads to better out-of-focus compensation for extending the dynamic ranges of digital projectors.

© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

Full Article  |  PDF Article
OSA Recommended Articles
Learned phase coded aperture for the benefit of depth of field extension

Shay Elmalem, Raja Giryes, and Emanuel Marom
Opt. Express 26(12) 15316-15331 (2018)

Passive depth estimation using chromatic aberration and a depth from defocus approach

Pauline Trouvé, Frédéric Champagnat, Guy Le Besnerais, Jacques Sabater, Thierry Avignon, and Jérôme Idier
Appl. Opt. 52(29) 7152-7164 (2013)

Depth-based defocus map estimation using off-axis apertures

Eunsung Lee, Eunjung Chae, Hejin Cheong, Semi Jeon, and Joonki Paik
Opt. Express 23(17) 21958-21971 (2015)

References

  • View by:
  • |
  • |
  • |

  1. Y. Wang, H. Zhao, H. Jiang, and X. Li, “Defocusing parameter selection strategies based on PSF measurement for square-binary defocusing fringe projection profilometry,” Opt. Express 26(16), 20351–20367 (2018).
    [Crossref]
  2. T. Hoang, B. Pan, D. Nguyen, and Z. Wang, “Generic gamma correction for accuracy enhancement in fringe-projection profilometry,” Opt. Lett. 35(12), 1992–1994 (2010).
    [Crossref]
  3. Z. Wang, H. Du, and H. Bi, “Out-of-plane shape determination in generalized fringe projection profilometry,” Opt. Express 14(25), 12122–12133 (2006).
    [Crossref]
  4. A. Doshi, R. T. Smith, B. H. Thomas, and C. Bouras, “Use of projector based augmented reality to improve manual spot-welding precision and accuracy for automotive manufacturing,” The Int. J. Adv. Manuf. Technol. 89(5-8), 1279–1293 (2017).
    [Crossref]
  5. A. E. Uva, M. Gattullo, V. M. Manghisi, D. Spagnulo, G. L. Cascella, and M. Fiorentino, “Evaluating the effectiveness of spatial augmented reality in smart manufacturing: a solution for manual working stations,” The Int. J. Adv. Manuf. Technol. 94(1-4), 509–521 (2018).
    [Crossref]
  6. M. Di Donato, M. Fiorentino, A. E. Uva, M. Gattullo, and G. Monno, “Text legibility for projected augmented reality on industrial workbenches,” Comput. Ind. 70(1), 70–78 (2015).
    [Crossref]
  7. M. S. Brown, P. Song, and T.-J. Cham, “Image pre-conditioning for out-of-focus projector blur," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2006), pp. 1956–1963.
  8. L. Zhang and S. Nayar, “Projection defocus analysis for scene capture and image display,” ACM Trans. Graph. 25(3), 907–915 (2006).
    [Crossref]
  9. J. Park and B.-U. Lee, “Defocus and geometric distortion correction for projected images on a curved surface,” Appl. Opt. 55(4), 896–902 (2016).
    [Crossref]
  10. H. Lin, J. Gao, Q. Mei, Y. He, J. Liu, and X. Wang, “Adaptive digital fringe projection technique for high dynamic range three-dimensional shape measurement,” Opt. Express 24(7), 7703–7718 (2016).
    [Crossref]
  11. J. Jurij, P. Franjo, L. Boštjan, and B. Miran, “2D sub-pixel point spread function measurement using a virtual point-like source,” Int. J. Comput. Vis. 121(3), 391–402 (2017).
    [Crossref]
  12. H. Du and K. J. Voss, “Effects of point-spread function on calibration and radiometric accuracy of CCD camera,” Appl. Opt. 43(3), 665–670 (2004).
    [Crossref]
  13. A. Mosleh, P. Green, E. Onzon, I. Begin, and J. M. P. Langlois, “Camera intrinsic blur kernel estimation: A reliable framework," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2015), pp. 4961–4968.
  14. Y. Oyamada and H. Saito, “Focal pre-correction of projected image for deblurring screen image," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2007), pp. 1–8.
  15. E. Kee, S. Paris, S. Chen, and J. Wang, “Modeling and removing spatially-varying optical blur," in Proceedings of the IEEE international conference on computational photography (IEEE, 2011), pp. 1–8.
  16. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2016), pp. 770–778.
  17. K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv:1409.1556 (2014).
  18. E. Shelhamer, J. Long, and T. Darrell, “Fully convolutional networks for semantic segmentation," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2015), pp. 3431–3440.
  19. H. Li, Z. Lin, X. Shen, J. Brandt, and G. Hua, “A convolutional neural network cascade for face detection," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2015), pp. 5325–5334.
  20. S. Ji, W. Xu, M. Yang, and K. Yu, “3D convolutional neural networks for human action recognition,” IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 221–231 (2013).
    [Crossref]
  21. P. Molchanov, S. Gupta, K. Kim, and J. Kautz, “Hand gesture recognition with 3D convolutional neural networks," in Proceedings of the IEEE conference on computer vision and pattern recognition workshops (IEEE, 2015), pp. 1–7.
  22. Q. Dou, H. Chen, L. Yu, L. Zhao, J. Qin, D. Wang, V. C. Mok, L. Shi, and P. Heng, “Automatic detection of cerebral microbleeds from MR images via 3D convolutional neural networks,” IEEE Trans. Med. Imaging 35(5), 1182–1195 (2016).
    [Crossref]
  23. E. Agustsson and R. Timofte, “NTIRE 2017 challenge on single image super-resolution: Dataset and study," in Proceedings of the IEEE international conference on computer vision workshops (IEEE, 2017), pp. 126–135.
  24. D. Moreno and G. Taubin, “Simple, accurate, and robust projector-camera calibration," in Proceedings of the Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission (IEEE, 2012), pp. 464–471.
  25. S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks," in Proceedings of Advances in neural information processing systems (NIPS, 2015), pp. 91–99.
  26. A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks," in Proceedings of Advances in neural information processing systems (NIPS, 2012), pp. 1097–1105.
  27. M. Trimeche, D. Paliy, M. Vehvilainen, and V. Katkovnic, “Multichannel image deblurring of raw color components," in Proceedings of the Computational Imaging III (International Society for Optics and Photonics, 2005), vol. 5674, pp. 169–178.
  28. S. Ladha, K. Smith-Miles, and S. Chandran, “Projection defocus correction using adaptive kernel sampling and geometric correction in dual-planar environments," in Proceedings of the IEEE international conference on computer vision workshops (IEEE, 2011), pp. 9–14.
  29. B. Lim, S. Son, H. Kim, S. Nah, and K. Mu Lee, “Enhanced deep residual networks for single image super-resolution," in Proceedings of the IEEE conference on computer vision and pattern recognition workshops (IEEE, 2017), pp. 136–144.
  30. H. Zhao, O. Gallo, I. Frosio, and J. Kautz, “Loss functions for image restoration with neural networks,” IEEE Trans. Comput. Imaging 3(1), 47–57 (2017).
    [Crossref]
  31. Z. Yunlun, L. Kunpeng, L. Kai, W. Lichen, Z. Bineng, and F. Yun, “Image super-resolution using very deep residual channel attention networks," in Proceedings of the European conference on computer vision (Springer, 2018), pp. 286–301.
  32. K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification," in Proceedings of the IEEE international conference on computer vision (IEEE, 2015), pp. 1026–1034.
  33. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. on Image Process. 13(4), 600–612 (2004).
    [Crossref]

2018 (2)

Y. Wang, H. Zhao, H. Jiang, and X. Li, “Defocusing parameter selection strategies based on PSF measurement for square-binary defocusing fringe projection profilometry,” Opt. Express 26(16), 20351–20367 (2018).
[Crossref]

A. E. Uva, M. Gattullo, V. M. Manghisi, D. Spagnulo, G. L. Cascella, and M. Fiorentino, “Evaluating the effectiveness of spatial augmented reality in smart manufacturing: a solution for manual working stations,” The Int. J. Adv. Manuf. Technol. 94(1-4), 509–521 (2018).
[Crossref]

2017 (3)

J. Jurij, P. Franjo, L. Boštjan, and B. Miran, “2D sub-pixel point spread function measurement using a virtual point-like source,” Int. J. Comput. Vis. 121(3), 391–402 (2017).
[Crossref]

A. Doshi, R. T. Smith, B. H. Thomas, and C. Bouras, “Use of projector based augmented reality to improve manual spot-welding precision and accuracy for automotive manufacturing,” The Int. J. Adv. Manuf. Technol. 89(5-8), 1279–1293 (2017).
[Crossref]

H. Zhao, O. Gallo, I. Frosio, and J. Kautz, “Loss functions for image restoration with neural networks,” IEEE Trans. Comput. Imaging 3(1), 47–57 (2017).
[Crossref]

2016 (3)

2015 (1)

M. Di Donato, M. Fiorentino, A. E. Uva, M. Gattullo, and G. Monno, “Text legibility for projected augmented reality on industrial workbenches,” Comput. Ind. 70(1), 70–78 (2015).
[Crossref]

2013 (1)

S. Ji, W. Xu, M. Yang, and K. Yu, “3D convolutional neural networks for human action recognition,” IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 221–231 (2013).
[Crossref]

2010 (1)

2006 (2)

Z. Wang, H. Du, and H. Bi, “Out-of-plane shape determination in generalized fringe projection profilometry,” Opt. Express 14(25), 12122–12133 (2006).
[Crossref]

L. Zhang and S. Nayar, “Projection defocus analysis for scene capture and image display,” ACM Trans. Graph. 25(3), 907–915 (2006).
[Crossref]

2004 (2)

H. Du and K. J. Voss, “Effects of point-spread function on calibration and radiometric accuracy of CCD camera,” Appl. Opt. 43(3), 665–670 (2004).
[Crossref]

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. on Image Process. 13(4), 600–612 (2004).
[Crossref]

Agustsson, E.

E. Agustsson and R. Timofte, “NTIRE 2017 challenge on single image super-resolution: Dataset and study," in Proceedings of the IEEE international conference on computer vision workshops (IEEE, 2017), pp. 126–135.

Begin, I.

A. Mosleh, P. Green, E. Onzon, I. Begin, and J. M. P. Langlois, “Camera intrinsic blur kernel estimation: A reliable framework," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2015), pp. 4961–4968.

Bi, H.

Bineng, Z.

Z. Yunlun, L. Kunpeng, L. Kai, W. Lichen, Z. Bineng, and F. Yun, “Image super-resolution using very deep residual channel attention networks," in Proceedings of the European conference on computer vision (Springer, 2018), pp. 286–301.

Boštjan, L.

J. Jurij, P. Franjo, L. Boštjan, and B. Miran, “2D sub-pixel point spread function measurement using a virtual point-like source,” Int. J. Comput. Vis. 121(3), 391–402 (2017).
[Crossref]

Bouras, C.

A. Doshi, R. T. Smith, B. H. Thomas, and C. Bouras, “Use of projector based augmented reality to improve manual spot-welding precision and accuracy for automotive manufacturing,” The Int. J. Adv. Manuf. Technol. 89(5-8), 1279–1293 (2017).
[Crossref]

Bovik, A. C.

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. on Image Process. 13(4), 600–612 (2004).
[Crossref]

Brandt, J.

H. Li, Z. Lin, X. Shen, J. Brandt, and G. Hua, “A convolutional neural network cascade for face detection," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2015), pp. 5325–5334.

Brown, M. S.

M. S. Brown, P. Song, and T.-J. Cham, “Image pre-conditioning for out-of-focus projector blur," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2006), pp. 1956–1963.

Cascella, G. L.

A. E. Uva, M. Gattullo, V. M. Manghisi, D. Spagnulo, G. L. Cascella, and M. Fiorentino, “Evaluating the effectiveness of spatial augmented reality in smart manufacturing: a solution for manual working stations,” The Int. J. Adv. Manuf. Technol. 94(1-4), 509–521 (2018).
[Crossref]

Cham, T.-J.

M. S. Brown, P. Song, and T.-J. Cham, “Image pre-conditioning for out-of-focus projector blur," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2006), pp. 1956–1963.

Chandran, S.

S. Ladha, K. Smith-Miles, and S. Chandran, “Projection defocus correction using adaptive kernel sampling and geometric correction in dual-planar environments," in Proceedings of the IEEE international conference on computer vision workshops (IEEE, 2011), pp. 9–14.

Chen, H.

Q. Dou, H. Chen, L. Yu, L. Zhao, J. Qin, D. Wang, V. C. Mok, L. Shi, and P. Heng, “Automatic detection of cerebral microbleeds from MR images via 3D convolutional neural networks,” IEEE Trans. Med. Imaging 35(5), 1182–1195 (2016).
[Crossref]

Chen, S.

E. Kee, S. Paris, S. Chen, and J. Wang, “Modeling and removing spatially-varying optical blur," in Proceedings of the IEEE international conference on computational photography (IEEE, 2011), pp. 1–8.

Darrell, T.

E. Shelhamer, J. Long, and T. Darrell, “Fully convolutional networks for semantic segmentation," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2015), pp. 3431–3440.

Di Donato, M.

M. Di Donato, M. Fiorentino, A. E. Uva, M. Gattullo, and G. Monno, “Text legibility for projected augmented reality on industrial workbenches,” Comput. Ind. 70(1), 70–78 (2015).
[Crossref]

Doshi, A.

A. Doshi, R. T. Smith, B. H. Thomas, and C. Bouras, “Use of projector based augmented reality to improve manual spot-welding precision and accuracy for automotive manufacturing,” The Int. J. Adv. Manuf. Technol. 89(5-8), 1279–1293 (2017).
[Crossref]

Dou, Q.

Q. Dou, H. Chen, L. Yu, L. Zhao, J. Qin, D. Wang, V. C. Mok, L. Shi, and P. Heng, “Automatic detection of cerebral microbleeds from MR images via 3D convolutional neural networks,” IEEE Trans. Med. Imaging 35(5), 1182–1195 (2016).
[Crossref]

Du, H.

Fiorentino, M.

A. E. Uva, M. Gattullo, V. M. Manghisi, D. Spagnulo, G. L. Cascella, and M. Fiorentino, “Evaluating the effectiveness of spatial augmented reality in smart manufacturing: a solution for manual working stations,” The Int. J. Adv. Manuf. Technol. 94(1-4), 509–521 (2018).
[Crossref]

M. Di Donato, M. Fiorentino, A. E. Uva, M. Gattullo, and G. Monno, “Text legibility for projected augmented reality on industrial workbenches,” Comput. Ind. 70(1), 70–78 (2015).
[Crossref]

Franjo, P.

J. Jurij, P. Franjo, L. Boštjan, and B. Miran, “2D sub-pixel point spread function measurement using a virtual point-like source,” Int. J. Comput. Vis. 121(3), 391–402 (2017).
[Crossref]

Frosio, I.

H. Zhao, O. Gallo, I. Frosio, and J. Kautz, “Loss functions for image restoration with neural networks,” IEEE Trans. Comput. Imaging 3(1), 47–57 (2017).
[Crossref]

Gallo, O.

H. Zhao, O. Gallo, I. Frosio, and J. Kautz, “Loss functions for image restoration with neural networks,” IEEE Trans. Comput. Imaging 3(1), 47–57 (2017).
[Crossref]

Gao, J.

Gattullo, M.

A. E. Uva, M. Gattullo, V. M. Manghisi, D. Spagnulo, G. L. Cascella, and M. Fiorentino, “Evaluating the effectiveness of spatial augmented reality in smart manufacturing: a solution for manual working stations,” The Int. J. Adv. Manuf. Technol. 94(1-4), 509–521 (2018).
[Crossref]

M. Di Donato, M. Fiorentino, A. E. Uva, M. Gattullo, and G. Monno, “Text legibility for projected augmented reality on industrial workbenches,” Comput. Ind. 70(1), 70–78 (2015).
[Crossref]

Girshick, R.

S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks," in Proceedings of Advances in neural information processing systems (NIPS, 2015), pp. 91–99.

Green, P.

A. Mosleh, P. Green, E. Onzon, I. Begin, and J. M. P. Langlois, “Camera intrinsic blur kernel estimation: A reliable framework," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2015), pp. 4961–4968.

Gupta, S.

P. Molchanov, S. Gupta, K. Kim, and J. Kautz, “Hand gesture recognition with 3D convolutional neural networks," in Proceedings of the IEEE conference on computer vision and pattern recognition workshops (IEEE, 2015), pp. 1–7.

He, K.

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2016), pp. 770–778.

S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks," in Proceedings of Advances in neural information processing systems (NIPS, 2015), pp. 91–99.

K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification," in Proceedings of the IEEE international conference on computer vision (IEEE, 2015), pp. 1026–1034.

He, Y.

Heng, P.

Q. Dou, H. Chen, L. Yu, L. Zhao, J. Qin, D. Wang, V. C. Mok, L. Shi, and P. Heng, “Automatic detection of cerebral microbleeds from MR images via 3D convolutional neural networks,” IEEE Trans. Med. Imaging 35(5), 1182–1195 (2016).
[Crossref]

Hinton, G. E.

A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks," in Proceedings of Advances in neural information processing systems (NIPS, 2012), pp. 1097–1105.

Hoang, T.

Hua, G.

H. Li, Z. Lin, X. Shen, J. Brandt, and G. Hua, “A convolutional neural network cascade for face detection," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2015), pp. 5325–5334.

Ji, S.

S. Ji, W. Xu, M. Yang, and K. Yu, “3D convolutional neural networks for human action recognition,” IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 221–231 (2013).
[Crossref]

Jiang, H.

Jurij, J.

J. Jurij, P. Franjo, L. Boštjan, and B. Miran, “2D sub-pixel point spread function measurement using a virtual point-like source,” Int. J. Comput. Vis. 121(3), 391–402 (2017).
[Crossref]

Kai, L.

Z. Yunlun, L. Kunpeng, L. Kai, W. Lichen, Z. Bineng, and F. Yun, “Image super-resolution using very deep residual channel attention networks," in Proceedings of the European conference on computer vision (Springer, 2018), pp. 286–301.

Katkovnic, V.

M. Trimeche, D. Paliy, M. Vehvilainen, and V. Katkovnic, “Multichannel image deblurring of raw color components," in Proceedings of the Computational Imaging III (International Society for Optics and Photonics, 2005), vol. 5674, pp. 169–178.

Kautz, J.

H. Zhao, O. Gallo, I. Frosio, and J. Kautz, “Loss functions for image restoration with neural networks,” IEEE Trans. Comput. Imaging 3(1), 47–57 (2017).
[Crossref]

P. Molchanov, S. Gupta, K. Kim, and J. Kautz, “Hand gesture recognition with 3D convolutional neural networks," in Proceedings of the IEEE conference on computer vision and pattern recognition workshops (IEEE, 2015), pp. 1–7.

Kee, E.

E. Kee, S. Paris, S. Chen, and J. Wang, “Modeling and removing spatially-varying optical blur," in Proceedings of the IEEE international conference on computational photography (IEEE, 2011), pp. 1–8.

Kim, H.

B. Lim, S. Son, H. Kim, S. Nah, and K. Mu Lee, “Enhanced deep residual networks for single image super-resolution," in Proceedings of the IEEE conference on computer vision and pattern recognition workshops (IEEE, 2017), pp. 136–144.

Kim, K.

P. Molchanov, S. Gupta, K. Kim, and J. Kautz, “Hand gesture recognition with 3D convolutional neural networks," in Proceedings of the IEEE conference on computer vision and pattern recognition workshops (IEEE, 2015), pp. 1–7.

Krizhevsky, A.

A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks," in Proceedings of Advances in neural information processing systems (NIPS, 2012), pp. 1097–1105.

Kunpeng, L.

Z. Yunlun, L. Kunpeng, L. Kai, W. Lichen, Z. Bineng, and F. Yun, “Image super-resolution using very deep residual channel attention networks," in Proceedings of the European conference on computer vision (Springer, 2018), pp. 286–301.

Ladha, S.

S. Ladha, K. Smith-Miles, and S. Chandran, “Projection defocus correction using adaptive kernel sampling and geometric correction in dual-planar environments," in Proceedings of the IEEE international conference on computer vision workshops (IEEE, 2011), pp. 9–14.

Langlois, J. M. P.

A. Mosleh, P. Green, E. Onzon, I. Begin, and J. M. P. Langlois, “Camera intrinsic blur kernel estimation: A reliable framework," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2015), pp. 4961–4968.

Lee, B.-U.

Li, H.

H. Li, Z. Lin, X. Shen, J. Brandt, and G. Hua, “A convolutional neural network cascade for face detection," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2015), pp. 5325–5334.

Li, X.

Lichen, W.

Z. Yunlun, L. Kunpeng, L. Kai, W. Lichen, Z. Bineng, and F. Yun, “Image super-resolution using very deep residual channel attention networks," in Proceedings of the European conference on computer vision (Springer, 2018), pp. 286–301.

Lim, B.

B. Lim, S. Son, H. Kim, S. Nah, and K. Mu Lee, “Enhanced deep residual networks for single image super-resolution," in Proceedings of the IEEE conference on computer vision and pattern recognition workshops (IEEE, 2017), pp. 136–144.

Lin, H.

Lin, Z.

H. Li, Z. Lin, X. Shen, J. Brandt, and G. Hua, “A convolutional neural network cascade for face detection," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2015), pp. 5325–5334.

Liu, J.

Long, J.

E. Shelhamer, J. Long, and T. Darrell, “Fully convolutional networks for semantic segmentation," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2015), pp. 3431–3440.

Manghisi, V. M.

A. E. Uva, M. Gattullo, V. M. Manghisi, D. Spagnulo, G. L. Cascella, and M. Fiorentino, “Evaluating the effectiveness of spatial augmented reality in smart manufacturing: a solution for manual working stations,” The Int. J. Adv. Manuf. Technol. 94(1-4), 509–521 (2018).
[Crossref]

Mei, Q.

Miran, B.

J. Jurij, P. Franjo, L. Boštjan, and B. Miran, “2D sub-pixel point spread function measurement using a virtual point-like source,” Int. J. Comput. Vis. 121(3), 391–402 (2017).
[Crossref]

Mok, V. C.

Q. Dou, H. Chen, L. Yu, L. Zhao, J. Qin, D. Wang, V. C. Mok, L. Shi, and P. Heng, “Automatic detection of cerebral microbleeds from MR images via 3D convolutional neural networks,” IEEE Trans. Med. Imaging 35(5), 1182–1195 (2016).
[Crossref]

Molchanov, P.

P. Molchanov, S. Gupta, K. Kim, and J. Kautz, “Hand gesture recognition with 3D convolutional neural networks," in Proceedings of the IEEE conference on computer vision and pattern recognition workshops (IEEE, 2015), pp. 1–7.

Monno, G.

M. Di Donato, M. Fiorentino, A. E. Uva, M. Gattullo, and G. Monno, “Text legibility for projected augmented reality on industrial workbenches,” Comput. Ind. 70(1), 70–78 (2015).
[Crossref]

Moreno, D.

D. Moreno and G. Taubin, “Simple, accurate, and robust projector-camera calibration," in Proceedings of the Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission (IEEE, 2012), pp. 464–471.

Mosleh, A.

A. Mosleh, P. Green, E. Onzon, I. Begin, and J. M. P. Langlois, “Camera intrinsic blur kernel estimation: A reliable framework," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2015), pp. 4961–4968.

Mu Lee, K.

B. Lim, S. Son, H. Kim, S. Nah, and K. Mu Lee, “Enhanced deep residual networks for single image super-resolution," in Proceedings of the IEEE conference on computer vision and pattern recognition workshops (IEEE, 2017), pp. 136–144.

Nah, S.

B. Lim, S. Son, H. Kim, S. Nah, and K. Mu Lee, “Enhanced deep residual networks for single image super-resolution," in Proceedings of the IEEE conference on computer vision and pattern recognition workshops (IEEE, 2017), pp. 136–144.

Nayar, S.

L. Zhang and S. Nayar, “Projection defocus analysis for scene capture and image display,” ACM Trans. Graph. 25(3), 907–915 (2006).
[Crossref]

Nguyen, D.

Onzon, E.

A. Mosleh, P. Green, E. Onzon, I. Begin, and J. M. P. Langlois, “Camera intrinsic blur kernel estimation: A reliable framework," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2015), pp. 4961–4968.

Oyamada, Y.

Y. Oyamada and H. Saito, “Focal pre-correction of projected image for deblurring screen image," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2007), pp. 1–8.

Paliy, D.

M. Trimeche, D. Paliy, M. Vehvilainen, and V. Katkovnic, “Multichannel image deblurring of raw color components," in Proceedings of the Computational Imaging III (International Society for Optics and Photonics, 2005), vol. 5674, pp. 169–178.

Pan, B.

Paris, S.

E. Kee, S. Paris, S. Chen, and J. Wang, “Modeling and removing spatially-varying optical blur," in Proceedings of the IEEE international conference on computational photography (IEEE, 2011), pp. 1–8.

Park, J.

Qin, J.

Q. Dou, H. Chen, L. Yu, L. Zhao, J. Qin, D. Wang, V. C. Mok, L. Shi, and P. Heng, “Automatic detection of cerebral microbleeds from MR images via 3D convolutional neural networks,” IEEE Trans. Med. Imaging 35(5), 1182–1195 (2016).
[Crossref]

Ren, S.

S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks," in Proceedings of Advances in neural information processing systems (NIPS, 2015), pp. 91–99.

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2016), pp. 770–778.

K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification," in Proceedings of the IEEE international conference on computer vision (IEEE, 2015), pp. 1026–1034.

Saito, H.

Y. Oyamada and H. Saito, “Focal pre-correction of projected image for deblurring screen image," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2007), pp. 1–8.

Sheikh, H. R.

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. on Image Process. 13(4), 600–612 (2004).
[Crossref]

Shelhamer, E.

E. Shelhamer, J. Long, and T. Darrell, “Fully convolutional networks for semantic segmentation," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2015), pp. 3431–3440.

Shen, X.

H. Li, Z. Lin, X. Shen, J. Brandt, and G. Hua, “A convolutional neural network cascade for face detection," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2015), pp. 5325–5334.

Shi, L.

Q. Dou, H. Chen, L. Yu, L. Zhao, J. Qin, D. Wang, V. C. Mok, L. Shi, and P. Heng, “Automatic detection of cerebral microbleeds from MR images via 3D convolutional neural networks,” IEEE Trans. Med. Imaging 35(5), 1182–1195 (2016).
[Crossref]

Simoncelli, E. P.

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. on Image Process. 13(4), 600–612 (2004).
[Crossref]

Simonyan, K.

K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv:1409.1556 (2014).

Smith, R. T.

A. Doshi, R. T. Smith, B. H. Thomas, and C. Bouras, “Use of projector based augmented reality to improve manual spot-welding precision and accuracy for automotive manufacturing,” The Int. J. Adv. Manuf. Technol. 89(5-8), 1279–1293 (2017).
[Crossref]

Smith-Miles, K.

S. Ladha, K. Smith-Miles, and S. Chandran, “Projection defocus correction using adaptive kernel sampling and geometric correction in dual-planar environments," in Proceedings of the IEEE international conference on computer vision workshops (IEEE, 2011), pp. 9–14.

Son, S.

B. Lim, S. Son, H. Kim, S. Nah, and K. Mu Lee, “Enhanced deep residual networks for single image super-resolution," in Proceedings of the IEEE conference on computer vision and pattern recognition workshops (IEEE, 2017), pp. 136–144.

Song, P.

M. S. Brown, P. Song, and T.-J. Cham, “Image pre-conditioning for out-of-focus projector blur," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2006), pp. 1956–1963.

Spagnulo, D.

A. E. Uva, M. Gattullo, V. M. Manghisi, D. Spagnulo, G. L. Cascella, and M. Fiorentino, “Evaluating the effectiveness of spatial augmented reality in smart manufacturing: a solution for manual working stations,” The Int. J. Adv. Manuf. Technol. 94(1-4), 509–521 (2018).
[Crossref]

Sun, J.

K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification," in Proceedings of the IEEE international conference on computer vision (IEEE, 2015), pp. 1026–1034.

S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks," in Proceedings of Advances in neural information processing systems (NIPS, 2015), pp. 91–99.

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2016), pp. 770–778.

Sutskever, I.

A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks," in Proceedings of Advances in neural information processing systems (NIPS, 2012), pp. 1097–1105.

Taubin, G.

D. Moreno and G. Taubin, “Simple, accurate, and robust projector-camera calibration," in Proceedings of the Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission (IEEE, 2012), pp. 464–471.

Thomas, B. H.

A. Doshi, R. T. Smith, B. H. Thomas, and C. Bouras, “Use of projector based augmented reality to improve manual spot-welding precision and accuracy for automotive manufacturing,” The Int. J. Adv. Manuf. Technol. 89(5-8), 1279–1293 (2017).
[Crossref]

Timofte, R.

E. Agustsson and R. Timofte, “NTIRE 2017 challenge on single image super-resolution: Dataset and study," in Proceedings of the IEEE international conference on computer vision workshops (IEEE, 2017), pp. 126–135.

Trimeche, M.

M. Trimeche, D. Paliy, M. Vehvilainen, and V. Katkovnic, “Multichannel image deblurring of raw color components," in Proceedings of the Computational Imaging III (International Society for Optics and Photonics, 2005), vol. 5674, pp. 169–178.

Uva, A. E.

A. E. Uva, M. Gattullo, V. M. Manghisi, D. Spagnulo, G. L. Cascella, and M. Fiorentino, “Evaluating the effectiveness of spatial augmented reality in smart manufacturing: a solution for manual working stations,” The Int. J. Adv. Manuf. Technol. 94(1-4), 509–521 (2018).
[Crossref]

M. Di Donato, M. Fiorentino, A. E. Uva, M. Gattullo, and G. Monno, “Text legibility for projected augmented reality on industrial workbenches,” Comput. Ind. 70(1), 70–78 (2015).
[Crossref]

Vehvilainen, M.

M. Trimeche, D. Paliy, M. Vehvilainen, and V. Katkovnic, “Multichannel image deblurring of raw color components," in Proceedings of the Computational Imaging III (International Society for Optics and Photonics, 2005), vol. 5674, pp. 169–178.

Voss, K. J.

Wang, D.

Q. Dou, H. Chen, L. Yu, L. Zhao, J. Qin, D. Wang, V. C. Mok, L. Shi, and P. Heng, “Automatic detection of cerebral microbleeds from MR images via 3D convolutional neural networks,” IEEE Trans. Med. Imaging 35(5), 1182–1195 (2016).
[Crossref]

Wang, J.

E. Kee, S. Paris, S. Chen, and J. Wang, “Modeling and removing spatially-varying optical blur," in Proceedings of the IEEE international conference on computational photography (IEEE, 2011), pp. 1–8.

Wang, X.

Wang, Y.

Wang, Z.

Xu, W.

S. Ji, W. Xu, M. Yang, and K. Yu, “3D convolutional neural networks for human action recognition,” IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 221–231 (2013).
[Crossref]

Yang, M.

S. Ji, W. Xu, M. Yang, and K. Yu, “3D convolutional neural networks for human action recognition,” IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 221–231 (2013).
[Crossref]

Yu, K.

S. Ji, W. Xu, M. Yang, and K. Yu, “3D convolutional neural networks for human action recognition,” IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 221–231 (2013).
[Crossref]

Yu, L.

Q. Dou, H. Chen, L. Yu, L. Zhao, J. Qin, D. Wang, V. C. Mok, L. Shi, and P. Heng, “Automatic detection of cerebral microbleeds from MR images via 3D convolutional neural networks,” IEEE Trans. Med. Imaging 35(5), 1182–1195 (2016).
[Crossref]

Yun, F.

Z. Yunlun, L. Kunpeng, L. Kai, W. Lichen, Z. Bineng, and F. Yun, “Image super-resolution using very deep residual channel attention networks," in Proceedings of the European conference on computer vision (Springer, 2018), pp. 286–301.

Yunlun, Z.

Z. Yunlun, L. Kunpeng, L. Kai, W. Lichen, Z. Bineng, and F. Yun, “Image super-resolution using very deep residual channel attention networks," in Proceedings of the European conference on computer vision (Springer, 2018), pp. 286–301.

Zhang, L.

L. Zhang and S. Nayar, “Projection defocus analysis for scene capture and image display,” ACM Trans. Graph. 25(3), 907–915 (2006).
[Crossref]

Zhang, X.

K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification," in Proceedings of the IEEE international conference on computer vision (IEEE, 2015), pp. 1026–1034.

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2016), pp. 770–778.

Zhao, H.

Y. Wang, H. Zhao, H. Jiang, and X. Li, “Defocusing parameter selection strategies based on PSF measurement for square-binary defocusing fringe projection profilometry,” Opt. Express 26(16), 20351–20367 (2018).
[Crossref]

H. Zhao, O. Gallo, I. Frosio, and J. Kautz, “Loss functions for image restoration with neural networks,” IEEE Trans. Comput. Imaging 3(1), 47–57 (2017).
[Crossref]

Zhao, L.

Q. Dou, H. Chen, L. Yu, L. Zhao, J. Qin, D. Wang, V. C. Mok, L. Shi, and P. Heng, “Automatic detection of cerebral microbleeds from MR images via 3D convolutional neural networks,” IEEE Trans. Med. Imaging 35(5), 1182–1195 (2016).
[Crossref]

Zisserman, A.

K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv:1409.1556 (2014).

ACM Trans. Graph. (1)

L. Zhang and S. Nayar, “Projection defocus analysis for scene capture and image display,” ACM Trans. Graph. 25(3), 907–915 (2006).
[Crossref]

Appl. Opt. (2)

Comput. Ind. (1)

M. Di Donato, M. Fiorentino, A. E. Uva, M. Gattullo, and G. Monno, “Text legibility for projected augmented reality on industrial workbenches,” Comput. Ind. 70(1), 70–78 (2015).
[Crossref]

IEEE Trans. Comput. Imaging (1)

H. Zhao, O. Gallo, I. Frosio, and J. Kautz, “Loss functions for image restoration with neural networks,” IEEE Trans. Comput. Imaging 3(1), 47–57 (2017).
[Crossref]

IEEE Trans. Med. Imaging (1)

Q. Dou, H. Chen, L. Yu, L. Zhao, J. Qin, D. Wang, V. C. Mok, L. Shi, and P. Heng, “Automatic detection of cerebral microbleeds from MR images via 3D convolutional neural networks,” IEEE Trans. Med. Imaging 35(5), 1182–1195 (2016).
[Crossref]

IEEE Trans. on Image Process. (1)

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. on Image Process. 13(4), 600–612 (2004).
[Crossref]

IEEE Trans. Pattern Anal. Mach. Intell. (1)

S. Ji, W. Xu, M. Yang, and K. Yu, “3D convolutional neural networks for human action recognition,” IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 221–231 (2013).
[Crossref]

Int. J. Comput. Vis. (1)

J. Jurij, P. Franjo, L. Boštjan, and B. Miran, “2D sub-pixel point spread function measurement using a virtual point-like source,” Int. J. Comput. Vis. 121(3), 391–402 (2017).
[Crossref]

Opt. Express (3)

Opt. Lett. (1)

The Int. J. Adv. Manuf. Technol. (2)

A. Doshi, R. T. Smith, B. H. Thomas, and C. Bouras, “Use of projector based augmented reality to improve manual spot-welding precision and accuracy for automotive manufacturing,” The Int. J. Adv. Manuf. Technol. 89(5-8), 1279–1293 (2017).
[Crossref]

A. E. Uva, M. Gattullo, V. M. Manghisi, D. Spagnulo, G. L. Cascella, and M. Fiorentino, “Evaluating the effectiveness of spatial augmented reality in smart manufacturing: a solution for manual working stations,” The Int. J. Adv. Manuf. Technol. 94(1-4), 509–521 (2018).
[Crossref]

Other (18)

M. S. Brown, P. Song, and T.-J. Cham, “Image pre-conditioning for out-of-focus projector blur," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2006), pp. 1956–1963.

P. Molchanov, S. Gupta, K. Kim, and J. Kautz, “Hand gesture recognition with 3D convolutional neural networks," in Proceedings of the IEEE conference on computer vision and pattern recognition workshops (IEEE, 2015), pp. 1–7.

Z. Yunlun, L. Kunpeng, L. Kai, W. Lichen, Z. Bineng, and F. Yun, “Image super-resolution using very deep residual channel attention networks," in Proceedings of the European conference on computer vision (Springer, 2018), pp. 286–301.

K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification," in Proceedings of the IEEE international conference on computer vision (IEEE, 2015), pp. 1026–1034.

A. Mosleh, P. Green, E. Onzon, I. Begin, and J. M. P. Langlois, “Camera intrinsic blur kernel estimation: A reliable framework," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2015), pp. 4961–4968.

Y. Oyamada and H. Saito, “Focal pre-correction of projected image for deblurring screen image," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2007), pp. 1–8.

E. Kee, S. Paris, S. Chen, and J. Wang, “Modeling and removing spatially-varying optical blur," in Proceedings of the IEEE international conference on computational photography (IEEE, 2011), pp. 1–8.

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2016), pp. 770–778.

K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv:1409.1556 (2014).

E. Shelhamer, J. Long, and T. Darrell, “Fully convolutional networks for semantic segmentation," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2015), pp. 3431–3440.

H. Li, Z. Lin, X. Shen, J. Brandt, and G. Hua, “A convolutional neural network cascade for face detection," in Proceedings of the IEEE conference on computer vision and pattern recognition (IEEE, 2015), pp. 5325–5334.

E. Agustsson and R. Timofte, “NTIRE 2017 challenge on single image super-resolution: Dataset and study," in Proceedings of the IEEE international conference on computer vision workshops (IEEE, 2017), pp. 126–135.

D. Moreno and G. Taubin, “Simple, accurate, and robust projector-camera calibration," in Proceedings of the Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission (IEEE, 2012), pp. 464–471.

S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks," in Proceedings of Advances in neural information processing systems (NIPS, 2015), pp. 91–99.

A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks," in Proceedings of Advances in neural information processing systems (NIPS, 2012), pp. 1097–1105.

M. Trimeche, D. Paliy, M. Vehvilainen, and V. Katkovnic, “Multichannel image deblurring of raw color components," in Proceedings of the Computational Imaging III (International Society for Optics and Photonics, 2005), vol. 5674, pp. 169–178.

S. Ladha, K. Smith-Miles, and S. Chandran, “Projection defocus correction using adaptive kernel sampling and geometric correction in dual-planar environments," in Proceedings of the IEEE international conference on computer vision workshops (IEEE, 2011), pp. 9–14.

B. Lim, S. Son, H. Kim, S. Nah, and K. Mu Lee, “Enhanced deep residual networks for single image super-resolution," in Proceedings of the IEEE conference on computer vision and pattern recognition workshops (IEEE, 2017), pp. 136–144.

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (10)

Fig. 1.
Fig. 1. The setup of an optical system to simultaneously capture screen-projected images and depth data at different projection distances.
Fig. 2.
Fig. 2. The data (in-focus, out-of-focus, and depth images) capturing process at different projection positions. We projected hundreds of images ($1280 \times 720$) from the publicly available DIV2K dataset [23] for capturing the training and testing images.
Fig. 3.
Fig. 3. Based on the established corner correspondences between a checkerboard pattern and its screen-projected image, a polynomial 2D geometric mapping function is computed to generate the viewpoint rectified images.
Fig. 4.
Fig. 4. An illustration of sub-pixel level alignment between in-focus and out-of-focus image pairs. (a) In-focus image; (b) Zoom-in view; (c) Alignment results based on 2D polynomial mapping; (d) Alignment results based on 2D displacement $X^{*}$ image warping. Note the red curves are presented in the same position in all images to highlight misalignments.
Fig. 5.
Fig. 5. Some well-aligned in-focus, out-of-focus, and depth images captured at different projection distances. We purposely use very different training and testing images to evaluate the generalization performance of our proposed method.
Fig. 6.
Fig. 6. An illustration of multi-channel input for our proposed MC-RDN model. RGB/depth image patches are integrated with two additional location maps to encode the $x$ and $y$ spatial coordinates.
Fig. 7.
Fig. 7. A full-size image is uniformly cropped into a number of small image patches. Selection A: image patches contain pixels of similar RGB values. Selection B: image patches contain abundant textures/structures. Only the image patches in Section B are utilized for deep network training.
Fig. 8.
Fig. 8. The architecture of our proposed MC-RDN model for accurate estimation of spatially-varying and depth-dependent defocus kernels.
Fig. 9.
Fig. 9. The predicted defocused images in the 50cm position using Gauss-NCC [9], Disk-NCC [14], 2D-Gauss [15], Non-para [15], and our MC-RDN model. Please zoom in to check details highlighted in red bounding box.
Fig. 10.
Fig. 10. Some comparative results of out-of-focus blurring effect compensation in the 50cm position using Gauss-NCC [9], Disk-NCC [14], 2D-Gauss [15], Non-para [15], and our MC-RDN model. Please zoom in to check details highlighted in red bounding box.

Tables (2)

Tables Icon

Table 1. Quantitative evaluation results at a number of projection positions where the training/calibration images are available. Red and blue indicate the best and the second-best performance, respectively.

Tables Icon

Table 2. Quantitative evaluation results at a number of projection positions where the training/calibration images are not provided. Red and blue indicate the best and the second-best performance, respectively.

Equations (9)

Equations on this page are rendered with MathJax. Learn more.

X = a r g   min X { p Ω ( I D F ( p + X ) I D F ( p ) ) 2 } ,
F 0 R = C o n v 1 × 1 ( I I F R ) ,
F 0 G = C o n v 1 × 1 ( I I F G ) ,
F 0 B = C o n v 1 × 1 ( I I F B ) ,
I D F R = C o n v 3 × 3 ( F N R ) ,
I D F G = C o n v 3 × 3 ( F N G ) ,
I D F B = C o n v 3 × 3 ( F N B ) ,
L = α p P | | I D F R ( p ) I D F R ( p ) | | 2 2 + β p P | | I D F G ( p ) I D F G ( p ) | | 2 2 + γ p P | | I D F B ( p ) I D F B ( p ) | | 2 2 ,
I = a r g   min I { p Ω ( D F ( I ( p ) ) + ϕ I I F ( p ) ) 2 } ,

Metrics