Abstract

We demonstrate a new digital cleaning technique which uses a neural network that is trained to learn the transformation from dirty to clean segments of a painting image. The inputs and outputs of the network are pixels belonging to dirty and clean segments found in Fernando Amorsolo’s Malacañang by the River. After digital cleaning we visualize the painting’s discoloration by assuming it to be a transmission filter superimposed on the clean painting. Using an RGB color-to-spectrum transformation to obtain the point-per-point spectra of the clean and dirty painting images, we calculate this “dirt” filter and render it for the whole image.

© 2011 OSA

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. M. Pappas and I. Pitas, “Digital color restoration of old paintings,” IEEE Trans. Image Process. 9(2), 291–294 (2000).
    [CrossRef] [PubMed]
  2. M. Barni, F. Bartolini, and V. Cappellini, “Image processing for virtual restoration of artworks,” IEEE Multimed. 7(2), 34–37 (2000).
    [CrossRef]
  3. R. Berns, F. Imai, and L. Taplin, “Rejuvenating Seurat’s A Sunday On La Grande Jatte- 1884 using color And imaging science techniques: A simulation,” in ICOM 14th Triennial Meeting The Hague: 12–16September,2005: Preprints, I. Verger. ed. (Maney Publishing, 2005), pp 452–458.
  4. C. M. Palomero and M. Soriano, “After digital cleaning: visualization of the dirt layer,” Proc. SPIE 7869, 78690O, 78690O-7 (2011), doi:.
    [CrossRef]
  5. P. Cotte and D. Dupraz, “Spectral imaging of Leonardo Da Vinci’s Mona Lisa: A true color smile without the influence of aged varnish,” in Proc. IS&T CGIV’06, University of Leeds UK, June 19–22, 2006.
  6. R. S. Berns, “Rejuvenating the appearance of cultural heritage using color and imaging science techniques,” in Proc. AIC Colour 05 (AIC, 2005), pp. 369–374.
  7. M. Bacci, F. Baldini, R. Carla, R. Linari, M. Picollo, and B. Radicati, “Color analysis of the Brancacci chapel frescoes: part II,” Appl. Spectrosc. 47(4), 399–402 (1993).
    [CrossRef]
  8. M. Bacci, A. Casini, C. Cucci, M. Picollo, B. Radicati, and M. Vervat, “Non-invasive spectroscopic measurements on the Il Ritratto della figliastra by Giovanni Fattori: identification of pigments and colourimetric analysis,” J. Cult. Herit. 4(4), 329–336 (2003).
    [CrossRef]
  9. C. M. Palomero and M. Soriano, “Neural network for the digital cleaning of an oil painting,” in Digital Image Processing and Analysis, OSA Technical Digest (CD) (Optical Society of America, 2010), paper DMD5. http://www.opticsinfobase.org/abstract.cfm?URI=DIPA-2010-DMD5
  10. A. Gascadi and P. Szolgay, “Image inpainting methods by using cellular neural networks,” in Int’l Workshop on Cellular Neural Networks and Their Applications (IEEE,2005), pp 198–201.
  11. Matlab 2007 Neural Network Toolbox Documentation page.
  12. M. J. Swain and D. H. Ballard, “Color indexing,” Int. J. Comput. Vis. 7(1), 11–32 (1991).
    [CrossRef]
  13. F. Imai and R. Berns, “Spectral estimation using trichromatic digital cameras,” in Proc. of the International Symposium on Multispectral Imaging and Color Reproduction for Digital Archives (AIC, 1999) pp. 42–49.
  14. H. Haneishi, T. Hasegawa, A. Hosoi, Y. Yokoyama, N. Tsumura, and Y. Miyake, “System design for accurately estimating the spectral reflectance of art paintings,” Appl. Opt. 39(35), 6621–6632 (2000).
    [CrossRef] [PubMed]
  15. M. Soriano, W. Oblefias, and C. Saloma, “Fluorescence spectrum estimation using multiple color images and minimum negativity constraint,” Opt. Express 10(25), 1458–1464 (2002).
    [PubMed]
  16. K. Martinez, J. Cupitt, D. Saunders, and R. Pillay, “Ten years of art imaging research,” in Proc. IEEE 90, 28–41 (2002).

2011 (1)

C. M. Palomero and M. Soriano, “After digital cleaning: visualization of the dirt layer,” Proc. SPIE 7869, 78690O, 78690O-7 (2011), doi:.
[CrossRef]

2003 (1)

M. Bacci, A. Casini, C. Cucci, M. Picollo, B. Radicati, and M. Vervat, “Non-invasive spectroscopic measurements on the Il Ritratto della figliastra by Giovanni Fattori: identification of pigments and colourimetric analysis,” J. Cult. Herit. 4(4), 329–336 (2003).
[CrossRef]

2002 (1)

2000 (3)

H. Haneishi, T. Hasegawa, A. Hosoi, Y. Yokoyama, N. Tsumura, and Y. Miyake, “System design for accurately estimating the spectral reflectance of art paintings,” Appl. Opt. 39(35), 6621–6632 (2000).
[CrossRef] [PubMed]

M. Pappas and I. Pitas, “Digital color restoration of old paintings,” IEEE Trans. Image Process. 9(2), 291–294 (2000).
[CrossRef] [PubMed]

M. Barni, F. Bartolini, and V. Cappellini, “Image processing for virtual restoration of artworks,” IEEE Multimed. 7(2), 34–37 (2000).
[CrossRef]

1993 (1)

1991 (1)

M. J. Swain and D. H. Ballard, “Color indexing,” Int. J. Comput. Vis. 7(1), 11–32 (1991).
[CrossRef]

Bacci, M.

M. Bacci, A. Casini, C. Cucci, M. Picollo, B. Radicati, and M. Vervat, “Non-invasive spectroscopic measurements on the Il Ritratto della figliastra by Giovanni Fattori: identification of pigments and colourimetric analysis,” J. Cult. Herit. 4(4), 329–336 (2003).
[CrossRef]

M. Bacci, F. Baldini, R. Carla, R. Linari, M. Picollo, and B. Radicati, “Color analysis of the Brancacci chapel frescoes: part II,” Appl. Spectrosc. 47(4), 399–402 (1993).
[CrossRef]

Baldini, F.

Ballard, D. H.

M. J. Swain and D. H. Ballard, “Color indexing,” Int. J. Comput. Vis. 7(1), 11–32 (1991).
[CrossRef]

Barni, M.

M. Barni, F. Bartolini, and V. Cappellini, “Image processing for virtual restoration of artworks,” IEEE Multimed. 7(2), 34–37 (2000).
[CrossRef]

Bartolini, F.

M. Barni, F. Bartolini, and V. Cappellini, “Image processing for virtual restoration of artworks,” IEEE Multimed. 7(2), 34–37 (2000).
[CrossRef]

Cappellini, V.

M. Barni, F. Bartolini, and V. Cappellini, “Image processing for virtual restoration of artworks,” IEEE Multimed. 7(2), 34–37 (2000).
[CrossRef]

Carla, R.

Casini, A.

M. Bacci, A. Casini, C. Cucci, M. Picollo, B. Radicati, and M. Vervat, “Non-invasive spectroscopic measurements on the Il Ritratto della figliastra by Giovanni Fattori: identification of pigments and colourimetric analysis,” J. Cult. Herit. 4(4), 329–336 (2003).
[CrossRef]

Cucci, C.

M. Bacci, A. Casini, C. Cucci, M. Picollo, B. Radicati, and M. Vervat, “Non-invasive spectroscopic measurements on the Il Ritratto della figliastra by Giovanni Fattori: identification of pigments and colourimetric analysis,” J. Cult. Herit. 4(4), 329–336 (2003).
[CrossRef]

Haneishi, H.

Hasegawa, T.

Hosoi, A.

Linari, R.

Miyake, Y.

Oblefias, W.

Palomero, C. M.

C. M. Palomero and M. Soriano, “After digital cleaning: visualization of the dirt layer,” Proc. SPIE 7869, 78690O, 78690O-7 (2011), doi:.
[CrossRef]

Pappas, M.

M. Pappas and I. Pitas, “Digital color restoration of old paintings,” IEEE Trans. Image Process. 9(2), 291–294 (2000).
[CrossRef] [PubMed]

Picollo, M.

M. Bacci, A. Casini, C. Cucci, M. Picollo, B. Radicati, and M. Vervat, “Non-invasive spectroscopic measurements on the Il Ritratto della figliastra by Giovanni Fattori: identification of pigments and colourimetric analysis,” J. Cult. Herit. 4(4), 329–336 (2003).
[CrossRef]

M. Bacci, F. Baldini, R. Carla, R. Linari, M. Picollo, and B. Radicati, “Color analysis of the Brancacci chapel frescoes: part II,” Appl. Spectrosc. 47(4), 399–402 (1993).
[CrossRef]

Pitas, I.

M. Pappas and I. Pitas, “Digital color restoration of old paintings,” IEEE Trans. Image Process. 9(2), 291–294 (2000).
[CrossRef] [PubMed]

Radicati, B.

M. Bacci, A. Casini, C. Cucci, M. Picollo, B. Radicati, and M. Vervat, “Non-invasive spectroscopic measurements on the Il Ritratto della figliastra by Giovanni Fattori: identification of pigments and colourimetric analysis,” J. Cult. Herit. 4(4), 329–336 (2003).
[CrossRef]

M. Bacci, F. Baldini, R. Carla, R. Linari, M. Picollo, and B. Radicati, “Color analysis of the Brancacci chapel frescoes: part II,” Appl. Spectrosc. 47(4), 399–402 (1993).
[CrossRef]

Saloma, C.

Soriano, M.

C. M. Palomero and M. Soriano, “After digital cleaning: visualization of the dirt layer,” Proc. SPIE 7869, 78690O, 78690O-7 (2011), doi:.
[CrossRef]

M. Soriano, W. Oblefias, and C. Saloma, “Fluorescence spectrum estimation using multiple color images and minimum negativity constraint,” Opt. Express 10(25), 1458–1464 (2002).
[PubMed]

Swain, M. J.

M. J. Swain and D. H. Ballard, “Color indexing,” Int. J. Comput. Vis. 7(1), 11–32 (1991).
[CrossRef]

Tsumura, N.

Vervat, M.

M. Bacci, A. Casini, C. Cucci, M. Picollo, B. Radicati, and M. Vervat, “Non-invasive spectroscopic measurements on the Il Ritratto della figliastra by Giovanni Fattori: identification of pigments and colourimetric analysis,” J. Cult. Herit. 4(4), 329–336 (2003).
[CrossRef]

Yokoyama, Y.

Appl. Opt. (1)

Appl. Spectrosc. (1)

IEEE Multimed. (1)

M. Barni, F. Bartolini, and V. Cappellini, “Image processing for virtual restoration of artworks,” IEEE Multimed. 7(2), 34–37 (2000).
[CrossRef]

IEEE Trans. Image Process. (1)

M. Pappas and I. Pitas, “Digital color restoration of old paintings,” IEEE Trans. Image Process. 9(2), 291–294 (2000).
[CrossRef] [PubMed]

Int. J. Comput. Vis. (1)

M. J. Swain and D. H. Ballard, “Color indexing,” Int. J. Comput. Vis. 7(1), 11–32 (1991).
[CrossRef]

J. Cult. Herit. (1)

M. Bacci, A. Casini, C. Cucci, M. Picollo, B. Radicati, and M. Vervat, “Non-invasive spectroscopic measurements on the Il Ritratto della figliastra by Giovanni Fattori: identification of pigments and colourimetric analysis,” J. Cult. Herit. 4(4), 329–336 (2003).
[CrossRef]

Opt. Express (1)

Proc. SPIE (1)

C. M. Palomero and M. Soriano, “After digital cleaning: visualization of the dirt layer,” Proc. SPIE 7869, 78690O, 78690O-7 (2011), doi:.
[CrossRef]

Other (8)

P. Cotte and D. Dupraz, “Spectral imaging of Leonardo Da Vinci’s Mona Lisa: A true color smile without the influence of aged varnish,” in Proc. IS&T CGIV’06, University of Leeds UK, June 19–22, 2006.

R. S. Berns, “Rejuvenating the appearance of cultural heritage using color and imaging science techniques,” in Proc. AIC Colour 05 (AIC, 2005), pp. 369–374.

K. Martinez, J. Cupitt, D. Saunders, and R. Pillay, “Ten years of art imaging research,” in Proc. IEEE 90, 28–41 (2002).

C. M. Palomero and M. Soriano, “Neural network for the digital cleaning of an oil painting,” in Digital Image Processing and Analysis, OSA Technical Digest (CD) (Optical Society of America, 2010), paper DMD5. http://www.opticsinfobase.org/abstract.cfm?URI=DIPA-2010-DMD5

A. Gascadi and P. Szolgay, “Image inpainting methods by using cellular neural networks,” in Int’l Workshop on Cellular Neural Networks and Their Applications (IEEE,2005), pp 198–201.

Matlab 2007 Neural Network Toolbox Documentation page.

F. Imai and R. Berns, “Spectral estimation using trichromatic digital cameras,” in Proc. of the International Symposium on Multispectral Imaging and Color Reproduction for Digital Archives (AIC, 1999) pp. 42–49.

R. Berns, F. Imai, and L. Taplin, “Rejuvenating Seurat’s A Sunday On La Grande Jatte- 1884 using color And imaging science techniques: A simulation,” in ICOM 14th Triennial Meeting The Hague: 12–16September,2005: Preprints, I. Verger. ed. (Maney Publishing, 2005), pp 452–458.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (7)

Fig. 1
Fig. 1

Malacañang by the River by Fernando Amorsolo, oil on canvas board, 43.7x56.2x4.0 cm before (left) and after (right) digital cleaning. The painting is from the UP Vargas Museum Collection.

Fig. 2
Fig. 2

Detail of Malacañang by the River before (left) and after (right) digital cleaning.

Fig. 3
Fig. 3

Detail of Malacañang by the River before (left) and with the over-cleaning (right).

Fig. 4
Fig. 4

Detail of Malacañang by the River before (left), with the over-cleaning (center) and after the post processing (right).

Fig. 5
Fig. 5

Malacañang by the River before (left) and after (right) digital cleaning and context-based post processing.

Fig. 6
Fig. 6

Reconstruction of the dirty, clean, and dirt spectra of a pixel in the sky portion of the painting. The patches show the corresponding image pixel.

Fig. 7
Fig. 7

Visualization of Malacañang by the River’s dirt layer.

Tables (1)

Tables Icon

Table 1 Distribution of Sample Pairs

Equations (1)

Equations on this page are rendered with MathJax. Learn more.

Dirt_spectra(λ)= Dirty_pixel_spectra(λ) Clean_pixel_spectra(λ)

Metrics