Abstract

The color histogram (or color cloud) of a digital image displays the colors present in an image regardless of their spatial location and can be visualized in (R,G,B) coordinates. Therefore, it contains essential information about the structure of colors in natural scenes. The analysis and visual exploration of this structure is difficult. The color cloud being thick, its more dense points are hidden in the clutter. Thus, it is impossible to properly visualize the cloud density. This paper proposes a visualization method that also enables one to validate a general model for color clouds. It argues first by physical arguments that the color cloud must be essentially a two-dimensional (2D) manifold. A color cloud-filtering algorithm is proposed to reveal this 2D structure. A quantitative analysis shows that the reconstructed 2D manifold is strikingly close to the color cloud and only marginally depends on the filtering parameter. Thanks to this algorithm, it is finally possible to visualize the color cloud density as a gray-level function defined on the 2D manifold.

© 2011 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. I. Omer and M. Werman, “Color lines: image specific color representation,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2004) pp. 946–953 ().
  2. G. J. Klinker, S. A. Shafer, and T. Kanade, “A physical approach to color image understanding,” Int. J. Comput. Vis. 4, 7–38(1990).
    [CrossRef]
  3. S. A. Shafer, “Using color to separate reflection components,” Color Res. Appl. 10, 210–218 (1985).
    [CrossRef]
  4. F. Chapeau-Blondeau, J. Chauveau, D. Rousseau, and P. Richard, “Fractal structure in the color distribution of natural images,” Chaos Solitons Fractals 42, 472–482 (2009).
    [CrossRef]
  5. D. Martin, C. Fowlkes, D. Tal, and J. Malik, “A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics,” in Proceedings of the 8th International Conference on Computer Vision (IEEE, 2002), pp. 416–423.
  6. M. Oren and S. K. Nayar, “Generalization of the Lambertian model and implications for machine vision,” Int. J. Comput. Vis. 14, 227–251 (1995).
    [CrossRef]
  7. K. Torrance and E. Sparrow, “Theory for off-specular reflection from rough surfaces,” J. Opt. Soc. Am. 57, 1105–1114(1967).
    [CrossRef]
  8. D. Levin, “The approximation power of moving least-squares,” Math. Comput. 67, 1517–1531 (1998).
    [CrossRef]
  9. J. Digne, J. M. Morel, C. Mehdi-Souzani, and C. Lartigue, “Scale space meshing of raw data point sets,” preprint CMLA no. 2009-30, http://www.cmla.ens-cachan.fr/fileadmin/Documentation/Prepublications/2009/CMLA2009-30.pdf.
  10. X. Huo and J. Chen, “Local linear projection (LLP),” in Proceedings of the First Workshop on Genomic Signal Processing and Statistics (IEEE, 2002).
  11. W. S. Torgerson, “Multidimensional scaling: I. Theory and method,” Psychometrika 17, 401–419 (1952).
    [CrossRef]
  12. S. T. Roweis and L. K. Saul, “Nonlinear dimensionality reduction by locally linear embedding,” Science 290, 2323–2326 (2000).
    [CrossRef] [PubMed]
  13. M. Belkin and P. Niyogi, “Laplacian eigenmaps and spectral techniques for embedding and clustering,” in Advances in Neural Information Processing Systems 14 (MIT Press), pp. 585–591.
  14. J. B. Tenenbaum, V. Silva, and J. C. Langford, “A global geometric framework for nonlinear dimensionality reduction,” Science 290, 2319–2323 (2000).
    [CrossRef] [PubMed]
  15. Universite Paris Descartes, http://www.mi.parisdescartes.fr/~buades/recerca.html.
  16. “Image color cube dimensional filtering and visualization,” http://www.ipol.im/pub/demo/blm_color_dimensional_filtering/.

2009 (1)

F. Chapeau-Blondeau, J. Chauveau, D. Rousseau, and P. Richard, “Fractal structure in the color distribution of natural images,” Chaos Solitons Fractals 42, 472–482 (2009).
[CrossRef]

2000 (2)

S. T. Roweis and L. K. Saul, “Nonlinear dimensionality reduction by locally linear embedding,” Science 290, 2323–2326 (2000).
[CrossRef] [PubMed]

J. B. Tenenbaum, V. Silva, and J. C. Langford, “A global geometric framework for nonlinear dimensionality reduction,” Science 290, 2319–2323 (2000).
[CrossRef] [PubMed]

1998 (1)

D. Levin, “The approximation power of moving least-squares,” Math. Comput. 67, 1517–1531 (1998).
[CrossRef]

1995 (1)

M. Oren and S. K. Nayar, “Generalization of the Lambertian model and implications for machine vision,” Int. J. Comput. Vis. 14, 227–251 (1995).
[CrossRef]

1990 (1)

G. J. Klinker, S. A. Shafer, and T. Kanade, “A physical approach to color image understanding,” Int. J. Comput. Vis. 4, 7–38(1990).
[CrossRef]

1985 (1)

S. A. Shafer, “Using color to separate reflection components,” Color Res. Appl. 10, 210–218 (1985).
[CrossRef]

1967 (1)

1952 (1)

W. S. Torgerson, “Multidimensional scaling: I. Theory and method,” Psychometrika 17, 401–419 (1952).
[CrossRef]

Belkin, M.

M. Belkin and P. Niyogi, “Laplacian eigenmaps and spectral techniques for embedding and clustering,” in Advances in Neural Information Processing Systems 14 (MIT Press), pp. 585–591.

Chapeau-Blondeau, F.

F. Chapeau-Blondeau, J. Chauveau, D. Rousseau, and P. Richard, “Fractal structure in the color distribution of natural images,” Chaos Solitons Fractals 42, 472–482 (2009).
[CrossRef]

Chauveau, J.

F. Chapeau-Blondeau, J. Chauveau, D. Rousseau, and P. Richard, “Fractal structure in the color distribution of natural images,” Chaos Solitons Fractals 42, 472–482 (2009).
[CrossRef]

Chen, J.

X. Huo and J. Chen, “Local linear projection (LLP),” in Proceedings of the First Workshop on Genomic Signal Processing and Statistics (IEEE, 2002).

Digne, J.

J. Digne, J. M. Morel, C. Mehdi-Souzani, and C. Lartigue, “Scale space meshing of raw data point sets,” preprint CMLA no. 2009-30, http://www.cmla.ens-cachan.fr/fileadmin/Documentation/Prepublications/2009/CMLA2009-30.pdf.

Fowlkes, C.

D. Martin, C. Fowlkes, D. Tal, and J. Malik, “A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics,” in Proceedings of the 8th International Conference on Computer Vision (IEEE, 2002), pp. 416–423.

Huo, X.

X. Huo and J. Chen, “Local linear projection (LLP),” in Proceedings of the First Workshop on Genomic Signal Processing and Statistics (IEEE, 2002).

Kanade, T.

G. J. Klinker, S. A. Shafer, and T. Kanade, “A physical approach to color image understanding,” Int. J. Comput. Vis. 4, 7–38(1990).
[CrossRef]

Klinker, G. J.

G. J. Klinker, S. A. Shafer, and T. Kanade, “A physical approach to color image understanding,” Int. J. Comput. Vis. 4, 7–38(1990).
[CrossRef]

Langford, J. C.

J. B. Tenenbaum, V. Silva, and J. C. Langford, “A global geometric framework for nonlinear dimensionality reduction,” Science 290, 2319–2323 (2000).
[CrossRef] [PubMed]

Lartigue, C.

J. Digne, J. M. Morel, C. Mehdi-Souzani, and C. Lartigue, “Scale space meshing of raw data point sets,” preprint CMLA no. 2009-30, http://www.cmla.ens-cachan.fr/fileadmin/Documentation/Prepublications/2009/CMLA2009-30.pdf.

Levin, D.

D. Levin, “The approximation power of moving least-squares,” Math. Comput. 67, 1517–1531 (1998).
[CrossRef]

Malik, J.

D. Martin, C. Fowlkes, D. Tal, and J. Malik, “A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics,” in Proceedings of the 8th International Conference on Computer Vision (IEEE, 2002), pp. 416–423.

Martin, D.

D. Martin, C. Fowlkes, D. Tal, and J. Malik, “A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics,” in Proceedings of the 8th International Conference on Computer Vision (IEEE, 2002), pp. 416–423.

Mehdi-Souzani, C.

J. Digne, J. M. Morel, C. Mehdi-Souzani, and C. Lartigue, “Scale space meshing of raw data point sets,” preprint CMLA no. 2009-30, http://www.cmla.ens-cachan.fr/fileadmin/Documentation/Prepublications/2009/CMLA2009-30.pdf.

Morel, J. M.

J. Digne, J. M. Morel, C. Mehdi-Souzani, and C. Lartigue, “Scale space meshing of raw data point sets,” preprint CMLA no. 2009-30, http://www.cmla.ens-cachan.fr/fileadmin/Documentation/Prepublications/2009/CMLA2009-30.pdf.

Nayar, S. K.

M. Oren and S. K. Nayar, “Generalization of the Lambertian model and implications for machine vision,” Int. J. Comput. Vis. 14, 227–251 (1995).
[CrossRef]

Niyogi, P.

M. Belkin and P. Niyogi, “Laplacian eigenmaps and spectral techniques for embedding and clustering,” in Advances in Neural Information Processing Systems 14 (MIT Press), pp. 585–591.

Omer, I.

I. Omer and M. Werman, “Color lines: image specific color representation,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2004) pp. 946–953 ().

Oren, M.

M. Oren and S. K. Nayar, “Generalization of the Lambertian model and implications for machine vision,” Int. J. Comput. Vis. 14, 227–251 (1995).
[CrossRef]

Richard, P.

F. Chapeau-Blondeau, J. Chauveau, D. Rousseau, and P. Richard, “Fractal structure in the color distribution of natural images,” Chaos Solitons Fractals 42, 472–482 (2009).
[CrossRef]

Rousseau, D.

F. Chapeau-Blondeau, J. Chauveau, D. Rousseau, and P. Richard, “Fractal structure in the color distribution of natural images,” Chaos Solitons Fractals 42, 472–482 (2009).
[CrossRef]

Roweis, S. T.

S. T. Roweis and L. K. Saul, “Nonlinear dimensionality reduction by locally linear embedding,” Science 290, 2323–2326 (2000).
[CrossRef] [PubMed]

Saul, L. K.

S. T. Roweis and L. K. Saul, “Nonlinear dimensionality reduction by locally linear embedding,” Science 290, 2323–2326 (2000).
[CrossRef] [PubMed]

Shafer, S. A.

G. J. Klinker, S. A. Shafer, and T. Kanade, “A physical approach to color image understanding,” Int. J. Comput. Vis. 4, 7–38(1990).
[CrossRef]

S. A. Shafer, “Using color to separate reflection components,” Color Res. Appl. 10, 210–218 (1985).
[CrossRef]

Silva, V.

J. B. Tenenbaum, V. Silva, and J. C. Langford, “A global geometric framework for nonlinear dimensionality reduction,” Science 290, 2319–2323 (2000).
[CrossRef] [PubMed]

Sparrow, E.

Tal, D.

D. Martin, C. Fowlkes, D. Tal, and J. Malik, “A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics,” in Proceedings of the 8th International Conference on Computer Vision (IEEE, 2002), pp. 416–423.

Tenenbaum, J. B.

J. B. Tenenbaum, V. Silva, and J. C. Langford, “A global geometric framework for nonlinear dimensionality reduction,” Science 290, 2319–2323 (2000).
[CrossRef] [PubMed]

Torgerson, W. S.

W. S. Torgerson, “Multidimensional scaling: I. Theory and method,” Psychometrika 17, 401–419 (1952).
[CrossRef]

Torrance, K.

Werman, M.

I. Omer and M. Werman, “Color lines: image specific color representation,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2004) pp. 946–953 ().

Chaos Solitons Fractals (1)

F. Chapeau-Blondeau, J. Chauveau, D. Rousseau, and P. Richard, “Fractal structure in the color distribution of natural images,” Chaos Solitons Fractals 42, 472–482 (2009).
[CrossRef]

Color Res. Appl. (1)

S. A. Shafer, “Using color to separate reflection components,” Color Res. Appl. 10, 210–218 (1985).
[CrossRef]

Int. J. Comput. Vis. (2)

G. J. Klinker, S. A. Shafer, and T. Kanade, “A physical approach to color image understanding,” Int. J. Comput. Vis. 4, 7–38(1990).
[CrossRef]

M. Oren and S. K. Nayar, “Generalization of the Lambertian model and implications for machine vision,” Int. J. Comput. Vis. 14, 227–251 (1995).
[CrossRef]

J. Opt. Soc. Am. (1)

Math. Comput. (1)

D. Levin, “The approximation power of moving least-squares,” Math. Comput. 67, 1517–1531 (1998).
[CrossRef]

Psychometrika (1)

W. S. Torgerson, “Multidimensional scaling: I. Theory and method,” Psychometrika 17, 401–419 (1952).
[CrossRef]

Science (2)

S. T. Roweis and L. K. Saul, “Nonlinear dimensionality reduction by locally linear embedding,” Science 290, 2323–2326 (2000).
[CrossRef] [PubMed]

J. B. Tenenbaum, V. Silva, and J. C. Langford, “A global geometric framework for nonlinear dimensionality reduction,” Science 290, 2319–2323 (2000).
[CrossRef] [PubMed]

Other (7)

Universite Paris Descartes, http://www.mi.parisdescartes.fr/~buades/recerca.html.

“Image color cube dimensional filtering and visualization,” http://www.ipol.im/pub/demo/blm_color_dimensional_filtering/.

I. Omer and M. Werman, “Color lines: image specific color representation,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2004) pp. 946–953 ().

M. Belkin and P. Niyogi, “Laplacian eigenmaps and spectral techniques for embedding and clustering,” in Advances in Neural Information Processing Systems 14 (MIT Press), pp. 585–591.

J. Digne, J. M. Morel, C. Mehdi-Souzani, and C. Lartigue, “Scale space meshing of raw data point sets,” preprint CMLA no. 2009-30, http://www.cmla.ens-cachan.fr/fileadmin/Documentation/Prepublications/2009/CMLA2009-30.pdf.

X. Huo and J. Chen, “Local linear projection (LLP),” in Proceedings of the First Workshop on Genomic Signal Processing and Statistics (IEEE, 2002).

D. Martin, C. Fowlkes, D. Tal, and J. Malik, “A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics,” in Proceedings of the 8th International Conference on Computer Vision (IEEE, 2002), pp. 416–423.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (14)

Fig. 1
Fig. 1

RGB cube on the left (corresponding to the top left image, from the Berkeley Segmentation Dataset [5]) has been iteratively filtered in a 2D cloud, with the standard parameter T = 10 used throughout the paper and explained in Section 3 . This processed RGB cube is displayed on the right, and the new image after filtering of the color is shown on the top right. A profile view of the cloud is shown to illustrate how it flattens under the 2D filtering.

Fig. 2
Fig. 2

Visualization of color densities. The image shows a view of the color cloud of the filtered image in Fig. 1 (top right), where each color point is represented by a gray level proportional to its density (lighter for higher densities). The displayed density for each color point is the logarithm of the number of color points in a neighborhood of radius T = 5 . The density is now visible because the cloud has been locally flattened into a 2D manifold. Otherwise, the dense voxels would be hidden, occluded by less dense ones.

Fig. 3
Fig. 3

Left, two patches of the same object (grass) under daylight and shadow (note that patch A is a subregion of patch B). Center, right, first principal view of the RGB cube of patches A and B. The elongation of this color histogram is due to the varying light intensity.

Fig. 4
Fig. 4

Pyramid example. Left, original image with selected patches A and B (note that patch A is a subregion of patch B). Center and right, first principal view of the RGB cube for patches A and B, respectively..

Fig. 5
Fig. 5

Ball example. Left, image with selected patches. Center and right, first principal views of the RGB cube of patch A (bottom part of the sphere, center) and patch B (full sphere, right). The color cloud of this monochromatic object is clearly 2D.

Fig. 6
Fig. 6

Top row, three examples of textured image patches. For each patch, σ H measures the dispersion of hue values within the color cluster. These dispersions are respectively σ H = 3.98 ° , 3.43 ° , and 4.96 ° confirming (in a scale of 0 to 360 ° !) that the hue is almost constant on each patch, and, therefore, the color cloud is flat. Bottom row, corresponding principal view of the RGB cube for each patch, showing its clear 2D spread. In short, these color clusters spread on an (approximately) constant hue plane, containing the gray-level axis.

Fig. 7
Fig. 7

Original image and the selected patches for the study of color interaction.

Fig. 8
Fig. 8

Top row: left, original patch A from Fig. 7; center, low hue gradient pixels; right, high hue gradient pixels. Bottom row, corresponding first principal view of the RGB cube for each patch.

Fig. 9
Fig. 9

Top row: left, original patch B from Fig. 7; center, low hue gradient pixels; right, high hue gradient pixels. Bottom row, corresponding first principal view of the RGB cube for each image.

Fig. 10
Fig. 10

Top row; left to right, three monochromatic regions of the image on the right. Bottom row, their color cubes. The color cube of each monochromatic region is an elongated 2D shape. The RGB cube of the whole image is the union of all these pieces, complemented by 2D surfaces connecting each pair of pieces in contact in the image.

Fig. 11
Fig. 11

Original image of Fig. 12 is filtered with LLP1 (first row) and LLP2 (second row), the radius T of the filter being 10, 15, and 20, respectively. While the 2D filter hardly alters the color cloud even with a large radius, the 1D filter transforms the cloud into a net of 1D curves strongly dependent on the filtering radius. Figure 12 shows that the color image itself is altered by the 1D filter, but not by the 2D filter.

Fig. 12
Fig. 12

Top row, the original image and the image whole color cloud has been filtered by LLP1, the radius T of the filter being 10, 15, and 20, respectively. The corresponding clouds are in Fig. 11. The 1D filter drastically alters the colors, bringing the green and the blue toward a grayish color. Bottom row, the original images followed by the image with its colors filtered by the 2D filter with T radius being 10, 15, and 20, respectively. No color alteration is visible, which is consistent with the small variation of the cloud itself. This visual experiment is complemented by the quantitative analysis of Table 1.

Fig. 13
Fig. 13

Four images, their original clouds, the LLP2-filtered ( T = 10 ) images, and their filtered clouds. The filtered cloud is a 2D numerical manifold, being a steady state for LLP2. There is no conspicuous alteration in the color cloud when passing in 2D, and the image with filtered colors looks identical to the original..

Fig. 14
Fig. 14

Left to right, densities of the filtered RGB cubes in Fig. 13 (top to bottom). Lighter gray levels indicate higher density values. The real density is now visible because the cloud has been flattened into a 2D manifold..

Tables (2)

Tables Icon

Table 1 Algorithm 1: Filtering LLP Algorithm

Tables Icon

Table 1 Statistical Results of Applying LLP1 and LLP2 to the 100 Images of the Berkeley Test Dataset a

Metrics