Abstract

In applications of principal component analysis (PCA) it has often been observed that the eigenvector with the largest eigenvalue has only nonnegative entries when the vectors of the underlying stochastic process have only nonnegative values. This has been used to show that the coordinate vectors in PCA are all located in a cone. We prove that the nonnegativity of the first eigenvector follows from the Perron–Frobenius (and Krein–Rutman theory). Experiments show also that for stochastic processes with nonnegative signals the mean vector is often very similar to the first eigenvector. This is not true in general, but we first give a heuristical explanation why we can expect such a similarity. We then derive a connection between the dominance of the first eigenvalue and the similarity between the mean and the first eigenvector and show how to check the relative size of the first eigenvalue without actually computing it. In the last part of the paper we discuss the implication of theoretical results for multispectral color processing.

© 2005 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. H. B. Barlow, “The coding of sensory messages,” in Current Problems in Animal Behaviour, W. H. Thorpe and O. L. Zangwill, eds. (Cambridge U. Press, Cambridge, UK, 1961) pp. 331–360.
  2. D. J. Field, “Relations between the statistics of natural images and the response properties of cortical cells,” J. Opt. Soc. Am. A 4, 2379–2394 (1987).
    [CrossRef] [PubMed]
  3. J. J. Atick, N. Redlich, “What does the retina know about natural scenes,” Neural Comput. 4, 449–572 (1992).
    [CrossRef]
  4. C. A. Párraga, G. Brelstaff, T. Troscianko, I. R. Moorehead, “Color and luminance information in natural scenes,” J. Opt. Soc. Am. A 15, 563–569 (1998).
    [CrossRef]
  5. P. Reinagel, S. Laughlin, “Natural stimulus statistics,” Network Comput. Neural Syst. 12, 237–240 (2001).
    [CrossRef]
  6. A. Levy, J. Rubinstein, “Hilbert-space Karhunen–Loeve transform with application to image analysis,” J. Opt. Soc. Am. A 16, 28–35 (1999).
    [CrossRef]
  7. F. R. Gantmacher, Matrizentheorie (Springer-Verlag, Berlin 1986).
    [CrossRef]
  8. N. Dunford, J. T. Schwartz, Part III, Spectral Operators (Interscience, New York, 1988).
  9. J. Toland, “Self-adjoint operators and cones,” J. London Math. Soc. Ser. 2 53, 167–183 (1996).
    [CrossRef]
  10. R. D. Nussbaum, “Eigenvectors of order-preserving linear operators,” J. London Math. Soc. Ser. 2 58, 480–496 (1998).
    [CrossRef]
  11. K. Yosida, Functional Analysis (Springer-Verlag, Berlin, 1978).
    [CrossRef]
  12. S. Mallat, A Wavelet Tour of Signal Processing (Academic, New York, 1999).
  13. R. Lenz, “Estimation of illumination characteristics,” IEEE Trans. Image Process. 10, 1031–1038 (2001).
    [CrossRef]
  14. R. Lenz, “Two stage principal component analysis of color,” IEEE Trans. Image Process. 11, 630–635 (2002).
    [CrossRef]
  15. H. Yilmaz, On Color Perception, Vol. XX of International School of Physics, Enrico Fermi (Academic, New York, 1962), pp. 239–251.
  16. H. L. Resnikoff, “Differential geometry and color perception,” J. Math. Biol. 1, 97–131 (1974).
    [CrossRef]
  17. M. Brill, G. West, “Group theory of chromatic adaptation,” Farbe 31, 4–22 (1983/1984).
  18. H. Fairman, M. Brill, “The principal components of reflectances,” Color Res. Appl. 29, 104–110 (2004).
    [CrossRef]
  19. J. Worthey, M. Brill, “Principal components applied to modeling: Dealing with the mean vector,” Color Res. Appl. 29, 261–266 (2004).
    [CrossRef]
  20. M. Brill, “A non-PC look at principal components,” Color Res. Appl. 28, 69–71 (2003).
    [CrossRef]
  21. S. Nascimento, F. Ferreira, D. Foster, “Statistics of spatial cone-excitation ratios in natural scenes,” J. Opt. Soc. Am. A 19, 1484–1490 (2002).
    [CrossRef]
  22. E. Oja, “A simplified neuron model as a principle component analyser,” J. Math. Biol. 15, 267–273 (1982).
    [CrossRef]
  23. J. Parkkinen, J. Hallikainen, T. Jaaskelainen, “Characteristic spectra of Munsell colors,” J. Opt. Soc. Am. A 6, 318–322 (1989).
    [CrossRef]

2004

H. Fairman, M. Brill, “The principal components of reflectances,” Color Res. Appl. 29, 104–110 (2004).
[CrossRef]

J. Worthey, M. Brill, “Principal components applied to modeling: Dealing with the mean vector,” Color Res. Appl. 29, 261–266 (2004).
[CrossRef]

2003

M. Brill, “A non-PC look at principal components,” Color Res. Appl. 28, 69–71 (2003).
[CrossRef]

2002

2001

R. Lenz, “Estimation of illumination characteristics,” IEEE Trans. Image Process. 10, 1031–1038 (2001).
[CrossRef]

P. Reinagel, S. Laughlin, “Natural stimulus statistics,” Network Comput. Neural Syst. 12, 237–240 (2001).
[CrossRef]

1999

1998

C. A. Párraga, G. Brelstaff, T. Troscianko, I. R. Moorehead, “Color and luminance information in natural scenes,” J. Opt. Soc. Am. A 15, 563–569 (1998).
[CrossRef]

R. D. Nussbaum, “Eigenvectors of order-preserving linear operators,” J. London Math. Soc. Ser. 2 58, 480–496 (1998).
[CrossRef]

1996

J. Toland, “Self-adjoint operators and cones,” J. London Math. Soc. Ser. 2 53, 167–183 (1996).
[CrossRef]

1992

J. J. Atick, N. Redlich, “What does the retina know about natural scenes,” Neural Comput. 4, 449–572 (1992).
[CrossRef]

1989

1987

1982

E. Oja, “A simplified neuron model as a principle component analyser,” J. Math. Biol. 15, 267–273 (1982).
[CrossRef]

1974

H. L. Resnikoff, “Differential geometry and color perception,” J. Math. Biol. 1, 97–131 (1974).
[CrossRef]

Atick, J. J.

J. J. Atick, N. Redlich, “What does the retina know about natural scenes,” Neural Comput. 4, 449–572 (1992).
[CrossRef]

Barlow, H. B.

H. B. Barlow, “The coding of sensory messages,” in Current Problems in Animal Behaviour, W. H. Thorpe and O. L. Zangwill, eds. (Cambridge U. Press, Cambridge, UK, 1961) pp. 331–360.

Brelstaff, G.

Brill, M.

H. Fairman, M. Brill, “The principal components of reflectances,” Color Res. Appl. 29, 104–110 (2004).
[CrossRef]

J. Worthey, M. Brill, “Principal components applied to modeling: Dealing with the mean vector,” Color Res. Appl. 29, 261–266 (2004).
[CrossRef]

M. Brill, “A non-PC look at principal components,” Color Res. Appl. 28, 69–71 (2003).
[CrossRef]

M. Brill, G. West, “Group theory of chromatic adaptation,” Farbe 31, 4–22 (1983/1984).

Dunford, N.

N. Dunford, J. T. Schwartz, Part III, Spectral Operators (Interscience, New York, 1988).

Fairman, H.

H. Fairman, M. Brill, “The principal components of reflectances,” Color Res. Appl. 29, 104–110 (2004).
[CrossRef]

Ferreira, F.

Field, D. J.

Foster, D.

Gantmacher, F. R.

F. R. Gantmacher, Matrizentheorie (Springer-Verlag, Berlin 1986).
[CrossRef]

Hallikainen, J.

Jaaskelainen, T.

Laughlin, S.

P. Reinagel, S. Laughlin, “Natural stimulus statistics,” Network Comput. Neural Syst. 12, 237–240 (2001).
[CrossRef]

Lenz, R.

R. Lenz, “Two stage principal component analysis of color,” IEEE Trans. Image Process. 11, 630–635 (2002).
[CrossRef]

R. Lenz, “Estimation of illumination characteristics,” IEEE Trans. Image Process. 10, 1031–1038 (2001).
[CrossRef]

Levy, A.

Mallat, S.

S. Mallat, A Wavelet Tour of Signal Processing (Academic, New York, 1999).

Moorehead, I. R.

Nascimento, S.

Nussbaum, R. D.

R. D. Nussbaum, “Eigenvectors of order-preserving linear operators,” J. London Math. Soc. Ser. 2 58, 480–496 (1998).
[CrossRef]

Oja, E.

E. Oja, “A simplified neuron model as a principle component analyser,” J. Math. Biol. 15, 267–273 (1982).
[CrossRef]

Parkkinen, J.

Párraga, C. A.

Redlich, N.

J. J. Atick, N. Redlich, “What does the retina know about natural scenes,” Neural Comput. 4, 449–572 (1992).
[CrossRef]

Reinagel, P.

P. Reinagel, S. Laughlin, “Natural stimulus statistics,” Network Comput. Neural Syst. 12, 237–240 (2001).
[CrossRef]

Resnikoff, H. L.

H. L. Resnikoff, “Differential geometry and color perception,” J. Math. Biol. 1, 97–131 (1974).
[CrossRef]

Rubinstein, J.

Schwartz, J. T.

N. Dunford, J. T. Schwartz, Part III, Spectral Operators (Interscience, New York, 1988).

Toland, J.

J. Toland, “Self-adjoint operators and cones,” J. London Math. Soc. Ser. 2 53, 167–183 (1996).
[CrossRef]

Troscianko, T.

West, G.

M. Brill, G. West, “Group theory of chromatic adaptation,” Farbe 31, 4–22 (1983/1984).

Worthey, J.

J. Worthey, M. Brill, “Principal components applied to modeling: Dealing with the mean vector,” Color Res. Appl. 29, 261–266 (2004).
[CrossRef]

Yilmaz, H.

H. Yilmaz, On Color Perception, Vol. XX of International School of Physics, Enrico Fermi (Academic, New York, 1962), pp. 239–251.

Yosida, K.

K. Yosida, Functional Analysis (Springer-Verlag, Berlin, 1978).
[CrossRef]

Color Res. Appl.

H. Fairman, M. Brill, “The principal components of reflectances,” Color Res. Appl. 29, 104–110 (2004).
[CrossRef]

J. Worthey, M. Brill, “Principal components applied to modeling: Dealing with the mean vector,” Color Res. Appl. 29, 261–266 (2004).
[CrossRef]

M. Brill, “A non-PC look at principal components,” Color Res. Appl. 28, 69–71 (2003).
[CrossRef]

Farbe

M. Brill, G. West, “Group theory of chromatic adaptation,” Farbe 31, 4–22 (1983/1984).

IEEE Trans. Image Process.

R. Lenz, “Estimation of illumination characteristics,” IEEE Trans. Image Process. 10, 1031–1038 (2001).
[CrossRef]

R. Lenz, “Two stage principal component analysis of color,” IEEE Trans. Image Process. 11, 630–635 (2002).
[CrossRef]

J. London Math. Soc. Ser. 2

J. Toland, “Self-adjoint operators and cones,” J. London Math. Soc. Ser. 2 53, 167–183 (1996).
[CrossRef]

R. D. Nussbaum, “Eigenvectors of order-preserving linear operators,” J. London Math. Soc. Ser. 2 58, 480–496 (1998).
[CrossRef]

J. Math. Biol.

H. L. Resnikoff, “Differential geometry and color perception,” J. Math. Biol. 1, 97–131 (1974).
[CrossRef]

E. Oja, “A simplified neuron model as a principle component analyser,” J. Math. Biol. 15, 267–273 (1982).
[CrossRef]

J. Opt. Soc. Am. A

Network Comput. Neural Syst.

P. Reinagel, S. Laughlin, “Natural stimulus statistics,” Network Comput. Neural Syst. 12, 237–240 (2001).
[CrossRef]

Neural Comput.

J. J. Atick, N. Redlich, “What does the retina know about natural scenes,” Neural Comput. 4, 449–572 (1992).
[CrossRef]

Other

H. B. Barlow, “The coding of sensory messages,” in Current Problems in Animal Behaviour, W. H. Thorpe and O. L. Zangwill, eds. (Cambridge U. Press, Cambridge, UK, 1961) pp. 331–360.

K. Yosida, Functional Analysis (Springer-Verlag, Berlin, 1978).
[CrossRef]

S. Mallat, A Wavelet Tour of Signal Processing (Academic, New York, 1999).

H. Yilmaz, On Color Perception, Vol. XX of International School of Physics, Enrico Fermi (Academic, New York, 1962), pp. 239–251.

F. R. Gantmacher, Matrizentheorie (Springer-Verlag, Berlin 1986).
[CrossRef]

N. Dunford, J. T. Schwartz, Part III, Spectral Operators (Interscience, New York, 1988).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (5)

Fig. 1
Fig. 1

Illumination spectrum, mean of color signals, and first eigenvectors of the correlation and covariance matrices, for the color signals derived from the Munsell chips under the illumination TLD18W35 White.

Fig. 2
Fig. 2

Mean and first eigenvector of the correlation matrix computed from the reflectance spectra in the Munsell database.

Fig. 3
Fig. 3

Mean and first eigenvector of the correlation matrix computed from the color signals in the multispectral image: Scene 1.

Fig. 4
Fig. 4

Ratio of first eigenvalue and trace and its estimation: color signals generated by Munsell spectra under different illuminations; see Table 3 for the corresponding illumination names.

Fig. 5
Fig. 5

Learning the eigenvector from the database.

Tables (3)

Tables Icon

Table 1 Scalar Product of the Normalized Mean and the First Eigenvector of the Correlation Matrix for the Munsell Chips and Multichannel Images of Natural Scenes

Tables Icon

Table 2 Scalar Product of the First Eigenvectors of the Correlation and Covariance Matrices for the Munsell Chips and Multichannel Images

Tables Icon

Table 3 Scalar Product of the First Eigenvectors of the Correlation and the Normalized Mean Vector for the Munsell Chips and Illumination Sources

Equations (17)

Equations on this page are rendered with MathJax. Learn more.

P 1 A P = P A P = [ C 1 0 C 21 C 2 ] .
P 1 A P = P A P = [ C 1 0 0 C 2 ] .
Σ = E ( ( s m ) ( s m ) ) = E ( s s ) m m = C m m ,
A C f ( λ 2 ) = f C ( λ 2 ) = C ( λ 1 , λ 2 ) , f ( λ 1 ) .
s = k = 0 K σ k b k , m = E ( s ) = k = 0 K μ k b k ,
C = E ( k = 0 K σ k b k l = 0 K σ l b l ) = k = 0 K l = 0 K E ( σ k σ l ) b k b l .
E ( σ k σ l ) = α k δ k l .
N = 1 Trace ( C ) C = [ x 0 0 . 0 0 x 1 . 0 0 . . 0 0 0 . x K ]
N 2 = [ x 0 2 0 . 0 0 x 1 2 . 0 0 . . 0 0 0 . x K 2 ] .
S = k = 2 K x k ,
1 = k = 0 K x k = x 0 + x 1 + S ,
γ = Trace ( N 2 ) = k = 0 K x k 2 = ( 1 x 1 S ) 2 + x 1 2 + k = 2 K x k 2 .
0 = γ x m = 2 ( 1 x 1 S ) + 2 x m ,
x 0 = 1 + 2 γ 1 2 .
Σ = C m m C m , b 0 2 b 0 b 0 ,
Σ b 0 = C b 0 m m b 0 β 0 b 0 m , b 0 2 b 0 = ( β 0 m , b 0 2 ) b 0 .
( T f ) ( y ) = K ( x , y ) f ( x ) d x .

Metrics