Abstract

Existing manifold learning algorithms use Euclidean distance to measure the proximity of data points. However, in high-dimensional space, Minkowski metrics are no longer stable because the ratio of distance of nearest and farthest neighbors to a given query is almost unit. It will degrade the performance of manifold learning algorithms when applied to dimensionality reduction of high-dimensional data. We introduce a new distance function named shrinkage-divergence-proximity (SDP) to manifold learning, which is meaningful in any high-dimensional space. An improved locally linear embedding (LLE) algorithm named SDP-LLE is proposed in light of the theoretical result. Experiments are conducted on a hyperspectral data set and an image segmentation data set. Experimental results show that the proposed method can efficiently reduce the dimensionality while getting higher classification accuracy.

© 2008 Chinese Optics Letters

PDF Article

References

  • View by:
  • |
  • |
  • |

  1. J. Huang, R. Zhu, J. Li, and Y. He, Chin. Opt. Lett. 5, 393 (2007).
  2. L. Xun, Y. Fang, and X. Li, Acta Opt. Sin. 27, 1178 (2007).
  3. Y. Zhu, P. K. Varshney, and H. Chen, in Proceedings of IEEE International Conference on Image Processing 2007 4, 97 (2007).
  4. Y. Chen, M. M. Crawford, and J. Ghosh, in Proceedings of 2005 IEEE International Geoscience and Remote Sensing Symposium 6, 4311 (2005).
  5. D. L. Donoho, "High dimensional data analysis: the curses and blessings of dimensionality" Lecture at the "Mathematical Challenges of the 21st Century" Conference of the American Math. Society (2000). Currently available on the web at http://www.stat.stanford.edu/~donoho/Lectures/ AMS2000/AMS2000.html (April 25, 2008).
  6. K. Beyer, J. Goldstein, R. Ramakrishnan, and U. Shaft, Lecture Notes in Computer Science 1540, 217 (1999).
  7. C.-M. Hsu and M.-S. Chen, in Proceedings of the 6th SIAM International Conference on Data Mining 12 (2006).
  8. A. Hinneburg, C. Aggarwal, and D. A. Keim, in Proceedings of the 26th International Conference on Very Large Data Bases 506 (2000).
  9. J. B. Tenenbaum, V. de Silva, and J. C. Langford, Science 290, 2319 (2000).
  10. S. T. Roweis and L. K. Saul, Science 290, 2323 (2000).
  11. M. Belkin and P. Niyogi, "Laplacian eigenmaps and spectral techniques for embedding and clustering" in Advances in Neural Information Processing Systems vol.14 (MIT Press, Cambridge, 2002) pp.585-591.
  12. G. Hinton and S. Roweis, "Stochastic neighbor embedding" in Advances in Neural Information Processing Systems vol.15 (MIT Press, Cambridge, 2003) pp.833-840.
  13. L. K. Saul and S. T. Roweis, "An introduction to locally linear embedding" http://www.cs.toronto.edu/~roweis/ lle/papers/lleintro.pdf (July 18, 2007).
  14. O. Kouropteva, O. Okun, and M. Pietikainen, in Proceedings of the 1st International Conference on Fuzzy Systems and Knowledge Discovery 359 (2002).
  15. F. Provost, T. Fawcett, and R. Kohavi, in Proceedings of the 15th International Conference on Machine Learning 445 (1998).

2007 (2)

L. Xun, Y. Fang, and X. Li, Acta Opt. Sin. 27, 1178 (2007).

J. Huang, R. Zhu, J. Li, and Y. He, Chin. Opt. Lett. 5, 393 (2007).

2000 (2)

J. B. Tenenbaum, V. de Silva, and J. C. Langford, Science 290, 2319 (2000).

S. T. Roweis and L. K. Saul, Science 290, 2323 (2000).

1999 (1)

K. Beyer, J. Goldstein, R. Ramakrishnan, and U. Shaft, Lecture Notes in Computer Science 1540, 217 (1999).

Beyer, K.

K. Beyer, J. Goldstein, R. Ramakrishnan, and U. Shaft, Lecture Notes in Computer Science 1540, 217 (1999).

Fang, Y.

L. Xun, Y. Fang, and X. Li, Acta Opt. Sin. 27, 1178 (2007).

Goldstein, J.

K. Beyer, J. Goldstein, R. Ramakrishnan, and U. Shaft, Lecture Notes in Computer Science 1540, 217 (1999).

He, Y.

Huang, J.

Langford, J. C.

J. B. Tenenbaum, V. de Silva, and J. C. Langford, Science 290, 2319 (2000).

Li, J.

Li, X.

L. Xun, Y. Fang, and X. Li, Acta Opt. Sin. 27, 1178 (2007).

Ramakrishnan, R.

K. Beyer, J. Goldstein, R. Ramakrishnan, and U. Shaft, Lecture Notes in Computer Science 1540, 217 (1999).

Roweis, S. T.

S. T. Roweis and L. K. Saul, Science 290, 2323 (2000).

Saul, L. K.

S. T. Roweis and L. K. Saul, Science 290, 2323 (2000).

Shaft, U.

K. Beyer, J. Goldstein, R. Ramakrishnan, and U. Shaft, Lecture Notes in Computer Science 1540, 217 (1999).

Silva, V. de

J. B. Tenenbaum, V. de Silva, and J. C. Langford, Science 290, 2319 (2000).

Tenenbaum, J. B.

J. B. Tenenbaum, V. de Silva, and J. C. Langford, Science 290, 2319 (2000).

Xun, L.

L. Xun, Y. Fang, and X. Li, Acta Opt. Sin. 27, 1178 (2007).

Zhu, R.

Acta Opt. Sin. (1)

L. Xun, Y. Fang, and X. Li, Acta Opt. Sin. 27, 1178 (2007).

Chin. Opt. Lett. (1)

Lecture Notes in Computer Science (1)

K. Beyer, J. Goldstein, R. Ramakrishnan, and U. Shaft, Lecture Notes in Computer Science 1540, 217 (1999).

Science (2)

J. B. Tenenbaum, V. de Silva, and J. C. Langford, Science 290, 2319 (2000).

S. T. Roweis and L. K. Saul, Science 290, 2323 (2000).

Other (10)

M. Belkin and P. Niyogi, "Laplacian eigenmaps and spectral techniques for embedding and clustering" in Advances in Neural Information Processing Systems vol.14 (MIT Press, Cambridge, 2002) pp.585-591.

G. Hinton and S. Roweis, "Stochastic neighbor embedding" in Advances in Neural Information Processing Systems vol.15 (MIT Press, Cambridge, 2003) pp.833-840.

L. K. Saul and S. T. Roweis, "An introduction to locally linear embedding" http://www.cs.toronto.edu/~roweis/ lle/papers/lleintro.pdf (July 18, 2007).

O. Kouropteva, O. Okun, and M. Pietikainen, in Proceedings of the 1st International Conference on Fuzzy Systems and Knowledge Discovery 359 (2002).

F. Provost, T. Fawcett, and R. Kohavi, in Proceedings of the 15th International Conference on Machine Learning 445 (1998).

Y. Zhu, P. K. Varshney, and H. Chen, in Proceedings of IEEE International Conference on Image Processing 2007 4, 97 (2007).

Y. Chen, M. M. Crawford, and J. Ghosh, in Proceedings of 2005 IEEE International Geoscience and Remote Sensing Symposium 6, 4311 (2005).

D. L. Donoho, "High dimensional data analysis: the curses and blessings of dimensionality" Lecture at the "Mathematical Challenges of the 21st Century" Conference of the American Math. Society (2000). Currently available on the web at http://www.stat.stanford.edu/~donoho/Lectures/ AMS2000/AMS2000.html (April 25, 2008).

C.-M. Hsu and M.-S. Chen, in Proceedings of the 6th SIAM International Conference on Data Mining 12 (2006).

A. Hinneburg, C. Aggarwal, and D. A. Keim, in Proceedings of the 26th International Conference on Very Large Data Bases 506 (2000).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.