Abstract

Laser-induced breakdown spectroscopy (LIBS) is an emerging technology that is suitable for a variety of material identification applications. For LIBS to successfully transition from the laboratory into field applications, the sensor must be paired with the appropriate algorithms for accurate and robust processing of the LIBS spectra. In this study we will report on the results of testing classification methods on eight distinct classification tasks using LIBS datasets. Results suggest that standard cross-validation techniques may not accurately estimate generalization performance and a proposed “leave-one-sample-out” approach to experiment design for classifier validation may provide a more robust measure of performance.

© 2012 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. R. Noll, I. Mönch, O. Klein, and A. Lamott, “Concept and operating performance of inspection machines for industrial use based on laser-induced breakdown spectroscopy,” Spectrochim. Acta, Part B: At. Spectrosc. 60, 1070–1075 (2005).
  2. F. R. Doucet, T. F. Belliveau, J. L. Fortier, and J. Hubert, “Use of chemometrics and laser-induced breakdown spectroscopy for quantitative analysis of major and minor elements in aluminum alloys,” Appl. Spectrosc. 61, 327–332 (2007).
    [CrossRef]
  3. R. S. Harmon, J. Remus, N. J. McMillan, C. McManus, L. Collins, J. L. Gottfried, F. C. DeLucia, and A. W. Miziolek, “LIBS analysis of geomaterials: Geochemical fingerprinting for the rapid analysis and discrimination of minerals,” Appl. Geochem. 24, 1125–1141 (2009).
    [CrossRef]
  4. R. C. Chinni, D. A. Cremers, L. J. Radziemski, M. Bostian, and C. Navarro-Northrup, “Detection of uranium using laser-induced breakdown spectroscopy,” Appl. Spectrosc. 63, 1238–1250 (2009).
    [CrossRef]
  5. E. C. Ferreira, D. M. B. P. Milori, E. J. Ferreira, R. M. Silva Da, and L. Martin-Neto, “Artificial neural network for Cu quantitative determination in soil using a portable Laser Induced Breakdown Spectroscopy system,” Spectrochim. Acta, Part B: At. Spectrosc. 63, 1216–1220 (2008).
  6. A. Ramil, A. López, and A. Yáñez, “Application of artificial neural networks for the rapid classification of archaeological ceramics by means of laser induced breakdown spectroscopy (LIBS),” Appl. Phys. A 92, 197–202 (2008).
    [CrossRef]
  7. G. Zadora, “Glass analysis for forensic purposes—a comparison of classification methods,” J. Chemom. 21, 174–186 (2007).
  8. R. Kohavi, “A study of cross-validation and bootstrap for accuracy estimation and model selection,” in Proceedings of the 14th International Joint Conference on Artificial Intelligence (Morgan Kaufmann, 1995), Vol. 2, pp. 1137–1145.
  9. S. de Jong, “SIMPLS: An alternative approach to partial least squares regression,” Chemometr. Intell. Lab. Syst. 18, 251–263 (1993).
  10. B. Matthew and R. William, “Partial least squares for discrimination,” J. Chemom. 17, 166–173 (2003).
    [CrossRef]
  11. L. Breiman, “Random Forests,” Mach. Learn. 45, 5–32 (2001).
    [CrossRef]

2009 (2)

R. S. Harmon, J. Remus, N. J. McMillan, C. McManus, L. Collins, J. L. Gottfried, F. C. DeLucia, and A. W. Miziolek, “LIBS analysis of geomaterials: Geochemical fingerprinting for the rapid analysis and discrimination of minerals,” Appl. Geochem. 24, 1125–1141 (2009).
[CrossRef]

R. C. Chinni, D. A. Cremers, L. J. Radziemski, M. Bostian, and C. Navarro-Northrup, “Detection of uranium using laser-induced breakdown spectroscopy,” Appl. Spectrosc. 63, 1238–1250 (2009).
[CrossRef]

2008 (2)

E. C. Ferreira, D. M. B. P. Milori, E. J. Ferreira, R. M. Silva Da, and L. Martin-Neto, “Artificial neural network for Cu quantitative determination in soil using a portable Laser Induced Breakdown Spectroscopy system,” Spectrochim. Acta, Part B: At. Spectrosc. 63, 1216–1220 (2008).

A. Ramil, A. López, and A. Yáñez, “Application of artificial neural networks for the rapid classification of archaeological ceramics by means of laser induced breakdown spectroscopy (LIBS),” Appl. Phys. A 92, 197–202 (2008).
[CrossRef]

2007 (2)

2005 (1)

R. Noll, I. Mönch, O. Klein, and A. Lamott, “Concept and operating performance of inspection machines for industrial use based on laser-induced breakdown spectroscopy,” Spectrochim. Acta, Part B: At. Spectrosc. 60, 1070–1075 (2005).

2003 (1)

B. Matthew and R. William, “Partial least squares for discrimination,” J. Chemom. 17, 166–173 (2003).
[CrossRef]

2001 (1)

L. Breiman, “Random Forests,” Mach. Learn. 45, 5–32 (2001).
[CrossRef]

1993 (1)

S. de Jong, “SIMPLS: An alternative approach to partial least squares regression,” Chemometr. Intell. Lab. Syst. 18, 251–263 (1993).

Belliveau, T. F.

Bostian, M.

Breiman, L.

L. Breiman, “Random Forests,” Mach. Learn. 45, 5–32 (2001).
[CrossRef]

Chinni, R. C.

Collins, L.

R. S. Harmon, J. Remus, N. J. McMillan, C. McManus, L. Collins, J. L. Gottfried, F. C. DeLucia, and A. W. Miziolek, “LIBS analysis of geomaterials: Geochemical fingerprinting for the rapid analysis and discrimination of minerals,” Appl. Geochem. 24, 1125–1141 (2009).
[CrossRef]

Cremers, D. A.

Da, R. M. Silva

E. C. Ferreira, D. M. B. P. Milori, E. J. Ferreira, R. M. Silva Da, and L. Martin-Neto, “Artificial neural network for Cu quantitative determination in soil using a portable Laser Induced Breakdown Spectroscopy system,” Spectrochim. Acta, Part B: At. Spectrosc. 63, 1216–1220 (2008).

de Jong, S.

S. de Jong, “SIMPLS: An alternative approach to partial least squares regression,” Chemometr. Intell. Lab. Syst. 18, 251–263 (1993).

DeLucia, F. C.

R. S. Harmon, J. Remus, N. J. McMillan, C. McManus, L. Collins, J. L. Gottfried, F. C. DeLucia, and A. W. Miziolek, “LIBS analysis of geomaterials: Geochemical fingerprinting for the rapid analysis and discrimination of minerals,” Appl. Geochem. 24, 1125–1141 (2009).
[CrossRef]

Doucet, F. R.

Ferreira, E. C.

E. C. Ferreira, D. M. B. P. Milori, E. J. Ferreira, R. M. Silva Da, and L. Martin-Neto, “Artificial neural network for Cu quantitative determination in soil using a portable Laser Induced Breakdown Spectroscopy system,” Spectrochim. Acta, Part B: At. Spectrosc. 63, 1216–1220 (2008).

Ferreira, E. J.

E. C. Ferreira, D. M. B. P. Milori, E. J. Ferreira, R. M. Silva Da, and L. Martin-Neto, “Artificial neural network for Cu quantitative determination in soil using a portable Laser Induced Breakdown Spectroscopy system,” Spectrochim. Acta, Part B: At. Spectrosc. 63, 1216–1220 (2008).

Fortier, J. L.

Gottfried, J. L.

R. S. Harmon, J. Remus, N. J. McMillan, C. McManus, L. Collins, J. L. Gottfried, F. C. DeLucia, and A. W. Miziolek, “LIBS analysis of geomaterials: Geochemical fingerprinting for the rapid analysis and discrimination of minerals,” Appl. Geochem. 24, 1125–1141 (2009).
[CrossRef]

Harmon, R. S.

R. S. Harmon, J. Remus, N. J. McMillan, C. McManus, L. Collins, J. L. Gottfried, F. C. DeLucia, and A. W. Miziolek, “LIBS analysis of geomaterials: Geochemical fingerprinting for the rapid analysis and discrimination of minerals,” Appl. Geochem. 24, 1125–1141 (2009).
[CrossRef]

Hubert, J.

Klein, O.

R. Noll, I. Mönch, O. Klein, and A. Lamott, “Concept and operating performance of inspection machines for industrial use based on laser-induced breakdown spectroscopy,” Spectrochim. Acta, Part B: At. Spectrosc. 60, 1070–1075 (2005).

Kohavi, R.

R. Kohavi, “A study of cross-validation and bootstrap for accuracy estimation and model selection,” in Proceedings of the 14th International Joint Conference on Artificial Intelligence (Morgan Kaufmann, 1995), Vol. 2, pp. 1137–1145.

Lamott, A.

R. Noll, I. Mönch, O. Klein, and A. Lamott, “Concept and operating performance of inspection machines for industrial use based on laser-induced breakdown spectroscopy,” Spectrochim. Acta, Part B: At. Spectrosc. 60, 1070–1075 (2005).

López, A.

A. Ramil, A. López, and A. Yáñez, “Application of artificial neural networks for the rapid classification of archaeological ceramics by means of laser induced breakdown spectroscopy (LIBS),” Appl. Phys. A 92, 197–202 (2008).
[CrossRef]

Martin-Neto, L.

E. C. Ferreira, D. M. B. P. Milori, E. J. Ferreira, R. M. Silva Da, and L. Martin-Neto, “Artificial neural network for Cu quantitative determination in soil using a portable Laser Induced Breakdown Spectroscopy system,” Spectrochim. Acta, Part B: At. Spectrosc. 63, 1216–1220 (2008).

Matthew, B.

B. Matthew and R. William, “Partial least squares for discrimination,” J. Chemom. 17, 166–173 (2003).
[CrossRef]

McManus, C.

R. S. Harmon, J. Remus, N. J. McMillan, C. McManus, L. Collins, J. L. Gottfried, F. C. DeLucia, and A. W. Miziolek, “LIBS analysis of geomaterials: Geochemical fingerprinting for the rapid analysis and discrimination of minerals,” Appl. Geochem. 24, 1125–1141 (2009).
[CrossRef]

McMillan, N. J.

R. S. Harmon, J. Remus, N. J. McMillan, C. McManus, L. Collins, J. L. Gottfried, F. C. DeLucia, and A. W. Miziolek, “LIBS analysis of geomaterials: Geochemical fingerprinting for the rapid analysis and discrimination of minerals,” Appl. Geochem. 24, 1125–1141 (2009).
[CrossRef]

Milori, D. M. B. P.

E. C. Ferreira, D. M. B. P. Milori, E. J. Ferreira, R. M. Silva Da, and L. Martin-Neto, “Artificial neural network for Cu quantitative determination in soil using a portable Laser Induced Breakdown Spectroscopy system,” Spectrochim. Acta, Part B: At. Spectrosc. 63, 1216–1220 (2008).

Miziolek, A. W.

R. S. Harmon, J. Remus, N. J. McMillan, C. McManus, L. Collins, J. L. Gottfried, F. C. DeLucia, and A. W. Miziolek, “LIBS analysis of geomaterials: Geochemical fingerprinting for the rapid analysis and discrimination of minerals,” Appl. Geochem. 24, 1125–1141 (2009).
[CrossRef]

Mönch, I.

R. Noll, I. Mönch, O. Klein, and A. Lamott, “Concept and operating performance of inspection machines for industrial use based on laser-induced breakdown spectroscopy,” Spectrochim. Acta, Part B: At. Spectrosc. 60, 1070–1075 (2005).

Navarro-Northrup, C.

Noll, R.

R. Noll, I. Mönch, O. Klein, and A. Lamott, “Concept and operating performance of inspection machines for industrial use based on laser-induced breakdown spectroscopy,” Spectrochim. Acta, Part B: At. Spectrosc. 60, 1070–1075 (2005).

Radziemski, L. J.

Ramil, A.

A. Ramil, A. López, and A. Yáñez, “Application of artificial neural networks for the rapid classification of archaeological ceramics by means of laser induced breakdown spectroscopy (LIBS),” Appl. Phys. A 92, 197–202 (2008).
[CrossRef]

Remus, J.

R. S. Harmon, J. Remus, N. J. McMillan, C. McManus, L. Collins, J. L. Gottfried, F. C. DeLucia, and A. W. Miziolek, “LIBS analysis of geomaterials: Geochemical fingerprinting for the rapid analysis and discrimination of minerals,” Appl. Geochem. 24, 1125–1141 (2009).
[CrossRef]

William, R.

B. Matthew and R. William, “Partial least squares for discrimination,” J. Chemom. 17, 166–173 (2003).
[CrossRef]

Yáñez, A.

A. Ramil, A. López, and A. Yáñez, “Application of artificial neural networks for the rapid classification of archaeological ceramics by means of laser induced breakdown spectroscopy (LIBS),” Appl. Phys. A 92, 197–202 (2008).
[CrossRef]

Zadora, G.

G. Zadora, “Glass analysis for forensic purposes—a comparison of classification methods,” J. Chemom. 21, 174–186 (2007).

Appl. Geochem. (1)

R. S. Harmon, J. Remus, N. J. McMillan, C. McManus, L. Collins, J. L. Gottfried, F. C. DeLucia, and A. W. Miziolek, “LIBS analysis of geomaterials: Geochemical fingerprinting for the rapid analysis and discrimination of minerals,” Appl. Geochem. 24, 1125–1141 (2009).
[CrossRef]

Appl. Phys. A (1)

A. Ramil, A. López, and A. Yáñez, “Application of artificial neural networks for the rapid classification of archaeological ceramics by means of laser induced breakdown spectroscopy (LIBS),” Appl. Phys. A 92, 197–202 (2008).
[CrossRef]

Appl. Spectrosc. (2)

Chemometr. Intell. Lab. Syst. (1)

S. de Jong, “SIMPLS: An alternative approach to partial least squares regression,” Chemometr. Intell. Lab. Syst. 18, 251–263 (1993).

J. Chemom. (2)

B. Matthew and R. William, “Partial least squares for discrimination,” J. Chemom. 17, 166–173 (2003).
[CrossRef]

G. Zadora, “Glass analysis for forensic purposes—a comparison of classification methods,” J. Chemom. 21, 174–186 (2007).

Mach. Learn. (1)

L. Breiman, “Random Forests,” Mach. Learn. 45, 5–32 (2001).
[CrossRef]

Spectrochim. Acta, Part B: At. Spectrosc. (2)

R. Noll, I. Mönch, O. Klein, and A. Lamott, “Concept and operating performance of inspection machines for industrial use based on laser-induced breakdown spectroscopy,” Spectrochim. Acta, Part B: At. Spectrosc. 60, 1070–1075 (2005).

E. C. Ferreira, D. M. B. P. Milori, E. J. Ferreira, R. M. Silva Da, and L. Martin-Neto, “Artificial neural network for Cu quantitative determination in soil using a portable Laser Induced Breakdown Spectroscopy system,” Spectrochim. Acta, Part B: At. Spectrosc. 63, 1216–1220 (2008).

Other (1)

R. Kohavi, “A study of cross-validation and bootstrap for accuracy estimation and model selection,” in Proceedings of the 14th International Joint Conference on Artificial Intelligence (Morgan Kaufmann, 1995), Vol. 2, pp. 1137–1145.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (5)

Fig. 1.
Fig. 1.

Illustration of the process of partitioning the data using either k-folds cross-validation (left) or the proposed leave-one-sample-out (right). The characters A, B, C, etc. represent samples, i.e., distinct physical objects that would be placed separately into the sample chamber. Subscripts are indices for the shots collected from that object. The grouped partitions constitute the data subsets that are held out one at a time for testing while the classifier is trained on the remaining data. Note that with the leave-one-sample-out approach, all of the data from one sample is sequestered during training.

Fig. 2.
Fig. 2.

Classification results using PLSDA on the eight classification tasks (one subplot per task). In each subplot, the percent correct classification is shown as a function of the number of PLSDA components using three experiment designs: Leave-One-Sample-Out cross-validation (dot markers), k-folds cross-validation (circle markers), and no cross-validation with training and testing on the same data (x markers).

Fig. 3.
Fig. 3.

Classification results using the Random Forest classifier on the eight classification tasks (one subplot per task). In each subplot, the percent correct classification is shown as a function of the number of component classification trees using three experiment designs: Leave-One-Sample-Out cross-validation (dot markers), k-folds cross-validation (circle markers), and no cross-validation with training and testing on the same data (x markers). Note that the Random Forest classifier, by definition, performs perfectly on its training data (the no cross-validation scenario).

Fig. 4.
Fig. 4.

Plots of the PLSDA weights projected into two-dimensions using principal components analysis. The PLSDA weights for each of the eight classification tasks are shown in a separate subplot. Within each subplot, there is one square marker representing the PLSDA weights generated using no cross-validation and all training data and N markers each for the k-folds and Leave-One-Sample-Out cross-validation methods (where N is the number of samples in the data set).

Fig. 5.
Fig. 5.

(Top) Matrix of correlation coefficients calculated for four arbitrarily-selected shots from each of nine samples from the Ink data set. The solid white lines identify samples within the same class (three classes are represented in the image) and black dashed lines distinguish between samples. The block-diagonal structure suggests high correlation between shots from the same sample. (Bottom) Histogram of the probabilities, based on the 36 shots shown in the image, that a shot from a different sample but within the same class will be less correlated than a shot from same sample. Higher probabilities suggest that the minimum within-sample correlation will be greater than between-sample correlation.

Tables (2)

Tables Icon

Table 1. Specifications for the Five LIBS Data Collections Used in this Study: Material Type, Number of Samples in Each Data Collection, Number of Lines in the Spectra, Laser Wavelength, Number of Shots Collected per Sample, and Categories for which Classification Labels can be Generated

Tables Icon

Table 2. Data Sets, Number of Classes, and the Number of Samples within Each Class for the Eight Classification Tasks Performed in This Study

Equations (1)

Equations on this page are rendered with MathJax. Learn more.

1Mm=1MI{ρAm<ρminAA},

Metrics