Abstract

A new detection algorithm for lidar applications has been developed. The detection is based on hyperspectral anomaly detection that is implemented for time anomaly where the question “is a target (aerosol cloud) present at range R within time t1 to t2” is addressed, and for range anomaly where the question “is a target present at time t within ranges R1 and R2” is addressed. A detection score significantly different in magnitude from the detection scores for background measurements suggests that an anomaly (interpreted as the presence of a target signal in space∕time) exists. The algorithm employs an option for a preprocessing stage where undesired oscillations and artifacts are filtered out with a low-rank orthogonal projection technique. The filtering technique adaptively removes the one over range-squared dependence of the background contribution of the lidar signal and also aids visualization of features in the data when the signal-to-noise ratio is low. A Gaussian-mixture probability model for two hypotheses (anomaly present or absent) is computed with an expectation-maximization algorithm to produce a detection threshold and probabilities of detection and false alarm. Results of the algorithm for CO2 lidar measurements of bioaerosol clouds Bacillus atrophaeus (formerly known as Bacillus subtilis niger, BG) and Pantoea agglomerans, Pa (formerly known as Erwinia herbicola, Eh) are shown and discussed.

© 2007 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |

  1. R. T. H. Collis and P. B. Russell, "Lidar measurement of particles and gases by elastic backscattering and differential absorption," in Laser Monitoring of the Atmosphere, E. D. Hinkley, ed. (Springer-Verlag, 1976), Chap. 4, pp. 71-151.
  2. R. M. Measures, Laser Remote Sensing: Fundamentals and Applications (Wiley, 1984).
  3. R. M. Measures, Laser Remote Chemical Analysis (Wiley, 1988).
  4. W. B. Grant, "Lidar for atmospheric and hydrospheric studies," in Tunable Laser Applications, F. J. Durate, ed. (Dekker, 1995), pp. 213-305.
  5. J. D. Klett, "Stable analytical inversion solution for processing lidar returns," Appl. Opt. 20, 211-220 (1981).
    [CrossRef] [PubMed]
  6. J. D. Klett, "Lidar inversion with variable backscatter/extinction ratios," Appl. Opt. 24, 1638-1643 (1985).
    [CrossRef] [PubMed]
  7. F. G. Fernald, "Analysis of atmospheric lidar observations: some comments," Appl. Opt. 23, 652-653 (1984).
    [CrossRef] [PubMed]
  8. V. A. Kovalev, "Lidar measurement of the vertical aerosol extinction profiles with range-dependent backscatter-to-extinction ratios," Appl. Opt. 32, 6053-6065 (1993).
    [CrossRef] [PubMed]
  9. L. R. Bissonnette, "Sensitivity analysis of lidar incersion algorithms," Appl. Opt. 25, 2122-2125 (1986).
    [CrossRef] [PubMed]
  10. F. Rocadenbosch and A. Comeron, "Error analysis for the lidar backward inversion algorithm," Appl. Opt. 38, 4461-4474 (1999).
    [CrossRef]
  11. R. T. H. Collis, "Lidar: a new atmospheric probe," Q. J. R. Meteorol. Soc. 92, 220-230 (1966).
    [CrossRef]
  12. G. J. Kunz and G. de Leeuw, "Inversion of lidar signals with the slope method," Appl. Opt. 32, 3249-3256 (1993).
    [CrossRef] [PubMed]
  13. A. Ben-David, "Mueller matrix for atmospheric aerosols at CO2 wavelengths from backscattering polarized lidar measurements," J. Geophys. Res. 103, 26041-26050 (1998).
    [CrossRef]
  14. A. Ben-David, "Backscattering measurements of atmospheric aerosols at CO2 laser wavelengths: implications of aerosol spectral structure on differential absorption lidar retrieval of molecular species," Appl. Opt. 38, 2616-2624 (1999).
    [CrossRef]
  15. F. Rocadenbosch, A. Comeron, and D. Pineda, "Assessment of lidar inversion errors for homogeneous atmospheres," Appl. Opt. 37, 2199-2206 (1998).
    [CrossRef]
  16. F. Rocadenbosch, C. Soriano, A. Comeron, and J.-M. Baldasano, "Lidar inversion of atmospheric backscatter and extinction-to-backscatter ratios by use of a Kalman filter," Appl. Opt. 38, 3175-3189 (1999).
    [CrossRef]
  17. A. Ansmann, U. Wandinger, M. Reibessel, C. Weitcamp, and M. Michaelis, "Independent measurements of extinction and backscatter profiles in cirrus clouds by using combined Raman elastic-backscatter lidar," Appl. Opt. 31, 7113-7131 (1992).
    [CrossRef] [PubMed]
  18. E. P. Zege, I. L. Katsev, and I. N. Polonsky, "Analytical solution to lidar return signals from clouds with regard to multiple scattering," Appl. Phys. B 60, 345-353 (1995).
    [CrossRef]
  19. S. V. Samoilova, Y. S. Balin, M. M. Krekova, and D. M. Winker, "Method for reconstructing atmospheric optical parameters from the data of polarized lidar sensing," Appl. Opt. 44, 3499-3509 (2005).
    [CrossRef] [PubMed]
  20. I. S. Reed and X. Yu, "Adaptive multiple-band CFAR detection of an optical pattern with unknown spectral distribution," IEEE Trans. Acoust. Speech Signal Process. 38, 1760-1770 (1990).
    [CrossRef]
  21. D. W. J. Stein, S. G. Beaven, L. E. Hoff, E. M. Winter, A. P. Schaum, and A. D. Stocker, "Anomaly detection from hyperspectral imagery," IEEE Signal Process Mag. 19, 58-69 (2002).
    [CrossRef]
  22. C.-I. Chang, Hyperspectral Imaging: Techniques for Spectral Detection and Classification (Kluwer Academic/Plenum, 2003).
  23. S. M. Kay, Fundamentals of Statistical Signal Processing: Detection Theory (Prentice Hall, 1998), Vol. II.
  24. J. Harsanyi and C.-I. Chang, "Hyperspectral image classification and dimensionality reduction: an orthogonal subspace projection approach," IEEE Trans. Geocsi. Remote Sens. 32, 779-785 (1994).
    [CrossRef]
  25. L. L. Scharf, Statistical Signal Processing, Detection, Estimation, and Time Series Analysis (Addison-Wesley, 1991).
  26. D. Manolakis, D. Marden, and G. A. Shaw, "Hyperspectral image processing for automatic target detection applications," Lincoln Lab. J. 14, 79-116 (2003).
  27. D. A. Landgrebe, Signal Theory Methods in Multispectral Remote Sensing (Wiley, 2003).
    [CrossRef]
  28. A. Ben-David, "Temperature dependence of water vapor absorption coefficients for CO2 differential absorption lidars," Appl. Opt. 32, 7479-7483 (1993).
    [CrossRef] [PubMed]
  29. A. Ben-David, "Backscattering measurements of atmospheric aerosols at CO2 laser wavelengths: implications of aerosol spectral structure on differential absorption lidar retrieval of molecular species," Appl. Opt. 38, 2616-2624 (1999).
    [CrossRef]
  30. A. Ben-David and H. Ren, "Detection, identification, and estimation of biological aerosols and vapors with Fourier transform infrared spectrometer," Appl. Opt. 42, 4887-4900 (2003).
    [CrossRef] [PubMed]
  31. N. L. Johnson, S. Kotz, and N. Balakrishnan, Continuous Univariate Distributions (Wiley, 1994), Vol. 1.
  32. C. M. Bishop, Neural Networks for Pattern Recognition (Oxford U. Press, 1995), Chap. 3.
  33. A. Ben-David, "Remote detection of biological aerosols at a distance of 3 km with a passive Fourier transform infrared (FTIR)," Opt. Express 11, 418-429 (2003).
    [CrossRef] [PubMed]
  34. H. Cramér, Mathematical Methods of Statistics (Princeton U. Press, 1999), Chap. 17.4.
  35. A. Ben-David, "Optimal bandwidth for topographical DIAL detection," Appl. Opt. 35, 1531-1536 (1996).
    [CrossRef] [PubMed]

2005 (1)

2003 (3)

2002 (1)

D. W. J. Stein, S. G. Beaven, L. E. Hoff, E. M. Winter, A. P. Schaum, and A. D. Stocker, "Anomaly detection from hyperspectral imagery," IEEE Signal Process Mag. 19, 58-69 (2002).
[CrossRef]

1999 (4)

1998 (2)

F. Rocadenbosch, A. Comeron, and D. Pineda, "Assessment of lidar inversion errors for homogeneous atmospheres," Appl. Opt. 37, 2199-2206 (1998).
[CrossRef]

A. Ben-David, "Mueller matrix for atmospheric aerosols at CO2 wavelengths from backscattering polarized lidar measurements," J. Geophys. Res. 103, 26041-26050 (1998).
[CrossRef]

1996 (1)

1995 (1)

E. P. Zege, I. L. Katsev, and I. N. Polonsky, "Analytical solution to lidar return signals from clouds with regard to multiple scattering," Appl. Phys. B 60, 345-353 (1995).
[CrossRef]

1994 (1)

J. Harsanyi and C.-I. Chang, "Hyperspectral image classification and dimensionality reduction: an orthogonal subspace projection approach," IEEE Trans. Geocsi. Remote Sens. 32, 779-785 (1994).
[CrossRef]

1993 (3)

1992 (1)

1990 (1)

I. S. Reed and X. Yu, "Adaptive multiple-band CFAR detection of an optical pattern with unknown spectral distribution," IEEE Trans. Acoust. Speech Signal Process. 38, 1760-1770 (1990).
[CrossRef]

1986 (1)

1985 (1)

1984 (1)

1981 (1)

1966 (1)

R. T. H. Collis, "Lidar: a new atmospheric probe," Q. J. R. Meteorol. Soc. 92, 220-230 (1966).
[CrossRef]

Appl. Opt. (16)

J. D. Klett, "Stable analytical inversion solution for processing lidar returns," Appl. Opt. 20, 211-220 (1981).
[CrossRef] [PubMed]

J. D. Klett, "Lidar inversion with variable backscatter/extinction ratios," Appl. Opt. 24, 1638-1643 (1985).
[CrossRef] [PubMed]

F. G. Fernald, "Analysis of atmospheric lidar observations: some comments," Appl. Opt. 23, 652-653 (1984).
[CrossRef] [PubMed]

V. A. Kovalev, "Lidar measurement of the vertical aerosol extinction profiles with range-dependent backscatter-to-extinction ratios," Appl. Opt. 32, 6053-6065 (1993).
[CrossRef] [PubMed]

L. R. Bissonnette, "Sensitivity analysis of lidar incersion algorithms," Appl. Opt. 25, 2122-2125 (1986).
[CrossRef] [PubMed]

F. Rocadenbosch and A. Comeron, "Error analysis for the lidar backward inversion algorithm," Appl. Opt. 38, 4461-4474 (1999).
[CrossRef]

A. Ben-David, "Backscattering measurements of atmospheric aerosols at CO2 laser wavelengths: implications of aerosol spectral structure on differential absorption lidar retrieval of molecular species," Appl. Opt. 38, 2616-2624 (1999).
[CrossRef]

F. Rocadenbosch, A. Comeron, and D. Pineda, "Assessment of lidar inversion errors for homogeneous atmospheres," Appl. Opt. 37, 2199-2206 (1998).
[CrossRef]

F. Rocadenbosch, C. Soriano, A. Comeron, and J.-M. Baldasano, "Lidar inversion of atmospheric backscatter and extinction-to-backscatter ratios by use of a Kalman filter," Appl. Opt. 38, 3175-3189 (1999).
[CrossRef]

A. Ansmann, U. Wandinger, M. Reibessel, C. Weitcamp, and M. Michaelis, "Independent measurements of extinction and backscatter profiles in cirrus clouds by using combined Raman elastic-backscatter lidar," Appl. Opt. 31, 7113-7131 (1992).
[CrossRef] [PubMed]

G. J. Kunz and G. de Leeuw, "Inversion of lidar signals with the slope method," Appl. Opt. 32, 3249-3256 (1993).
[CrossRef] [PubMed]

S. V. Samoilova, Y. S. Balin, M. M. Krekova, and D. M. Winker, "Method for reconstructing atmospheric optical parameters from the data of polarized lidar sensing," Appl. Opt. 44, 3499-3509 (2005).
[CrossRef] [PubMed]

A. Ben-David, "Temperature dependence of water vapor absorption coefficients for CO2 differential absorption lidars," Appl. Opt. 32, 7479-7483 (1993).
[CrossRef] [PubMed]

A. Ben-David, "Backscattering measurements of atmospheric aerosols at CO2 laser wavelengths: implications of aerosol spectral structure on differential absorption lidar retrieval of molecular species," Appl. Opt. 38, 2616-2624 (1999).
[CrossRef]

A. Ben-David and H. Ren, "Detection, identification, and estimation of biological aerosols and vapors with Fourier transform infrared spectrometer," Appl. Opt. 42, 4887-4900 (2003).
[CrossRef] [PubMed]

A. Ben-David, "Optimal bandwidth for topographical DIAL detection," Appl. Opt. 35, 1531-1536 (1996).
[CrossRef] [PubMed]

Appl. Phys. B (1)

E. P. Zege, I. L. Katsev, and I. N. Polonsky, "Analytical solution to lidar return signals from clouds with regard to multiple scattering," Appl. Phys. B 60, 345-353 (1995).
[CrossRef]

IEEE Signal Process Mag. (1)

D. W. J. Stein, S. G. Beaven, L. E. Hoff, E. M. Winter, A. P. Schaum, and A. D. Stocker, "Anomaly detection from hyperspectral imagery," IEEE Signal Process Mag. 19, 58-69 (2002).
[CrossRef]

IEEE Trans. Acoust. Speech Signal Process. (1)

I. S. Reed and X. Yu, "Adaptive multiple-band CFAR detection of an optical pattern with unknown spectral distribution," IEEE Trans. Acoust. Speech Signal Process. 38, 1760-1770 (1990).
[CrossRef]

IEEE Trans. Geocsi. Remote Sens. (1)

J. Harsanyi and C.-I. Chang, "Hyperspectral image classification and dimensionality reduction: an orthogonal subspace projection approach," IEEE Trans. Geocsi. Remote Sens. 32, 779-785 (1994).
[CrossRef]

J. Geophys. Res. (1)

A. Ben-David, "Mueller matrix for atmospheric aerosols at CO2 wavelengths from backscattering polarized lidar measurements," J. Geophys. Res. 103, 26041-26050 (1998).
[CrossRef]

Lincoln Lab. J. (1)

D. Manolakis, D. Marden, and G. A. Shaw, "Hyperspectral image processing for automatic target detection applications," Lincoln Lab. J. 14, 79-116 (2003).

Opt. Express (1)

Q. J. R. Meteorol. Soc. (1)

R. T. H. Collis, "Lidar: a new atmospheric probe," Q. J. R. Meteorol. Soc. 92, 220-230 (1966).
[CrossRef]

Other (11)

R. T. H. Collis and P. B. Russell, "Lidar measurement of particles and gases by elastic backscattering and differential absorption," in Laser Monitoring of the Atmosphere, E. D. Hinkley, ed. (Springer-Verlag, 1976), Chap. 4, pp. 71-151.

R. M. Measures, Laser Remote Sensing: Fundamentals and Applications (Wiley, 1984).

R. M. Measures, Laser Remote Chemical Analysis (Wiley, 1988).

W. B. Grant, "Lidar for atmospheric and hydrospheric studies," in Tunable Laser Applications, F. J. Durate, ed. (Dekker, 1995), pp. 213-305.

H. Cramér, Mathematical Methods of Statistics (Princeton U. Press, 1999), Chap. 17.4.

D. A. Landgrebe, Signal Theory Methods in Multispectral Remote Sensing (Wiley, 2003).
[CrossRef]

L. L. Scharf, Statistical Signal Processing, Detection, Estimation, and Time Series Analysis (Addison-Wesley, 1991).

C.-I. Chang, Hyperspectral Imaging: Techniques for Spectral Detection and Classification (Kluwer Academic/Plenum, 2003).

S. M. Kay, Fundamentals of Statistical Signal Processing: Detection Theory (Prentice Hall, 1998), Vol. II.

N. L. Johnson, S. Kotz, and N. Balakrishnan, Continuous Univariate Distributions (Wiley, 1994), Vol. 1.

C. M. Bishop, Neural Networks for Pattern Recognition (Oxford U. Press, 1995), Chap. 3.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (8)

Fig. 1
Fig. 1

Two-dimensional scatter plot for two-band ( b 1 , b 2 ) anomaly detection. (a) Background X H 0 measurements in the measurements space in arbitrary units. (b) Background X H 0 measurements, X w H 0 = W ( X H 0 E [ X H 0 ] ) , using the symmetric whitening operator W = U S 1 / 2 U T , in the whitened space; 10% of the background data lies outside the threshold distance. (c) Signal X H 1 (noted as three anomalies clusters, c1, c2, and c3) superimposed on background X H 0 in the measurement space in arbitrary units. (d) Signal X H 1 X w H 1 and background X H 0 X w H 0 in the whitened space using X w H 1 = W ( X H 1 E [ X H 0 ] ) and X w H 0 = W ( X H 0 E [ X H 0 ] ) computed with the symmetric whitening operator. All data points outside the threshold radius are declared as “detection,” background data points outside the threshold radius cause false alarm (10%), and signal data points within the threshold radius cause a missed detection. (e) Same as (d) but computed with the nonsymmetric whitening W = S 1 / 2 U T to show that the signal X w H 1 (anomaly data, noted as c1, c2, and c3 in the figure) is not in the same orientation (due to rotation and reflection affected by U T ) with respect to the original X H 1 orientation of the three anomalies c1, c2, and c3 in (c).

Fig. 2
Fig. 2

Detection algorithm flow chart. The lidar detection algorithm is comprised of three steps: a preprocessing∕filtering stage, anomaly detection, and a probability model. The preprocessing filter is optional for the range-anomaly detector, but is mandatory for time anomaly.

Fig. 3
Fig. 3

Lidar signals (Utah, 16 June 2005) from an airborne BG line release at 20 m above the ground and a range of 2200   m measured at wavelength 10.260 μ m (10R18) as a function of range where large oscillations (hardware problems) are seen. (a) Raw data where oscillations as a function of range are shown. (b) Filtered data (using three eigenvectors) where the oscillations are removed and the BG cloud at 2200   m are clearly seen.

Fig. 4
Fig. 4

Same as Fig. 3 but for a weak CO 2 wavelength 9.695 μ m (9P36) for which the SNR is lower and the raw signal from the cloud (a) is less visible than the filtered signal (b). Only one eigenvector was used in the filtering due to the lower SNR.

Fig. 5
Fig. 5

Lidar signals (Utah, 28 April 2006) for a ground release of a Pa cloud at 900   m at wavelength 10.260 μ m (10R18) as a function of range. (a) Raw lidar data where the atmospheric 1 / r 2 signal dependence is shown, (b) filtered lidar data where the 1 / r 2 signal dependence is absent and only the Pa cloud is seen.

Fig. 6
Fig. 6

Lidar measurements of airborne line release of BG bioaerosol cloud at ∼2200 m and time 315 s. (a) Range-anomaly scores and threshold for detection where the cloud is detected (based on threshold location) at 331 s. (b) Time-anomaly scores and threshold for detection where the cloud is detected (based on threshold location) at 2210 m. (c) Gaussian pdf mixture probability model for the range-detection scores with parameter set w 0 = 0.801 , μ 0 = 0.151 , σ 0 = 0.0563 , μ 1 = 0.433 , σ 1 = 0.194 , and threshold γ = 0.287 for range anomaly for which the detection and false alarm probabilities are P D = 0.77 and P F A = 0.008 . (d) Gaussian pdf mixture probability model for the time-detection scores with parameter set w 0 = 0.886 , μ 0 = 0.0426 , σ 0 = 0.0209 , μ 1 = 0.356 , σ 1 = 0.267 , and threshold γ = 0.109 for which the detection and false alarm probabilities are P D = 0.82 and P F A = 0.0007 , respectively.

Fig. 7
Fig. 7

ROC curves for range-anomaly detection of BG cloud at distance of 2.2 km. (a) Cloud with varying concentration (0– 50 cm 3 in the experiment [Fig. 6(a)]), ROC area = 0.92 , (b) prediction for a constant concentration of 32 cm 3 , the ROC area is 0.9999. Note the x axis is logarithmic for (b).

Fig. 8
Fig. 8

Anomaly detection scores applied to raw data X ( r , t ) and filtered data X f ( r , t ) for Pa cloud (Fig. 5) released at 900   m at time 189 s. (a) Range-anomaly scores s ( t ) ; (b) Time-anomaly scores s ( r ) .

Equations (25)

Equations on this page are rendered with MathJax. Learn more.

X f = F X .
F = I P U ,
P U = U ( U T U ) 1 U T = U U T ,
C ( r i , r j ) = E [ { X ( r i , t ) E [ X ( r i ) ] } { X ( r j , t ) E [ X ( r j ) ] } ] = 1 n 0 t = 1 n 0 { [ X ( r i , t ) 1 n 0 t = 1 n 0 X ( r i , t ) ] [ X ( r j , t ) 1 n 0 t = 1 n 0 X ( r j , t ) ] } .
C r = E [ X 0 X 0 T ] E [ X 0 ] E [ X 0 ] T ,
s ( t ) = [ X ( t ) E ( X 0 ) ] T C r 1 [ X ( t ) E ( X 0 ) ] ,
C t = E [ X 0 , f T X 0 , f ] E [ X 0 , f ] T E [ X 0 , f ] ,
s ( r ) = [ X f ( r ) E ( X 0 , f ) ] C t 1 [ X f ( r ) E ( X 0 , f ) ] T ,
P ( s ) = P ( H 0 ) P ( s H 0 ) + P ( H 1 ) P ( s H 1 ) = w 0 P ( s H 0 ) + ( 1 w 0 ) P ( s H 1 ) = P ( s , H 0 ) + P ( s , H 1 ) ,
P ( s H 0 ) = N ( s ; μ 0 , σ 0 2 ) ,
P ( s H 1 ) = N ( s ; μ 1 , σ 1 2 ) ,
N ( s ; μ , σ 2 ) = 1 2 π σ 2   exp ( ( s μ ) 2 2 σ 2 ) ,
P ( H 0 s ) = P ( s , H 0 ) P ( s ) = w 0 P ( s H 0 ) P ( s ) ,
P ( H 1 s ) = P ( s , H 1 ) P ( s ) = ( 1 w 0 ) P ( s H 1 ) P ( s ) .
w 0 = P ( H 0 s ) d s ,
μ 0 = P ( H 0 s ) s d s P ( H 0 s ) d s = w 0 1 P ( H 0 s ) s d s ,
μ 1 = P ( H 1 s ) s d s P ( H 1 s ) d s = ( 1 w 0 ) 1 P ( H 1 s ) s d s ,
σ 0 2 = w 0 1 P ( H 0 s ) ( s μ 0 ) 2 d s ,
σ 1 2 = ( 1 w 0 ) 1 P ( H 1 s ) ( s μ 1 ) 2 d s .
μ 1 = [ 1000 ( 1 w 0 ) ] 1 i = 1 1000 P ( H 1 s i ) s i .
w 0 N ( γ ; μ 0 , σ 0 2 ) = ( 1 w 0 ) N ( γ ; μ 1 , σ 1 2 ) ,
γ = μ 0 σ 1 2 μ 1 σ 0 2 ± σ 0 σ 1 ( μ 1 μ 0 ) 2 + 2 ( σ 1 2 σ 0 2 ) ln   w 0 σ 1 ( 1 w 0 ) σ 0 σ 1 2 σ 0 2 ,
P D ( γ ) = γ P ( s H 1 ) d s = 1 2 [ 1 erf ( γ μ 1 2 1 / 2 σ 1 ) ] ,
P F A ( γ ) = γ P ( s H 0 ) d s = 1 2 [ 1 erf ( γ μ 0 2 1 / 2 σ 0 ) ] ,
ζ ( s ) = ζ ( s max ) s μ 0 s max μ 0 ,

Metrics