Willem J. Marais, Robert E. Holz, Yu Hen Hu, Ralph E. Kuehn, Edwin E. Eloranta, and Rebecca M. Willett, "Approach to simultaneously denoise and invert backscatter and extinction from photon-limited atmospheric lidar observations," Appl. Opt. 55, 8316-8334 (2016)
Atmospheric lidar observations provide a unique capability to directly observe the vertical column of cloud and aerosol scattering properties. Detector and solar-background noise, however, hinder the ability of lidar systems to provide reliable backscatter and extinction cross-section estimates. Standard methods for solving this inverse problem are most effective with high signal-to-noise ratio observations that are only available at low resolution in uniform scenes. This paper describes a novel method for solving the inverse problem with high-resolution, lower signal-to-noise ratio observations that are effective in non-uniform scenes. The novelty is twofold. First, the inferences of the backscatter and extinction are applied to images, whereas current lidar algorithms only use the information content of single profiles. Hence, the latent spatial and temporal information in noisy images are utilized to infer the cross-sections. Second, the noise associated with photon-counting lidar observations can be modeled using a Poisson distribution, and state-of-the-art tools for solving Poisson inverse problems are adapted to the atmospheric lidar problem. It is demonstrated through photon-counting high spectral resolution lidar (HSRL) simulations that the proposed algorithm yields inverted backscatter and extinction cross-sections (per unit volume) with smaller mean squared error values at higher spatial and temporal resolutions, compared to the standard approach. Two case studies of real experimental data are also provided where the proposed algorithm is applied on HSRL observations and the inverted backscatter and extinction cross-sections are compared against the standard approach.
S. P. Burton, C. A. Hostetler, A. L. Cook, J. W. Hair, S. T. Seaman, S. Scola, D. B. Harper, J. A. Smith, M. A. Fenn, R. A. Ferrare, P. E. Saide, E. V. Chemyakin, and D. Müller Appl. Opt. 57(21) 6061-6075 (2018)
You do not have subscription access to this journal. Cited by links are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.
You do not have subscription access to this journal. Figure files are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.
You do not have subscription access to this journal. Article tables are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.
You do not have subscription access to this journal. Equations are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.
A short description of each symbol is given, as well as the type of variable (matrix or scalar) and the location where the symbol was introduced.
Algorithm 1
The standard approach in inferring the backscatter and extinction cross-sections, the lidar ratio , and the optical depth . The tuning parameters of are set by a lidar expert. Note that whenever the unknown parameters are estimated with this algorithm, it is indicated by a superscript
Let be a low-pass filter function, and let be a discrete derivative operator.
The standard approach with averaging in inferring the backscatter and extinction cross-sections, the lidar ratio , and the optical depth . The tuning parameters of are set by a lidar expert or an automated system, which are determined by some objective criteria. Note that whenever the unknown parameters are estimated with this algorithm, it is indicated by a superscript
Let be an averaging function, i.e., block averaging.
1: /*Average photon-count images*/
2:
3:
4: Execute Algorithm 1 using and to get estimates of the backscatter and extinction cross-sections, the lidar ratio , and the optical depth . Denote these estimates by , , , and .
5: return, , , .
Algorithm 2
The new approach in inverting for the backscatter and extinction cross-sections, along with the lidar ratio , and the optical depth . Note that whenever the unknown parameters are estimated with this algorithm, it is indicated by a superscript
1: /*Invert the parallel backscatter cross-section*/
2: Set
3: for alldo
4: Algorithm 2a [, , , ]
5: end for
6:
7:
8: /*Invert the extinction cross-section*/
9: Set
10: = Algorithm 2a [, , ]
11:
12:
13:
14: return
Algorithm 2a
The TV-PMLE with cross-validation to estimate either or ; see Algorithm 2
Input: A loss function such as , the Poisson noisy matrices , and the constraint set for .
1: , ← Poisson Thinning ()
2: , ← Poisson Thinning ()
3: for all in do
4:
5: end for
6:
7: return
Algorithm 3
Another approach to inverting the particulate extinction cross-section . Note that whenever the unknown parameters are estimated with this algorithm, it is indicated by a superscript
Input: Previously inverted parallel backscatter cross-section , such as , with the depolarization coefficient to compute .
1: /*Invert the extinction cross-section*/
2: Set
3: Algorithm 2a [, , ]
4:
5:
6: return
Table 2.
Root Mean Square Error, Bias, and Standard Deviation (Std) Results of the First Experimenta
Backscatter Cross-section
RMSE
Bias
Std
Relative to standard approach, reduced by a factor of
Standard approach
1
1
1
Standard approach with block avg.
6
6
6
New approach
190
104
200
Optical Depth
RMSE
Bias
Std
Relative to standard approach, reduced by a factor of
Standard approach
1
1
1
Standard approach with block avg.
2
2
2
New approach
7
2
13
All results are expressed by a factor of reduction relative to the standard approach (Algorithm 1), i.e., the RMSE backscatter cross-section of the new approach is 190 times less than the RMSE backscatter cross-section of the standard approach; the larger the number is, the better. The reduction of, say, the RMSE in regards to the backscatter cross-section was calculated by taking the ratio of RMSE of the standard approach and the RMSE of the new approach (Algorithm 2). From the first five rows, we see that the backscatter cross-section of the new approach attains better performance, compared to the standard approach without (Algorithm 1) and with block averaging (Algorithm 1a). The optical depth estimates are also improved. This is because the new approach uses the actual noise model to find an estimates of the backscatter cross-section and optical depth, and these parameters are inferred as a piecewise smooth images rather than a collection of unrelated values. In contrast, the standard approach infers the backscatter cross-section and optical depth from individual profiles and any spatial and temporal information is not utilized, except for when block averaging is employed, which is clearly suboptimal.
Table 3.
Root Mean Square Error, Bias, and Standard Deviation (Std) Results of the Second Experimenta
Backscatter Cross-section
RMSE
Bias
Std
Relative to standard approach, reduced by a factor of
Standard approach
1
1
1
New approach
48
12
80
Optical Depth
RMSE
Bias
Std
Relative to standard approach, reduced by a factor of
Standard approach
1
1
1
New approach
16
5
25
Alternative approach
6
4
7
Extinction Cross-section
RMSE
Bias
Std
Relative to standard approach, reduced by a factor of
Standard approach
1
1
1
New approach
5
8
3
Alternative approach
0
1
0
Lidar Ratio
RMSE
Bias
Std
Relative to standard approach, reduced by a factor of
Standard approach
1
1
1
New approach
Alternative approach
All results are expressed by a factor of reduction relative to the standard approach (Algorithm 1), i.e., the RMSE backscatter cross-section of the new approach is 48 times less than the RMSE backscatter cross-section of the standard approach; the larger the number is, the better. The reduction of, say, the RMSE in regards to the extinction cross-section was calculated by taking the ratio of RMSE of the standard approach and the RMSE of the new approach. In all accounts, the new approach (Algorithm 2) is able to infer the optical depth , extinction cross-section , and lidar ratio at a better accuracy compared to the standard and alternative approaches. The new approach is able to achieve smaller RMSE values for the extinction cross-section because it uses the previously estimated backscatter cross-section to infer the lidar ratio. The lidar ratio is constrained to be spatial piecewise constant using the total variation smoothness constraint. It follows, then, that the inferred optical depth of the new approach also has a higher accuracy, compared to the other approaches. The standard approach lidar ratio estimates are very inaccurate relative to the new and alternative approaches. This is because the lidar ratio of the standard approach is the ratio of the extinction and backscatter cross-sections, and residual noise in the backscatter cross-section increases the standard deviation of the lidar ratio estimate.
Tables (8)
Table 1.
Non-Exhaustive List of Symbols Used in this Papera
A short description of each symbol is given, as well as the type of variable (matrix or scalar) and the location where the symbol was introduced.
Algorithm 1
The standard approach in inferring the backscatter and extinction cross-sections, the lidar ratio , and the optical depth . The tuning parameters of are set by a lidar expert. Note that whenever the unknown parameters are estimated with this algorithm, it is indicated by a superscript
Let be a low-pass filter function, and let be a discrete derivative operator.
The standard approach with averaging in inferring the backscatter and extinction cross-sections, the lidar ratio , and the optical depth . The tuning parameters of are set by a lidar expert or an automated system, which are determined by some objective criteria. Note that whenever the unknown parameters are estimated with this algorithm, it is indicated by a superscript
Let be an averaging function, i.e., block averaging.
1: /*Average photon-count images*/
2:
3:
4: Execute Algorithm 1 using and to get estimates of the backscatter and extinction cross-sections, the lidar ratio , and the optical depth . Denote these estimates by , , , and .
5: return, , , .
Algorithm 2
The new approach in inverting for the backscatter and extinction cross-sections, along with the lidar ratio , and the optical depth . Note that whenever the unknown parameters are estimated with this algorithm, it is indicated by a superscript
1: /*Invert the parallel backscatter cross-section*/
2: Set
3: for alldo
4: Algorithm 2a [, , , ]
5: end for
6:
7:
8: /*Invert the extinction cross-section*/
9: Set
10: = Algorithm 2a [, , ]
11:
12:
13:
14: return
Algorithm 2a
The TV-PMLE with cross-validation to estimate either or ; see Algorithm 2
Input: A loss function such as , the Poisson noisy matrices , and the constraint set for .
1: , ← Poisson Thinning ()
2: , ← Poisson Thinning ()
3: for all in do
4:
5: end for
6:
7: return
Algorithm 3
Another approach to inverting the particulate extinction cross-section . Note that whenever the unknown parameters are estimated with this algorithm, it is indicated by a superscript
Input: Previously inverted parallel backscatter cross-section , such as , with the depolarization coefficient to compute .
1: /*Invert the extinction cross-section*/
2: Set
3: Algorithm 2a [, , ]
4:
5:
6: return
Table 2.
Root Mean Square Error, Bias, and Standard Deviation (Std) Results of the First Experimenta
Backscatter Cross-section
RMSE
Bias
Std
Relative to standard approach, reduced by a factor of
Standard approach
1
1
1
Standard approach with block avg.
6
6
6
New approach
190
104
200
Optical Depth
RMSE
Bias
Std
Relative to standard approach, reduced by a factor of
Standard approach
1
1
1
Standard approach with block avg.
2
2
2
New approach
7
2
13
All results are expressed by a factor of reduction relative to the standard approach (Algorithm 1), i.e., the RMSE backscatter cross-section of the new approach is 190 times less than the RMSE backscatter cross-section of the standard approach; the larger the number is, the better. The reduction of, say, the RMSE in regards to the backscatter cross-section was calculated by taking the ratio of RMSE of the standard approach and the RMSE of the new approach (Algorithm 2). From the first five rows, we see that the backscatter cross-section of the new approach attains better performance, compared to the standard approach without (Algorithm 1) and with block averaging (Algorithm 1a). The optical depth estimates are also improved. This is because the new approach uses the actual noise model to find an estimates of the backscatter cross-section and optical depth, and these parameters are inferred as a piecewise smooth images rather than a collection of unrelated values. In contrast, the standard approach infers the backscatter cross-section and optical depth from individual profiles and any spatial and temporal information is not utilized, except for when block averaging is employed, which is clearly suboptimal.
Table 3.
Root Mean Square Error, Bias, and Standard Deviation (Std) Results of the Second Experimenta
Backscatter Cross-section
RMSE
Bias
Std
Relative to standard approach, reduced by a factor of
Standard approach
1
1
1
New approach
48
12
80
Optical Depth
RMSE
Bias
Std
Relative to standard approach, reduced by a factor of
Standard approach
1
1
1
New approach
16
5
25
Alternative approach
6
4
7
Extinction Cross-section
RMSE
Bias
Std
Relative to standard approach, reduced by a factor of
Standard approach
1
1
1
New approach
5
8
3
Alternative approach
0
1
0
Lidar Ratio
RMSE
Bias
Std
Relative to standard approach, reduced by a factor of
Standard approach
1
1
1
New approach
Alternative approach
All results are expressed by a factor of reduction relative to the standard approach (Algorithm 1), i.e., the RMSE backscatter cross-section of the new approach is 48 times less than the RMSE backscatter cross-section of the standard approach; the larger the number is, the better. The reduction of, say, the RMSE in regards to the extinction cross-section was calculated by taking the ratio of RMSE of the standard approach and the RMSE of the new approach. In all accounts, the new approach (Algorithm 2) is able to infer the optical depth , extinction cross-section , and lidar ratio at a better accuracy compared to the standard and alternative approaches. The new approach is able to achieve smaller RMSE values for the extinction cross-section because it uses the previously estimated backscatter cross-section to infer the lidar ratio. The lidar ratio is constrained to be spatial piecewise constant using the total variation smoothness constraint. It follows, then, that the inferred optical depth of the new approach also has a higher accuracy, compared to the other approaches. The standard approach lidar ratio estimates are very inaccurate relative to the new and alternative approaches. This is because the lidar ratio of the standard approach is the ratio of the extinction and backscatter cross-sections, and residual noise in the backscatter cross-section increases the standard deviation of the lidar ratio estimate.
Add to BibSonomy