Abstract

A method for probability density function (PDF) estimation using Bayesian mixtures of weighted gamma distributions, called the Dirichlet process gamma mixture model (DP-GaMM), is presented and applied to the analysis of a laser beam in turbulence. The problem is cast in a Bayesian setting, with the mixture model itself treated as random process. A stick-breaking interpretation of the Dirichlet process is employed as the prior distribution over the random mixture model. The number and underlying parameters of the gamma distribution mixture components as well as the associated mixture weights are learned directly from the data during model inference. A hybrid Metropolis–Hastings and Gibbs sampling parameter inference algorithm is developed and presented in its entirety. Results on several sets of controlled data are shown, and comparisons of PDF estimation fidelity are conducted with favorable results.

© 2014 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. E. Jakeman and P. Pusey, “Significance of the k-distribution in scattering experiments,” Phys. Rev. Lett. 40, 546–550 (1978).
    [CrossRef]
  2. V. Gudimetla and J. Holmes, “Probability density function of the intensity for a laser-generated speckle field after propagation through the turbulent atmosphere,” J. Opt. Soc. Am. 72, 1213–1218 (1982).
    [CrossRef]
  3. R. Phillips and L. Andrews, “Universal statistical model for irradiance fluctuations in a turbulent medium,” J. Opt. Soc. Am. 72, 864–870 (1982).
    [CrossRef]
  4. L. Bissonette and P. Wizniowich, “Probability distribution of turbulent irradiance in a saturation regime,” Appl. Opt. 18, 1590–1599 (1979).
    [CrossRef]
  5. W. Strohbein, T. Wang, and J. Speck, “On the probability distribution of line-of-sight fluctuations for optical signals,” Radio Sci. 10, 59–70 (1975).
  6. J. Churnside and R. Hill, “Probability density of irradiance scintillations for strong path-integrated refractive turbulence,” J. Opt. Soc. Am. 4, 727–733 (1987).
    [CrossRef]
  7. D. Mudge, A. Wedd, J. Craig, and J. Thomas, “Statistical measurements of irradiance fluctuations produced by a reflective membrane optical scintillator,” J. Opt. Laser Technol. 28, 381–387 (1996).
    [CrossRef]
  8. P. Orbanz and Y. Teh, “Bayesian nonparametric models,” in Encyclopedia of Machine Learning (Springer, 2010), pp. 81–89.
  9. J. McLaren, J. Thomas, J. Mackintosh, K. Mudge, K. Grant, B. Clare, and W. Cowley, “Comparison of probability density functions for analyzing irradiance statistics due to atmospheric turbulence,” J. Appl. Opt. 51, 5996–6002 (2012).
    [CrossRef]
  10. Y. Jiang, J. Ma, L. Tan, S. Yu, and W. Du, “Measurement of optical intensity fluctuation over an 11.8  km turbulent path,” Opt. Express 16, 6963–6973 (2008).
    [CrossRef]
  11. M. Al-Habash, L. Andrews, and R. Phillips, “Mathematical model for the irradiance probability density function of a laser beam propagating through turbulent media,” J. Opt. Eng. 40, 1554–1562 (2001).
    [CrossRef]
  12. R. Barakat, “Second-order statistics of integrated intensities and detected photons, the exact analysis,” J. Mod. Opt. 43, 1237–1252 (1996).
    [CrossRef]
  13. R. Barakat, “First-order intensity and log-intensity probability density functions of light scattered by the turbulent atmosphere in terms of lower-order moments,” J. Opt. Soc. Am. A 16, 2269–2274 (1999).
    [CrossRef]
  14. O. Korotkova, S. Avramov-Zamurovic, R. Malek-Madani, and C. Nelson, “Probability density function of the intensity of a laser beam propagating the maritime environment,” Opt. Express 19, 20322–20331 (2011).
    [CrossRef]
  15. M. Welling, “Robust series expansions for probability density estimation,” Technical Note (Department of Electrical and Computer Engineering, California Institute of Technology, 2001).
  16. B. Silverman, “Survey of existing methods,” in Density Estimates for Statistics and Data Analysis, Monographs on Statistics and Applied Probability (Chapman & Hall, 1986), pp. 1–22.
  17. C. Nelson, S. Avramov-Zamurovic, R. Malek-Madani, O. Korotkova, R. Sova, and F. Davidson, “Measurements and comparison of the probability density and covariance functions of laser beam intensity fluctuations in a hot-air turbulence emulator with the maritime atmospheric environment,” Proc. SPIE 8517, 851707 (2012).
  18. M. Escobar and M. West, “Bayesian density estimation and inference using mixtures,” J. Am. Stat. Assoc. 90, 577–588 (1995).
    [CrossRef]
  19. M. Rosenblatt, “Remarks on some nonparametric estimates of density function,” Ann. Math. Sci. 27, 832–837 (1956).
    [CrossRef]
  20. E. Parzen, “On estimation of a probability density function and mode,” Ann. Math. Sci. 33, 1065–1076 (1962).
    [CrossRef]
  21. T. Ferguson, “Bayesian density estimation by mixtures of normal distributions,” Recent Advances in Statistics (Academic, 1983), pp. 287–302.
  22. P. Orbanz and J. Buhmann, “Smooth image segmentation by nonparametric Bayesian inference,” in European Conference on Computer Vision (Springer, 2006), Vol. 1, pp. 444–457.
  23. Y. Teh, M. Jordan, M. Beal, and M. Jordan, “Hierarchical Dirichlet processes,” J. Am. Stat. Assoc. 101, 1566–1581 (2005).
    [CrossRef]
  24. E. Wang, D. Liu, J. Silva, and L. Carin, “Joint analysis of time-evolving binary matrices and associated documents,” in Advances in Neural Information Processing Systems (Curran Associates, 2010), pp. 2370–2378.
  25. J. Sethuraman, “A constructive definition of Dirichlet priors,” Statistica Sinica 4, 639–650 (1994).
  26. A. Webb, “Gamma mixture models for target recognition,” Pattern Recogn. 33, 2045–2054 (2000).
    [CrossRef]
  27. K. Corsey and A. Webb, “Bayesian gamma mixture model approach to radar target recognition,” IEEE Trans. Aerosp. Electron. Syst. 39, 1201–1217 (2003).
  28. M. Wiper, D. Insua, and F. Ruggeri, “Mixtures of gamma distributions with applications,” J. Comput. Graph. Stat. 10, 440–454 (2001).
    [CrossRef]
  29. R. Hill and J. Churnside, “Observational challenges of strong scintillations of irradiance,” J. Opt. Soc. Am. A 5, 445–447 (1988).
    [CrossRef]
  30. D. Blei, A. Ng, and M. Jordan, “Latent Dirichlet allocation,” J. Mach. Learn. Res. 3, 993–1022 (2003).
  31. J. Neyman and E. Pearson, “On the problem of the most efficient tests of statistical hypothesis,” Philos. Trans. R. Soc. London 231, 289–337 (1933).
    [CrossRef]
  32. M. J. Beal, “Variational algorithms for approximate Bayesian inference,” Ph.D. Thesis (University College London, 2003).
  33. A. Gelfand and A. Smith, “Sampling-based approaches to calculating marginal densities,” J. Am. Stat. Assoc. 85, 398–409 (1990).
    [CrossRef]
  34. A. Dempster, N. Laird, and D. Rubin, “Maximum likelihood from incomplete data via the EM algorithm,” J. R. Stat. Soc. 39, 1–38 (1977).
  35. M. Aitkin and D. Rubin, “Estimation and hypothesis testing in finite mixture models,” J. R. Stat. Soc. 47, 67–75 (1985).
  36. J. Marin, K. Mengersen, and C. Roberts, Handbook of Statistics: Bayesian Thinking - Modeling and Computation (Elsevier, 2011), Chap. 25.
  37. H. Ishwaran and M. Zarepour, “Exact and approximate sum representations for the Dirichlet process,” Can. J. Stat. 30, 269–283 (2002).
    [CrossRef]
  38. C. Andrieu, N. de Freitas, A. Doucet, and M. Jordan, “An introduction to MCMC for machine learning,” Mach. Learn. 50, 5–43 (2003).
    [CrossRef]
  39. N. Metropolis, A. Rosenbluth, M. Rosenbluth, and A. Teller, “Equations of state calculations by fast computing machines,” J. Chem. Phys. 21, 1087–1092 (1953).
    [CrossRef]
  40. W. Hastings, “Monte Carlo sampling methods using Markov chains and their applications,” Biometrika 57, 97–109 (1970).
    [CrossRef]
  41. Q. An, C. Wang, I. Shterev, E. Wang, L. Carin, and D. Dunson, “Hierarchical kernel stick-breaking process for multi-task image analysis,” in International Conference on Machine Learning (ICML) (Omnipress, 2008), pp. 17–24.
  42. D. Dunson and N. Pillai, “Bayesian density regression,” J. R. Stat. Soc. 69, 163–183 (2007).
    [CrossRef]
  43. I. Pruteanu-Malcini, L. Ren, J. Paisley, E. Wang, and L. Carin, “Hierarchical Bayesian modeling of topics in time-stamped documents,” IEEE Trans. Pattern Anal. Mach. Intell. 32, 996–1011 (2010).
    [CrossRef]

2012

J. McLaren, J. Thomas, J. Mackintosh, K. Mudge, K. Grant, B. Clare, and W. Cowley, “Comparison of probability density functions for analyzing irradiance statistics due to atmospheric turbulence,” J. Appl. Opt. 51, 5996–6002 (2012).
[CrossRef]

C. Nelson, S. Avramov-Zamurovic, R. Malek-Madani, O. Korotkova, R. Sova, and F. Davidson, “Measurements and comparison of the probability density and covariance functions of laser beam intensity fluctuations in a hot-air turbulence emulator with the maritime atmospheric environment,” Proc. SPIE 8517, 851707 (2012).

2011

2010

I. Pruteanu-Malcini, L. Ren, J. Paisley, E. Wang, and L. Carin, “Hierarchical Bayesian modeling of topics in time-stamped documents,” IEEE Trans. Pattern Anal. Mach. Intell. 32, 996–1011 (2010).
[CrossRef]

2008

2007

D. Dunson and N. Pillai, “Bayesian density regression,” J. R. Stat. Soc. 69, 163–183 (2007).
[CrossRef]

2005

Y. Teh, M. Jordan, M. Beal, and M. Jordan, “Hierarchical Dirichlet processes,” J. Am. Stat. Assoc. 101, 1566–1581 (2005).
[CrossRef]

2003

K. Corsey and A. Webb, “Bayesian gamma mixture model approach to radar target recognition,” IEEE Trans. Aerosp. Electron. Syst. 39, 1201–1217 (2003).

D. Blei, A. Ng, and M. Jordan, “Latent Dirichlet allocation,” J. Mach. Learn. Res. 3, 993–1022 (2003).

C. Andrieu, N. de Freitas, A. Doucet, and M. Jordan, “An introduction to MCMC for machine learning,” Mach. Learn. 50, 5–43 (2003).
[CrossRef]

2002

H. Ishwaran and M. Zarepour, “Exact and approximate sum representations for the Dirichlet process,” Can. J. Stat. 30, 269–283 (2002).
[CrossRef]

2001

M. Al-Habash, L. Andrews, and R. Phillips, “Mathematical model for the irradiance probability density function of a laser beam propagating through turbulent media,” J. Opt. Eng. 40, 1554–1562 (2001).
[CrossRef]

M. Wiper, D. Insua, and F. Ruggeri, “Mixtures of gamma distributions with applications,” J. Comput. Graph. Stat. 10, 440–454 (2001).
[CrossRef]

2000

A. Webb, “Gamma mixture models for target recognition,” Pattern Recogn. 33, 2045–2054 (2000).
[CrossRef]

1999

1996

R. Barakat, “Second-order statistics of integrated intensities and detected photons, the exact analysis,” J. Mod. Opt. 43, 1237–1252 (1996).
[CrossRef]

D. Mudge, A. Wedd, J. Craig, and J. Thomas, “Statistical measurements of irradiance fluctuations produced by a reflective membrane optical scintillator,” J. Opt. Laser Technol. 28, 381–387 (1996).
[CrossRef]

1995

M. Escobar and M. West, “Bayesian density estimation and inference using mixtures,” J. Am. Stat. Assoc. 90, 577–588 (1995).
[CrossRef]

1994

J. Sethuraman, “A constructive definition of Dirichlet priors,” Statistica Sinica 4, 639–650 (1994).

1990

A. Gelfand and A. Smith, “Sampling-based approaches to calculating marginal densities,” J. Am. Stat. Assoc. 85, 398–409 (1990).
[CrossRef]

1988

1987

J. Churnside and R. Hill, “Probability density of irradiance scintillations for strong path-integrated refractive turbulence,” J. Opt. Soc. Am. 4, 727–733 (1987).
[CrossRef]

1985

M. Aitkin and D. Rubin, “Estimation and hypothesis testing in finite mixture models,” J. R. Stat. Soc. 47, 67–75 (1985).

1982

1979

1978

E. Jakeman and P. Pusey, “Significance of the k-distribution in scattering experiments,” Phys. Rev. Lett. 40, 546–550 (1978).
[CrossRef]

1977

A. Dempster, N. Laird, and D. Rubin, “Maximum likelihood from incomplete data via the EM algorithm,” J. R. Stat. Soc. 39, 1–38 (1977).

1975

W. Strohbein, T. Wang, and J. Speck, “On the probability distribution of line-of-sight fluctuations for optical signals,” Radio Sci. 10, 59–70 (1975).

1970

W. Hastings, “Monte Carlo sampling methods using Markov chains and their applications,” Biometrika 57, 97–109 (1970).
[CrossRef]

1962

E. Parzen, “On estimation of a probability density function and mode,” Ann. Math. Sci. 33, 1065–1076 (1962).
[CrossRef]

1956

M. Rosenblatt, “Remarks on some nonparametric estimates of density function,” Ann. Math. Sci. 27, 832–837 (1956).
[CrossRef]

1953

N. Metropolis, A. Rosenbluth, M. Rosenbluth, and A. Teller, “Equations of state calculations by fast computing machines,” J. Chem. Phys. 21, 1087–1092 (1953).
[CrossRef]

1933

J. Neyman and E. Pearson, “On the problem of the most efficient tests of statistical hypothesis,” Philos. Trans. R. Soc. London 231, 289–337 (1933).
[CrossRef]

Aitkin, M.

M. Aitkin and D. Rubin, “Estimation and hypothesis testing in finite mixture models,” J. R. Stat. Soc. 47, 67–75 (1985).

Al-Habash, M.

M. Al-Habash, L. Andrews, and R. Phillips, “Mathematical model for the irradiance probability density function of a laser beam propagating through turbulent media,” J. Opt. Eng. 40, 1554–1562 (2001).
[CrossRef]

An, Q.

Q. An, C. Wang, I. Shterev, E. Wang, L. Carin, and D. Dunson, “Hierarchical kernel stick-breaking process for multi-task image analysis,” in International Conference on Machine Learning (ICML) (Omnipress, 2008), pp. 17–24.

Andrews, L.

M. Al-Habash, L. Andrews, and R. Phillips, “Mathematical model for the irradiance probability density function of a laser beam propagating through turbulent media,” J. Opt. Eng. 40, 1554–1562 (2001).
[CrossRef]

R. Phillips and L. Andrews, “Universal statistical model for irradiance fluctuations in a turbulent medium,” J. Opt. Soc. Am. 72, 864–870 (1982).
[CrossRef]

Andrieu, C.

C. Andrieu, N. de Freitas, A. Doucet, and M. Jordan, “An introduction to MCMC for machine learning,” Mach. Learn. 50, 5–43 (2003).
[CrossRef]

Avramov-Zamurovic, S.

C. Nelson, S. Avramov-Zamurovic, R. Malek-Madani, O. Korotkova, R. Sova, and F. Davidson, “Measurements and comparison of the probability density and covariance functions of laser beam intensity fluctuations in a hot-air turbulence emulator with the maritime atmospheric environment,” Proc. SPIE 8517, 851707 (2012).

O. Korotkova, S. Avramov-Zamurovic, R. Malek-Madani, and C. Nelson, “Probability density function of the intensity of a laser beam propagating the maritime environment,” Opt. Express 19, 20322–20331 (2011).
[CrossRef]

Barakat, R.

Beal, M.

Y. Teh, M. Jordan, M. Beal, and M. Jordan, “Hierarchical Dirichlet processes,” J. Am. Stat. Assoc. 101, 1566–1581 (2005).
[CrossRef]

Beal, M. J.

M. J. Beal, “Variational algorithms for approximate Bayesian inference,” Ph.D. Thesis (University College London, 2003).

Bissonette, L.

Blei, D.

D. Blei, A. Ng, and M. Jordan, “Latent Dirichlet allocation,” J. Mach. Learn. Res. 3, 993–1022 (2003).

Buhmann, J.

P. Orbanz and J. Buhmann, “Smooth image segmentation by nonparametric Bayesian inference,” in European Conference on Computer Vision (Springer, 2006), Vol. 1, pp. 444–457.

Carin, L.

I. Pruteanu-Malcini, L. Ren, J. Paisley, E. Wang, and L. Carin, “Hierarchical Bayesian modeling of topics in time-stamped documents,” IEEE Trans. Pattern Anal. Mach. Intell. 32, 996–1011 (2010).
[CrossRef]

E. Wang, D. Liu, J. Silva, and L. Carin, “Joint analysis of time-evolving binary matrices and associated documents,” in Advances in Neural Information Processing Systems (Curran Associates, 2010), pp. 2370–2378.

Q. An, C. Wang, I. Shterev, E. Wang, L. Carin, and D. Dunson, “Hierarchical kernel stick-breaking process for multi-task image analysis,” in International Conference on Machine Learning (ICML) (Omnipress, 2008), pp. 17–24.

Churnside, J.

R. Hill and J. Churnside, “Observational challenges of strong scintillations of irradiance,” J. Opt. Soc. Am. A 5, 445–447 (1988).
[CrossRef]

J. Churnside and R. Hill, “Probability density of irradiance scintillations for strong path-integrated refractive turbulence,” J. Opt. Soc. Am. 4, 727–733 (1987).
[CrossRef]

Clare, B.

J. McLaren, J. Thomas, J. Mackintosh, K. Mudge, K. Grant, B. Clare, and W. Cowley, “Comparison of probability density functions for analyzing irradiance statistics due to atmospheric turbulence,” J. Appl. Opt. 51, 5996–6002 (2012).
[CrossRef]

Corsey, K.

K. Corsey and A. Webb, “Bayesian gamma mixture model approach to radar target recognition,” IEEE Trans. Aerosp. Electron. Syst. 39, 1201–1217 (2003).

Cowley, W.

J. McLaren, J. Thomas, J. Mackintosh, K. Mudge, K. Grant, B. Clare, and W. Cowley, “Comparison of probability density functions for analyzing irradiance statistics due to atmospheric turbulence,” J. Appl. Opt. 51, 5996–6002 (2012).
[CrossRef]

Craig, J.

D. Mudge, A. Wedd, J. Craig, and J. Thomas, “Statistical measurements of irradiance fluctuations produced by a reflective membrane optical scintillator,” J. Opt. Laser Technol. 28, 381–387 (1996).
[CrossRef]

Davidson, F.

C. Nelson, S. Avramov-Zamurovic, R. Malek-Madani, O. Korotkova, R. Sova, and F. Davidson, “Measurements and comparison of the probability density and covariance functions of laser beam intensity fluctuations in a hot-air turbulence emulator with the maritime atmospheric environment,” Proc. SPIE 8517, 851707 (2012).

de Freitas, N.

C. Andrieu, N. de Freitas, A. Doucet, and M. Jordan, “An introduction to MCMC for machine learning,” Mach. Learn. 50, 5–43 (2003).
[CrossRef]

Dempster, A.

A. Dempster, N. Laird, and D. Rubin, “Maximum likelihood from incomplete data via the EM algorithm,” J. R. Stat. Soc. 39, 1–38 (1977).

Doucet, A.

C. Andrieu, N. de Freitas, A. Doucet, and M. Jordan, “An introduction to MCMC for machine learning,” Mach. Learn. 50, 5–43 (2003).
[CrossRef]

Du, W.

Dunson, D.

D. Dunson and N. Pillai, “Bayesian density regression,” J. R. Stat. Soc. 69, 163–183 (2007).
[CrossRef]

Q. An, C. Wang, I. Shterev, E. Wang, L. Carin, and D. Dunson, “Hierarchical kernel stick-breaking process for multi-task image analysis,” in International Conference on Machine Learning (ICML) (Omnipress, 2008), pp. 17–24.

Escobar, M.

M. Escobar and M. West, “Bayesian density estimation and inference using mixtures,” J. Am. Stat. Assoc. 90, 577–588 (1995).
[CrossRef]

Ferguson, T.

T. Ferguson, “Bayesian density estimation by mixtures of normal distributions,” Recent Advances in Statistics (Academic, 1983), pp. 287–302.

Gelfand, A.

A. Gelfand and A. Smith, “Sampling-based approaches to calculating marginal densities,” J. Am. Stat. Assoc. 85, 398–409 (1990).
[CrossRef]

Grant, K.

J. McLaren, J. Thomas, J. Mackintosh, K. Mudge, K. Grant, B. Clare, and W. Cowley, “Comparison of probability density functions for analyzing irradiance statistics due to atmospheric turbulence,” J. Appl. Opt. 51, 5996–6002 (2012).
[CrossRef]

Gudimetla, V.

Hastings, W.

W. Hastings, “Monte Carlo sampling methods using Markov chains and their applications,” Biometrika 57, 97–109 (1970).
[CrossRef]

Hill, R.

R. Hill and J. Churnside, “Observational challenges of strong scintillations of irradiance,” J. Opt. Soc. Am. A 5, 445–447 (1988).
[CrossRef]

J. Churnside and R. Hill, “Probability density of irradiance scintillations for strong path-integrated refractive turbulence,” J. Opt. Soc. Am. 4, 727–733 (1987).
[CrossRef]

Holmes, J.

Insua, D.

M. Wiper, D. Insua, and F. Ruggeri, “Mixtures of gamma distributions with applications,” J. Comput. Graph. Stat. 10, 440–454 (2001).
[CrossRef]

Ishwaran, H.

H. Ishwaran and M. Zarepour, “Exact and approximate sum representations for the Dirichlet process,” Can. J. Stat. 30, 269–283 (2002).
[CrossRef]

Jakeman, E.

E. Jakeman and P. Pusey, “Significance of the k-distribution in scattering experiments,” Phys. Rev. Lett. 40, 546–550 (1978).
[CrossRef]

Jiang, Y.

Jordan, M.

Y. Teh, M. Jordan, M. Beal, and M. Jordan, “Hierarchical Dirichlet processes,” J. Am. Stat. Assoc. 101, 1566–1581 (2005).
[CrossRef]

Y. Teh, M. Jordan, M. Beal, and M. Jordan, “Hierarchical Dirichlet processes,” J. Am. Stat. Assoc. 101, 1566–1581 (2005).
[CrossRef]

C. Andrieu, N. de Freitas, A. Doucet, and M. Jordan, “An introduction to MCMC for machine learning,” Mach. Learn. 50, 5–43 (2003).
[CrossRef]

D. Blei, A. Ng, and M. Jordan, “Latent Dirichlet allocation,” J. Mach. Learn. Res. 3, 993–1022 (2003).

Korotkova, O.

C. Nelson, S. Avramov-Zamurovic, R. Malek-Madani, O. Korotkova, R. Sova, and F. Davidson, “Measurements and comparison of the probability density and covariance functions of laser beam intensity fluctuations in a hot-air turbulence emulator with the maritime atmospheric environment,” Proc. SPIE 8517, 851707 (2012).

O. Korotkova, S. Avramov-Zamurovic, R. Malek-Madani, and C. Nelson, “Probability density function of the intensity of a laser beam propagating the maritime environment,” Opt. Express 19, 20322–20331 (2011).
[CrossRef]

Laird, N.

A. Dempster, N. Laird, and D. Rubin, “Maximum likelihood from incomplete data via the EM algorithm,” J. R. Stat. Soc. 39, 1–38 (1977).

Liu, D.

E. Wang, D. Liu, J. Silva, and L. Carin, “Joint analysis of time-evolving binary matrices and associated documents,” in Advances in Neural Information Processing Systems (Curran Associates, 2010), pp. 2370–2378.

Ma, J.

Mackintosh, J.

J. McLaren, J. Thomas, J. Mackintosh, K. Mudge, K. Grant, B. Clare, and W. Cowley, “Comparison of probability density functions for analyzing irradiance statistics due to atmospheric turbulence,” J. Appl. Opt. 51, 5996–6002 (2012).
[CrossRef]

Malek-Madani, R.

C. Nelson, S. Avramov-Zamurovic, R. Malek-Madani, O. Korotkova, R. Sova, and F. Davidson, “Measurements and comparison of the probability density and covariance functions of laser beam intensity fluctuations in a hot-air turbulence emulator with the maritime atmospheric environment,” Proc. SPIE 8517, 851707 (2012).

O. Korotkova, S. Avramov-Zamurovic, R. Malek-Madani, and C. Nelson, “Probability density function of the intensity of a laser beam propagating the maritime environment,” Opt. Express 19, 20322–20331 (2011).
[CrossRef]

Marin, J.

J. Marin, K. Mengersen, and C. Roberts, Handbook of Statistics: Bayesian Thinking - Modeling and Computation (Elsevier, 2011), Chap. 25.

McLaren, J.

J. McLaren, J. Thomas, J. Mackintosh, K. Mudge, K. Grant, B. Clare, and W. Cowley, “Comparison of probability density functions for analyzing irradiance statistics due to atmospheric turbulence,” J. Appl. Opt. 51, 5996–6002 (2012).
[CrossRef]

Mengersen, K.

J. Marin, K. Mengersen, and C. Roberts, Handbook of Statistics: Bayesian Thinking - Modeling and Computation (Elsevier, 2011), Chap. 25.

Metropolis, N.

N. Metropolis, A. Rosenbluth, M. Rosenbluth, and A. Teller, “Equations of state calculations by fast computing machines,” J. Chem. Phys. 21, 1087–1092 (1953).
[CrossRef]

Mudge, D.

D. Mudge, A. Wedd, J. Craig, and J. Thomas, “Statistical measurements of irradiance fluctuations produced by a reflective membrane optical scintillator,” J. Opt. Laser Technol. 28, 381–387 (1996).
[CrossRef]

Mudge, K.

J. McLaren, J. Thomas, J. Mackintosh, K. Mudge, K. Grant, B. Clare, and W. Cowley, “Comparison of probability density functions for analyzing irradiance statistics due to atmospheric turbulence,” J. Appl. Opt. 51, 5996–6002 (2012).
[CrossRef]

Nelson, C.

C. Nelson, S. Avramov-Zamurovic, R. Malek-Madani, O. Korotkova, R. Sova, and F. Davidson, “Measurements and comparison of the probability density and covariance functions of laser beam intensity fluctuations in a hot-air turbulence emulator with the maritime atmospheric environment,” Proc. SPIE 8517, 851707 (2012).

O. Korotkova, S. Avramov-Zamurovic, R. Malek-Madani, and C. Nelson, “Probability density function of the intensity of a laser beam propagating the maritime environment,” Opt. Express 19, 20322–20331 (2011).
[CrossRef]

Neyman, J.

J. Neyman and E. Pearson, “On the problem of the most efficient tests of statistical hypothesis,” Philos. Trans. R. Soc. London 231, 289–337 (1933).
[CrossRef]

Ng, A.

D. Blei, A. Ng, and M. Jordan, “Latent Dirichlet allocation,” J. Mach. Learn. Res. 3, 993–1022 (2003).

Orbanz, P.

P. Orbanz and J. Buhmann, “Smooth image segmentation by nonparametric Bayesian inference,” in European Conference on Computer Vision (Springer, 2006), Vol. 1, pp. 444–457.

P. Orbanz and Y. Teh, “Bayesian nonparametric models,” in Encyclopedia of Machine Learning (Springer, 2010), pp. 81–89.

Paisley, J.

I. Pruteanu-Malcini, L. Ren, J. Paisley, E. Wang, and L. Carin, “Hierarchical Bayesian modeling of topics in time-stamped documents,” IEEE Trans. Pattern Anal. Mach. Intell. 32, 996–1011 (2010).
[CrossRef]

Parzen, E.

E. Parzen, “On estimation of a probability density function and mode,” Ann. Math. Sci. 33, 1065–1076 (1962).
[CrossRef]

Pearson, E.

J. Neyman and E. Pearson, “On the problem of the most efficient tests of statistical hypothesis,” Philos. Trans. R. Soc. London 231, 289–337 (1933).
[CrossRef]

Phillips, R.

M. Al-Habash, L. Andrews, and R. Phillips, “Mathematical model for the irradiance probability density function of a laser beam propagating through turbulent media,” J. Opt. Eng. 40, 1554–1562 (2001).
[CrossRef]

R. Phillips and L. Andrews, “Universal statistical model for irradiance fluctuations in a turbulent medium,” J. Opt. Soc. Am. 72, 864–870 (1982).
[CrossRef]

Pillai, N.

D. Dunson and N. Pillai, “Bayesian density regression,” J. R. Stat. Soc. 69, 163–183 (2007).
[CrossRef]

Pruteanu-Malcini, I.

I. Pruteanu-Malcini, L. Ren, J. Paisley, E. Wang, and L. Carin, “Hierarchical Bayesian modeling of topics in time-stamped documents,” IEEE Trans. Pattern Anal. Mach. Intell. 32, 996–1011 (2010).
[CrossRef]

Pusey, P.

E. Jakeman and P. Pusey, “Significance of the k-distribution in scattering experiments,” Phys. Rev. Lett. 40, 546–550 (1978).
[CrossRef]

Ren, L.

I. Pruteanu-Malcini, L. Ren, J. Paisley, E. Wang, and L. Carin, “Hierarchical Bayesian modeling of topics in time-stamped documents,” IEEE Trans. Pattern Anal. Mach. Intell. 32, 996–1011 (2010).
[CrossRef]

Roberts, C.

J. Marin, K. Mengersen, and C. Roberts, Handbook of Statistics: Bayesian Thinking - Modeling and Computation (Elsevier, 2011), Chap. 25.

Rosenblatt, M.

M. Rosenblatt, “Remarks on some nonparametric estimates of density function,” Ann. Math. Sci. 27, 832–837 (1956).
[CrossRef]

Rosenbluth, A.

N. Metropolis, A. Rosenbluth, M. Rosenbluth, and A. Teller, “Equations of state calculations by fast computing machines,” J. Chem. Phys. 21, 1087–1092 (1953).
[CrossRef]

Rosenbluth, M.

N. Metropolis, A. Rosenbluth, M. Rosenbluth, and A. Teller, “Equations of state calculations by fast computing machines,” J. Chem. Phys. 21, 1087–1092 (1953).
[CrossRef]

Rubin, D.

M. Aitkin and D. Rubin, “Estimation and hypothesis testing in finite mixture models,” J. R. Stat. Soc. 47, 67–75 (1985).

A. Dempster, N. Laird, and D. Rubin, “Maximum likelihood from incomplete data via the EM algorithm,” J. R. Stat. Soc. 39, 1–38 (1977).

Ruggeri, F.

M. Wiper, D. Insua, and F. Ruggeri, “Mixtures of gamma distributions with applications,” J. Comput. Graph. Stat. 10, 440–454 (2001).
[CrossRef]

Sethuraman, J.

J. Sethuraman, “A constructive definition of Dirichlet priors,” Statistica Sinica 4, 639–650 (1994).

Shterev, I.

Q. An, C. Wang, I. Shterev, E. Wang, L. Carin, and D. Dunson, “Hierarchical kernel stick-breaking process for multi-task image analysis,” in International Conference on Machine Learning (ICML) (Omnipress, 2008), pp. 17–24.

Silva, J.

E. Wang, D. Liu, J. Silva, and L. Carin, “Joint analysis of time-evolving binary matrices and associated documents,” in Advances in Neural Information Processing Systems (Curran Associates, 2010), pp. 2370–2378.

Silverman, B.

B. Silverman, “Survey of existing methods,” in Density Estimates for Statistics and Data Analysis, Monographs on Statistics and Applied Probability (Chapman & Hall, 1986), pp. 1–22.

Smith, A.

A. Gelfand and A. Smith, “Sampling-based approaches to calculating marginal densities,” J. Am. Stat. Assoc. 85, 398–409 (1990).
[CrossRef]

Sova, R.

C. Nelson, S. Avramov-Zamurovic, R. Malek-Madani, O. Korotkova, R. Sova, and F. Davidson, “Measurements and comparison of the probability density and covariance functions of laser beam intensity fluctuations in a hot-air turbulence emulator with the maritime atmospheric environment,” Proc. SPIE 8517, 851707 (2012).

Speck, J.

W. Strohbein, T. Wang, and J. Speck, “On the probability distribution of line-of-sight fluctuations for optical signals,” Radio Sci. 10, 59–70 (1975).

Strohbein, W.

W. Strohbein, T. Wang, and J. Speck, “On the probability distribution of line-of-sight fluctuations for optical signals,” Radio Sci. 10, 59–70 (1975).

Tan, L.

Teh, Y.

Y. Teh, M. Jordan, M. Beal, and M. Jordan, “Hierarchical Dirichlet processes,” J. Am. Stat. Assoc. 101, 1566–1581 (2005).
[CrossRef]

P. Orbanz and Y. Teh, “Bayesian nonparametric models,” in Encyclopedia of Machine Learning (Springer, 2010), pp. 81–89.

Teller, A.

N. Metropolis, A. Rosenbluth, M. Rosenbluth, and A. Teller, “Equations of state calculations by fast computing machines,” J. Chem. Phys. 21, 1087–1092 (1953).
[CrossRef]

Thomas, J.

J. McLaren, J. Thomas, J. Mackintosh, K. Mudge, K. Grant, B. Clare, and W. Cowley, “Comparison of probability density functions for analyzing irradiance statistics due to atmospheric turbulence,” J. Appl. Opt. 51, 5996–6002 (2012).
[CrossRef]

D. Mudge, A. Wedd, J. Craig, and J. Thomas, “Statistical measurements of irradiance fluctuations produced by a reflective membrane optical scintillator,” J. Opt. Laser Technol. 28, 381–387 (1996).
[CrossRef]

Wang, C.

Q. An, C. Wang, I. Shterev, E. Wang, L. Carin, and D. Dunson, “Hierarchical kernel stick-breaking process for multi-task image analysis,” in International Conference on Machine Learning (ICML) (Omnipress, 2008), pp. 17–24.

Wang, E.

I. Pruteanu-Malcini, L. Ren, J. Paisley, E. Wang, and L. Carin, “Hierarchical Bayesian modeling of topics in time-stamped documents,” IEEE Trans. Pattern Anal. Mach. Intell. 32, 996–1011 (2010).
[CrossRef]

Q. An, C. Wang, I. Shterev, E. Wang, L. Carin, and D. Dunson, “Hierarchical kernel stick-breaking process for multi-task image analysis,” in International Conference on Machine Learning (ICML) (Omnipress, 2008), pp. 17–24.

E. Wang, D. Liu, J. Silva, and L. Carin, “Joint analysis of time-evolving binary matrices and associated documents,” in Advances in Neural Information Processing Systems (Curran Associates, 2010), pp. 2370–2378.

Wang, T.

W. Strohbein, T. Wang, and J. Speck, “On the probability distribution of line-of-sight fluctuations for optical signals,” Radio Sci. 10, 59–70 (1975).

Webb, A.

K. Corsey and A. Webb, “Bayesian gamma mixture model approach to radar target recognition,” IEEE Trans. Aerosp. Electron. Syst. 39, 1201–1217 (2003).

A. Webb, “Gamma mixture models for target recognition,” Pattern Recogn. 33, 2045–2054 (2000).
[CrossRef]

Wedd, A.

D. Mudge, A. Wedd, J. Craig, and J. Thomas, “Statistical measurements of irradiance fluctuations produced by a reflective membrane optical scintillator,” J. Opt. Laser Technol. 28, 381–387 (1996).
[CrossRef]

Welling, M.

M. Welling, “Robust series expansions for probability density estimation,” Technical Note (Department of Electrical and Computer Engineering, California Institute of Technology, 2001).

West, M.

M. Escobar and M. West, “Bayesian density estimation and inference using mixtures,” J. Am. Stat. Assoc. 90, 577–588 (1995).
[CrossRef]

Wiper, M.

M. Wiper, D. Insua, and F. Ruggeri, “Mixtures of gamma distributions with applications,” J. Comput. Graph. Stat. 10, 440–454 (2001).
[CrossRef]

Wizniowich, P.

Yu, S.

Zarepour, M.

H. Ishwaran and M. Zarepour, “Exact and approximate sum representations for the Dirichlet process,” Can. J. Stat. 30, 269–283 (2002).
[CrossRef]

Ann. Math. Sci.

M. Rosenblatt, “Remarks on some nonparametric estimates of density function,” Ann. Math. Sci. 27, 832–837 (1956).
[CrossRef]

E. Parzen, “On estimation of a probability density function and mode,” Ann. Math. Sci. 33, 1065–1076 (1962).
[CrossRef]

Appl. Opt.

Biometrika

W. Hastings, “Monte Carlo sampling methods using Markov chains and their applications,” Biometrika 57, 97–109 (1970).
[CrossRef]

Can. J. Stat.

H. Ishwaran and M. Zarepour, “Exact and approximate sum representations for the Dirichlet process,” Can. J. Stat. 30, 269–283 (2002).
[CrossRef]

IEEE Trans. Aerosp. Electron. Syst.

K. Corsey and A. Webb, “Bayesian gamma mixture model approach to radar target recognition,” IEEE Trans. Aerosp. Electron. Syst. 39, 1201–1217 (2003).

IEEE Trans. Pattern Anal. Mach. Intell.

I. Pruteanu-Malcini, L. Ren, J. Paisley, E. Wang, and L. Carin, “Hierarchical Bayesian modeling of topics in time-stamped documents,” IEEE Trans. Pattern Anal. Mach. Intell. 32, 996–1011 (2010).
[CrossRef]

J. Am. Stat. Assoc.

A. Gelfand and A. Smith, “Sampling-based approaches to calculating marginal densities,” J. Am. Stat. Assoc. 85, 398–409 (1990).
[CrossRef]

Y. Teh, M. Jordan, M. Beal, and M. Jordan, “Hierarchical Dirichlet processes,” J. Am. Stat. Assoc. 101, 1566–1581 (2005).
[CrossRef]

M. Escobar and M. West, “Bayesian density estimation and inference using mixtures,” J. Am. Stat. Assoc. 90, 577–588 (1995).
[CrossRef]

J. Appl. Opt.

J. McLaren, J. Thomas, J. Mackintosh, K. Mudge, K. Grant, B. Clare, and W. Cowley, “Comparison of probability density functions for analyzing irradiance statistics due to atmospheric turbulence,” J. Appl. Opt. 51, 5996–6002 (2012).
[CrossRef]

J. Chem. Phys.

N. Metropolis, A. Rosenbluth, M. Rosenbluth, and A. Teller, “Equations of state calculations by fast computing machines,” J. Chem. Phys. 21, 1087–1092 (1953).
[CrossRef]

J. Comput. Graph. Stat.

M. Wiper, D. Insua, and F. Ruggeri, “Mixtures of gamma distributions with applications,” J. Comput. Graph. Stat. 10, 440–454 (2001).
[CrossRef]

J. Mach. Learn. Res.

D. Blei, A. Ng, and M. Jordan, “Latent Dirichlet allocation,” J. Mach. Learn. Res. 3, 993–1022 (2003).

J. Mod. Opt.

R. Barakat, “Second-order statistics of integrated intensities and detected photons, the exact analysis,” J. Mod. Opt. 43, 1237–1252 (1996).
[CrossRef]

J. Opt. Eng.

M. Al-Habash, L. Andrews, and R. Phillips, “Mathematical model for the irradiance probability density function of a laser beam propagating through turbulent media,” J. Opt. Eng. 40, 1554–1562 (2001).
[CrossRef]

J. Opt. Laser Technol.

D. Mudge, A. Wedd, J. Craig, and J. Thomas, “Statistical measurements of irradiance fluctuations produced by a reflective membrane optical scintillator,” J. Opt. Laser Technol. 28, 381–387 (1996).
[CrossRef]

J. Opt. Soc. Am.

J. Opt. Soc. Am. A

J. R. Stat. Soc.

D. Dunson and N. Pillai, “Bayesian density regression,” J. R. Stat. Soc. 69, 163–183 (2007).
[CrossRef]

A. Dempster, N. Laird, and D. Rubin, “Maximum likelihood from incomplete data via the EM algorithm,” J. R. Stat. Soc. 39, 1–38 (1977).

M. Aitkin and D. Rubin, “Estimation and hypothesis testing in finite mixture models,” J. R. Stat. Soc. 47, 67–75 (1985).

Mach. Learn.

C. Andrieu, N. de Freitas, A. Doucet, and M. Jordan, “An introduction to MCMC for machine learning,” Mach. Learn. 50, 5–43 (2003).
[CrossRef]

Opt. Express

Pattern Recogn.

A. Webb, “Gamma mixture models for target recognition,” Pattern Recogn. 33, 2045–2054 (2000).
[CrossRef]

Philos. Trans. R. Soc. London

J. Neyman and E. Pearson, “On the problem of the most efficient tests of statistical hypothesis,” Philos. Trans. R. Soc. London 231, 289–337 (1933).
[CrossRef]

Phys. Rev. Lett.

E. Jakeman and P. Pusey, “Significance of the k-distribution in scattering experiments,” Phys. Rev. Lett. 40, 546–550 (1978).
[CrossRef]

Proc. SPIE

C. Nelson, S. Avramov-Zamurovic, R. Malek-Madani, O. Korotkova, R. Sova, and F. Davidson, “Measurements and comparison of the probability density and covariance functions of laser beam intensity fluctuations in a hot-air turbulence emulator with the maritime atmospheric environment,” Proc. SPIE 8517, 851707 (2012).

Radio Sci.

W. Strohbein, T. Wang, and J. Speck, “On the probability distribution of line-of-sight fluctuations for optical signals,” Radio Sci. 10, 59–70 (1975).

Statistica Sinica

J. Sethuraman, “A constructive definition of Dirichlet priors,” Statistica Sinica 4, 639–650 (1994).

Other

E. Wang, D. Liu, J. Silva, and L. Carin, “Joint analysis of time-evolving binary matrices and associated documents,” in Advances in Neural Information Processing Systems (Curran Associates, 2010), pp. 2370–2378.

T. Ferguson, “Bayesian density estimation by mixtures of normal distributions,” Recent Advances in Statistics (Academic, 1983), pp. 287–302.

P. Orbanz and J. Buhmann, “Smooth image segmentation by nonparametric Bayesian inference,” in European Conference on Computer Vision (Springer, 2006), Vol. 1, pp. 444–457.

P. Orbanz and Y. Teh, “Bayesian nonparametric models,” in Encyclopedia of Machine Learning (Springer, 2010), pp. 81–89.

M. Welling, “Robust series expansions for probability density estimation,” Technical Note (Department of Electrical and Computer Engineering, California Institute of Technology, 2001).

B. Silverman, “Survey of existing methods,” in Density Estimates for Statistics and Data Analysis, Monographs on Statistics and Applied Probability (Chapman & Hall, 1986), pp. 1–22.

M. J. Beal, “Variational algorithms for approximate Bayesian inference,” Ph.D. Thesis (University College London, 2003).

J. Marin, K. Mengersen, and C. Roberts, Handbook of Statistics: Bayesian Thinking - Modeling and Computation (Elsevier, 2011), Chap. 25.

Q. An, C. Wang, I. Shterev, E. Wang, L. Carin, and D. Dunson, “Hierarchical kernel stick-breaking process for multi-task image analysis,” in International Conference on Machine Learning (ICML) (Omnipress, 2008), pp. 17–24.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (7)

Fig. 1.
Fig. 1.

Mixture model with different numbers of mixture components (clockwise from top left K=2, 5, 20, 10). Mixture components shown in red and estimated PDF shown in black for Run 1, SI=7.99×102. The normalized histogram in gray has area 1.

Fig. 2.
Fig. 2.

Estimated DP-GaMM PDFs for three different runs. Mixture components shown in red and estimated PDF shown in black. The normalized histogram in gray has area 1. Top left: Run 1, SI=7.99×102. Top right: Run 2, SI=8.42×102. Bottom: Run 3, SI=10.75×102.

Fig. 3.
Fig. 3.

Estimated DP-GaMM PDFs using data sampled from gamma–gamma (top left), gamma (top right), and log–normal (bottom). DP-GaMM PDF is shown in black, sampling PDF is shown in red, and normalized frequency (normalized to area 1) of the sampled data is shown in gray.

Fig. 4.
Fig. 4.

PDFs from various models for Run 1. The normalized histogram in gray has area 1.

Fig. 5.
Fig. 5.

PDFs from various models for Run 2. The normalized histogram in gray has area 1.

Fig. 6.
Fig. 6.

PDFs from various models for Run 3. The normalized histogram in gray has area 1.

Fig. 7.
Fig. 7.

PDF of raw voltage readings from sensors. The red PDFs are samples from the collections iterations, and the blue PDF is the sample with highest training set likelihood. The normalized histogram in gray has area 1.

Tables (1)

Tables Icon

Table 1. Log-Likelihood Comparison of Various Models on the Three Runsa

Equations (20)

Equations on this page are rendered with MathJax. Learn more.

p(x|π,Φ)=π1f(x|ϕ1)+π2f(x|ϕ2)+π3f(x|ϕ3)+,
f(x|ϕk)=Gamma(x;ηk,ηk/μk)=(ηk/μk)ηkΓ(ηk)xηk1exp(ηkμkx),
SI=I2I21,
xG,GDP(α,G0),
xf(ϕz),zMult(π),G=k=1πkδϕk,πk=Vkl<k(1Vl),VkBeta(1,α),ϕkG0,
xn=In1Nn=1NIn,
f(xn|ϕk)=(ηk/μk)ηkΓ(ηk)xnηk1exp(ηkμkxn).
xnGamma(ηk,ηk/μk),znMult(π),πk=Vkl<k(1Vl),VkBeta(1,α),ηkExp(a),μkInvGamma(b,c),
G0=p(ϕk)=p(ηk,μk|a,b,c)=p(ηk|a)p(μk|b,c)=Exp(ηk;a)InvGamma(μk;b,c),
p(A|X)=p(X|A)p(A)p(X|A)p(A)dA,
n=1Np(xn|zn,{γk,Vk,ηk,μk}k=1:K)=n=1Nk=1K{p(xn|zn,ηk,μk)p(zn|Vk)p(Vk|α)p(ηk|a)p(μk|b,c)}.
znMult(ν1,ν2,,νK),
νk=πkGamma(xn;ηk,ηk/μk)k=1KπkGamma(xn;ηk,ηl/μk).
VkBeta(1+Nk,α+k=k+1KNk),
μkInvGamma(b+n=1N1(zn=k),c+n=1N1(zn=k)xn).
p(ηk|{xn,zn}n=1:N,{μk}k=1:K)ηkn=1N1(zn=k)ηkΓ(ηk)n=1N1(zn=k)exp(ηk(a+n=1N1(zn=k)xnμk+ηklogμklogn=1N1(zn=k)xn)),
min{1,p(η˜k|{xn,zn}n=1:N,{μk}k=1:K)p(η˜k|r,ηk)p(ηk|{xn,zn}n=1:N,{μk}k=1:K)p(ηk|r,η˜k)},
αkGamma(d+1,elog(1Vk)),
H=m=1Mlogp(xm(held-out)|Ω),
H=t=1Tm=1Mlogp(xm(held-out)|Ωt)/T

Metrics