Abstract

We present new high-resolution methods for the problem of retrieving sinusoidal processes from noisy measurements. The approach taken is by use of the so-called principal-components method, which is a singular-value-decomposition- based approximate modeling method. The low-rank property and the algebraic structure of both the data matrix and the covariance matrix (under noise-free conditions) form the basis of exact modeling methods. In a noisy environment, however, the rank property is often perturbed, and singular-value decomposition is used to obtain a low-rank approximant in factored form. The underlying algebraic structure of these factors leads naturally to least-squares estimates of the state-space parameters of the sinusoidal process. This forms the basis of the Toeplitz approximation method, which offers a robust Pisarenko-like spectral estimate from the covariance sequence. Furthermore, the principle of Pisarenko's method is extended to harmonic retrieval directly from timeseries data, which leads to a direct-data approximation method. Our simulation results indicate that favorable resolution capability (compared with existing methods) can be achieved by the above methods. The application of these principles to two-dimensional signals is also discussed.

© 1983 Optical Society of America

PDF Article

References

  • View by:
  • |
  • |
  • |

  1. O. S. Halpeny and D. G. Childers, "Composite wavefront decomposition vai multidimensional digital filtering of array data," IEEE Trans. Circuits Syst. CAS-22, 552–562 (1975).
  2. S. M. Kay and S. L. Marple, Jr., "Spectrum analysis—a modern perspective," Proc. IEEE 69, 1380–1418 (1981).
  3. J. A. Cadzow, "Spectral estimation: an overdetermined rational model equation approach," Proc. IEEE 70, 907–939 (1982).
  4. D. W. Tufts and R. Kumaresen, "Estimation of frequencies of multiple sinusoids: making linear prediction perform like maximum likelihood," Proc. IEEE 70, 975–989 (1982).
  5. G. M. Jenkins and D. G. Watts, Spectral Analysis and its Applications (Holden-Day, San Francisco, Calif., 1966).
  6. A. Papoulis, Probability, Random Variables, and Stochastic Processes (McGraw-Hill, New York, 1965).
  7. N. Levinson, "The Wiener rms (root mean square) error criterion in filter design and prediction," J. Math. Phys. 25, 261–278 (1947).
  8. J. P. Burg, "Maximum entropy spectral analysis," Ph.D Thesis (Stanford University, Stanford, Calif., 1975).
  9. W. Gersh, "Estimation of the autoregressive parameters of mixed autoregressive moving-average time series," IEEE Trans. Autom. Control AC-15, 583–588 (1970).
  10. T. Kailath, Linear Systems (Prentice-Hall, Englewood Cliffs, N.J., 1980).
  11. L. Kronecker, "Zur Theorie der Elmination einer Variabeln aus zwei Algebraischen Gleichungen," Trans. Royal Prussian Academy of Science (see collected works, Vol. 2), 1881.
  12. B. L. Ho and R. E. Kalman, "Effective construction of linear state variable models from input/output data," in Proceedings of the 3rd Allerton Conference on Circuits and System Theory (U. Illinois Press, Urbana, Ill., 1965), pp. 449–459.
  13. J. Rissanen, "Recursive identification of linear systems," J. SIAM Control 9, 420–430 (1971).
  14. P. Faurre, "Stochastic realization algorithms" in System Iden-tification: Advances and Case Studies, R. K. Mehra and D. G. Lainiotis, eds. (Academic, New York, 1976).
  15. V. C. Klemma and A. J. Laub, "The singular value decomposition: its computation and some applications," IEEE Trans. Autom. Control AC-25, 164–176 (1980).
  16. S. Y. Kung, "A new identification and model reduction algorithm via singular value decomposition", in Proceedings of the 12th Asilomar Conference on Circuits, Systems and Computers, (Institute of Electrical and Electronics Engineers, New York, 1978), pp. 705–714.
  17. B. C. Moore, "Principal component analysis in linear systems: controllability, observability, and model reduction," IEEE Trans. Autom. Control AC-26, 17–32 (1981).
  18. C. T. Mullis and R. A. Roberts, "Synthesis of minimum round-off noise fixed point digital filters," IEEE Trans. Circuits Syst. CAS-23, 551–562 (1976).
  19. S. Y. Kung and K. S. Arun, "A novel Hankel approximation method for ARMA pole-zero estimation from noisy covariance data," in Digest of the Topical Meeting on Signal Recovery and Synthesis with Incomplete Information and Partial Constraints (Optical Society of America, Washington, D.C., 1983), pp. WA19-1–WA19-5.
  20. U. B. Desai and D. Pal, "A realization approach to stochastic model reduction and balanced stochastic realizations," in Proceedings of the Twenty-First IEEE Conference on Decision and Control (Institute of Electronics and Electrical Engineers, New York, 1982), pp. 1105–1112.
  21. S. Y. Kung and K. S. Arun, "Approximate realization methods for ARMA spectral estimation," in Proceedings of the IEEE International Symposium on Circuits and Systems (Institute of Electrical and Electronics Engineers, New York, 1983).
  22. V. F. Pisarenko, "The retrieval of harmonics from a covariance function," Geophys. J. R. Astron. Soc. Can. 33, 347–366 (1973).
  23. D. V. Bhaskar Rao, "Adaptive notch filtering for the retrieval of harmonics," Ph.D Thesis (University of Southern California, Los Angeles, Calif., 1983).
  24. G. R. B. Prony, "Essai experimental et analytique, etc.," J. Ec. Polytech. 1, 24–76 (1795).
  25. T. J. Ulyrch and R. W. Clayton, "Time series modeling and maximum entropy," Phys. Earth Planet. Inter. 12, 188–200 (1976).
  26. S. Y. Kung, "A Toeplitz approximation method and some applications," in Proceedings of the International Symposium on Mathematical Theory of Networks and Systems (Western, North Hollywood, Calif., 1981), pp. 262–266.
  27. J. P. Burg, "A new analysis technique for time series data," presented at NATO Advanced Study Institute on Signal Processing with Emphasis on Underwater Acoustics, August 12–23, 1968.
  28. G. H. Golub and C. Reinsch, "Singular value decomposition and least squares solutions," Numer. Math. 14, 403–420 (1970).
  29. S. Attasi, "Modelling and recursive estimation for double indexed sequences," in System Identification: Advances and Case Studies, R. K. Mehra and D. G. Lainiotis, eds. (Academic, New York, 1976).

1983 (3)

S. Y. Kung and K. S. Arun, "A novel Hankel approximation method for ARMA pole-zero estimation from noisy covariance data," in Digest of the Topical Meeting on Signal Recovery and Synthesis with Incomplete Information and Partial Constraints (Optical Society of America, Washington, D.C., 1983), pp. WA19-1–WA19-5.

S. Y. Kung and K. S. Arun, "Approximate realization methods for ARMA spectral estimation," in Proceedings of the IEEE International Symposium on Circuits and Systems (Institute of Electrical and Electronics Engineers, New York, 1983).

D. V. Bhaskar Rao, "Adaptive notch filtering for the retrieval of harmonics," Ph.D Thesis (University of Southern California, Los Angeles, Calif., 1983).

1982 (3)

U. B. Desai and D. Pal, "A realization approach to stochastic model reduction and balanced stochastic realizations," in Proceedings of the Twenty-First IEEE Conference on Decision and Control (Institute of Electronics and Electrical Engineers, New York, 1982), pp. 1105–1112.

J. A. Cadzow, "Spectral estimation: an overdetermined rational model equation approach," Proc. IEEE 70, 907–939 (1982).

D. W. Tufts and R. Kumaresen, "Estimation of frequencies of multiple sinusoids: making linear prediction perform like maximum likelihood," Proc. IEEE 70, 975–989 (1982).

1981 (3)

S. M. Kay and S. L. Marple, Jr., "Spectrum analysis—a modern perspective," Proc. IEEE 69, 1380–1418 (1981).

B. C. Moore, "Principal component analysis in linear systems: controllability, observability, and model reduction," IEEE Trans. Autom. Control AC-26, 17–32 (1981).

S. Y. Kung, "A Toeplitz approximation method and some applications," in Proceedings of the International Symposium on Mathematical Theory of Networks and Systems (Western, North Hollywood, Calif., 1981), pp. 262–266.

1980 (1)

V. C. Klemma and A. J. Laub, "The singular value decomposition: its computation and some applications," IEEE Trans. Autom. Control AC-25, 164–176 (1980).

1978 (1)

S. Y. Kung, "A new identification and model reduction algorithm via singular value decomposition", in Proceedings of the 12th Asilomar Conference on Circuits, Systems and Computers, (Institute of Electrical and Electronics Engineers, New York, 1978), pp. 705–714.

1976 (4)

C. T. Mullis and R. A. Roberts, "Synthesis of minimum round-off noise fixed point digital filters," IEEE Trans. Circuits Syst. CAS-23, 551–562 (1976).

S. Attasi, "Modelling and recursive estimation for double indexed sequences," in System Identification: Advances and Case Studies, R. K. Mehra and D. G. Lainiotis, eds. (Academic, New York, 1976).

P. Faurre, "Stochastic realization algorithms" in System Iden-tification: Advances and Case Studies, R. K. Mehra and D. G. Lainiotis, eds. (Academic, New York, 1976).

T. J. Ulyrch and R. W. Clayton, "Time series modeling and maximum entropy," Phys. Earth Planet. Inter. 12, 188–200 (1976).

1975 (2)

O. S. Halpeny and D. G. Childers, "Composite wavefront decomposition vai multidimensional digital filtering of array data," IEEE Trans. Circuits Syst. CAS-22, 552–562 (1975).

J. P. Burg, "Maximum entropy spectral analysis," Ph.D Thesis (Stanford University, Stanford, Calif., 1975).

1973 (1)

V. F. Pisarenko, "The retrieval of harmonics from a covariance function," Geophys. J. R. Astron. Soc. Can. 33, 347–366 (1973).

1971 (1)

J. Rissanen, "Recursive identification of linear systems," J. SIAM Control 9, 420–430 (1971).

1970 (2)

W. Gersh, "Estimation of the autoregressive parameters of mixed autoregressive moving-average time series," IEEE Trans. Autom. Control AC-15, 583–588 (1970).

G. H. Golub and C. Reinsch, "Singular value decomposition and least squares solutions," Numer. Math. 14, 403–420 (1970).

1968 (1)

J. P. Burg, "A new analysis technique for time series data," presented at NATO Advanced Study Institute on Signal Processing with Emphasis on Underwater Acoustics, August 12–23, 1968.

1965 (1)

B. L. Ho and R. E. Kalman, "Effective construction of linear state variable models from input/output data," in Proceedings of the 3rd Allerton Conference on Circuits and System Theory (U. Illinois Press, Urbana, Ill., 1965), pp. 449–459.

1947 (1)

N. Levinson, "The Wiener rms (root mean square) error criterion in filter design and prediction," J. Math. Phys. 25, 261–278 (1947).

1881 (1)

L. Kronecker, "Zur Theorie der Elmination einer Variabeln aus zwei Algebraischen Gleichungen," Trans. Royal Prussian Academy of Science (see collected works, Vol. 2), 1881.

1795 (1)

G. R. B. Prony, "Essai experimental et analytique, etc.," J. Ec. Polytech. 1, 24–76 (1795).

Arun, K. S.

S. Y. Kung and K. S. Arun, "A novel Hankel approximation method for ARMA pole-zero estimation from noisy covariance data," in Digest of the Topical Meeting on Signal Recovery and Synthesis with Incomplete Information and Partial Constraints (Optical Society of America, Washington, D.C., 1983), pp. WA19-1–WA19-5.

S. Y. Kung and K. S. Arun, "Approximate realization methods for ARMA spectral estimation," in Proceedings of the IEEE International Symposium on Circuits and Systems (Institute of Electrical and Electronics Engineers, New York, 1983).

Attasi, S.

S. Attasi, "Modelling and recursive estimation for double indexed sequences," in System Identification: Advances and Case Studies, R. K. Mehra and D. G. Lainiotis, eds. (Academic, New York, 1976).

Burg, J. P.

J. P. Burg, "Maximum entropy spectral analysis," Ph.D Thesis (Stanford University, Stanford, Calif., 1975).

J. P. Burg, "A new analysis technique for time series data," presented at NATO Advanced Study Institute on Signal Processing with Emphasis on Underwater Acoustics, August 12–23, 1968.

Cadzow, J. A.

J. A. Cadzow, "Spectral estimation: an overdetermined rational model equation approach," Proc. IEEE 70, 907–939 (1982).

Childers, D. G.

O. S. Halpeny and D. G. Childers, "Composite wavefront decomposition vai multidimensional digital filtering of array data," IEEE Trans. Circuits Syst. CAS-22, 552–562 (1975).

Clayton, R. W.

T. J. Ulyrch and R. W. Clayton, "Time series modeling and maximum entropy," Phys. Earth Planet. Inter. 12, 188–200 (1976).

Desai, U. B.

U. B. Desai and D. Pal, "A realization approach to stochastic model reduction and balanced stochastic realizations," in Proceedings of the Twenty-First IEEE Conference on Decision and Control (Institute of Electronics and Electrical Engineers, New York, 1982), pp. 1105–1112.

Faurre, P.

P. Faurre, "Stochastic realization algorithms" in System Iden-tification: Advances and Case Studies, R. K. Mehra and D. G. Lainiotis, eds. (Academic, New York, 1976).

Gersh, W.

W. Gersh, "Estimation of the autoregressive parameters of mixed autoregressive moving-average time series," IEEE Trans. Autom. Control AC-15, 583–588 (1970).

Golub, G. H.

G. H. Golub and C. Reinsch, "Singular value decomposition and least squares solutions," Numer. Math. 14, 403–420 (1970).

Halpeny, O. S.

O. S. Halpeny and D. G. Childers, "Composite wavefront decomposition vai multidimensional digital filtering of array data," IEEE Trans. Circuits Syst. CAS-22, 552–562 (1975).

Ho, B. L.

B. L. Ho and R. E. Kalman, "Effective construction of linear state variable models from input/output data," in Proceedings of the 3rd Allerton Conference on Circuits and System Theory (U. Illinois Press, Urbana, Ill., 1965), pp. 449–459.

Jenkins, G. M.

G. M. Jenkins and D. G. Watts, Spectral Analysis and its Applications (Holden-Day, San Francisco, Calif., 1966).

Kailath, T.

T. Kailath, Linear Systems (Prentice-Hall, Englewood Cliffs, N.J., 1980).

Kalman, R. E.

B. L. Ho and R. E. Kalman, "Effective construction of linear state variable models from input/output data," in Proceedings of the 3rd Allerton Conference on Circuits and System Theory (U. Illinois Press, Urbana, Ill., 1965), pp. 449–459.

Kay, S. M.

S. M. Kay and S. L. Marple, Jr., "Spectrum analysis—a modern perspective," Proc. IEEE 69, 1380–1418 (1981).

Klemma, V. C.

V. C. Klemma and A. J. Laub, "The singular value decomposition: its computation and some applications," IEEE Trans. Autom. Control AC-25, 164–176 (1980).

Kronecker, L.

L. Kronecker, "Zur Theorie der Elmination einer Variabeln aus zwei Algebraischen Gleichungen," Trans. Royal Prussian Academy of Science (see collected works, Vol. 2), 1881.

Kumaresen, R.

D. W. Tufts and R. Kumaresen, "Estimation of frequencies of multiple sinusoids: making linear prediction perform like maximum likelihood," Proc. IEEE 70, 975–989 (1982).

Kung, S. Y.

S. Y. Kung and K. S. Arun, "A novel Hankel approximation method for ARMA pole-zero estimation from noisy covariance data," in Digest of the Topical Meeting on Signal Recovery and Synthesis with Incomplete Information and Partial Constraints (Optical Society of America, Washington, D.C., 1983), pp. WA19-1–WA19-5.

S. Y. Kung and K. S. Arun, "Approximate realization methods for ARMA spectral estimation," in Proceedings of the IEEE International Symposium on Circuits and Systems (Institute of Electrical and Electronics Engineers, New York, 1983).

S. Y. Kung, "A Toeplitz approximation method and some applications," in Proceedings of the International Symposium on Mathematical Theory of Networks and Systems (Western, North Hollywood, Calif., 1981), pp. 262–266.

S. Y. Kung, "A new identification and model reduction algorithm via singular value decomposition", in Proceedings of the 12th Asilomar Conference on Circuits, Systems and Computers, (Institute of Electrical and Electronics Engineers, New York, 1978), pp. 705–714.

Laub, A. J.

V. C. Klemma and A. J. Laub, "The singular value decomposition: its computation and some applications," IEEE Trans. Autom. Control AC-25, 164–176 (1980).

Levinson, N.

N. Levinson, "The Wiener rms (root mean square) error criterion in filter design and prediction," J. Math. Phys. 25, 261–278 (1947).

Marple, Jr., S. L.

S. M. Kay and S. L. Marple, Jr., "Spectrum analysis—a modern perspective," Proc. IEEE 69, 1380–1418 (1981).

Moore, B. C.

B. C. Moore, "Principal component analysis in linear systems: controllability, observability, and model reduction," IEEE Trans. Autom. Control AC-26, 17–32 (1981).

Mullis, C. T.

C. T. Mullis and R. A. Roberts, "Synthesis of minimum round-off noise fixed point digital filters," IEEE Trans. Circuits Syst. CAS-23, 551–562 (1976).

Pal, D.

U. B. Desai and D. Pal, "A realization approach to stochastic model reduction and balanced stochastic realizations," in Proceedings of the Twenty-First IEEE Conference on Decision and Control (Institute of Electronics and Electrical Engineers, New York, 1982), pp. 1105–1112.

Papoulis, A.

A. Papoulis, Probability, Random Variables, and Stochastic Processes (McGraw-Hill, New York, 1965).

Pisarenko, V. F.

V. F. Pisarenko, "The retrieval of harmonics from a covariance function," Geophys. J. R. Astron. Soc. Can. 33, 347–366 (1973).

Prony, G. R. B.

G. R. B. Prony, "Essai experimental et analytique, etc.," J. Ec. Polytech. 1, 24–76 (1795).

Rao, D. V. Bhaskar

D. V. Bhaskar Rao, "Adaptive notch filtering for the retrieval of harmonics," Ph.D Thesis (University of Southern California, Los Angeles, Calif., 1983).

Reinsch, C.

G. H. Golub and C. Reinsch, "Singular value decomposition and least squares solutions," Numer. Math. 14, 403–420 (1970).

Rissanen, J.

J. Rissanen, "Recursive identification of linear systems," J. SIAM Control 9, 420–430 (1971).

Roberts, R. A.

C. T. Mullis and R. A. Roberts, "Synthesis of minimum round-off noise fixed point digital filters," IEEE Trans. Circuits Syst. CAS-23, 551–562 (1976).

Tufts, D. W.

D. W. Tufts and R. Kumaresen, "Estimation of frequencies of multiple sinusoids: making linear prediction perform like maximum likelihood," Proc. IEEE 70, 975–989 (1982).

Ulyrch, T. J.

T. J. Ulyrch and R. W. Clayton, "Time series modeling and maximum entropy," Phys. Earth Planet. Inter. 12, 188–200 (1976).

Watts, D. G.

G. M. Jenkins and D. G. Watts, Spectral Analysis and its Applications (Holden-Day, San Francisco, Calif., 1966).

Geophys. J. R. Astron. Soc. Can. (1)

V. F. Pisarenko, "The retrieval of harmonics from a covariance function," Geophys. J. R. Astron. Soc. Can. 33, 347–366 (1973).

IEEE Trans. Autom. Control (3)

V. C. Klemma and A. J. Laub, "The singular value decomposition: its computation and some applications," IEEE Trans. Autom. Control AC-25, 164–176 (1980).

W. Gersh, "Estimation of the autoregressive parameters of mixed autoregressive moving-average time series," IEEE Trans. Autom. Control AC-15, 583–588 (1970).

B. C. Moore, "Principal component analysis in linear systems: controllability, observability, and model reduction," IEEE Trans. Autom. Control AC-26, 17–32 (1981).

IEEE Trans. Circuits Syst. (2)

C. T. Mullis and R. A. Roberts, "Synthesis of minimum round-off noise fixed point digital filters," IEEE Trans. Circuits Syst. CAS-23, 551–562 (1976).

O. S. Halpeny and D. G. Childers, "Composite wavefront decomposition vai multidimensional digital filtering of array data," IEEE Trans. Circuits Syst. CAS-22, 552–562 (1975).

J. Ec. Polytech. (1)

G. R. B. Prony, "Essai experimental et analytique, etc.," J. Ec. Polytech. 1, 24–76 (1795).

J. Math. Phys. (1)

N. Levinson, "The Wiener rms (root mean square) error criterion in filter design and prediction," J. Math. Phys. 25, 261–278 (1947).

J. SIAM Control (1)

J. Rissanen, "Recursive identification of linear systems," J. SIAM Control 9, 420–430 (1971).

Numer. Math. (1)

G. H. Golub and C. Reinsch, "Singular value decomposition and least squares solutions," Numer. Math. 14, 403–420 (1970).

Phys. Earth Planet. Inter. (1)

T. J. Ulyrch and R. W. Clayton, "Time series modeling and maximum entropy," Phys. Earth Planet. Inter. 12, 188–200 (1976).

Other (18)

S. Y. Kung, "A Toeplitz approximation method and some applications," in Proceedings of the International Symposium on Mathematical Theory of Networks and Systems (Western, North Hollywood, Calif., 1981), pp. 262–266.

J. P. Burg, "A new analysis technique for time series data," presented at NATO Advanced Study Institute on Signal Processing with Emphasis on Underwater Acoustics, August 12–23, 1968.

S. Y. Kung, "A new identification and model reduction algorithm via singular value decomposition", in Proceedings of the 12th Asilomar Conference on Circuits, Systems and Computers, (Institute of Electrical and Electronics Engineers, New York, 1978), pp. 705–714.

D. V. Bhaskar Rao, "Adaptive notch filtering for the retrieval of harmonics," Ph.D Thesis (University of Southern California, Los Angeles, Calif., 1983).

S. Attasi, "Modelling and recursive estimation for double indexed sequences," in System Identification: Advances and Case Studies, R. K. Mehra and D. G. Lainiotis, eds. (Academic, New York, 1976).

P. Faurre, "Stochastic realization algorithms" in System Iden-tification: Advances and Case Studies, R. K. Mehra and D. G. Lainiotis, eds. (Academic, New York, 1976).

J. P. Burg, "Maximum entropy spectral analysis," Ph.D Thesis (Stanford University, Stanford, Calif., 1975).

S. Y. Kung and K. S. Arun, "A novel Hankel approximation method for ARMA pole-zero estimation from noisy covariance data," in Digest of the Topical Meeting on Signal Recovery and Synthesis with Incomplete Information and Partial Constraints (Optical Society of America, Washington, D.C., 1983), pp. WA19-1–WA19-5.

U. B. Desai and D. Pal, "A realization approach to stochastic model reduction and balanced stochastic realizations," in Proceedings of the Twenty-First IEEE Conference on Decision and Control (Institute of Electronics and Electrical Engineers, New York, 1982), pp. 1105–1112.

S. Y. Kung and K. S. Arun, "Approximate realization methods for ARMA spectral estimation," in Proceedings of the IEEE International Symposium on Circuits and Systems (Institute of Electrical and Electronics Engineers, New York, 1983).

S. M. Kay and S. L. Marple, Jr., "Spectrum analysis—a modern perspective," Proc. IEEE 69, 1380–1418 (1981).

J. A. Cadzow, "Spectral estimation: an overdetermined rational model equation approach," Proc. IEEE 70, 907–939 (1982).

D. W. Tufts and R. Kumaresen, "Estimation of frequencies of multiple sinusoids: making linear prediction perform like maximum likelihood," Proc. IEEE 70, 975–989 (1982).

G. M. Jenkins and D. G. Watts, Spectral Analysis and its Applications (Holden-Day, San Francisco, Calif., 1966).

A. Papoulis, Probability, Random Variables, and Stochastic Processes (McGraw-Hill, New York, 1965).

T. Kailath, Linear Systems (Prentice-Hall, Englewood Cliffs, N.J., 1980).

L. Kronecker, "Zur Theorie der Elmination einer Variabeln aus zwei Algebraischen Gleichungen," Trans. Royal Prussian Academy of Science (see collected works, Vol. 2), 1881.

B. L. Ho and R. E. Kalman, "Effective construction of linear state variable models from input/output data," in Proceedings of the 3rd Allerton Conference on Circuits and System Theory (U. Illinois Press, Urbana, Ill., 1965), pp. 449–459.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.