Hartung-Gorre Verlag

Inh.: Dr. Renate Gorre

D-78465 Konstanz

Fon: +49 (0)7533 97227

Fax: +49 (0)7533 97228




Series in Signal and Information Processing, Vol. 23
edited by Hans-Andrea Loeliger

Christoph Reller

State-Space Methods in Statistical Signal Processing:

New Ideas and Applications.

1. Auflage/1st edition 2013, XXIV, 264 Seiten/pages, € 64,00.

ISBN 3-86628-447-0, 978-3-86628-447-0



This thesis is about several extensions of a general framework and about the application of these extensions to various problems arising in signal processing. The general framework is a graphical modeling technique, more precisely factor graphs, which provides the basis for the development of message passing algorithms. Such algorithms can be used to solve many statistical inference problems, most notably, estimation and detection problems.

Most of the problems addressed in this thesis are in some way linked to one or several discrete-time state-space models. While state-space representations of systems are widely used in control theory and somewhat less in statistics, a factor graph approach to such models seems to be neglected. Indeed, this thesis shows how the interaction between these two topics leads to a powerful framework for devising novel algorithms in a systematic yet uncomplicated manner.

This thesis is partitioned into two parts. The first part focuses on Gaussian message passing in linear models, and parameter estimation for such models. The second part is concerned with message-passing based computation of likelihoods or related quantities. We start with the first part.

The factor graph representation of a linear model leads to Gaussian message passing in the case of known model parameters. In a first extension we consider several variants and enhancements of recursive-least-squares-type algorithms that incorporate a forgetting factor. An application to outlier detection in a noisy quasi-periodic signal with known period is shown.

Infinite impulse response systems are treated in depth with a focus on the real-valued Jordan canonical form. For the autonomous second-order case, analytic solutions for Gaussian messages across the whole state-space model are derived and the relation between a forgetting factor and state noise is shown. Continuous time signals and two interpolation models are touched upon. In a further extension, the local factor graph view of three approximate inference principles (cyclic maximization, expectation maximization, and local Taylor approximation) is shown. These principles are applied to the estimation of a state-transition matrix that is given in real-valued Jordan canonical form. The same principles are used to estimate covariance matrices of a state-space model. The resulting algorithms are iterative in nature and the resulting messages are members of the exponential family. We show an application to the estimation of the time-varying fundamental frequency of a quasi-periodic signal.

The second part of this thesis starts with exposing connections between model likelihood and scale factors of sum-product messages in a factor graph. A main result is the derivation of message passing update rules for two types of such scale factors that arise in sum-product message passing. First, different types of general factors and general messages are considered. Then the setup is narrowed down to linear factors and Gaussian messages.

Since sum-product message passing is intimately connected with the computation of likelihoods, likelihood functions, and log-likelihood ratios, such quantities can be neatly expressed in terms of messages or message scale factors. The latter need, however, not in all cases be computed, and this case distinction is made precise.

Next, we consider a factor graph representation of linear state-space models augmented with an additional factor - the “glue factor” - connecting state variables of several models. This leads to the notion of a family of factor graphs parametrized by the glue factor parameters and its position on the time axis. A surprising variety of problems such as array processing and pulse modeling can be treated in this framework.

The glue factor view of likelihood computation by means of sum-product message passing leads to the novel concept of likelihood filtering. In essence, this is a message passing algorithm for computing efficiently likelihood-related quantities for each member in the family under consideration. This procedure can be considered as traditional sum-product message passing on several graphs, but without neglecting scale factors, followed by a likelihood computation. Both offline (block based) and online algorithms are thus formulated for estimation and detection of model changes and for locating pulses-like events. Finally, we propose a hierarchical likelihood filter architecture for general signal analysis.



Keywords: State-space model, factor graph, sum-product message passing, parameter estimation, detection, recursive least squares, cyclo-stationary signal, quasi-periodic signal, frequency estimation, expectation maximization, cyclic maximization, real Jordan canonical form, variance estimation, parameter selection, hypothesis testing, glue factor, likelihood filtering, change-point estimation, hierarchical likelihood filtering.

Direkt bestellen bei / to order directly from:

Reihe "Series in Signal and Information Processing" im Hartung-Gorre Verlag

Hartung-Gorre Verlag / D-78465 Konstanz / Germany

Telefon: +49 (0) 7533 97227  Telefax: +49 (0) 7533 97228
http://www.hartung-gorre.de   eMail: verlag@hartung-gorre.de