Dynamic textures are sequences of images of moving scenes that exhibit certain stationarity properties in time; these include sea-waves, smoke, foliage, whirlwind etc. We present a novel characterization of dynamic textures that poses the problems of modeling, learning, recognizing and synthesizing dynamic textures on a firm analytical footing. We borrow tools from system identification to capture the "essence" of dynamic textures; we do so by learning (i.e. identifying) models that are optimal in the sense of maximum likelihood or minimum prediction error variance. For the special case of second-order stationary processes, we identify the model sub-optimally in closed-form. Once learned, a model has predictive power and can be used for extrapolating synthetic sequences to infinite length with negligible computational cost. We present experimental evidence that, within our framework, even low-dimensional models can capture very complex visual phenomena.

This paper reviews a significant component of the rich field of statistical multiresolution (MR) modeling and processing. These MR methods have found application and permeated the literature of a widely scattered set of disciplines, and one of our principal objectives is to present a single, coherent picture of this framework. A second goal is to describe how this topic fits into the even larger field of MR methods and concepts–in particular making ties to topics such as wavelets and multigrid methods. A third is to provide several alternate viewpoints for this body of work, as the methods and concepts we describe intersect with a number of other fields. The principle focus of our presentation is the class of MR Markov processes defined on pyramidally organized trees. The attractiveness of these models stems from both the very efficient algorithms they admit and their expressive power and broad applicability. We show how a variety of methods and models relate to this framework including models for self-similar and 1/f processes. We also illustrate how these methods have been used in practice. We discuss the construction of MR models on trees and show how questions that arise in this context make contact with wavelets, state space modeling of time series, system and parameter identification, and hidden

Recently a great deal of attention has been given to numerical algorithms for subspace state space system identification (N4SID). In this paper, we derive two new N4SID algorithms to identify mixed deterministic-stochastic systems. Both algorithms determine state sequences through the projection of input and output data. These state sequences are shown to be outputs of non-steady state Kalman filter banks. From these it is easy to determine the state space system matrices. The N4SID algorithms are always convergent (non-iterative) and numerically stable since they only make use of QR and Singular Value Decompositions. Both N4SID algorithms are similar, but the second one trades off accuracy for simplicity. These new algorithms are compared with existing subspace algorithms in theory and in practice. Key words : Subspace identification, non-steady state Kalman filter, Riccati difference equations, QR and Singular Value Decomposition 1 Introduction The greater part of the systems ide...

In this paper, we derive a new subspace algorithm to consistently identify stochastic state space models from given output data without forming the covariance matrix and using only semi-infinite block Hankel matrices. The algorithm is based on the concept of principal angles and directions. We describe how they can be calculated with QR and Quotient Singular Value Decomposition. We also provide an interpretation of the principal directions as states of a non-steady state Kalman filter bank. Key Words Principal angles and directions, QR and quotient singular value decomposition, Kalman filter, Riccati difference equation, stochastic balancing, stochastic realization. 1 Introduction Let y k 2 ! l ; k = 0; 1; : : : ; K be a data sequence that is generated by the following system : x k+1 = Ax k + w k (1) y k = Cx k + v k (2) where x k 2 ! n is a state vector. The vector sequence w k 2 ! n is a process noise while the vector sequence v k 2 ! l is a measurement noise. They are bo...

The multiscale autoregressive (MAR) framework was introduced to support the development of optimal multiscale statistical signal processing. Its power resides in the fast and flexible algorithms to which it leads. While the MAR framework was originally motivated by wavelets, the link between these two worlds has been previously established only in the simple case of the Haar wavelet. The first contribution of this paper is to provide a unification of the MAR framework and all compactly supported wavelets as well as a new view of the multiscale stochastic realization problem. The second contribution of this paper is to develop wavelet-based approximate internal MAR models for stochastic processes. This will be done by incorporating a powerful synthesis algorithm for the detail coefficients which complements the usual wavelet reconstruction algorithm for the scaling coefficients. Taking advantage of the statistical machinery provided by the MAR framework, we will illustrate the application of our models to sample-path generation and estimation from noisy, irregular, and sparse measurements.

Abstract. We address the problem of modeling the spatial and temporal second-order statistics of video sequences that exhibit both spatial and temporal regularity, intended in a statistical sense. We model such sequences as dynamic multiscale autoregressive models, and introduce an efficient algorithm to learn the model parameters. We then show how the model can be used to synthesize novel sequences that extend the original ones in both space and time, and illustrate the power, and limitations, of the models we propose with a number of real image sequences. 1

The scope of identification theory is to construct algorithms for automatic model building from observed data. In these lectures we shall only discuss the case where the data are collected in one irrepetible experiment and no preparation of the experiment is possible (i.e. we cannot choose the experimental

In this paper we develop a stochastic realization theory for multiscale autoregressive (MAR) processes that leads to computationally efficient realization algorithms. The utility of MAR processes has been limited by the fact that the previously known general purpose realization algorithm, based on canonical correlations, leads to model inconsistencies and has complexity quartic in problem size. Our realization theory and algorithms addresses these issues by focusing on the estimation-theoretic concept of predictive efficiency and by exploiting the scale-recursive structure of so-called internal MAR processes. Our realization algorithm has complexity quadratic in problem size and with an approximation we also obtain an algorithm that has complexity linear in problem size.

Abstract—We propose a model of the joint variation of shape and appearance of portions of an image sequence. The model is conditionally linear, and can be thought of as an extension of active appearance models to exploit the temporal correlation of adjacent image frames. Inference of the model parameters can be performed efficiently using established numerical optimization techniques borrowed from finite-element analysis and system identification techniques. Index Terms—Active appearance models, linear dynamical systems, video analysis, image motion, dynamic textures. 1

In this report, two algorithms for the state inference and parameter learning of a discrete-time time-invariant linear dynamical system are compared: expectation-maximisation (EM) and subspace state space system identi- cation (4SID). Particular implementations used are the EM algorithm due to Ghahramani and Hinton [8], and a 4SID algorithm due to Van Overschee and De Moor (algorithm3, chapter 3, [26]). The report sets this within a probabilistic framework, noting the inherent phase and state space basis ambiguities, and the requirements for positive realness of the estimated covariance sequence. The EM and 4SID algorithms are similar. Both involve two-stage estimation procedures: state inference using Kalman devices and parameter learning using a form of least-squares. Both assume a minimum-phase system and cast the state into a balanced or frequency-weighted balanced basis. However the two algorithms also dier and the main dierences are as follows. EM is iterative, whereas 4SID ...