System identification is the art and science of building mathematical models of dynamic systems from observed input-output data. It can be seen as the interface between the real world of applications and the mathematical world of control theory and model abstractions. As such, it is an ubiquitous necessity for successful applications. System identification is a very large topic, with different techniques that depend on the character of the models to be estimated: linear, nonlinear, hybrid, nonparametric etc. At the same time, the area can be characterized by a small number of leading principles, e.g. to look for sustainable descriptions by proper decisions in the triangle of model complexity, information contents in the data, and effective validation. The area has many facets and there are many approaches and methods. A tutorial or a survey in a few pages is not quite possible. Instead, this presentation aims at giving an overview of the “science ” side, i.e. basic principles and results and at pointing to open problem areas in the practical, “art”, side of how to approach and solve a real problem. 1.

Spatiotemporal connectionist networks (STCN's) comprise an important class of neural models that can deal with patterns distributed both in time and space. In this paper, we widen the application domain of the taxonomy for supervised STCN's recently proposed by Kremer (2001) to the unsupervised case. This is possible through a reinterpretation of the state vector as a vector of latent (hidden) variables, as proposed by Meinicke (2000). The goal of this generalized taxonomy is then to provide a nonlinear generative framework for describing unsupervised spatiotemporal networks, making it easier to compare and contrast their representational and operational characteristics. Computational properties, representational issues and learning are also discussed and a number of references to the relevant source publications are provided. It is argued that the proposed approach is simple and more powerful than the previous attempts, from a descriptive and predictive viewpoint. We also discuss the relation of this taxonomy with automata theory and state space modeling, and suggest directions for further work.

In this paper we discuss several aspects of the mathematical foundations of non-linear black-box identification problem. As we shall see that the quality of the identification procedure is always a result of a certain trade-off between the expressive power of the model we try to identify (the larger is the number of parameters used to describe the model, more flexible would be the approximation), and the stochastic error (which is proportional to the number of parameters). A consequence of this trade-off is a simple fact that good approximation technique can be a basis of good identification algorithm. From this point of view we consider different approximation methods, and pay special attention to spatially adaptive approximants. We introduce wavelet and "neuron" approximations and show that they are spatially adaptive. Then we apply the acquired approximation experience to estimation problems. Finally, we consider some implications of these theoretic developments for the practically...

We describe both the key principles and real application examples of a unified theory which allows us to perform the on-board incipient fault detection and isolation tasks involved in monitoring for condition-based maintenance. We stress that, when designing detection algorithms, the main conceptual task is to select a convenient estimating function. ml, ls, iv and subspace identification methods are addressed in this perspective.

In previous papers we have described a co-evolutionary algorithm (EEA), the estimation-exploration algorithm, that infers the hidden inner structure of systems using minimal testing. In this paper we introduce the concept of ‘managed challenge ’ to alleviate the problem of disengagement in this and other co-evol-utionary algorithms. A known problem in co-evolutionary dynamics occurs when one population systematically outperforms the other, resulting in a loss of selection pressure for both populations. In system identification (which deals with determining the inner structure of a system using only input/output data), multiple trials (a test that causes the system to produce some output) on the system to be identified must be performed. When such trials are costly, this disengagement results in wasted data that is not utilized by the evolutionary process. Here we propose that data from futile interactions should be stored during disengagement and automatically re-introduced later, when the population re-engages: we refer to this as the test bank. We demonstrate that the advantage of the test bank is twofold: it allows for the discovery of more accurate models, and it reduces the amount of required training data for both parametric identification – parameterizing inner structure – and symbolic identification – approximating inner structure using symbolic equations – of nonlinear systems.

Understanding the dynamic behavior of parallel programs is key to developing efficient system software and runtime environments; this is even more true on emerging computational Grids where resource availability and performance can change in unpredictable ways. Event tracing provides details on behavioral dynamics, albeit often at great cost. We describe an intermediate approach, based on curve fitting, that retains many of the advantages of event tracing but with lower overhead. These compact "application signatures" summarize the time-varying resource needs of scientific codes from historical trace data.

Although feedforward neural networks are well suited to function approximation, in some applications networks experience problems when learning a desired function. One problem is interference which occurs when learning in one area of the input space causes unlearning in another area. Networks that are less susceptible to interference are referred to as spatially local networks. To understand these properties, a theoretical framework, consisting of a measure of interference and a measure of network localization, is developed that incorporates not only the network weights and architecture but also the learning algorithm. Using this framework to analyze sigmoidal multi-layer perceptron (MLP) networks that employ the back-prop learning algorithm, we address a familiar misconception that sigmoidal networks are inherently non-local by demonstrating that given a sufficiently large number of adjustable parameters, sigmoidal MLPs can be made arbitrarily local while retaining the ability to repr...

A general-purpose useful parameteri tia serie forecasti# i the regressorsire correspondiS to themijhTq number ofvariSPhj necessary to forecast the future values of thetiS seriTh If the models are nonlihKjR thechoiK ofthi regressor becomes very diy #cult. We present aquasijRIPE#Sji methodusio a nonliIjR projectiP namedcurvi66hIj component analysi tobuiK thi regressor. The sij ofthi regressorwir bedetermiRI by the estihjRIq of theiejhPSII dijhPSII of anover-siRI regressor.Thi methodwih be appliq toelectri consumpti# of Polandusin systemati cross-valijRI#TT ThenonliqSE model used for the prediqEjR i a Kohonen map(self-organiTPj map). c 2002Publi6#S byElsevij Sciij B.V. Keywords:Ti4 seri0 prediWjhIK NonliWj projectiIK CurvitiIK component analysit Self-organiPIS map; ElectriqjR consumptiR 1. I41pz122 TiI serii forecastii i a great challengei many #elds. In #nance, one forecasts stock exchange courses orijII6P of stock markets; dataprocessij specisijR forecast the #ow ofihK#TEjREh onthei networks; producers ofelectriREh forecast the load of thefollowiE day. The commonpoio tothei problemsi Correspondi author.

We developed in this paper a method to predict time series with non-linear tools. The specificity of the method is to use as much information as possible as input to the model (many past values of the series, many exogenous variables), to compress this information (by a non-linear method) in order to obtain a state vector of limited size, facilitating the subsequent regression and the generalization ability of the forecasting algorithm and to fit a non-linear regressor (here a RBF neural network) on the reduced vectors. We show that this method is able to find non-linear relationships in artificial and real-world financial series. On a difficult task, which consists in forecasting the tendency of the Bel 20 stock market index, we show that this method improves the results compared both to linear models and to non-linear ones where the non-linear compression is not used.

The concept of just in time models is introduced for models that are not estimated until they are really needed. The idea is to store all observations of the process in a database, and then estimate a local model at the current working point. The variance/bias tradeoff is optimized locally by adapting the number of data and their relative weighting. This is in contrast to general non-linear black-box models, like neural networks, where the performance is optimized globally. 1 Introduction Consider a non-linear dynamical system described by ae z(t) = f('(t)) y(t) = z(t) + e(t) (1) Here e(t) is measurement noise and the regression vector '(t) typically consists of lagged inputs and outputs for dynamical systems, but could also be or include a working point vector. The problem considered here is to find a local model to be used for control or prediction. A standard approach is to divide the '(t)-space into a number of regions and once for all compute a model or controller in each of...