A nonlinear black box structure for a dynamical system is a model structure that is prepared to describe virtually any nonlinear dynamics. There has been considerable recent interest in this area with structures based on neural networks, radial basis networks, wavelet networks, hinging hyperplanes, as well as wavelet transform based methods and models based on fuzzy sets and fuzzy rules. This paper describes all these approaches in a common framework, from a user's perspective. It focuses on what are the common features in the different approaches, the choices that have to be made and what considerations are relevant for a successful system identification application of these techniques. It is pointed out that the nonlinear structures can be seen as a concatenation of a mapping from observed data to a regression vector and a nonlinear mapping from the regressor space to the output space. These mappings are discussed separately. The latter mapping is usually formed as a basis function expansion. The basis functions are typically formed from one simple scalar function which is modified in terms of scale and location. The expansion from the scalar argument to the regressor space is achieved by a radial or a ridge type approach. Basic techniques for estimating the parameters in the structures are criterion minimization, as well as two step procedures, where first the relevant basis functions are determined, using data, and then a linear least squares step to determine the coordinates of the function approximation. A particular problem is to deal with the large number of potentially necessary parameters. This is handled by making the number of "used " parameters considerably less than the number of "offered " parameters, by regularization, shrinking, pruning or regressor selection. A more mathematically comprehensive treatment is given in a companion paper (Juditsky et al., 1995).
|
4364
|
Elements of Information Theory
– Cover, Thomas
- 1991
|
|
1871
|
Neural Networks: A Comprehensive Foundation
– Haykin
- 1999
|
|
938
|
Density Estimation for Statistics and Data Analysis
– Silverman
- 1986
|
|
742
|
System Identification: Theory for the User
– Ljung
- 1987
|
|
599
|
Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Prentice-Hall
– DENNIS, SCHNABEL
- 1983
|
|
530
|
Approximation by superposition of a sigmoidal function
– Cybenko
- 1989
|
|
524
|
Networks for approximation and learning
– Poggio, Girosi
- 1990
|
|
431
|
Learning representation by back propagating errors
– Rumelhart, Hinton, et al.
- 1986
|
|
414
|
Fuzzy identification of systems and its applications to modeling and control
– Takagi, Sugeno
- 1985
|
|
401
|
Matching pursuits with timefrequency dictionaries
– Mallat, Zhang
- 1993
|
|
366
|
Beyond Regression: New Tools for Prediction and Analysis
– Werbos
- 1974
|
|
324
|
Bayesian interpolation
– MacKay
- 1992
|
|
309
|
Multiresolution approximation and wavelet orthonormal bases of L2
– Mallat
- 1989
|
|
289
|
Identification and Control of Dynamical Systems Using Neural Networks
– Narendra, Parthasarathy
- 1990
|
|
268
|
Projection pursuit regression
– Friedman, Stuetzle
- 1981
|
|
259
|
Theory and Practice of Recursive Identification
– Ljung, Söderström
- 1983
|
|
257
|
Universal approximation bounds for superposition of a sigmoid function
– Barron
- 1993
|
|
214
|
Applied Regression Analysis
– Draper, Smith
- 1980
|
|
210
|
Regularization algorithms for learning that are equivalent to multilayer networks
– Poggio, Girosi
- 1990
|
|
171
|
Orthogonal Least-Squares Learning Algorithms for Radial Basis Function Network
– Chen, Cowan, et al.
- 1991
|
|
145
|
Ondelettes et Opérateurs
– Meyer
- 1990
|
|
145
|
The Effective Number of Parameters: An Analysis of Generalization and Regularization in Nonlinear Learning Systems
– Moody
- 1992
|
|
118
|
Optimal global rates of convergence for nonparametric regression
– Stone
- 1982
|
|
117
|
Bayesian Methods for Adaptive Models
– MacKay
- 1992
|
|
116
|
A course in density estimation
– Devroye
- 1987
|
|
93
|
Neurofuzzy Adaptive Modelling and Control
– Brown, Harris
- 1994
|
|
91
|
Adaptive Fuzzy Systems and Control: Design and Stability Analysis
– Wang
- 1994
|
|
86
|
Smooth regression analysis
– Watson
- 1964
|
|
76
|
Wavelets, A Tutorial in Theory and Applications
– Chui
- 1992
|
|
75
|
Fuzzy systems are universal approximators
– Wang
- 1992
|
|
73
|
Orthogonal least squares methods and their application to nonlinear system identification
– Chen, Billings, et al.
- 1989
|
|
73
|
Nonlinear regulation: The piecewise linear approach
– Sontag
- 1981
|
|
68
|
Hinging hyperplanes for regression, classification, and function approximation
– Breiman
- 1993
|
|
68
|
System identification using laguerre models
– Wahlberg
- 1991
|
|
59
|
Fuzzy logic, neural networks, and soft computing
– Zadeh
- 1994
|
|
55
|
Projection pursuit (with discussion
– Huber
- 1985
|
|
50
|
Nonlinear system identification using neural networks
– Chen, Billings, et al.
- 1990
|
|
47
|
Digital Neural Networks
– Kung
- 1993
|
|
45
|
System identification using Kautz models
– WAHLBERG
- 1994
|
|
43
|
Supervised learning of probability distributions by neural networks
– Baum, Wilczek
- 1988
|
|
40
|
Neural Networks and Nonlinear Adaptive Filtering: Unifying Concepts and
– Nerrand, Roussel-Ragot, et al.
- 1993
|
|
34
|
Ill-Conditioning in Neural Network Training Problems
– Saarinen, Bramley
- 1993
|
|
32
|
A fuzzy logic based approach to qualitative modeling
– Sugeno, Yasukawa
- 1993
|
|
32
|
The collinearity problem in linear regression, the partial least squares approach to generalized inverse
– Wold, Ruhe, et al.
- 1984
|
|
31
|
Pruning algorithms-a survey
– Reed
- 1993
|
|
26
|
Neural networks for control
– Sontag
- 1993
|
|
24
|
Overtraining, regularization, and searching for minimum in neural networks
– Sjoberg, Ljung
- 1992
|
|
22
|
Using Wavelet Network in Nonparametric Estimation
– Zhang
- 1997
|
|
21
|
Modeling of Dynamic Systems
– Ljung
- 1994
|
|
20
|
Neural networks for nonlinear dynamic system modelling and identification
– Chen, Billings
- 1992
|
