Fundamental and advanced developments in neuro-fuzzy synergisms for modeling and control are reviewed. The essential part of neuro-fuzzy synergisms comes from a common framework called adaptive networks, which unifies both neural networks and fuzzy models. The fuzzy models under the framework of adaptive networks is called ANFIS (Adaptive-Network-based Fuzzy Inference System), which possess certain advantages over neural networks. We introduce the design methods for ANFIS in both modeling and control applications. Current problems and future directions for neuro-fuzzy approaches are also addressed.

Animation through the numerical simulation of physics-based graphics models offers unsurpassed realism, but it can be computationally demanding. Likewise, finding controllers that enable physics-based models to produce desired animations usually entails formidable computational cost. This paper demonstrates the possibility of replacing the numerical simulation and control of model dynamics with a dramatically more efficient alternative. In particular, we propose the NeuroAnimator, a novel approach to creating physically realistic animation that exploits neural networks. NeuroAnimators are automatically trained off-line to emulate physical dynamics through the observation of physics-based models in action. Depending on the model, its neural network emulator can yield physically realistic animation one or two orders of magnitude faster than conventional numerical simulation. Furthermore, by exploiting the network structure of the NeuroAnimator, we introduce a fast algorithm for learning controllers that enables either physics-based models or their neural network emulators to synthesize motions satisfying prescribed animation goals. We demonstrate NeuroAnimators for passive and active (actuated) rigid body, articulated, and deformable physics-based models.

We discuss a variety of Adaptive Critic Designs (ACDs) for neurocontrol. These are suitable for learning in noisy, nonlinear, and nonstationary environments. They have common roots as generalizations of dynamic programming for neural reinforcement learning approaches. Our discussion of these origins leads to an explanation of three design families: Heuristic Dynamic Programming (HDP), Dual Heuristic Programming (DHP), and Globalized Dual Heuristic Programming (GDHP). The main emphasis is on DHP and GDHP as advanced ACDs. We suggest two new modifications of the original GDHP design that are currently the only working implementations of GDHP. They promise to be useful for many engineering applications in the areas of optimization and optimal control. Based on one of these modifications, we present a unified approach to all ACDs. This leads to a generalized training procedure for ACDs. 1 The authors gratefully acknowledge support from the Texas Tech Center for Applied Research, Ford Moto...

In this paper, we study the theoretical properties of a new kind of artificial neural network, which is able to adapt its activation functions by varying the control points of a Catmull --Rom cubic spline. Most of all, we are interested in generalization capability, and we can show that our architecture presents several advantages. First of all, it can be seen as a sub-optimal realization of the additive spline based model obtained by the reguralization theory. Besides, simulations confirm that the special learning mechanism allows to use in a very effective way the network's free parameters, keeping their total number at lower values than in networks with sigmoidal activation functions. Other notable properties are a shorter training time and a reduced hardware complexity, due to the surplus in the number of neurons. # 1998 Elsevier Science Ltd. All rights reserved. Keywords: Spline neural networks; Multilayer perceptron; Generalized sigmoidal functions; Adaptive activation functions...

We present a learning algorithm for neural networks, called Alopex. Instead of error gradient, Alopex uses local correlations between changes in individual weights and changes in the global error measure. The algorithm does not make any assump-tions about transfer functions of individual neurons, and does not explicitly depend on the functional form of the error measure. Hence, it can be used in networks with arbi-trary transfer functions and for minimizing a large class of error measures. The learn-ing algorithm is the same for feed-forward and recurrent networks. All the weights in a network are updated simultaneously, using only local computations. This allows com-plete parallelization of the algorithm. The algorithm is stochastic and it uses a ‘tem-perature ’ parameter in a manner similar to that in simulated annealing. A heuristic ‘ annealing schedule ’ is presented which is effective in finding global minima of error surfaces. In this paper, we report extensive simulation studies illustrating these advan-tages and show that learning times are comparable to those for standard gradient des-cent methods. Feed-forward networks trained with Alopex are used to solve the MONK’s problems and symmetry problems. Recurrent networks trained with the same algorithm are used for solving temporal XOR problems. Scaling properties of the algorithm are demonstrated using encoder problems of different sizes and advantages of appropriate error measures are illustrated using a variety of problems.

In this paper, a new complex-valued neural network based on adaptive activation functions is proposed. By varying the control points of a pair of Catmull–Rom cubic splines, which are used as an adaptable activation function, this new kind of neural network can be implemented as a very simple structure that is able to improve the generalization capabilities using few training samples. Due to its low architectural complexity (low overhead with respect to a simple FIR filter), this network can be used to cope with several nonlinear DSP problems at a high symbol rate. In particular, this work addresses the problem of nonlinear channel equalization. In fact, although several authors have already recognized the usefulness of a neural network as a channel equalizer, one problem has not yet been addressed: the high complexity and the very long data sequence needed to train the network. Several experimental results using a realistic channel model are reported that prove the effectiveness of the proposed network on equalizing a digital satellite radio link in the presence of noise, nonlinearities, and intersymbol interference (ISI).

In this paper, a new adaptive spline activation function neural network (ASNN) is presented. Due to the ASNN's high representation capabilities, networks with a small number of interconnections can be trained to solve both pattern recognition and data processing real-time problems. The main idea is to use a Catmull--Rom cubic spline as the neuron's activation function, which ensures a simple structure suitable for both software and hardware implementation. Experimental results demonstrate improvements in terms of generalization capability and of learning speed in both pattern recognition and data processing tasks. Index Terms--- Adaptive activation functions, function shape autotuning, generalization, generalized sigmoidal functions, multilayer perceptron, neural networks, spline neural networks. I. INTRODUCTION I N both hardware and software neural network (NN) implementations, the complexity, both structural, in terms of interconnections, and computational, in terms of the number...

The algorithm presented performs gradient descent on the weight space of an Artificial Neural Network (ANN), using a finite difference to approximate the gradient. The method is novel in that it achieves a computational complexity similar to that of Node Perturbation, O(N 3 ), but does not require access to the activity of hidden or internal neurons. This is possible due to a stochastic relation between perturbations at the weights and the neurons of an ANN. The algorithm is also similar to Weight Perturbation in that it is optimal in terms of hardware requirements when used for the training of VLSI implementations of ANN's. 1 INTRODUCTION Optimization of the weights of an ANN may be performed by, the application of a gradient descent technique. The gradient may be calculated directly as in Backpropagation, or it may be approximated by a Finite Difference Method which is what we concern ourselves with in this paper. These methods lend themselves to the task of training hardware imp...

A generic, modular, neural network-based feature extraction and pattern classification system is proposed for finding essentially 2dimensional objects or object parts from digital images in a distortion tolerant manner. The distortion tolerance is built up gradually by successive blocks in a pipeline architecture. The system consists of only feedforward neural networks, allowing efficient parallel implementation. The most time and data-consuming stage, learning the relevant features, is wholly unsupervised and can be made off-line. The consequent supervised stage where the object classes are learned is simple and fast. The feature extraction is based on distortion tolerant Gabor transformations, followed by minimum distortion clustering by Multilayer Self-Organizing Maps (MSOM). Due to the unsupervised learning strategy, there is no need for preclassified training samples or other explicit selection for training patterns during the training, which allows a large amount of training mate...

Abstract — Set-membership identification (SMI) theory is extended to the more general problem of linear-in-parameters filtering by defining a set-membership specification, as opposed to a bounded noise assumption. This sets the framework for several important filtering problems that are not modeled by a “true ” unknown system with bounded noise, such as adaptive equalization, to exploit the unique advantages of SMI algorithms. A recursive solution for set membership filtering is derived that resembles a variable step size normalized least mean squares (NLMS) algorithm. Interesting properties of the algorithm, such as asymptotic cessation of updates and monotonically nonincreasing parameter error, are established. Simulations show significant performance improvement in varied environments with a greatly reduced number of updates. I.