Results 1  10
of
229
Waveletbased statistical signal processing using hidden Markov models
 IEEE TRANSACTIONS ON SIGNAL PROCESSING
, 1998
"... Waveletbased statistical signal processing techniques such as denoising and detection typically model the wavelet coefficients as independent or jointly Gaussian. These models are unrealistic for many realworld signals. In this paper, we develop a new framework for statistical signal processing b ..."
Abstract

Cited by 417 (55 self)
 Add to MetaCart
Waveletbased statistical signal processing techniques such as denoising and detection typically model the wavelet coefficients as independent or jointly Gaussian. These models are unrealistic for many realworld signals. In this paper, we develop a new framework for statistical signal processing based on waveletdomain hidden Markov models (HMM’s) that concisely models the statistical dependencies and nonGaussian statistics encountered in realworld signals. Waveletdomain HMM’s are designed with the intrinsic properties of the wavelet transform in mind and provide powerful, yet tractable, probabilistic signal models. Efficient expectation maximization algorithms are developed for fitting the HMM’s to observational signal data. The new framework is suitable for a wide range of applications, including signal estimation, detection, classification, prediction, and even synthesis. To demonstrate the utility of waveletdomain HMM’s, we develop novel algorithms for signal denoising, classification, and detection.
Universal approximation using incremental constructive feedforward networks with . . .
 IEEE TRANSACTIONS ON NEURAL NETWORKS
, 2005
"... According to conventional neural network theories, singlehiddenlayer feedforward networks (SLFNs) with additive or radial basis function (RBF) hidden nodes are universal approximators when all the parameters of the networks are allowed adjustable. However, as observed in most neural network implem ..."
Abstract

Cited by 89 (15 self)
 Add to MetaCart
According to conventional neural network theories, singlehiddenlayer feedforward networks (SLFNs) with additive or radial basis function (RBF) hidden nodes are universal approximators when all the parameters of the networks are allowed adjustable. However, as observed in most neural network implementations, tuning all the parameters of the networks may cause learning complicated and inefficient, and it may be difficult to train networks with nondifferential activation functions such as threshold networks. Unlike conventional neural network theories, this paper proves in an incremental constructive method that in order to let SLFNs work as universal approximators, one may simply randomly choose hidden nodes and then only need to adjust the output weights linking the hidden layer and the output layer. In such SLFNs implementations, the activation functions for additive nodes can be any bounded nonconstant piecewise continuous functions X and the activation functions for RBF nodes can be any integrable piecewise continuous functions X and @ A aH. The proposed incremental method is efficient not only for SFLNs with continuous (including nondifferentiable) activation functions but also for SLFNs with piecewise continuous (such as threshold) activation functions. Compared to other popular methods such a new network is fully automatic and users need not intervene the learning process by manually tuning control parameters.
Constructive Algorithms for Structure Learning in Feedforward Neural Networks for Regression Problems
 IEEE Transactions on Neural Networks
, 1997
"... In this survey paper, we review the constructive algorithms for structure learning in feedforward neural networks for regression problems. The basic idea is to start with a small network, then add hidden units and weights incrementally until a satisfactory solution is found. By formulating the whole ..."
Abstract

Cited by 87 (2 self)
 Add to MetaCart
(Show Context)
In this survey paper, we review the constructive algorithms for structure learning in feedforward neural networks for regression problems. The basic idea is to start with a small network, then add hidden units and weights incrementally until a satisfactory solution is found. By formulating the whole problem as a state space search, we first describe the general issues in constructive algorithms, with special emphasis on the search strategy. A taxonomy, based on the differences in the state transition mapping, the training algorithm and the network architecture, is then presented. Keywords Constructive algorithm, structure learning, state space search, dynamic node creation, projection pursuit regression, cascadecorrelation, resourceallocating network, group method of data handling. I. Introduction A. Problems with Fixed Size Networks I N recent years, many neural network models have been proposed for pattern classification, function approximation and regression problems. Among...
A fast and accurate online sequential learning algorithm for feedforward networks
 IEEE Trans. Neural Netw
, 2006
"... Abstract—In this paper, we develop an online sequential learning algorithm for single hidden layer feedforward networks (SLFNs) with additive or radial basis function (RBF) hidden nodes in a unified framework. The algorithm is referred to as online sequential extreme learning machine (OSELM) and ca ..."
Abstract

Cited by 45 (7 self)
 Add to MetaCart
Abstract—In this paper, we develop an online sequential learning algorithm for single hidden layer feedforward networks (SLFNs) with additive or radial basis function (RBF) hidden nodes in a unified framework. The algorithm is referred to as online sequential extreme learning machine (OSELM) and can learn data onebyone or chunkbychunk (a block of data) with fixed or varying chunk size. The activation functions for additive nodes in OSELM can be any bounded nonconstant piecewise continuous functions and the activation functions for RBF nodes can be any integrable piecewise continuous functions. In OSELM, the parameters of hidden nodes (the input weights and biases of additive nodes or the centers and impact factors of RBF nodes) are randomly selected and the output weights are analytically determined based on the sequentially arriving data. The algorithm uses the ideas of ELM of Huang et al. developed for batch learning which has been shown to be extremely fast with generalization performance better than other batch training methods. Apart from selecting the number of hidden nodes, no other control parameters have to be manually chosen. Detailed performance comparison of OSELM is done with other popular sequential learning algorithms on benchmark problems drawn from the regression, classification and time series prediction areas. The results show that the OSELM is faster than the other sequential algorithms and produces better generalization performance. Index Terms—Extreme learning machine (ELM), growing and pruning RBF network (GAPRBF), GGAPRBF, minimal resource allocation network (MRAN), online sequential ELM (OSELM), resource allocation network (RAN), resource allocation network via extended kalman filter (RANEKF), stochastic gradient descent backpropagation (SGBP). I.
Regularisation in the Selection of Radial Basis Function Centres
 NEURAL COMPUTATION
, 1995
"... Subset selection and regularisation are two well known techniques which can improve the generalisation performance of nonparametric linear regression estimators, such as radial basis function networks. This paper examines regularised forward selection (RFS)  a combination of forward subset selecti ..."
Abstract

Cited by 43 (7 self)
 Add to MetaCart
Subset selection and regularisation are two well known techniques which can improve the generalisation performance of nonparametric linear regression estimators, such as radial basis function networks. This paper examines regularised forward selection (RFS)  a combination of forward subset selection and zeroorder regularisation. An efficient implementation of RFS into which either delete1 or generalised crossvalidation can be incorporated and a reestimation formula for the regularisation parameter are also discussed. Simulation studies are presented which demonstrate improved generalisation performance due to regularisation in the forward selection of radial basis function centres.
Combined Genetic Algorithm Optimization and Regularized Orthogonal Least Squares Learning for Radial Basis Function Networks
, 1999
"... The paper presents a twolevel learning method for radial basis function (RBF) networks. A regularized orthogonal least squares (ROLS) algorithm is employed at the lower level to construct RBF networks while the two key learning parameters, the regularization parameter and the RBF width, are optimiz ..."
Abstract

Cited by 40 (14 self)
 Add to MetaCart
The paper presents a twolevel learning method for radial basis function (RBF) networks. A regularized orthogonal least squares (ROLS) algorithm is employed at the lower level to construct RBF networks while the two key learning parameters, the regularization parameter and the RBF width, are optimized using a genetic algorithm (GA) at the upper level. Nonlinear time series modeling and prediction is used as an example to demonstrate the effectiveness of this hierarchical learning approach.
New Neural Transfer Functions
 Neural Computing Surveys
, 1997
"... In this article advantages of various neural transfer functions are discussed. ..."
Abstract

Cited by 38 (29 self)
 Add to MetaCart
In this article advantages of various neural transfer functions are discussed.
Identification and Control of Nonlinear Systems Using Neural Network Models: Design and Stability Analysis
 ELECTRICAL ENGINEERING—SYSTEMS REP
, 1991
"... The feasibility of applying neural network learning techniques in problems of system identification and control has been demonstrated through several empirical studies. These studies are based for the most part on gradient techniques for deriving parameter adjustment laws. While such schemes perf ..."
Abstract

Cited by 31 (2 self)
 Add to MetaCart
The feasibility of applying neural network learning techniques in problems of system identification and control has been demonstrated through several empirical studies. These studies are based for the most part on gradient techniques for deriving parameter adjustment laws. While such schemes perform well in many cases, in general, problems arise in attempting to prove stability of the overall system, or convergence of the output error to zero. This paper presents a stability theory approach to synthesizing and analyzing identification and control schemes for nonlinear dynamical systems using neural network models. The nonlinearities of the dynamical system are assumed to be unknown and are modelled by neural network architectures. Multilayer networks with sigmoidal activation functions and radial basis function networks are the two types of neural network models that are considered. These static network architectures are combined with dynamical elements, in the form of stable filters, to construct a type of recurrent network configuration which is shown to be capable of approximating a large class of dynamical systems.
Learning without Local Minima in Radial Basis Function Networks
 IEEE Transactions on Neural Networks
, 1995
"... Learning from examples plays a central role in artificial neural networks (ANN). However, the success of many learning schemes is not guaranteed, since algorithms like Backpropagation (BP) may get stuck in local minima, thus providing suboptimal solutions. For feedforward networks, the theoretical ..."
Abstract

Cited by 31 (6 self)
 Add to MetaCart
(Show Context)
Learning from examples plays a central role in artificial neural networks (ANN). However, the success of many learning schemes is not guaranteed, since algorithms like Backpropagation (BP) may get stuck in local minima, thus providing suboptimal solutions. For feedforward networks, the theoretical results reported in [5,6,15,20] show that optimal learning can be achieved provided that certain conditions on the network and the learning environment are met. A similar investigation is put forward in this paper for the case of networks using radial basis functions (RBF) [10,14]. The analysis proposed in [6] is extended naturally under the assumption that the patterns of the learning environment are separable by hyperspheres. In that case, we prove that the attached cost function is local minima free with respect to all the weights. This provides us with some theoretical foundations for a massive application of RBF in pattern recognition. Keywords Backpropagation, multilayered networks...