D. R. Hush and B. G. Horne, "Progress in Supervised Neural Networks: What's New Since Lippmann?," IEEE Signal Proc. Mag., pp 8-39, Jan. 1993.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....with x(t) denoting the input vector, o(t) denoting the output vector, w(i,j) denoting the weights and W denoting the connection weight. Held in the relation: where x(t) is the input vector and o(t) is the output vector. For more information on the specifics of the MLP neural network, please see [8]. Note that the MLP has neither time delay elements nor any recurrent input. The class of MLP networks is among the simplest and most powerful type of neural networks. Figure 1: MLP diagram Two matrices characterize an MLP neural network: The MLP search was constrained to the three layer model ....

D.R. Hush and G.B. Horne, "Progress in supervised neural networks: What's new since Lippmann?," IEEE Signal Processing Magazine, pp 8-38, 1993.

....or from expert s knowledge or from experimental analysis of the system. As these data can also contain uncertainty, a suitable approach is required so as to include any information about the system. In such situations, the system identification can be done by a clustering technique. Clustering [1] is one of the most fundamental issues in pattern recognition. It plays a significant role in searching for structures in data. Given a finite set of data X, the problem of clustering in X is to find several cluster centers that can properly characterize relevant classes of X. In classic cluster ....

Hush D.R., and Horne B.G, "Progress in Supervised Neural Networks. What's new since Lippmann?", IEEE Signal Processing Magazine, January 1993, vol. 10, No 1, p.32-43.

....the most plausible and challenging motivation and framework for the design of such intelligent machines [GR94, RHW86] 1 CHAPTER 1. INTRODUCTION 2 Artificial Neural Network (NN) systems are computational models which mimic the functionality of the human brain in a very fundamental manner [HH93] NN classifiers have some merits over traditional pattern recognition systems with respect to capability of adaptive learning, generalization ability with noisy or sparse learning data, and feasibility for hardware implementation. Theoretically, a NN can approximate any function with any desired ....

....some merits over traditional pattern recognition systems with respect to capability of adaptive learning, generalization ability with noisy or sparse learning data, and feasibility for hardware implementation. Theoretically, a NN can approximate any function with any desired degree of accuracy [HH93, HSW89] 1.2 Problem Practically, however, there are several performance problems in the current NN designs. This thesis considers problems of two NN classifier types; the current generation of non modular NN classifiers, and the recently attempted Modular Neural Network (MNN) alternatives. ....

[Article contains additional citation context not shown here]

D. Hush and B. Horne. Progress in supervised neural networks: What's new since lippmann ? IEEE Signal Processing Magazine, 10(2):8--39, January 1993. BIBLIOGRAPHY 135

....Extended Kalman Filtering (NDEKF) BPTT(h) uses h copies of the network with the same weights but different node activations corresponding to different time steps. We typically used h=10. The Recurrent Neural Networks (RNN) we use (Figure 2) are Discrete Time Recurrent Multi layer Perceptrons [5]. In BPTT(h) multiple copies of the same network weights are made. This can be pictured as one larger network, with each time step representing a hidden layer passing its output to the next layer. While the copies weights are usually the same, the node activations are different and time ....

Hush, D. R., and Horne B. "Progress in Supervised Neural Networks - What's New since Lippman?," IEEE Signal Processing Mag., Jan. 1993. pp. 8-34.

....when the Euclidean norm of the gradient vector reaches a sufficiently small gradient threshold. Another criterion could be, the absolute rate of change in the average squared error per epoch to be sufficiently small. Another very important characteristic of neural networks is the generalization [8, 9]. A network is said to generalize well when the input output relationship computed by the network is correct (or nearly so) for input output patterns (test data) never used in training the network. Of course it is assumed that the test data have the same population characteristics as the training ....

Hush, D.R., and B.G. Horne, 1993. "Progress in supervised neural networks: What's new since Lippmann?" IEEE Signal Processing Magazine 10, 8-39

....Sixteen different journal types consisting of 66 issues were selected for the experiment for a total of 2176 binary images. These images are 8.5 x 11 inches and scanned at 300 dpi resolution. 5. 2 Cross Validation Method For purposes of generalization, the cross validation (CV) technique [12] is used by randomly dividing the training data set into five data groups of which four data groups create a CV train set and one remaining data group is considered as a CV test set. As a result, there are five pairs of a CV train set and a CV test set that are used to train and test the ....

D. R. Hush and B. G. Horne, Progress in Supervised Neural Networks - What's New Since Lippmann? IEEE Signal Processing Magazine: pp. 8-39, 1993.

....different journal types consisting of 107 issues were selected for the experiment for a total of 2948 binary images. These images are 8.5 x 11 inches in size, and scanned at 300 dpi resolution. 4. 2 Cross Validation Method For purposes of generalization, the cross validation (CV) technique [12] is used by randomly dividing the training data set into five data groups of which four data groups create a CV train set and the one remaining data group is considered as a CV test set. As a result, there are five pairs of a CV train set and a CV test set that are used to train and test the ....

D. R. Hush and B. G. Horne, Progress in Supervised Neural Networks - What's New Since Lippmann? IEEE Signal Processing Magazine: pp. 8-39, 1993.

....classification benchmarks using comparable size of weight parameters. 1 Introduction Feedforward multilayer perceptrons (MLPs) 22] have been the most popular neural networks for a wide variety of classification (pattern recognition) and regression (functional approximation) applications [10]. Two major issues remain unsolved: one is the proper selection of the network topology (size and depth) and the other is the efficiency (time complexity) of learning algorithm. Proper and systematic selection of network topologies can be achieved in two different ways, i.e. the network pruning ....

D. R. Hush and B. G. Horne. Progress in supervised neural networks: what's new since Lippmann? IEEE Signal Processing Magazine, pp. 8-39, January 1993.

....stimulus (standard) That is, they produce a response significantly different from zero only when the input standard is within a small region located in the input space. For this reason this category of networks is sometimes referred to in literature as a network of localised receptive fields [22]. The input 4 is done from the source nodes (sensorial units) Each activation function requires a centre and a numeric parameter. A function which can be used as activation is the Gauss function, while this network can be used to make decisions of maximum hood, determining which of the various ....

HUSH, D. R. , HORNE, B. G. Progress in Supervised Neural Networks: What's New Since Lippmann. IEEE Signal Processing Magazine, p.8-39, January, 1993.

....1 2 in this paper. If such statistics are not available a priori, one can choose a reasonable value for OE with a trade off between transmission efficiency and protocol processing efficiency. 3. 2 TDNN based Prediction As an alternative approach we introduce a TDNN based prediction scheme [18, 19]. ANNs have adaptation capability that can accommodate nonstationarity. ANNs have generalization capability which makes them flexible and robust when faced with new and or noisy data patterns. Once the training is completed, an ANN can be computationally inexpensive even if it continues to adapt ....

D. Hush and B. Horone, "Progress in Supervised Neural Networks: What's New since Lippmann?", IEEE Signal Processing Magazine, pp. 8-38, Jan. 1993.

....a TDNN based prediction scheme. TDNN is a class of Sequence Processing Neural Networks (SPN) where the temporal input sequences are associated through weighted connections and then a classification (e.g. recognition, decision) or a regression (e.g. transformation, prediction) is performed [16, 17]. ANNs have adaptation capability that can accommodate nonstationarity. ANNs have generalization capability which makes them flexible and robust when faced with new and or noisy data patterns. Once the training is completed, an ANN can be computationally inexpensive even if it continues to adapt ....

D. Hush and B. Horone, "Progress in Supervised Neural Networks: What's New since Lippmann ?", IEEE Signal Processing Magazine, pp.8-38, Jan. 1993.

....operation can significantly reduce the bandwidth adaptation frequency at the expense of increased transmission bandwidth. As an example, we set OE=E[x(t) V ar[x(t) 1 2 in this paper. 3. 2 TDNN based Prediction As an alternative approach we introduce a TDNN based prediction scheme [18, 19]. ANNs have adaptation capability that can accommodate nonstationarity. ANNs have generalization capability which makes them flexible and robust when faced with new and or noisy data patterns. Once the training is completed, an ANN can be computationally inexpensive even if it continues to adapt ....

D. Hush and B. Horone, "Progress in Supervised Neural Networks: What's New since Lippmann?", IEEE Signal Processing Magazine, pp. 8-38, Jan. 1993.

....fewer errors in downgrading the quality than in upgrading the quality. However, in this first stage of the processing it is safer to downgrade the quality, and to run the additional processing stages on a view if there is the possibility that it contains a defect. As suggested by Hush and Horne [2], better generalisation is achieved if the number of training samples is at least ten times larger then the number of weights in a multilayer back propagation network. This neural network, and the neural networks at later stages of processing, is trained from a database that contains over 2000 ....

D.R. Hysh and B.G. Horne. Progress in supervised neural networks-what's new since lippmann? IEEE Signal Processing, pages 8--39, 1993.

....p(C; X) If this is known it is easy to marginalize to get any conditional probability. It is necessary to model the joint probability to be able to handle missing data in a principled way [Ahmad Tresp, 1993] We adopt this approach. The first two can be modeled with feed forward neural networks [Hush Horne, 1993; Richard Lippmann, 1991] This will however not work for subsets of X since a neural network classifier needs all input values at the same time. Classification with unknown inputs has recently been addressed in the neural network context [Ahmad Tresp, 1993; Ghahramani Jordan, 1994] They ....

Hush, Don R. & Horne, Bill G. (1993). Progress in supervised neural networks --- What's new since Lippmann? IEEE Signal Processing Magazine, pp 8--39, January.

....networks are tested. In a DPCM scheme (see Figure 1) the present signal sample is predicted based on the past ones; the prediction error is coded by entropic techniques (Huffmann and Arithmetic coding) 4] The results of comparing the performance of predictors based on dynamic neural networks [5, 6] with that obtained with predictors based on classical linear adaptive schemes (ADPCM) and on static feedforward neural networks [3] are exposed. The compression ratio which can be achieved by coding the resulting prediction error is also reported. Predictor Predictor Q Q z 1 z 1 ....

....feedforward neural networks are a particular case of these dynamic networks. The presence of the feedback makes dynamic networks quite suitable for time series processing [7] For training the dynamic neural predictors the Real Time Recurrent Learning RTRL rule proposed by Williams and Zipser [5, 6, 8] is used. It belongs to the class of the supervised learning algorithms. The training examples are constituted by segments of the input signal of length T and by the corresponding desired outputs. This sequence of input output pairs is called RUN. In our case the RUN is made up of a segment of EEG ....

[Article contains additional citation context not shown here]

D.R. Hush and B.G. Horne, "Progress in supervised neural networks (what's new since Lippmann ?)," IEEE Signal Processing Magazine, January 1993, pp. 8-39.

....outputs ( 1 0] ND; 0 1] D) All activation functions were logistic sigmoids, and the outputs had a competitive layer used only after training. With targets that are 0s and 1s rather than bipolar, this network automatically estimates the posteriori probability distributions of each class [5]. Thus if the decoder smooth outputs add up to about one, we also obtain a degree of certainty in the classification. To keep sight of the generalization capabilities of the final network, the smooth mean squared error (MSE) per output on the crossvalidation set VAL was monitored, and is shown ....

D.R. Hush and B.G. Horne, "Progress in supervised neural networks: What's new since Lippmann?," IEEE Signal Processing Magazine, Jan. 1993.

No context found.

D. R. Hush and B. G. Horne, "Progress in Supervised Neural Networks: What's New Since Lippmann?," IEEE Signal Proc. Mag., pp 8-39, Jan. 1993.

No context found.

D. R. Hush and B. G. Horne, "Progress in Supervised Neural Networks, What's New Since Lippmann?", IEEE Signal Processing Magazine, January 1993, pp.8-39.

No context found.

Hush, D.R. and Horne, G.B. 1993. Progress in supervised neural networks: What's new since Lippmann? IEEE Signal Processing Magazine, pp. 8--38.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC