Use of Bias Term in Projection Pursuit Learning Improves Approximation and Convergence Properties

Use of Bias Term in Projection Pursuit Learning Improves Approximation and Convergence Properties (1996)

Cached

Download Links

by Tin-yau Kwok , Dit-Yan Yeung

Venue:	IEEE Trans. Neural Networks
Citations:	6 - 1 self

BibTeX

@ARTICLE{Kwok96useof,
    author = {Tin-yau Kwok and Dit-Yan Yeung},
    title = {Use of Bias Term in Projection Pursuit Learning Improves Approximation and Convergence Properties},
    journal = {IEEE Trans. Neural Networks},
    year = {1996},
    volume = {7},
    pages = {1168--1183}
}

Bookmark

OpenURL

Abstract

In a regression problem, one is given a d- dimensional random vector X, the components of which are called predictor variables, and a random variable, Y , called response. A regression surface describes a general relationship between variables X and Y . One nonparametric regression technique that has been successfully applied to highdimensional data is projection pursuit regression (PPR). In this method, the regression surface is approximated by a sum of empirically determined univariate functions of linear combinations of the predictors. Projection pursuit learning (PPL) proposed by Hwang et al. formulates PPR using a two-layer feedforward neural network. One of the main differences between PPR and PPL is that the smoothers in PPR are nonparametric, whereas those in PPL are based on Hermite functions of some predefined highest order R. While the convergence property of PPR is already known, that for PPL has not been thoroughly studied. In this paper, we demonstrate that PPL networks...

Citations

1131	A new look at the statistical model identification - Akaike - 1974
590	The cascade-correlation learning architecture - Fahlman, Lebiere - 1990
537	Neural networks and the bias/variance dilemma - Geman, Bienenstock, et al. - 1992
375	Optimum brain damage - Cun, Denker, et al. - 1990
327	Projection pursuit regression - Friedman, Stuetzle - 1981
235	Faster-learning variations on back-propagation: An empirical study - Fahlman - 1988
206	Exploratory projection pursuit - Friedman - 1987
191	Adaptive Control Processes - Bellman - 1961
185	Functional Analysis - Riesz, Sz-Nagy - 1990
169	Pruning algorithms ± a survey - Reed - 1993
158	Approximation capabilities of multilayer feedforward networks, Neural Networks 4 - Hornik - 1991
152	Nonlinear Signal Processing using Neural Networks - Lapedes, Farber - 1987
148	Second order derivatives for network pruning: Optimal brain surgeon - Hassibi, Stork - 1993
125	Generalization by weight–elimination with applications to forecasting - Weigend, Rumelhart, et al. - 1991
115	Learning with localized receptive fields - Moody, Darken - 1988
112	A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training. The Annals of Statistics - Jones - 1992
111	A technique for trimming the fat from a network via relevance assessment - Mozer, Smolensky, et al. - 1988
107	Dynamic node creation in backpropagation networks - Ash - 1989
107	Approximation and estimation bounds for artificial neural networks - BARRON - 1994
84	Universal Approximation using radial basis functions network - Park, Sandberg - 1991
75	Back-propagation algorithm which varies the number of hidden units - HIROSE, YAMSHITA, et al. - 1991
61	Regression modeling in back-propagation and projection pursuit learning - Hwang, Lay, et al. - 1994
59	A simple procedure for pruning back-propagation trained neural networks - Karnin - 1990
52	A Comparison of Biases For Minimal Network Construction with Back-Propagation - Hanson, Pratt - 1989
45	A back-propagation algorithm with optimal use ofhidden units, in: D.S. Touretzky (Ed - Chauvin - 1990
43	A variable span smoother - Friedman - 1984
43	On a conjecture of huber concerning the convergence of projection pursuit regression - Jones - 1987
38	Some new results on neural network approximation - Hornik - 1993
23	Predicting the future: Advantages of semilocal units - Hartman, Keeler - 1991
17	Predicting the Mackey-Glass time series with cascade-correlation learning - Crowder - 1990
16	Orthogonal functions - Sansone - 1991
15	Automatic smoothing spline projection pursuit - Roosen, Hastie - 1994
13	Multidimensional additive spline approximation - Friedman, Grosse, et al. - 1983
11	Statistical Inference - Rohatgi - 1984
10	Generalised Additive Models. Monographs on Statistics and Applied Probability Number 43 - Hastie, Tibshirani - 1994
9	Networks with learned unit response functions - Moody, Yarvin - 1992
7	Constructive neural networks as estimators of bayesian discriminant functions - Yeung - 1993
7	What’s wrong with a cascaded correlation learning network: A projection pursuit learning perspective - Hwang, You, et al. - 1993
7	Approximation, dimension reduction and nonconvex optimization using linear superpositions of Gaussians - Saha, Wu - 1993
6	Nonmonotonic activation functions in multilayer perceptrons - Flake - 1993
6	Projection pursuit learning - Zhao, Atkeson - 1991
5	Projection pursuit learning networks for regression - Maechler, Martin, et al. - 1990
5	Logisitc response projection pursuit - Roosen, Hastie - 1993
4	A comparison of projection pursuit and neural network regression modeling - Hwang, Li, et al. - 1992
3	A Conceptual Approach to Generalization in Dynamic Neural Networks - SjOEgaard - 1991
3	The Learning Parsimony of Projection Pursuit and Back-propagation Networks - Hwang, Li, et al. - 1991
2	Projection pursuit (with discussion)", The - Huber - 1985
1	Pattern classificationusing projection pursuit - Flick, Jones, et al. - 1990