Results 1  10
of
28
Bayesian Treed Gaussian Process Models with an Application to Computer Modeling
 Journal of the American Statistical Association
, 2007
"... This paper explores nonparametric and semiparametric nonstationary modeling methodologies that couple stationary Gaussian processes and (limiting) linear models with treed partitioning. Partitioning is a simple but effective method for dealing with nonstationarity. Mixing between full Gaussian proce ..."
Abstract

Cited by 87 (19 self)
 Add to MetaCart
This paper explores nonparametric and semiparametric nonstationary modeling methodologies that couple stationary Gaussian processes and (limiting) linear models with treed partitioning. Partitioning is a simple but effective method for dealing with nonstationarity. Mixing between full Gaussian processes and simple linear models can yield a more parsimonious spatial model while significantly reducing computational effort. The methodological developments and statistical computing details which make this approach efficient are described in detail. Illustrations of our model are given for both synthetic and real datasets. Key words: recursive partitioning, nonstationary spatial model, nonparametric regression, Bayesian model averaging 1
Support vector machines for dyadic data
 Neural Computation
"... We describe a new technique for the analysis of dyadic data, where two sets of objects (“row ” and “column ” objects) are characterized by a matrix of numerical values which describe their mutual relationships. The new technique, called “Potential Support Vector Machine ” (PSVM), is a largemargin ..."
Abstract

Cited by 24 (7 self)
 Add to MetaCart
(Show Context)
We describe a new technique for the analysis of dyadic data, where two sets of objects (“row ” and “column ” objects) are characterized by a matrix of numerical values which describe their mutual relationships. The new technique, called “Potential Support Vector Machine ” (PSVM), is a largemargin method for the construction of classifiers and regression functions for the “column ” objects. Contrary to standard support vector machine approaches, the PSVM minimizes a scaleinvariant capacity measure and requires a new set of constraints. As a result, the PSVM method leads to a usually sparse expansion of the classification and regression functions in terms of the “row ” rather than the “column ” objects and can handle data and kernel matrices which are neither positive definite nor square. We then describe two complementary regularization schemes. The first scheme improves generalization performance for classification and regression tasks, the second scheme leads to the selection of a small, informative set of “row ” “support ” objects and can be applied to feature selection. Benchmarks for classification, regression, and feature selection tasks are performed with toy data as well as with several real world data sets. The results show, that the new method is at least competitive with but often performs better than the benchmarked standard methods for standard vectorial as well as for true dyadic data sets. In addition, a theoretical justification is provided for the new approach. 1
Support vector regression
 Neural Information Processing Letters and Reviews
, 2007
"... Abstract − Instead of minimizing the observed training error, Support Vector Regression (SVR) attempts to minimize the generalization error bound so as to achieve generalized performance. The idea of SVR is based on the computation of a linear regression function in a high dimensional feature space ..."
Abstract

Cited by 19 (0 self)
 Add to MetaCart
(Show Context)
Abstract − Instead of minimizing the observed training error, Support Vector Regression (SVR) attempts to minimize the generalization error bound so as to achieve generalized performance. The idea of SVR is based on the computation of a linear regression function in a high dimensional feature space where the input data are mapped via a nonlinear function. SVR has been applied in various fields – time series and financial (noisy and risky) prediction, approximation of complex engineering analyses, convex quadratic programming and choices of loss functions, etc. In this paper, an attempt has been made to review the existing theory, methods, recent developments and scopes of SVR.
Extreme learning machine: RBF network case
 in Proc. 8th Int. Conf. Control, Autom., Robot., Vis. (ICARCV 2004
"... Abstract – A new learning algorithm called extreme learning machine (ELM) has recently been proposed for singlehidden layer feedforward neural networks (SLFNs) to easily achieve good generalization performance at extremely fast learning speed. ELM randomly chooses the input weights and analytically ..."
Abstract

Cited by 18 (9 self)
 Add to MetaCart
(Show Context)
Abstract – A new learning algorithm called extreme learning machine (ELM) has recently been proposed for singlehidden layer feedforward neural networks (SLFNs) to easily achieve good generalization performance at extremely fast learning speed. ELM randomly chooses the input weights and analytically determines the output weightsofSLFNs.ThispapershowsthatELMcanbe extended to radial basis function (RBF) network case, which allows the centers and impact widths of RBF kernels to be randomly generated and the output weights to be simply analytically calculated instead of iteratively tuned. Interestingly, the experimental results show that the ELM algorithm for RBF networks can complete learning at extremely fast speed and produce generalization performance very close to that of SVM in many artifical and real benchmarking function approximation and classification problems. Since ELM does not require validation and humanintervened parameters for given network architectures, ELM can be easily used. Index terms Radial basis function network, feedforward neural networks, SLFN, real time learning, extreme learning machine, ELM. I.
Simple probabilistic predictions for support vector regression
, 2004
"... Support vector regression (SVR) has been popular in the past decade, but it provides only an estimated target value instead of predictive probability intervals. Many work have addressed this issue but sometimes the SVR formula must be modified. This paper presents a rather simple and direct approach ..."
Abstract

Cited by 13 (1 self)
 Add to MetaCart
(Show Context)
Support vector regression (SVR) has been popular in the past decade, but it provides only an estimated target value instead of predictive probability intervals. Many work have addressed this issue but sometimes the SVR formula must be modified. This paper presents a rather simple and direct approach to construct such intervals. We assume that the conditional distribution of the target value depends on its input only through the predicted value, and propose to model this distribution by simple functions. Experiments show that the proposed approach gives predictive intervals with competitive coverages with Bayesian SVR methods. I.
Support vector echostate machine for chaotic timeseries prediction
 IEEE Transactions on Neural Networks
"... Abstract: A novel chaotic time series prediction method based on support vector machines and echo state mechanisms is proposed. The basic idea is replacing “kernel trick ” with “reservoir trick ” in dealing with nonlinearity, that is, performing linear support vector regression in the high dimension ..."
Abstract

Cited by 11 (0 self)
 Add to MetaCart
(Show Context)
Abstract: A novel chaotic time series prediction method based on support vector machines and echo state mechanisms is proposed. The basic idea is replacing “kernel trick ” with “reservoir trick ” in dealing with nonlinearity, that is, performing linear support vector regression in the high dimension “reservoir ” state space, and the solution benefits from the advantages from structural risk minimization principle, and we call it SVESMs (Support Vector Echo State Machines). SVESMs belong to a special kind of recurrent neural networks with convex objective function, and its solution is global optimal and unique. SVESMs are especially efficient in dealing with real life nonlinear time series, and its generalization ability and robustness are obtained by regularization operator and robust loss function. The method is tested on the benchmark prediction problem of MackeyGlass time series and applied to some real life time series such as monthly sunspots time series and runoff time series of the Yellow River, and the prediction results are promising.
Gaussian processes and limiting linear models
 In Proceedings of the American Statistical Association, Section on Bayesian Statistical Science
, 2005
"... Gaussian processes (GPs) retain the linear model (LM) either as a special case, or in the limit. We show how this relationship can be exploited when the data are at least partially linear. However from the prospective of the Bayesian posterior, the GPs which encode the LM either have probability of ..."
Abstract

Cited by 6 (4 self)
 Add to MetaCart
(Show Context)
Gaussian processes (GPs) retain the linear model (LM) either as a special case, or in the limit. We show how this relationship can be exploited when the data are at least partially linear. However from the prospective of the Bayesian posterior, the GPs which encode the LM either have probability of nearly zero or are otherwise unattainable without the explicit construction of a prior with the limiting linear model (LLM) in mind. We develop such a prior, and show that its practical benefits extend well beyond the computational and conceptual simplicity of the LM. For example, linearity can be extracted on a perdimension basis, or can be combined with treed partition models to yield a highly efficient nonstationary model. Our approach is demonstrated on synthetic and real datasets of varying linearity and dimensionality. Comparisons are made to other approaches in the literature.
Bayesian Support Vector Machines for feature ranking and selection
 Feature Extraction, Foundations and Applications
, 2006
"... In this chapter, we develop and evaluate a feature selection algorithm for Bayesian support vector machines. The relevance level of features are represented by ARD (automatic relevance determination) parameters, which are optimized by maximizing the model evidence in the Bayesian framework. The feat ..."
Abstract

Cited by 4 (1 self)
 Add to MetaCart
In this chapter, we develop and evaluate a feature selection algorithm for Bayesian support vector machines. The relevance level of features are represented by ARD (automatic relevance determination) parameters, which are optimized by maximizing the model evidence in the Bayesian framework. The features are ranked in descending order using the optimal ARD values, and then forward selection is carried out to determine the minimal set of relevant features. In the numerical experiments, our approach using ARD for feature ranking can achieve a more compact feature set than standard ranking techniques, along with better generalization performance. 1
A naïve support vector regression benchmark for the NN3 forecasting competition
 In 2007 IEEE international
, 2007
"... AbstractSupport Vector Regression is one of the promising contenders in predicting the 111 time series of the NN3 Neural Forecasting Competition. As they offer substantial degrees of freedom in the modeling process, in selecting the kernel function and its parameters, cost and epsilon parameters, i ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
(Show Context)
AbstractSupport Vector Regression is one of the promising contenders in predicting the 111 time series of the NN3 Neural Forecasting Competition. As they offer substantial degrees of freedom in the modeling process, in selecting the kernel function and its parameters, cost and epsilon parameters, issues of model parameterization and model selection arise. In lack of an established methodology or comprehensive empirical evidence on their modeling, a number of heuristics and adhoc rules have emerged, that result in selecting different models, which show different performance. In order to determine a lower bound for Support Vector Regression accuracy in the NN3 competition, this paper seeks to compute benchmark results using a naive methodology with a fixed parameter gridsearch and exponentially increasing step sizes for radial basis function kernels, estimating 43,725 candidate models for each of the 111 time series. The naive approach attempts to mimic many of the common mistakes in model building, providing error as a lower bound to support vector regression accuracy. The insample results parameters are evaluated to estimate the impact of potential shortcomings in the grid search heuristic and the interaction effects of the parameters.
OnLine Learning of the Transition Model for Recursive Bayesian Estimation
"... Recursive Bayesian Estimation (RBE) is a widespread solution for visual tracking as well as for applications in other domains requiring hidden state estimation. Although theoretically sound and unquestionably powerful, from a practical point of view RBE suffers from the assumption of complete a prio ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
(Show Context)
Recursive Bayesian Estimation (RBE) is a widespread solution for visual tracking as well as for applications in other domains requiring hidden state estimation. Although theoretically sound and unquestionably powerful, from a practical point of view RBE suffers from the assumption of complete a priori knowledge of the transition model, that is typically unknown. The use of wrong a priori transition model may lead to large estimation errors or even to divergence. This work proposes to prevent these problems, in case of fully observable systems, learning the transition model online via Support Vector Regression. An application of this general framework is proposed in the context of linear/Gaussian systems and shown to be superior to a standard, non adaptive solution. 1.