Results 1  10
of
10
Fast learning rates in statistical inference through aggregation
 SUBMITTED TO THE ANNALS OF STATISTICS
, 2008
"... We develop minimax optimal risk bounds for the general learning task consisting in predicting as well as the best function in a reference set G up to the smallest possible additive term, called the convergence rate. When the reference set is finite and when n denotes the size of the training data, w ..."
Abstract

Cited by 42 (8 self)
 Add to MetaCart
(Show Context)
We develop minimax optimal risk bounds for the general learning task consisting in predicting as well as the best function in a reference set G up to the smallest possible additive term, called the convergence rate. When the reference set is finite and when n denotes the size of the training data, we provide minimax convergence rates of the form C () log G  v with tight evaluation of the positive constant C and with n exact 0 < v ≤ 1, the latter value depending on the convexity of the loss function and on the level of noise in the output distribution. The risk upper bounds are based on a sequential randomized algorithm, which at each step concentrates on functions having both low risk and low variance with respect to the previous step prediction function. Our analysis puts forward the links between the probabilistic and worstcase viewpoints, and allows to obtain risk bounds unachievable with the standard statistical learning approach. One of the key idea of this work is to use probabilistic inequalities with respect to appropriate (Gibbs) distributions on the prediction function space instead of using them with respect to the distribution generating the data. The risk lower bounds are based on refinements of the Assouad lemma taking particularly into account the properties of the loss function. Our key example to illustrate the upper and lower bounds is to consider the Lqregression setting for which an exhaustive analysis of the convergence rates is given while q ranges in [1; +∞[.
Tight conditions for consistent variable selection in high dimensional nonparametric regression
"... ..."
6 DENSITY ESTIMATION WITH QUADRATIC LOSS: A CONFIDENCE INTERVALS METHOD
"... cc sd ..."
(Show Context)
JeanYves Audibert CERTIS Ecole des Ponts
"... A randomized online learning algorithm for ..."
(Show Context)
unknown title
, 2006
"... Model selection type aggregation with better variance control or Fast learning rates in statistical inference through aggregation ..."
Abstract
 Add to MetaCart
(Show Context)
Model selection type aggregation with better variance control or Fast learning rates in statistical inference through aggregation
Submitted arXiv:math.ST/1106.4293v2 TIGHT CONDITIONS FOR CONSISTENCY OF VARIABLE SELECTION IN THE CONTEXT OF HIGH DIMENSIONALITY
, 2012
"... We address the issue of variable selection in the regression model with very high ambient dimension, i.e., when the number of variables is very large. The main focus is on the situation where the number of relevant variables, called intrinsic dimension and denoted by d ∗ , is much smaller than the a ..."
Abstract
 Add to MetaCart
We address the issue of variable selection in the regression model with very high ambient dimension, i.e., when the number of variables is very large. The main focus is on the situation where the number of relevant variables, called intrinsic dimension and denoted by d ∗ , is much smaller than the ambient dimension d. Without assuming any parametric form of the underlying regression function, we get tight conditions making it possible to consistently estimate the set of relevant variables. These conditions relate the intrinsic dimension to the ambient dimension and to the sample size. The procedure that is provably consistent under these tight conditions is based on comparing quadratic functionals of the empirical Fourier coefficients with appropriately chosen threshold values. The asymptotic analysis reveals the presence of two quite different regimes. The first regime is when d ∗ is fixed. In this case the situation in nonparametric regression is the same as in linear regression, i.e., consistent variable selection is possible if and only if logd is small compared to the sample size n. The picture is different in the second regime, d ∗ →∞as n→∞, where we prove that consistent variable selection in nonparametric setup is possible only if d ∗ + loglogd is small compared to logn. We apply these results to derive minimax separation rates for the problem of variable selection. 1. Introduction. Realworld
Submitted to the Annals of Statistics FAST LEARNING RATES IN STATISTICAL INFERENCE THROUGH AGGREGATION By JeanYves Audibert1,2
"... We develop minimax optimal risk bounds for the general learning task consisting in predicting as well as the best function in a reference set G up to the smallest possible additive term, called the convergence rate. When the reference set is finite and when n denotes the size of the training data, w ..."
Abstract
 Add to MetaCart
(Show Context)
We develop minimax optimal risk bounds for the general learning task consisting in predicting as well as the best function in a reference set G up to the smallest possible additive term, called the convergence rate. When the reference set is finite and when n denotes the size of the training data, we provide minimax convergence rates of the form
<hal00139030v2>
, 2008
"... Fast learning rates in statistical inference through aggregation JeanYves Audibert ..."
Abstract
 Add to MetaCart
(Show Context)
Fast learning rates in statistical inference through aggregation JeanYves Audibert