Results 1  10
of
41
Optimal aggregation of classifiers in statistical learning
 Ann. Statist
, 2004
"... Classification can be considered as nonparametric estimation of sets, where the risk is defined by means of a specific distance between sets associated with misclassification error. It is shown that the rates of convergence of classifiers depend on two parameters: the complexity of the class of cand ..."
Abstract

Cited by 225 (7 self)
 Add to MetaCart
Classification can be considered as nonparametric estimation of sets, where the risk is defined by means of a specific distance between sets associated with misclassification error. It is shown that the rates of convergence of classifiers depend on two parameters: the complexity of the class of candidate sets and the margin parameter. The dependence is explicitly given, indicating that optimal fast rates approaching O(n−1) can be attained, where n is the sample size, and that the proposed classifiers have the property of robustness to the margin. The main result of the paper concerns optimal aggregation of classifiers: we suggest a classifier that automatically adapts both to the complexity and to the margin, and attains the optimal fast rates, up to a logarithmic factor. 1. Introduction. Let (Xi,Yi)
Smooth Discrimination Analysis
 Ann. Statist
, 1998
"... Discriminant analysis for two data sets in IR d with probability densities f and g can be based on the estimation of the set G = fx : f(x) g(x)g. We consider applications where it is appropriate to assume that the region G has a smooth boundary. In particular, this assumption makes sense if di ..."
Abstract

Cited by 154 (3 self)
 Add to MetaCart
Discriminant analysis for two data sets in IR d with probability densities f and g can be based on the estimation of the set G = fx : f(x) g(x)g. We consider applications where it is appropriate to assume that the region G has a smooth boundary. In particular, this assumption makes sense if discriminant analysis is used as a data analytic tool. We discuss optimal rates for estimation of G. 1991 AMS: primary 62G05 , secondary 62G20 Keywords and phrases: discrimination analysis, minimax rates, Bayes risk Short title: Smooth discrimination analysis This research was supported by the Deutsche Forschungsgemeinschaft, Sonderforschungsbereich 373 "Quantifikation und Simulation okonomischer Prozesse", HumboldtUniversitat zu Berlin 1 Introduction Assume that one observes two independent samples X = (X 1 ; : : : ; X n ) and Y = (Y 1 ; : : : ; Ym ) of IR d valued i.i.d. observations with densities f or g, respectively. The densities f and g are unknown. An additional random variabl...
Rademacher Processes And Bounding The Risk Of Function Learning
 High Dimensional Probability II
, 1999
"... We construct data dependent upper bounds on the risk in function learning problems. The bounds are based on the local norms of the Rademacher process indexed by the underlying function class and they do not require prior knowledge about the distribution of training examples or any specific propertie ..."
Abstract

Cited by 58 (7 self)
 Add to MetaCart
We construct data dependent upper bounds on the risk in function learning problems. The bounds are based on the local norms of the Rademacher process indexed by the underlying function class and they do not require prior knowledge about the distribution of training examples or any specific properties of the function class. Using Talagrand's type concentration inequalities for empirical and Rademacher processes, we show that the bounds hold with high probability that decreases exponentially fast when the sample size grows. In typical situations that are frequently encountered in the theory of function learning, the bounds give nearly optimal rate of convergence of the risk to zero. 1. Local Rademacher norms and bounds on the risk: main results Let (S; A) be a measurable space and let F be a class of Ameasurable functions from S into [0; 1]: Denote P(S) the set of all probability measures on (S; A): Let f 0 2 F be an unknown target function. Given a probability measure P 2 P(S) (also unknown), let (X 1 ; : : : ; Xn ) be an i.i.d. sample in (S; A) with common distribution P (defined on a probability space(\Omega ; \Sigma; P)). In computer learning theory, the problem of estimating f 0 ; based on the labeled sample (X 1 ; Y 1 ); : : : ; (Xn ; Yn ); where Y j := f 0 (X j ); j = 1; : : : ; n; is referred to as function learning problem. The so called concept learning is a special case of function learning. In this case, F := fI C : C 2 Cg; where C ae A is called a class of concepts (see Vapnik (1998), Vidyasagar (1996), Devroye, Gyorfi and Lugosi (1996) for the account on statistical learning theory). The goal of function learning is to find an estimate
On Estimation of Monotone and Concave Frontier Functions
 Journal of the American Statistical Association
, 1998
"... When analyzing the productivity of firms, one may want to compare how the firms transform a set of inputs x (typically labor, energy or capital) into an output y (typically a quantity of goods produced). The economic efficiency of a firm is then defined in terms of its ability of operating close to ..."
Abstract

Cited by 51 (6 self)
 Add to MetaCart
When analyzing the productivity of firms, one may want to compare how the firms transform a set of inputs x (typically labor, energy or capital) into an output y (typically a quantity of goods produced). The economic efficiency of a firm is then defined in terms of its ability of operating close to or on the production frontier which is the boundary of the production set. The frontier function gives the maximal level of output attainable by a firm for a given combination of its inputs. The efficiency of a firm may then be estimated via the distance between the attained production level and the optimal level given by the frontier function. From a statistical point of view, the frontier function may be viewed as the upper boundary of the support of the population of firms density in the input and output space. It is often reasonable to assume that the production frontier is a concave monotone function. Then, a famous estimator, in the univariate input and output case, is the data envelop...
Image Denoising: Pointwise Adaptive Approach
 Annals of Statistics
, 1998
"... A new method of pointwise adaptation has been proposed and studied in Spokoiny (1998) in context of estimation of piecewise smooth univariate functions. The present paper extends that method to estimation of bivariate greyscale images composed of large homogeneous regions with smooth edges and obse ..."
Abstract

Cited by 31 (0 self)
 Add to MetaCart
A new method of pointwise adaptation has been proposed and studied in Spokoiny (1998) in context of estimation of piecewise smooth univariate functions. The present paper extends that method to estimation of bivariate greyscale images composed of large homogeneous regions with smooth edges and observed with noise on a gridded design. The proposed estimator # f(x) at a point x is simply the average of observations over a window # U(x) selected in a datadriven way. The theoretical properties of the procedure are studied for the case of piecewise constant images. We present a nonasymptotic bound for the accuracy of estimation at a specific grid point x as a function of the number of pixel n, of the distance from the point of estimation to the closest boundary and of smoothness properties and orientation of this boundary. It is also shown that the proposed method provides a near optimal rate of estimation near edges and inside homogeneous regions. We briefly discuss algorithmic aspects and the complexity of the procedure. The numerical examples demonstrate a reasonable performance of the method and they are in agreement with the theoretical issues. An example from satellite (SAR) imaging illustrates the applicability of the method. # The authors thank A.Juditski, O. Lepski, A.Tsybakov and Yu.Golubev for important remarks and discussion. polzehl, j. and spokoiny, v. 1 1
Minimax optimal level set estimation
 in Proc. SPIE, Wavelets XI, 31 July  4
, 2005
"... Abstract — This paper describes a new methodology and associated theoretical analysis for rapid and accurate extraction of level sets of a multivariate function from noisy data. The identification of the boundaries of such sets is an important theoretical problem with applications for digital elevat ..."
Abstract

Cited by 21 (5 self)
 Add to MetaCart
(Show Context)
Abstract — This paper describes a new methodology and associated theoretical analysis for rapid and accurate extraction of level sets of a multivariate function from noisy data. The identification of the boundaries of such sets is an important theoretical problem with applications for digital elevation maps, medical imaging, and pattern recognition. This problem is significantly different from classical segmentation because level set boundaries may not correspond to singularities or edges in the underlying function; as a result, segmentation methods which rely upon detecting boundaries would be potentially ineffective in this regime. This issue is addressed in this paper through a novel error metric sensitive to both the error in the location of the level set estimate and the deviation of the function from the critical level. Hoeffding’s inequality is used to derive a novel regularization
Simultaneous adaptation to the margin and to complexity in classification
, 2005
"... We consider the problem of adaptation to the margin and to complexity in binary classification. We suggest a learning method with a numerically easy aggregation step. Adaptivity both to the margin and complexity in classification, usually involves empirical risk minimization or Rademacher complexiti ..."
Abstract

Cited by 20 (6 self)
 Add to MetaCart
We consider the problem of adaptation to the margin and to complexity in binary classification. We suggest a learning method with a numerically easy aggregation step. Adaptivity both to the margin and complexity in classification, usually involves empirical risk minimization or Rademacher complexities which lead to numerical difficulties. On the other hand there exist classifiers that are easy to compute and that converge with fast rates but are not adaptive. Combining these classifiers by our aggregation procedure we get numerically realizable adaptive classifiers that converge with fast rates.
Spectral clustering based on local linear approximations
 ELECTRONIC JOURNAL OF STATISTICS
, 2011
"... Abstract: In the context of clustering, we assume a generative model where each cluster is the result of sampling points in the neighborhood of an embedded smooth surface; the sample may be contaminated with outliers, which are modeled as points sampled in space away from the clusters. We consider a ..."
Abstract

Cited by 18 (5 self)
 Add to MetaCart
(Show Context)
Abstract: In the context of clustering, we assume a generative model where each cluster is the result of sampling points in the neighborhood of an embedded smooth surface; the sample may be contaminated with outliers, which are modeled as points sampled in space away from the clusters. We consider a prototype for a higherorder spectral clustering method based on the residual from a local linear approximation. We obtain theoretical guarantees for this algorithm and show that, in terms of both separation and robustness to outliers, it outperforms the standard spectral clustering algorithm (based on pairwise distances) of Ng, Jordan and Weiss (NIPS ’01). The optimal choice for some of the tuning parameters depends on the dimension and thickness of the clusters. We provide estimators that come close enough for our theoretical purposes. We also discuss the cases of clusters of mixed dimensions and of clusters that are generated from smoother surfaces. In our experiments, this algorithm is shown to outperform pairwise spectral clustering on both simulated and real data.
Extreme values and Haar series estimates of point processes boundaries
 Scandinavian Journal of Statistics
, 2002
"... We present a new method for estimating the edge of a twodimensional bounded set, given a finite random set of points drawn from the interior. The estimator is based both on Haar series and extreme values of the point process. We give conditions for various kind of convergence and we obtain remarkab ..."
Abstract

Cited by 18 (7 self)
 Add to MetaCart
We present a new method for estimating the edge of a twodimensional bounded set, given a finite random set of points drawn from the interior. The estimator is based both on Haar series and extreme values of the point process. We give conditions for various kind of convergence and we obtain remarkably different possible limit distributions. We propose a method of reducing the negative bias, illustrated by a simulation. Keywords: Haar basis, Extreme values, Poisson process, Shape estimation. AMS Subject Classification: Primary 60G70; Secondary 62M30, 62G05, 62G20. 1 1 Introduction Many proposals are given in the literature for estimating a bounded set S of R 2 , given a finite random set N of points drawn from the interior. Such a diversity follows from crossing properties of the observed random set N (sample, point process, random field on a grid, ...), properties of the unknown bounded set S (convex sets, starshaped domains, pieces of support under a given curve, images, ...), ...
EXTREME VALUES AND KERNEL ESTIMATES OF POINT PROCESSES BOUNDARIES
, 2004
"... We present a method for estimating the edge of a twodimensional bounded set, given a finite random set of points drawn from the interior. The estimator is based both on a ParzenRosenblatt kernel and extreme values of point processes. We give conditions for various kinds of convergence and asympt ..."
Abstract

Cited by 17 (6 self)
 Add to MetaCart
We present a method for estimating the edge of a twodimensional bounded set, given a finite random set of points drawn from the interior. The estimator is based both on a ParzenRosenblatt kernel and extreme values of point processes. We give conditions for various kinds of convergence and asymptotic normality. We propose a method of reducing the negative bias and edge effects, illustrated by some simulations.