Results 1  10
of
985
Piecewise linear regularized solution paths,
 The Annals of Statistics,
, 2007
"... Abstract We consider the generic regularized optimization problemβ(λ) = arg min β L(y, Xβ) + λJ(β). Recently, ..."
Abstract

Cited by 140 (9 self)
 Add to MetaCart
(Show Context)
Abstract We consider the generic regularized optimization problemβ(λ) = arg min β L(y, Xβ) + λJ(β). Recently,
A scalable modular convex solver for regularized risk minimization
 In KDD. ACM
, 2007
"... A wide variety of machine learning problems can be described as minimizing a regularized risk functional, with different algorithms using different notions of risk and different regularizers. Examples include linear Support Vector Machines (SVMs), Logistic Regression, Conditional Random Fields (CRFs ..."
Abstract

Cited by 78 (16 self)
 Add to MetaCart
(Show Context)
A wide variety of machine learning problems can be described as minimizing a regularized risk functional, with different algorithms using different notions of risk and different regularizers. Examples include linear Support Vector Machines (SVMs), Logistic Regression, Conditional Random Fields (CRFs), and Lasso amongst others. This paper describes the theory and implementation of a highly scalable and modular convex solver which solves all these estimation problems. It can be parallelized on a cluster of workstations, allows for datalocality, and can deal with regularizers such as ℓ1 and ℓ2 penalties. At present, our solver implements 20 different estimation problems, can be easily extended, scales to millions of observations, and is up to 10 times faster than specialized solvers for many applications. The open source code is freely available as part of the ELEFANT toolbox.
Bundle Methods for Regularized Risk Minimization
"... A wide variety of machine learning problems can be described as minimizing a regularized risk functional, with different algorithms using different notions of risk and different regularizers. Examples include linear Support Vector Machines (SVMs), Gaussian Processes, Logistic Regression, Conditional ..."
Abstract

Cited by 76 (4 self)
 Add to MetaCart
A wide variety of machine learning problems can be described as minimizing a regularized risk functional, with different algorithms using different notions of risk and different regularizers. Examples include linear Support Vector Machines (SVMs), Gaussian Processes, Logistic Regression, Conditional Random Fields (CRFs), and Lasso amongst others. This paper describes the theory and implementation of a scalable and modular convex solver which solves all these estimation problems. It can be parallelized on a cluster of workstations, allows for datalocality, and can deal with regularizers such as L1 and L2 penalties. In addition to the unified framework we present tight convergence bounds, which show that our algorithm converges in O(1/ɛ) steps to ɛ precision for general convex problems and in O(log(1/ɛ)) steps for continuously differentiable problems. We demonstrate the performance of our general purpose solver on a variety of publicly available datasets.
Unconditional quantile regressions
 Technical Working Paper 339, National Bureau of Economic Research
, 2007
"... Preliminary Paper, Comments Welcome We propose a new regression method for modelling unconditional quantiles of an outcome variable as a function of explanatory variables. The method consists of running a regression of the (recentered) influence function of the unconditional quantile of the dependen ..."
Abstract

Cited by 68 (0 self)
 Add to MetaCart
(Show Context)
Preliminary Paper, Comments Welcome We propose a new regression method for modelling unconditional quantiles of an outcome variable as a function of explanatory variables. The method consists of running a regression of the (recentered) influence function of the unconditional quantile of the dependent variable on the explanatory variables. The influence function is a widely used tool in robust estimation that can easily be computed for each quantile of interest. The estimated regression model can be used to infer the impact of various explanatory variable on a given unconditional quantile, just like the regression coefficients are used in the case of the mean. Our approach can thus be used, for example, to decompose quantiles as a function of the different explanatory variables (as in a standard OaxacaBlinder mean decomposition), or to predict the effect of changes in policy or other variables on quantiles.
Innovation and Firm Growth in HighTech Sectors: A Quantile Regression Approach
 RESEARCH POLICY
, 2008
"... We relate innovation to sales growth for incumbent firms in hightech sectors. A firm, on average, experiences only modest growth and may grow for a number of reasons that may or may not be related to ‘innovativeness’. However, given that the returns to innovation are highly skewed and that growth r ..."
Abstract

Cited by 56 (8 self)
 Add to MetaCart
We relate innovation to sales growth for incumbent firms in hightech sectors. A firm, on average, experiences only modest growth and may grow for a number of reasons that may or may not be related to ‘innovativeness’. However, given that the returns to innovation are highly skewed and that growth rates distributions are heavytailed, it may be misleading to use regression techniques that focus on the ‘average effect for the average firm’. Using a quantile regression approach, we observe that innovation is of crucial importance for a handful of ‘superstar’ fastgrowth firms. We also discuss policy implications of our results.
Nonparametric quantile estimation
, 2006
"... In regression, the desired estimate of yx is not always given by a conditional mean, although this is most common. Sometimes one wants to obtain a good estimate that satisfies the property that a proportion, τ, of yx, will be below the estimate. For τ = 0.5 this is an estimate of the median. What ..."
Abstract

Cited by 55 (9 self)
 Add to MetaCart
(Show Context)
In regression, the desired estimate of yx is not always given by a conditional mean, although this is most common. Sometimes one wants to obtain a good estimate that satisfies the property that a proportion, τ, of yx, will be below the estimate. For τ = 0.5 this is an estimate of the median. What might be called median regression, is subsumed under the term quantile regression. We present a nonparametric version of a quantile estimator, which can be obtained by solving a simple quadratic programming problem and provide uniform convergence statements and bounds on the quantile property of our estimator. Experimental results show the feasibility of the approach and competitiveness of our method with existing ones. We discuss several types of extensions including an approach to solve the quantile crossing problems, as well as a method to incorporate prior qualitative knowledge such as monotonicity constraints. 1.
Quantile Regression under Misspecification, with an Application to the U.S
 Wage Structure. Econometrica
, 2006
"... Quantile regression (QR) fits a linear model for conditional quantiles, just as ordinary least squares (OLS) fits a linear model for conditional means. An attractive feature of OLS is that it gives the minimum mean square error linear approximation to the conditional expectation function even when t ..."
Abstract

Cited by 51 (5 self)
 Add to MetaCart
Quantile regression (QR) fits a linear model for conditional quantiles, just as ordinary least squares (OLS) fits a linear model for conditional means. An attractive feature of OLS is that it gives the minimum mean square error linear approximation to the conditional expectation function even when the linear model is misspecified. Empirical research using quantile regression with discrete covariates suggests that QR may have a similar property, but the exact nature of the linear approximation has remained elusive. In this paper, we show that QR minimizes a weighted meansquared error loss function for specification error. The weighting function is an average density of the dependent variable near the true conditional quantile. The weighted least squares interpretation of QR is used to derive an omitted variables bias formula and a partial quantile regression concept, similar to the relationship between partial regression and OLS. We also present asymptotic theory for the QR process under misspecification of the conditional quantile function. The approximation properties of QR are illustrated using wage data from the US census. These results point to major changes in inequality from 19902000.
Making and evaluating point forecasts
 Journal of the American Statistical Association
"... iv ..."
(Show Context)
Quantile Regression Forests
 JOURNAL OF MACHINE LEARNING RESEARCH
, 2006
"... Random Forests were introduced as a Machine Learning tool in Breiman (2001) and have since proven to be very popular and powerful for highdimensional regression and classification. For regression, Random Forests give an accurate approximation of the conditional mean of a response variable. It is sh ..."
Abstract

Cited by 47 (0 self)
 Add to MetaCart
Random Forests were introduced as a Machine Learning tool in Breiman (2001) and have since proven to be very popular and powerful for highdimensional regression and classification. For regression, Random Forests give an accurate approximation of the conditional mean of a response variable. It is shown here that Random Forests provide information about the full conditional distribution of the response variable, not only about the conditional mean. Conditional quantiles can be inferred with Quantile Regression Forests, a generalisation of Random Forests. Quantile Regression Forests give a nonparametric and accurate way of estimating conditional quantiles for highdimensional predictor variables. The algorithm is shown to be consistent. Numerical examples suggest that the algorithm is competitive in terms of predictive power.
Dualism and crosscountry growth regressions
 Journal of Economic Growth
, 2006
"... This paper considers how growth regressions should incorporate dualism and structural change. If there is a differential across sectors in the marginal product of labour, changes in the structure of employment can raise aggregate total factor productivity. This paper develops empirical growth models ..."
Abstract

Cited by 44 (5 self)
 Add to MetaCart
This paper considers how growth regressions should incorporate dualism and structural change. If there is a differential across sectors in the marginal product of labour, changes in the structure of employment can raise aggregate total factor productivity. This paper develops empirical growth models that allow for this effect in a more flexible way than previous work. Estimates of the models imply sizeable marginal product differentials, and reveal that structural change can explain a significant fraction of the international variation in TFP growth. Preliminary and incomplete. Do not quote nor circulate without authors ’ explicit permission.