Results 1  10
of
926
Piecewise linear regularized solution paths
 Ann. Statist
, 2007
"... We consider the generic regularized optimization problem ˆ β(λ) = arg minβ L(y, Xβ) + λJ(β). Recently, Efron et al. (2004) have shown that for the Lasso – that is, if L is squared error loss and J(β) = ‖β‖1 is the l1 norm of β – the optimal coefficient path is piecewise linear, i.e., ∂ ˆ β(λ)/∂λ i ..."
Abstract

Cited by 138 (9 self)
 Add to MetaCart
(Show Context)
We consider the generic regularized optimization problem ˆ β(λ) = arg minβ L(y, Xβ) + λJ(β). Recently, Efron et al. (2004) have shown that for the Lasso – that is, if L is squared error loss and J(β) = ‖β‖1 is the l1 norm of β – the optimal coefficient path is piecewise linear, i.e., ∂ ˆ β(λ)/∂λ is piecewise constant. We derive a general characterization of the properties of (loss L, penalty J) pairs which give piecewise linear coefficient paths. Such pairs allow for efficient generation of the full regularized coefficient paths. We investigate the nature of efficient path following algorithms which arise. We use our results to suggest robust versions of the Lasso for regression and classification, and to develop new, efficient algorithms for existing problems in the literature, including Mammen & van de Geer’s Locally Adaptive Regression Splines. 1
2012), “The Short and Longterm Career Effects of Graduating in a Recession
 American Economic Journal: Applied Economics
"... The standard neoclassical model of wage setting predicts shortterm effects of temporary labor market shocks on careers and low costs of recessions for both more and less advantaged workers. In contrast, a vast range of alternative career models based on frictions in the labor market suggests that ..."
Abstract

Cited by 87 (5 self)
 Add to MetaCart
The standard neoclassical model of wage setting predicts shortterm effects of temporary labor market shocks on careers and low costs of recessions for both more and less advantaged workers. In contrast, a vast range of alternative career models based on frictions in the labor market suggests that labor market shocks can have persistent effects on the entire earnings profile. This paper analyzes the longterm effects of graduating in a recession on earnings, job mobility, and employer characteristics for a large sample of Canadian college graduates with different predicted earnings using matched universityemployeremployee data from 1982 to 1999, and uses its results to assess the importance of alternative career models. We find that young graduates entering the labor market in a recession suffer significant initial earnings losses that eventually fade, but after 8 to 10 years. We also document substantial heterogeneity in the costs of recessions and important effects on job mobility and employer characteristics, but small effects on time worked. These adjustment patterns are neither consistent with a neoclassical spot market nor a complete scarring effect, but could be explained by a combination of time intensive search for better employers and longterm wage contracting. All results are robust to an extensive sensitivity analysis including controls for correlated business cycle shocks after labor market entry, endogenous timing of graduation, permanent cohort
Bundle Methods for Regularized Risk Minimization
"... A wide variety of machine learning problems can be described as minimizing a regularized risk functional, with different algorithms using different notions of risk and different regularizers. Examples include linear Support Vector Machines (SVMs), Gaussian Processes, Logistic Regression, Conditional ..."
Abstract

Cited by 78 (4 self)
 Add to MetaCart
A wide variety of machine learning problems can be described as minimizing a regularized risk functional, with different algorithms using different notions of risk and different regularizers. Examples include linear Support Vector Machines (SVMs), Gaussian Processes, Logistic Regression, Conditional Random Fields (CRFs), and Lasso amongst others. This paper describes the theory and implementation of a scalable and modular convex solver which solves all these estimation problems. It can be parallelized on a cluster of workstations, allows for datalocality, and can deal with regularizers such as L1 and L2 penalties. In addition to the unified framework we present tight convergence bounds, which show that our algorithm converges in O(1/ɛ) steps to ɛ precision for general convex problems and in O(log(1/ɛ)) steps for continuously differentiable problems. We demonstrate the performance of our general purpose solver on a variety of publicly available datasets.
A scalable modular convex solver for regularized risk minimization
 In KDD. ACM
, 2007
"... A wide variety of machine learning problems can be described as minimizing a regularized risk functional, with different algorithms using different notions of risk and different regularizers. Examples include linear Support Vector Machines (SVMs), Logistic Regression, Conditional Random Fields (CRFs ..."
Abstract

Cited by 78 (15 self)
 Add to MetaCart
(Show Context)
A wide variety of machine learning problems can be described as minimizing a regularized risk functional, with different algorithms using different notions of risk and different regularizers. Examples include linear Support Vector Machines (SVMs), Logistic Regression, Conditional Random Fields (CRFs), and Lasso amongst others. This paper describes the theory and implementation of a highly scalable and modular convex solver which solves all these estimation problems. It can be parallelized on a cluster of workstations, allows for datalocality, and can deal with regularizers such as ℓ1 and ℓ2 penalties. At present, our solver implements 20 different estimation problems, can be easily extended, scales to millions of observations, and is up to 10 times faster than specialized solvers for many applications. The open source code is freely available as part of the ELEFANT toolbox.
Unconditional quantile regressions
 Technical Working Paper 339, National Bureau of Economic Research
, 2007
"... Preliminary Paper, Comments Welcome We propose a new regression method for modelling unconditional quantiles of an outcome variable as a function of explanatory variables. The method consists of running a regression of the (recentered) influence function of the unconditional quantile of the dependen ..."
Abstract

Cited by 61 (0 self)
 Add to MetaCart
(Show Context)
Preliminary Paper, Comments Welcome We propose a new regression method for modelling unconditional quantiles of an outcome variable as a function of explanatory variables. The method consists of running a regression of the (recentered) influence function of the unconditional quantile of the dependent variable on the explanatory variables. The influence function is a widely used tool in robust estimation that can easily be computed for each quantile of interest. The estimated regression model can be used to infer the impact of various explanatory variable on a given unconditional quantile, just like the regression coefficients are used in the case of the mean. Our approach can thus be used, for example, to decompose quantiles as a function of the different explanatory variables (as in a standard OaxacaBlinder mean decomposition), or to predict the effect of changes in policy or other variables on quantiles.
Nonparametric quantile estimation
, 2006
"... In regression, the desired estimate of yx is not always given by a conditional mean, although this is most common. Sometimes one wants to obtain a good estimate that satisfies the property that a proportion, τ, of yx, will be below the estimate. For τ = 0.5 this is an estimate of the median. What ..."
Abstract

Cited by 52 (9 self)
 Add to MetaCart
(Show Context)
In regression, the desired estimate of yx is not always given by a conditional mean, although this is most common. Sometimes one wants to obtain a good estimate that satisfies the property that a proportion, τ, of yx, will be below the estimate. For τ = 0.5 this is an estimate of the median. What might be called median regression, is subsumed under the term quantile regression. We present a nonparametric version of a quantile estimator, which can be obtained by solving a simple quadratic programming problem and provide uniform convergence statements and bounds on the quantile property of our estimator. Experimental results show the feasibility of the approach and competitiveness of our method with existing ones. We discuss several types of extensions including an approach to solve the quantile crossing problems, as well as a method to incorporate prior qualitative knowledge such as monotonicity constraints. 1.
Innovation and Firm Growth in HighTech Sectors: A Quantile Regression Approach
 RESEARCH POLICY
, 2008
"... We relate innovation to sales growth for incumbent firms in hightech sectors. A firm, on average, experiences only modest growth and may grow for a number of reasons that may or may not be related to ‘innovativeness’. However, given that the returns to innovation are highly skewed and that growth r ..."
Abstract

Cited by 52 (8 self)
 Add to MetaCart
We relate innovation to sales growth for incumbent firms in hightech sectors. A firm, on average, experiences only modest growth and may grow for a number of reasons that may or may not be related to ‘innovativeness’. However, given that the returns to innovation are highly skewed and that growth rates distributions are heavytailed, it may be misleading to use regression techniques that focus on the ‘average effect for the average firm’. Using a quantile regression approach, we observe that innovation is of crucial importance for a handful of ‘superstar’ fastgrowth firms. We also discuss policy implications of our results.
Making and evaluating point forecasts
 Journal of the American Statistical Association
"... iv ..."
(Show Context)
Quantile Regression under Misspecification, with an Application to the U.S
 Wage Structure. Econometrica
, 2006
"... Quantile regression (QR) fits a linear model for conditional quantiles, just as ordinary least squares (OLS) fits a linear model for conditional means. An attractive feature of OLS is that it gives the minimum mean square error linear approximation to the conditional expectation function even when t ..."
Abstract

Cited by 47 (4 self)
 Add to MetaCart
Quantile regression (QR) fits a linear model for conditional quantiles, just as ordinary least squares (OLS) fits a linear model for conditional means. An attractive feature of OLS is that it gives the minimum mean square error linear approximation to the conditional expectation function even when the linear model is misspecified. Empirical research using quantile regression with discrete covariates suggests that QR may have a similar property, but the exact nature of the linear approximation has remained elusive. In this paper, we show that QR minimizes a weighted meansquared error loss function for specification error. The weighting function is an average density of the dependent variable near the true conditional quantile. The weighted least squares interpretation of QR is used to derive an omitted variables bias formula and a partial quantile regression concept, similar to the relationship between partial regression and OLS. We also present asymptotic theory for the QR process under misspecification of the conditional quantile function. The approximation properties of QR are illustrated using wage data from the US census. These results point to major changes in inequality from 19902000.
Quantile Regression Forests
 JOURNAL OF MACHINE LEARNING RESEARCH
, 2006
"... Random Forests were introduced as a Machine Learning tool in Breiman (2001) and have since proven to be very popular and powerful for highdimensional regression and classification. For regression, Random Forests give an accurate approximation of the conditional mean of a response variable. It is sh ..."
Abstract

Cited by 46 (0 self)
 Add to MetaCart
Random Forests were introduced as a Machine Learning tool in Breiman (2001) and have since proven to be very popular and powerful for highdimensional regression and classification. For regression, Random Forests give an accurate approximation of the conditional mean of a response variable. It is shown here that Random Forests provide information about the full conditional distribution of the response variable, not only about the conditional mean. Conditional quantiles can be inferred with Quantile Regression Forests, a generalisation of Random Forests. Quantile Regression Forests give a nonparametric and accurate way of estimating conditional quantiles for highdimensional predictor variables. The algorithm is shown to be consistent. Numerical examples suggest that the algorithm is competitive in terms of predictive power.