Results 1  10
of
23
PACBayesian bounds for sparse regression estimation with exponential weights
 Electronic Journal of Statistics
"... Abstract. We consider the sparse regression model where the number of parameters p is larger than the sample size n. The difficulty when considering highdimensional problems is to propose estimators achieving a good compromise between statistical and computational performances. The BIC estimator ..."
Abstract

Cited by 31 (5 self)
 Add to MetaCart
(Show Context)
Abstract. We consider the sparse regression model where the number of parameters p is larger than the sample size n. The difficulty when considering highdimensional problems is to propose estimators achieving a good compromise between statistical and computational performances. The BIC estimator for instance performs well from the statistical point of view [11] but can only be computed for values of p of at most a few tens. The Lasso estimator is solution of a convex minimization problem, hence computable for large value of p. However stringent conditions on the design are required to establish fast rates of convergence for this estimator. Dalalyan and Tsybakov [19] propose a method achieving a good compromise between the statistical and computational aspects of the problem. Their estimator can be computed for reasonably large p and satisfies nice statistical properties under weak assumptions on the design. However, [19] proposes sparsity oracle inequalities in expectation for the empirical excess risk only. In this paper, we propose an aggregation procedure similar to that of [19] but with improved statistical performances. Our main theoretical result is a sparsity oracle inequality in probability for the true excess risk for a version of exponential weight estimator. We also propose a MCMC method to compute our estimator for reasonably large values of p.
Posterior contraction in sparse Bayesian factor models for massive covariance matrices
, 2012
"... ar ..."
(Show Context)
Variable Selection for Partially Linear Models With Measurement Errors
"... This article focuses on variable selection for partially linear models when the covariates are measured with additive errors. We propose two classes of variable selection procedures, penalized least squares and penalized quantile regression, using the nonconvex penalized principle. The first procedu ..."
Abstract

Cited by 10 (0 self)
 Add to MetaCart
This article focuses on variable selection for partially linear models when the covariates are measured with additive errors. We propose two classes of variable selection procedures, penalized least squares and penalized quantile regression, using the nonconvex penalized principle. The first procedure corrects the bias in the loss function caused by the measurement error by applying the socalled correctionforattenuation approach, whereas the second procedure corrects the bias by using orthogonal regression. The sampling properties for the two procedures are investigated. The rate of convergence and the asymptotic normality of the resulting estimates are established. We further demonstrate that, with proper choices of the penalty functions and the regularization parameter, the resulting estimates perform asymptotically as well as an oracle property. Choice of smoothing parameters is also discussed. Finite sample performance of the proposed variable selection procedures is assessed by Monte Carlo simulation studies. We further illustrate the proposed procedures by an application.
Bayesian factorizations of big sparse tensors
, 2013
"... It has become routine to collect data that are structured as multiway arrays (tensors). There is an enormous literature on low rank and sparse matrix factorizations, but limited consideration of extensions to the tensor case in statistics. The most common low rank tensor factorization relies on par ..."
Abstract

Cited by 5 (0 self)
 Add to MetaCart
It has become routine to collect data that are structured as multiway arrays (tensors). There is an enormous literature on low rank and sparse matrix factorizations, but limited consideration of extensions to the tensor case in statistics. The most common low rank tensor factorization relies on parallel factor analysis (PARAFAC), which expresses a rank k tensor as a sum of rank one tensors. When observations are only available for a tiny subset of the cells of a big tensor, the low rank assumption is not sufficient and PARAFAC has poor performance. We induce an additional layer of dimension reduction by allowing the effective rank to vary across dimensions of the table. For concreteness, we focus on a contingency table application. Taking a Bayesian approach, we place priors on terms in the factorization and develop an efficient Gibbs sampler for posterior computation. Theory is provided showing posterior concentration rates in highdimensional settings, and the methods are shown to have excellent performance in simulations and several real data applications.
Bayesian conditional tensor factorizations for highdimensional classification
, 2012
"... ar ..."
(Show Context)
Consistency of Bayesian linear model selection with a growing number of parameters
 J. Statist. Plann. Inference
, 2011
"... Linear models with a growing number of parameters have been widely used in modern statistics. One important problem about this kind of model is the variable selection issue. Bayesian approaches, which provide a stochastic search of informative variables, have gained popularity. In this paper, we wil ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
Linear models with a growing number of parameters have been widely used in modern statistics. One important problem about this kind of model is the variable selection issue. Bayesian approaches, which provide a stochastic search of informative variables, have gained popularity. In this paper, we will study the asymptotic properties related to Bayesian model selection when the model dimension p is growing with the sample size n. We consider p ≤ n and provide sufficient conditions under which: (1) with large probability, the posterior probability of the true model (from which samples are drawn) uniformly dominates the posterior probability of any incorrect models; and (2) with large probability, the posterior probability of the true model converges to one. Both (1) and (2) guarantee that the true model will be selected under a Bayesian framework. We also demonstrate several situations when (1) holds but (2) fails, which illustrates the difference between these two properties. Simulated examples are provided to illustrate the main results.
An Application of Bayesian Variable Selection to Spatial Concurrent Linear Models
"... Spatial concurrent linear models, in which the model coefficients are spatial processes varying at a local level, are flexible and useful tools for analyzing spatial data. One existing approach is to use wavelet tools to represent the spatial processes and use LASSO to perform estimation, but this a ..."
Abstract

Cited by 2 (2 self)
 Add to MetaCart
Spatial concurrent linear models, in which the model coefficients are spatial processes varying at a local level, are flexible and useful tools for analyzing spatial data. One existing approach is to use wavelet tools to represent the spatial processes and use LASSO to perform estimation, but this approach does not lead easily to making inferences. In this article, we propose a Bayesian variable selection approach based on wavelet tools to address this problem. We impose a mixture prior directly on each wavelet coefficient and introduce an option to control the priors such that high resolution coefficients are more likely to be zero. Computationally efficient MCMC procedures are provided to estimate posteriors. Examples based on simulated data demonstrate the estimation accuracy and advantages of the proposed method. We also illustrate the performance of the proposed method for real data obtained through remote sensing.
BAYESIAN MODEL CHOICE AND INFORMATION CRITERIA IN SPARSE GENERALIZED LINEAR MODELS
"... Abstract. We consider Bayesian model selection in generalized linear models that are highdimensional, with the number of covariates p being large relative to the sample size n, but sparse in that the number of active covariates is small compared to p. Treating the covariates as random and adopting ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
Abstract. We consider Bayesian model selection in generalized linear models that are highdimensional, with the number of covariates p being large relative to the sample size n, but sparse in that the number of active covariates is small compared to p. Treating the covariates as random and adopting an asymptotic scenario in which p increases with n, we show that Bayesian model selection using certain priors on the set of models is asymptotically equivalent to selecting a model using an extended Bayesian information criterion. Moreover, we prove that the smallest true model is selected by either of these methods with probability tending to one. Having addressed random covariates, we are also able to give a consistency result for pseudolikelihood approaches to highdimensional sparse graphical modeling. Experiments on real data demonstrate good performance of the extended Bayesian information criterion for regression and for graphical models. 1.
Bayesian Variable Selection: Theory and Applications By
"... I, I introduce some theoretical results related to BVS. Linear models with a growing number of parameters have been widely used in modern statistics. One important problem about this kind of model is the variable selection issue. Bayesian approaches, which provide a stochastic search of informative ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
I, I introduce some theoretical results related to BVS. Linear models with a growing number of parameters have been widely used in modern statistics. One important problem about this kind of model is the variable selection issue. Bayesian approaches, which provide a stochastic search of informative variables, have gained popularity. Here, we studied the asymptotic properties related to BVS when the model dimension is growing with the sample size. We provide sufficient conditions under which the posterior probability of the true model converges to one. This will guarantee that the true model will be selected under a Bayesian framework. In part II, I introduce the application of BVS in spatial concurrent linear models. Spatial concurrent linear models, in which the model coefficients are spatial processes varying at a local level, are flexible and useful tools for analyzing spatial data. One approach places stationary Gaussian process priors on the spatial processes, but in applications the data
Model Selection for Likelihoodfree Bayesian Methods Based on Moment Conditions: Theory and Numerical Examples. ArXiv eprints
, 2014
"... An important practice in statistics is to use robust likelihoodfree methods, such as the estimating equations, which only require assumptions on the moments instead of specifying the full probabilistic model. We propose a Bayesian flavored model selection approach for such likelihoodfree methods, ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
An important practice in statistics is to use robust likelihoodfree methods, such as the estimating equations, which only require assumptions on the moments instead of specifying the full probabilistic model. We propose a Bayesian flavored model selection approach for such likelihoodfree methods, based on (quasi)posterior probabilities from the Bayesian Generalized Method of Moments (BGMM). This novel concept allows us to incorporate two important advantages of a Bayesian approach: the expressiveness of posterior distributions and the convenient computational method of MCMC. Many different applications are possible, including modeling the correlated longitudinal data, the quantile regression, and the graphical models based on partial correlation. We demonstrate numerically how our method works in these applications. Under mild conditions, we show that theoretically the BGMM can achieve the posterior consistency for selecting the unknown true model, and that it possesses a Bayesian version of the oracle property, i.e. the posterior distribution for the parameter of interest is asymptotically normal and is as informative as if the true model were known. In addition, we show that the proposed quasiposterior is valid to be interpreted as an approximate conditional distribution given a data summary.