Results 1  10
of
47
A sparse pls for variable selection when integrating omics data
 Statistical Applications in Genetics and Molecular Biology, 7(1):Article 35
, 2008
"... Recent biotechnology advances allow for the collection of multiple types of omics data sets, such as transcriptomic, proteomic or metabolomic data to be integrated. The problem of feature selection has been addressed several times in the context of classification, but has to be handled in a specific ..."
Abstract

Cited by 35 (0 self)
 Add to MetaCart
Recent biotechnology advances allow for the collection of multiple types of omics data sets, such as transcriptomic, proteomic or metabolomic data to be integrated. The problem of feature selection has been addressed several times in the context of classification, but has to be handled in a specific manner when integrating data. In this study, we focus on the integration of twoblock data sets that are measured on the same samples. Our goal is to combine integration and simultaneous variable selection on the two data sets in a onestep procedure using a PLS variant to facilitate the biologists interpretation. A novel computational methodology called “sparse PLS ” is introduced for a predictive purpose analysis to deal with these newly arisen problems. The sparsity of our approach is obtained by softthresholding penalization of the loading vectors during the SVD decomposition. Sparse PLS is shown to be effective and biologically meaningful. Comparisons with classical PLS are performed on simulated and real data sets and a thorough biological interpretation of the results obtained on one data set is provided. We show that sparse PLS provides a valuable variable selection tool for high dimensional data sets.
Local shrinkage rules, Lévy processes, and regularized regression
, 2010
"... We use Lévy processes to generate joint prior distributions, and therefore penalty functions, for a location parameter β = (β1,...,βp) as p grows large. This generalizes the class of localglobal shrinkage rules based on scale mixtures of normals, illuminates new connections among disparate methods, ..."
Abstract

Cited by 14 (4 self)
 Add to MetaCart
(Show Context)
We use Lévy processes to generate joint prior distributions, and therefore penalty functions, for a location parameter β = (β1,...,βp) as p grows large. This generalizes the class of localglobal shrinkage rules based on scale mixtures of normals, illuminates new connections among disparate methods, and leads to new results for computing posterior means and modes under a wide class of priors. We extend this framework to largescale regularized regression problems where p> n, and provide comparisons with other methodologies.
Revisiting useful approaches to datarich macroeconomic forecasting. Federal Reserve Bank of
, 2008
"... This paper presents preliminary findings and is being distributed to economists and other interested readers solely to stimulate discussion and elicit comments. The views expressed in this paper are those of the authors and are not necessarily reflective of views at the Federal Reserve Bank of New Y ..."
Abstract

Cited by 13 (3 self)
 Add to MetaCart
This paper presents preliminary findings and is being distributed to economists and other interested readers solely to stimulate discussion and elicit comments. The views expressed in this paper are those of the authors and are not necessarily reflective of views at the Federal Reserve Bank of New York or the Federal Reserve System. Any errors or omissions are the responsibility of the authors.
Correlated component regression: A prediction/classification methodology for possibly many features
 In Proceedings of the American Statistical Association
, 2010
"... Abstract A new ensemble dimension reduction regression technique, called Correlated Component Regression (CCR), is proposed that predicts the dependent variable based on K correlated components. For K = 1, CCR is equivalent to the corresponding Naïve Bayes solution, and for K = P, CCR is equivalent ..."
Abstract

Cited by 10 (3 self)
 Add to MetaCart
Abstract A new ensemble dimension reduction regression technique, called Correlated Component Regression (CCR), is proposed that predicts the dependent variable based on K correlated components. For K = 1, CCR is equivalent to the corresponding Naïve Bayes solution, and for K = P, CCR is equivalent to traditional regression with P predictors. An optional stepdown variable selection procedure provides a sparse solution, with each component defined as a linear combination of only P* < P predictors. For highdimensional data, simulation results suggest that good prediction is generally attainable for K = 3 or 4 regardless of the number of predictors, and estimation is fast. When predictors include one or more suppressor variables, common with gene expression data, simulations based on linear regression, logistic regression and discriminant analysis suggest that CCR predicts outside the sample better than comparable approaches based on stepwise regression, penalized regression and/or PLS regression. A major reason for the improvement is that the CCR/stepdown algorithm is much better than other sparse techniques in capturing important suppressor variables among the final predictors.
Sparse ReducedRank Regression for Simultaneous Dimension Reduction and Variable Selection in Multivariate Regression
, 2012
"... The reducedrank regression is an effective method to predict multiple response variables from the same set of predictor variables, because it can reduce the number of model parameters as well as take advantage of interrelations between the response variables and therefore improve predictive accurac ..."
Abstract

Cited by 9 (0 self)
 Add to MetaCart
(Show Context)
The reducedrank regression is an effective method to predict multiple response variables from the same set of predictor variables, because it can reduce the number of model parameters as well as take advantage of interrelations between the response variables and therefore improve predictive accuracy. We propose to add a new feature to the reducedrank regression that allows selection of relevant variables using sparsity inducing penalties. By treating each row of the matrix of the regression coefficients as a group, we propose a grouplasso type penalty and show that this penalty satisfies certain desirable invariance property. We develop two numerical algorithms to solve the penalized regression problem and establish the asymptotic consistency of the proposed method. In particular, the manifold structure of the reducedrank regression coefficient matrix is respected and carefully studied in our theoretical analysis. In a simulation study and real data analysis, the new method is compared with several existing variable selection methods for multivariate regression and exhibits competitive performance in prediction and variable selection.
Learning query and document similarities from clickthrough bipartite graph with metadata
 In Proceedings of the sixth ACM international conference on WSDM
, 2013
"... ABSTRACT We consider learning query and document similarities from a clickthrough bipartite graph with metadata on the nodes. The metadata contains multiple types of features of queries and documents. We aim to leverage both the clickthrough bipartite graph and the features to learn querydocument ..."
Abstract

Cited by 9 (3 self)
 Add to MetaCart
(Show Context)
ABSTRACT We consider learning query and document similarities from a clickthrough bipartite graph with metadata on the nodes. The metadata contains multiple types of features of queries and documents. We aim to leverage both the clickthrough bipartite graph and the features to learn querydocument, documentdocument, and queryquery similarities. The challenges include how to model and learn the similarity functions based on the graph data. We propose solving the problems in a principled way. Specifically, we use two different linear mappings to project the queries and documents in two different feature spaces into the same latent space, and take the dot product in the latent space as their similarity. Queryquery and documentdocument similarities can also be naturally defined as dot products in the latent space. We formalize the learning of similarity functions as learning of the mappings that maximize the similarities of the observed querydocument pairs on the enriched clickthrough bipartite graph. When queries and documents have multiple types of features, the similarity function is defined as a linear combination of multiple similarity functions, each based on one type of features. We further solve the learning problem by using a new technique called Multiview Partial Least Squares (MPLS). The advantages include the global optimum which can be obtained through Singular Value Decomposition (SVD) and the capability of finding high quality similar queries. We conducted large scale experiments on enterprise search data and web search data. The experimental results on relevance ranking and similar query finding demonstrate that the proposed method works significantly better than the baseline methods.
Envelope models for parsimonious and efficient multivariate liner regression
 Statist. Sinica
, 2010
"... Abstract: We propose a new parsimonious version of the classical multivariate normal linear model, yielding a maximum likelihood estimator (MLE) that is asymptotically less variable than the MLE based on the usual model. Our approach is based on the construction of a link between the mean function a ..."
Abstract

Cited by 8 (3 self)
 Add to MetaCart
Abstract: We propose a new parsimonious version of the classical multivariate normal linear model, yielding a maximum likelihood estimator (MLE) that is asymptotically less variable than the MLE based on the usual model. Our approach is based on the construction of a link between the mean function and the covariance matrix, using the minimal reducing subspace of the latter that accommodates the former. This leads to a multivariate regression model that we call the envelope model, where the number of parameters is maximally reduced. The MLE from the envelope model can be substantially less variable than the usual MLE, especially when the mean function varies in directions that are orthogonal to the directions of maximum variation for the covariance matrix. Key words and phrases: Discriminant analysis, functional data analysis, grassmann manifolds, invariant subspaces, principal components, reduced rank regression, reducing subspaces, sufficient dimension reduction. 1.
Regularized partial least squares with an application to nmr spectroscopy
 Statistical Analysis and Data Mining
"... Highdimensional data common in genomics, proteomics, and chemometrics often contains complicated correlation structures. Recently, partial least squares (PLS) and Sparse PLS methods have gained attention in these areas as dimension reduction techniques in the context of supervised data analysis. W ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
Highdimensional data common in genomics, proteomics, and chemometrics often contains complicated correlation structures. Recently, partial least squares (PLS) and Sparse PLS methods have gained attention in these areas as dimension reduction techniques in the context of supervised data analysis. We introduce a framework for Regularized PLS by solving a relaxation of the SIMPLS optimization problem with penalties on the PLS loadings vectors. Our approach enjoys many advantages including flexibility, general penalties, easy interpretation of results, and fast computation in highdimensional settings. We also outline extensions of our methods leading to novel methods for Nonnegative PLS and Generalized PLS, an adaption of PLS for structured data. We demonstrate the utility of our methods through simulations and a case study on proton Nuclear Magnetic Resonance (NMR) spectroscopy data.
A PRESS statistic for twoblock partial least squares regression
 In Proceedings of the 10th Annual Workshop on Computational Intelligence, 2010. REFERENCES 216
"... iv ..."
(Show Context)