Results 1  10
of
11
Solving Structured Sparsity Regularization with Proximal Methods
 Proc. of ECML
, 2010
"... Abstract. Proximal methods have recently been shown to provide effective optimization procedures to solve the variational problems defining the!1 regularization algorithms. The goal of the paper is twofold. First we discuss how proximal methods can be applied to solve a large class of machine lear ..."
Abstract

Cited by 20 (6 self)
 Add to MetaCart
(Show Context)
Abstract. Proximal methods have recently been shown to provide effective optimization procedures to solve the variational problems defining the!1 regularization algorithms. The goal of the paper is twofold. First we discuss how proximal methods can be applied to solve a large class of machine learning algorithms which can be seen as extensions of!1 regularization, namely structured sparsity regularization. For all these algorithms, it is possible to derive an optimization procedure which corresponds to an iterative projection algorithm. Second, we discuss the effect of a preconditioning of the optimization procedure achieved by adding a strictly convex functional to the objective function. Structured sparsity algorithms are usually based on minimizing a convex (not strictly convex) objective function and this might lead to undesired unstable behavior. We show that by perturbing the objective function by a small strictly convex term we often reduce substantially the number of required computations without affecting the prediction performance of the obtained solution. 1
ELASTICNET REGULARIZATION IN LEARNING THEORY
, 2008
"... Abstract. Within the framework of statistical learning theory we analyze in detail the socalled elasticnet regularization scheme proposed by Zou and Hastie [45] for the selection of groups of correlated variables. To investigate on the statistical properties of this scheme and in particular on its ..."
Abstract

Cited by 18 (7 self)
 Add to MetaCart
(Show Context)
Abstract. Within the framework of statistical learning theory we analyze in detail the socalled elasticnet regularization scheme proposed by Zou and Hastie [45] for the selection of groups of correlated variables. To investigate on the statistical properties of this scheme and in particular on its consistency properties, we set up a suitable mathematical framework. Our setting is randomdesign regression where we allow the response variable to be vectorvalued and we consider prediction functions which are linear combination of elements (features) in an infinitedimensional dictionary. Under the assumption that the regression function admits a sparse representation on the dictionary, we prove that there exists a particular “elasticnet representation ” of the regression function such that, if the number of data increases, the elasticnet estimator is consistent not only for prediction but also for variable/feature selection. Our results include finitesample bounds and an adaptive scheme to select the regularization parameter. Moreover, using convex analysis tools, we derive an iterative thresholding algorithm for computing the elasticnet solution which is different from the optimization procedure originally proposed in [45]. 1.
Iterative projection methods for structured sparsity regularization
, 2009
"... In this paper we propose a general framework to characterize and solve the optimization problems underlying a large class of sparsity based regularization algorithms. More precisely, we study the minimization of learning functionals that are sums of a differentiable data term and a convex non differ ..."
Abstract

Cited by 11 (4 self)
 Add to MetaCart
(Show Context)
In this paper we propose a general framework to characterize and solve the optimization problems underlying a large class of sparsity based regularization algorithms. More precisely, we study the minimization of learning functionals that are sums of a differentiable data term and a convex non differentiable penalty. These latter penalties have recently become popular in machine learning since they allow to enforce various kinds of sparsity properties in the solution. Leveraging on the theory of Fenchel duality and subdifferential calculus, we derive explicit optimality conditions for the regularized solution and propose a general iterative projection algorithm whose convergence to the optimal solution can be proved. The generality of the framework is illustrated, considering several examples of regularization schemes, including ℓ1 regularization (and several variants), multiple kernel learning and multitask learning. Finally, some features of the proposed framework are empirically studied. 1
A Regularization Approach to Nonlinear Variable Selection
"... In this paper we consider a regularization approach to variable selection when the regression function depends nonlinearly on a few input variables. The proposed method is based on a regularized least square estimator penalizing large values of the partial derivatives. An efficient iterative procedu ..."
Abstract

Cited by 6 (3 self)
 Add to MetaCart
In this paper we consider a regularization approach to variable selection when the regression function depends nonlinearly on a few input variables. The proposed method is based on a regularized least square estimator penalizing large values of the partial derivatives. An efficient iterative procedure is proposed to solve the underlying variational problem, and its convergence is proved. The empirical properties of the obtained estimator are tested both for prediction and variable selection. The algorithm compares favorably to more standard ridge regression and ℓ1 regularization schemes. 1
Vector valued regression for iron overload estimation
 in 19th International Conference on Pattern Recognition
, 2008
"... Abstract In this work we present and discuss in detail a novel vectorvalued regression technique: our approach allows for an allatonce estimation, as opposed to solve a number of scalarvalued regression tasks. Despite its general purpose nature, the method has been designed to solve a delicate ..."
Abstract

Cited by 5 (1 self)
 Add to MetaCart
(Show Context)
Abstract In this work we present and discuss in detail a novel vectorvalued regression technique: our approach allows for an allatonce estimation, as opposed to solve a number of scalarvalued regression tasks. Despite its general purpose nature, the method has been designed to solve a delicate medical issue: a reliable and noninvasive assessment of bodyiron overload. The Magnetic Iron Detector (MID) measures the magnetic track of a person, which depends on the anthropometric characteristics and the bodyiron burden. We aim to provide an estimate of this signal in absence of iron overload. We show how this question can be formulated as the estimation of a vectorvalued function which encompasses the prior knowledge on the shape of the magnetic track. This is accomplished by designing an appropriate vectorvalued feature map. We successfully applied the method on a dataset of 84 volunteers.
A Regularized Framework for Feature Selection in Face Detection and Authentication
"... Abstract This paper proposes a general framework for selecting features in the computer vision domain—i.e., learning descriptions from data—where the prior knowledge related to the application is confined in the early stages. The main building block is a regularization algorithm based on a penalty t ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
Abstract This paper proposes a general framework for selecting features in the computer vision domain—i.e., learning descriptions from data—where the prior knowledge related to the application is confined in the early stages. The main building block is a regularization algorithm based on a penalty term enforcing sparsity. The overall strategy we propose is also effective for training sets of limited size and reaches competitive performances with respect to the stateoftheart. To show the versatility of the proposed strategy we apply it to both face detection and authentication, implementing two modules of a monitoring system working in real time in our lab. Aside from the choices of the feature dictionary and the training data, which require prior knowledge on the problem, the proposed method is fully automatic. The very good results obtained in different applications speak for the generality and the robustness of the framework.
A Machine Learning Pipeline for Discriminant Pathways Identification
 In N.K. Kasabov, editor, Springer Handbook of Bio/Neuroinformatics, chapter 53
, 2013
"... 2 System and Methods 2 ..."
Methodology Report Identification of Multiple Hypoxia Signatures in Neuroblastoma Cell Lines by l 1 l 2 Regularization and Data Reduction
"... Hypoxia is a condition of low oxygen tension occurring in the tumor and negatively correlated with the progression of the disease. We studied the gene expression profiles of nine neuroblastoma cell lines grown under hypoxic conditions to define gene signatures that characterize hypoxic neuroblastom ..."
Abstract
 Add to MetaCart
(Show Context)
Hypoxia is a condition of low oxygen tension occurring in the tumor and negatively correlated with the progression of the disease. We studied the gene expression profiles of nine neuroblastoma cell lines grown under hypoxic conditions to define gene signatures that characterize hypoxic neuroblastoma. The l 1 l 2 regularization applied to the entire transcriptome identified a single signature of 11 probesets discriminating the hypoxic state. We demonstrate that new hypoxia signatures, with similar discriminatory power, can be generated by a prior knowledgebased filtering in which a much smaller number of probesets, characterizing hypoxiarelated biochemical pathways, are analyzed. l 1 l 2 regularization identified novel and robust hypoxia signatures within apoptosis, glycolysis, and oxidative phosphorylation Gene Ontology classes. We conclude that the filtering approach overcomes the noisy nature of the microarray data and allows generating robust signatures suitable for biomarker discovery and patients risk assessment in a fraction of computer time.
DATA
, 2010
"... ASD symptoms are heterogeneous and hard to discriminate in distinct subtypes. Although candidate loci and CNV regions have been recently identified by integration of large ASD cohorts [1] [2] [3], new bioinformatics methods are needed to cope with high individual variability. The L1L2 regularization ..."
Abstract
 Add to MetaCart
(Show Context)
ASD symptoms are heterogeneous and hard to discriminate in distinct subtypes. Although candidate loci and CNV regions have been recently identified by integration of large ASD cohorts [1] [2] [3], new bioinformatics methods are needed to cope with high individual variability. The L1L2 regularization [4] is a feature selection technique capable to generate a specific signature in biologically complex settings. It was proposed for predicting quantitative phenotypes traits from high dimensional genetic data [5]. Here we studied its first large scale application to whole genome data from the AGRE research program. Social Responsiveness Scale (SRS) score [6] is a good predictor of autism diagnosis in the AGRE [7] cohort (see section Data). Therefore we aim is to predict the SRS scores from genotype as a quantitative trait and use such predictions for classification of cases from controls. We set a bioinformatics experiment in which all unfiltered variant positions in the genome are used as potential markers and training is based on extreme value cases.
Identification of Multiple Hypoxia Signatures in Neuroblastoma
"... Copyright © 2010 Paolo Fardin et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Hypoxia is a condition of low oxygen tension occ ..."
Abstract
 Add to MetaCart
(Show Context)
Copyright © 2010 Paolo Fardin et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Hypoxia is a condition of low oxygen tension occurring in the tumor and negatively correlated with the progression of the disease. We studied the gene expression profiles of nine neuroblastoma cell lines grown under hypoxic conditions to define gene signatures that characterize hypoxic neuroblastoma. The l1l2 regularization applied to the entire transcriptome identified a single signature of 11 probesets discriminating the hypoxic state. We demonstrate that new hypoxia signatures, with similar discriminatory power, can be generated by a prior knowledgebased filtering in which a much smaller number of probesets, characterizing hypoxiarelated biochemical pathways, are analyzed. l1l2 regularization identified novel and robust hypoxia signatures within apoptosis, glycolysis, and oxidative phosphorylation Gene Ontology classes. We conclude that the filtering approach overcomes the noisy nature of the microarray data and allows generating robust signatures suitable for biomarker discovery and patients risk assessment in a fraction of computer time. 1. Background Neuroblastoma is the most common pediatric solid tumor, deriving from immature or precursor cells of the ganglionic lineage of the sympathetic nervous system [1, 2] endowed