Results 1  10
of
87
Structured Sparsity through Convex Optimization
, 2012
"... Sparse estimation methods are aimed at using or obtaining parsimonious representations of data or models. While naturally cast as a combinatorial optimization problem, variable or feature selection admits a convex relaxation through the regularization by the ℓ1norm. In this paper, we consider sit ..."
Abstract

Cited by 47 (6 self)
 Add to MetaCart
Sparse estimation methods are aimed at using or obtaining parsimonious representations of data or models. While naturally cast as a combinatorial optimization problem, variable or feature selection admits a convex relaxation through the regularization by the ℓ1norm. In this paper, we consider situations where we are not only interested in sparsity, but where some structural prior knowledge is available as well. We show that the ℓ1norm can then be extended to structured norms built on either disjoint or overlapping groups of variables, leading to a flexible framework that can deal with various structures. We present applications to unsupervised learning, for structured sparse principal component analysis and hierarchical dictionary learning, and to supervised learning in the context of nonlinear variable selection.
1 Invariant Scattering Convolution Networks
, 1203
"... Abstract—A wavelet scattering network computes a translation invariant image representation, which is stable to deformations and preserves high frequency information for classification. It cascades wavelet transform convolutions with nonlinear modulus and averaging operators. The first network laye ..."
Abstract

Cited by 44 (7 self)
 Add to MetaCart
(Show Context)
Abstract—A wavelet scattering network computes a translation invariant image representation, which is stable to deformations and preserves high frequency information for classification. It cascades wavelet transform convolutions with nonlinear modulus and averaging operators. The first network layer outputs SIFTtype descriptors whereas the next layers provide complementary invariant information which improves classification. The mathematical analysis of wavelet scattering networks explain important properties of deep convolution networks for classification. A scattering representation of stationary processes incorporates higher order moments and can thus discriminate textures having same Fourier power spectrum. State of the art classification results are obtained for handwritten digits and texture discrimination, with a Gaussian kernel SVM and a generative PCA classifier. 1
TopDown Visual Saliency via Joint CRF and Dictionary Learning
"... Topdown visual saliency facilities object localization by providing a discriminative representation of target objects and a probability map for reducing the search space. In this paper, we propose a novel topdown saliency model that jointly learns a Conditional Random Field (CRF) and a discriminat ..."
Abstract

Cited by 17 (2 self)
 Add to MetaCart
(Show Context)
Topdown visual saliency facilities object localization by providing a discriminative representation of target objects and a probability map for reducing the search space. In this paper, we propose a novel topdown saliency model that jointly learns a Conditional Random Field (CRF) and a discriminative dictionary. The proposed model is formulated based on a CRF with latent variables. By using sparse codes as latent variables, we train the dictionary modulated by CRF, and meanwhile a CRF with sparse coding. We propose a maxmargin approach to train our model via fast inference algorithms. We evaluate our model on the Graz02 and PASCAL VOC 2007 datasets. Experimental results show that our model performs favorably against the stateoftheart topdown saliency methods. We also observe that the dictionary update significantly improves the model performance. 1.
Subspace Interpolation via Dictionary Learning for Unsupervised Domain Adaptation
"... Domain adaptation addresses the problem where data instances of a source domain have different distributions from that of a target domain, which occurs frequently in many real life scenarios. This work focuses on unsupervised domain adaptation, where labeled data are only available in the source dom ..."
Abstract

Cited by 15 (1 self)
 Add to MetaCart
(Show Context)
Domain adaptation addresses the problem where data instances of a source domain have different distributions from that of a target domain, which occurs frequently in many real life scenarios. This work focuses on unsupervised domain adaptation, where labeled data are only available in the source domain. We propose to interpolate subspaces through dictionary learning to link the source and target domains. These subspaces are able to capture the intrinsic domain shift and form a shared feature representation for cross domain recognition. Further, we introduce a quantitative measure to characterize the shift between two domains, which enables us to select the optimal domain to adapt to the given multiple source domains. We present experiments on face recognition across pose, illumination and blur variations, cross dataset object recognition, and report improved performance over the state of the art. 1.
Learning Separable Filters ⋆
, 2012
"... Abstract. While learned image features can achieve great accuracy on different Computer Vision problems, their use in realworld situations is still very limited as their extraction is typically timeconsuming. We therefore propose a method to learn image features that can be extracted very efficien ..."
Abstract

Cited by 10 (2 self)
 Add to MetaCart
(Show Context)
Abstract. While learned image features can achieve great accuracy on different Computer Vision problems, their use in realworld situations is still very limited as their extraction is typically timeconsuming. We therefore propose a method to learn image features that can be extracted very efficiently using separable filters, by looking for low rank filters. We evaluate our approach on both the image categorization and the pixel classification tasks and show that we obtain similar accuracy as stateoftheart methods, at a fraction of the computational cost. 1
SparsityBased Generalization Bounds for Predictive Sparse Coding
"... The goal of predictive sparse coding is to learn a representation of examples as sparse linear combinations of elements from a dictionary, such that a learned hypothesis linear in the new representation performs well on a predictive task. Predictive sparse coding has demonstrated impressive performa ..."
Abstract

Cited by 9 (0 self)
 Add to MetaCart
(Show Context)
The goal of predictive sparse coding is to learn a representation of examples as sparse linear combinations of elements from a dictionary, such that a learned hypothesis linear in the new representation performs well on a predictive task. Predictive sparse coding has demonstrated impressive performance on a variety of supervised tasks, but its generalization properties have not been studied. We establish the first generalization error bounds for predictive sparse coding, in the overcomplete setting, where the number of features k exceeds the original dimensionality d. The learning bound decays as Õ(√dk/m) with respect to d, k, and the size m of the training sample. It depends intimately on stability properties of the learned sparse encoder, as measured on the training sample. Consequently, we also present a fundamental stability result for the LASSO, a result that characterizes the stability of the sparse codes with respect to dictionary perturbations. 1.
Sparse Variation Dictionary Learning for Face Recognition with A Single Training Sample Per Person
"... Face recognition (FR) with a single training sample per person (STSPP) is a very challenging problem due to the lack of information to predict the variations in the query sample. Sparse representation based classification has shown interesting results in robust FR; however, its performance will dete ..."
Abstract

Cited by 9 (1 self)
 Add to MetaCart
Face recognition (FR) with a single training sample per person (STSPP) is a very challenging problem due to the lack of information to predict the variations in the query sample. Sparse representation based classification has shown interesting results in robust FR; however, its performance will deteriorate much for FR with STSPP. To address this issue, in this paper we learn a sparse variation dictionary from a generic training set to improve the query sample representation by STSPP. Instead of learning from the generic training set independently w.r.t. the gallery set, the proposed sparse variation dictionary learning (SVDL) method is adaptive to the gallery set by jointly learning a projection to connect the generic training set with the gallery set. The learnt sparse variation dictionary can be easily integrated into the framework of sparse representation based classification so that various variations in face images, including illumination, expression, occlusion, pose, etc., can be better handled. Experiments on the largescale CMU MultiPIE, FRGC and LFW databases demonstrate the promising performance of SVDL on FR with STSPP. 1.
Efficient convolutional sparse coding
 in Proc. IEEE Int. Conf. Acoust. Speech Signal Process. (ICASSP
, 2014
"... When applying sparse representation techniques to images, the standard approach is to independently compute the representations for a set of overlapping image patches. This method performs very well in a variety of applications, but the independent sparse coding of each patch results in a represen ..."
Abstract

Cited by 8 (6 self)
 Add to MetaCart
(Show Context)
When applying sparse representation techniques to images, the standard approach is to independently compute the representations for a set of overlapping image patches. This method performs very well in a variety of applications, but the independent sparse coding of each patch results in a representation that is not optimal for the image as a whole. A recent development is convolutional sparse coding, in which a sparse representation for an entire image is computed by replacing the linear combination of a set of dictionary vectors by the sum of a set of convolutions with dictionary filters. A disadvantage of this formulation is its computational expense, but the development of efficient algorithms has received some attention in the literature, with the current leading method exploiting a Fourier domain approach. The present paper introduces a new way of solving the problem in the Fourier domain, leading to substantially reduced computational cost.
Deep unfolding: Modelbased inspiration of novel deep architectures. arXiv preprint arXiv:1409.2574
, 2014
"... Modelbased methods and deep neural networks have both been tremendously successful paradigms in machine learning. In modelbased methods, problem domain knowledge can be built into the constraints of the model, typically at the expense of difficulties during inference. In contrast, deterministic d ..."
Abstract

Cited by 8 (2 self)
 Add to MetaCart
Modelbased methods and deep neural networks have both been tremendously successful paradigms in machine learning. In modelbased methods, problem domain knowledge can be built into the constraints of the model, typically at the expense of difficulties during inference. In contrast, deterministic deep neural networks are constructed in such a way that inference is straightforward, but their architectures are generic and it is unclear how to incorporate knowledge. This work aims to obtain the advantages of both approaches. To do so, we start with a modelbased approach and an associated inference algorithm, and unfold the inference iterations as layers in a deep network. Rather than optimizing the original model, we untie the model parameters across layers, in order to create a more powerful network. The resulting architecture can be trained discriminatively to perform accurate inference within a fixed network size. We show how this framework allows us to interpret conventional networks as meanfield inference in Markov random fields, and to obtain new architectures by instead using belief propagation as the inference algorithm. We then show its application to a nonnegative matrix factorization model that incorporates the problemdomain knowledge that sound sources are additive. Deep unfolding of this model yields a new kind of nonnegative deep neural network, that can be trained using a multiplicative backpropagationstyle update algorithm. We present speech enhancement experiments showing that our approach is competitive with conventional neural networks despite using far fewer parameters. 1
On the Identifiability of Overcomplete Dictionaries via the Minimisation Principle Underlying KSVD
, 2013
"... This article gives theoretical insights into the performance of KSVD, a dictionary learning algorithm that has gained significant popularity in practical applications. The particular question studied here is when a dictionary Φ ∈ Rd×K can be recovered as local minimum of the minimisation criterion ..."
Abstract

Cited by 8 (1 self)
 Add to MetaCart
This article gives theoretical insights into the performance of KSVD, a dictionary learning algorithm that has gained significant popularity in practical applications. The particular question studied here is when a dictionary Φ ∈ Rd×K can be recovered as local minimum of the minimisation criterion underlying KSVD from a set of N training signals yn = Φxn. A theoretical analysis of the problem leads to two types of identifiability results assuming the training signals are generated from a tight frame with coefficients drawn from a random symmetric distribution. First asymptotic results showing, that in expectation the generating dictionary can be recovered exactly as a local minimum of the KSVD criterion if the coefficient distribution exhibits sufficient decay. This decay can be characterised by the coherence of the dictionary and the `1norm of the coefficients. Based on the asymptotic results it is further demonstrated that given a finite number of training samples N, such that N / logN = O(K3d), except with probability O(N−Kd) there is a local minimum of the KSVD criterion within distance O(KN−1/4) to the generating dictionary.