Results 11  20
of
28
Blind Separation of QuasiStationary Sources: Exploiting Convex Geometry in Covariance Domain
"... Abstract—This paper revisits blind source separation of instantaneously mixed quasistationary sources (BSSQSS), motivated by the observation that in certain applications (e.g., speech) there exist time frames during which only one source is active, or locally dominant. Combined with nonnegativity ..."
Abstract

Cited by 3 (3 self)
 Add to MetaCart
(Show Context)
Abstract—This paper revisits blind source separation of instantaneously mixed quasistationary sources (BSSQSS), motivated by the observation that in certain applications (e.g., speech) there exist time frames during which only one source is active, or locally dominant. Combined with nonnegativity of source powers, this endows the problem with a nice convex geometry that enables elegant and efficient BSS solutions. Local dominance is tantamount to the socalled pure pixel/separability assumption in hyperspectral unmixing/nonnegative matrix factorization, respectively. Building on this link, a very simple algorithm called successive projection algorithm (SPA) is considered for estimating the mixing system in closed form. To complement SPA in the specific BSSQSS context, an algebraic preprocessing procedure is proposed to suppress shortterm source crosscorrelation interference. The proposed procedure is simple, effective, and supported by theoretical analysis. Solutions based on volume minimization (VolMin) are also considered. By theoretical analysis, it is shown that VolMin guarantees perfect mixing system identifiability under an assumption more relaxed than (exact) local dominance—which means wider applicability in practice. Exploiting the specific structure of BSSQSS, a fast VolMin algorithm is proposed for the overdetermined case. Careful simulations using real speech sources showcase the simplicity, efficiency, and accuracy of the proposed algorithms. Index Terms—Blind source separation, local dominance, purepixel, separability, volume minimization, identifiability, speech, audio. I.
A Vavasis, “Semidefinite programming based preconditioning for more robust nearseparable nonnegative matrix factorization,” arXiv preprint arXiv:1310.2273
, 2013
"... ar ..."
(Show Context)
Hierarchical Clustering of Hyperspectral Images Using RankTwo Nonnegative Matrix Factorization
 IEEE, Transactions on Geoscience and Remote Sensing
, 2015
"... In this paper, we design a hierarchical clustering algorithm for highresolution hyperspectral images. At the core of the algorithm, a new ranktwo nonnegative matrix factorizations (NMF) algorithm is used to split the clusters, which is motivated by convex geometry concepts. The method starts with ..."
Abstract

Cited by 3 (2 self)
 Add to MetaCart
(Show Context)
In this paper, we design a hierarchical clustering algorithm for highresolution hyperspectral images. At the core of the algorithm, a new ranktwo nonnegative matrix factorizations (NMF) algorithm is used to split the clusters, which is motivated by convex geometry concepts. The method starts with a single cluster containing all pixels, and, at each step, (i) selects a cluster in such a way that the error at the next step is minimized, and (ii) splits the selected cluster into two disjoint clusters using ranktwo NMF in such a way that the clusters are well balanced and stable. The proposed method can also be used as an endmember extraction algorithm in the presence of pure pixels. The effectiveness of this approach is illustrated on several synthetic and realworld hyperspectral images, and shown to outperform standard clustering techniques such as kmeans, spherical kmeans and standard NMF.
Ellipsoidal Rounding for Nonnegative Matrix Factorization Under Noisy Separability
, 2013
"... We present a numerical algorithm for nonnegative matrix factorization (NMF) problems under noisy separability. An NMF problem under separability can be stated as one of finding all vertices of the convex hull of data points. The research interest of this paper is to find the vectors as close to the ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
(Show Context)
We present a numerical algorithm for nonnegative matrix factorization (NMF) problems under noisy separability. An NMF problem under separability can be stated as one of finding all vertices of the convex hull of data points. The research interest of this paper is to find the vectors as close to the vertices as possible in a situation in which noise is added to the data points. Our algorithm is designed to capture the shape of the convex hull of data points by using its enclosing ellipsoid. We show that the algorithm has correctness and robustness properties from theoretical and practical perspectives; correctness here means that if the data points do not contain any noise, the algorithm can find the vertices of their convex hull; robustness means that if the data points contain noise, the algorithm can find the nearvertices. Finally, we apply the algorithm to document clustering, and report the experimental results.
Provable Algorithms for Machine Learning Problems
, 2013
"... Modern machine learning algorithms can extract useful information from text, images and videos. All these applications involve solving NPhard problems in average case using heuristics. What properties of the input allow it to be solved efficiently? Theoretically analyzing the heuristics is often v ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
(Show Context)
Modern machine learning algorithms can extract useful information from text, images and videos. All these applications involve solving NPhard problems in average case using heuristics. What properties of the input allow it to be solved efficiently? Theoretically analyzing the heuristics is often very challenging. Few results were known. This thesis takes a different approach: we identify natural properties of the input, then design new algorithms that provably works assuming the input has these properties. We are able to give new, provable and sometimes practical algorithms for learning tasks related to text corpus, images and social networks. The first part of the thesis presents new algorithms for learning thematic structure in documents. We show under a reasonable assumption, it is possible to provably learn many topic models, including the famous Latent Dirichlet Allocation. Our algorithm is the first provable algorithms for topic modeling. An implementation runs 50 times faster than latest MCMC implementation and produces comparable results. The second part of the thesis provides ideas for provably learning deep, sparse representations. We start with sparse linear representations, and give the first algorithm for dictionary learning problem with provable guarantees. Then we apply similar ideas to deep learning: under reasonable assumptions our algorithms can learn a deep network built by denoising autoencoders. The final part of the thesis develops a framework for learning latent variable models. We demonstrate how various latent variable models can be reduced to orthogonal tensor decomposition, and then be solved using tensor power method. We give a tight perturbation analysis for tensor power method, which reduces the number of samples required to learn many latent variable models. In theory, the assumptions in this thesis help us understand why intractable problems in machine learning can often be solved; in practice, the results suggest inherently new approaches for machine learning. We hope the assumptions and algorithms inspire new research problems and learning algorithms. iii
Successive Nonnegative Projection Algorithm for Robust Nonnegative Blind Source Separation
"... ar ..."
(Show Context)
Scalable Methods for Nonnegative Matrix Factorizations of Nearseparable Tallandskinny Matrices
"... Numerous algorithms are used for nonnegative matrix factorization under the assumption that the matrix is nearly separable. In this paper, we show how to make these algorithms scalable for data matrices that have many more rows than columns, socalled “tallandskinny matrices. ” One key component ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
(Show Context)
Numerous algorithms are used for nonnegative matrix factorization under the assumption that the matrix is nearly separable. In this paper, we show how to make these algorithms scalable for data matrices that have many more rows than columns, socalled “tallandskinny matrices. ” One key component to these improved methods is an orthogonal matrix transformation that preserves the separability of the NMF problem. Our final methods need to read the data matrix only once and are suitable for streaming, multicore, and MapReduce architectures. We demonstrate the efficacy of these algorithms on terabytesized matrices from scientific computing and bioinformatics. 1 Nonnegative matrix factorizations at scale A nonnegative matrix factorization (NMF) for an m × n matrix X with realvalued, nonnegative entries is X = WH (1) where W is m × r, H is r × n, r < min(m, n), and both factors have nonnegative entries. While
Scalable Methods for Nonnegative Matrix Factorizations of Nearseparable Tallandskinny Matrices
"... • NMF Problem: X ∈ Rm×n+ is a matrix with nonnegative entries, and we want to compute a nonnegative matrix factorization (NMF) X = WH, where W ∈ Rm×r+ and H ∈ Rr×n+. When r < m, this problem is NPhard. • A separable matrix is one that admits a nonnegative factorization where ..."
Abstract
 Add to MetaCart
(Show Context)
• NMF Problem: X ∈ Rm×n+ is a matrix with nonnegative entries, and we want to compute a nonnegative matrix factorization (NMF) X = WH, where W ∈ Rm×r+ and H ∈ Rr×n+. When r < m, this problem is NPhard. • A separable matrix is one that admits a nonnegative factorization where
Elsevier online version:
"... of industrial source identification”, Applied Numerical Mathematics, vol. 85, ..."
Abstract
 Add to MetaCart
(Show Context)
of industrial source identification”, Applied Numerical Mathematics, vol. 85,