Results 1  10
of
49
FINITE SAMPLE APPROXIMATION RESULTS FOR PRINCIPAL COMPONENT ANALYSIS: A MATRIX PERTURBATION APPROACH
"... Principal Component Analysis (PCA) is a standard tool for dimensional reduction of a set of n observations (samples), each with p variables. In this paper, using a matrix perturbation approach, we study the nonasymptotic relation between the eigenvalues and eigenvectors of PCA computed on a finite ..."
Abstract

Cited by 66 (15 self)
 Add to MetaCart
(Show Context)
Principal Component Analysis (PCA) is a standard tool for dimensional reduction of a set of n observations (samples), each with p variables. In this paper, using a matrix perturbation approach, we study the nonasymptotic relation between the eigenvalues and eigenvectors of PCA computed on a finite sample of size n, to those of the limiting population PCA as n → ∞. As in machine learning, we present a finite sample theorem which holds with high probability for the closeness between the leading eigenvalue and eigenvector of sample PCA and population PCA under a spiked covariance model. In addition, we also consider the relation between finite sample PCA and the asymptotic results in the joint limit p, n → ∞, with p/n = c. We present a matrix perturbation view of the “phase transition phenomenon”, and a simple linearalgebra based derivation of the eigenvalue and eigenvector overlap in this asymptotic limit. Moreover, our analysis also applies for finite p, n where we show that although there is no sharp phase transition as in the infinite case, either as a function of noise level or as a function of sample size n, the eigenvector of sample PCA may exhibit a sharp ”loss of tracking”, suddenly losing its relation to the (true) eigenvector of the population PCA matrix. This occurs due to a crossover between the eigenvalue due to the signal and the largest eigenvalue due to noise, whose eigenvector points in a random direction.
Tighter Bounds for Random Projections of Manifolds
, 2008
"... The JohnsonLindenstrauss random projection lemma gives a simple way to reduce the dimensionality of a set of points while approximately preserving their pairwise distances. The most direct application of the lemma applies to a finite set of points, but recent work has extended the technique to affi ..."
Abstract

Cited by 46 (0 self)
 Add to MetaCart
The JohnsonLindenstrauss random projection lemma gives a simple way to reduce the dimensionality of a set of points while approximately preserving their pairwise distances. The most direct application of the lemma applies to a finite set of points, but recent work has extended the technique to affine subspaces, curves, and general smooth manifolds. Here the case of random projection of smooth manifolds is considered, and a previous analysis is sharpened, reducing the dependence on such properties as the manifold’s maximum curvature. 1
Sparse Recovery of Positive Signals with Minimal Expansion
, 902
"... We investigate the sparse recovery problem of reconstructing a highdimensional nonnegative sparse vector from lower dimensional linear measurements. While much work has focused on dense measurement matrices, sparse measurement schemes are crucial in applications, such as DNA microarrays and sensor ..."
Abstract

Cited by 22 (3 self)
 Add to MetaCart
We investigate the sparse recovery problem of reconstructing a highdimensional nonnegative sparse vector from lower dimensional linear measurements. While much work has focused on dense measurement matrices, sparse measurement schemes are crucial in applications, such as DNA microarrays and sensor networks, where dense measurements are not practically feasible. One possible construction uses the adjacency matrices of expander graphs, which often leads to recovery algorithms much more efficient than ℓ1 minimization. However, to date, constructions based on expanders have required very high expansion coefficients which can potentially make the construction of such graphs difficult and the size of the recoverable sets small. In this paper, we construct sparse measurement matrices for the recovery of nonnegative vectors, using perturbations of the adjacency matrix of an expander graph with much smaller expansion coefficient. We present a necessary and sufficient condition for ℓ1 optimization to successfully recover the unknown vector and obtain expressions for the recovery threshold. For certain classes of measurement matrices, this necessary and sufficient condition is further equivalent to the existence of a “unique ” vector in the constraint set, which opens the door to
A highresolution technique for multidimensional NMR spectroscopy
 IEEE Transactions on Biomedical Engineering
, 1998
"... Abstract — In this paper, a scheme for estimating frequencies and damping factors of multidimensional nuclear magnetic resonance (NMR) data is presented. multidimensional NMR data can be modeled as the sum of several multidimensional damped sinusoids. The estimated frequencies and damping factors of ..."
Abstract

Cited by 14 (0 self)
 Add to MetaCart
(Show Context)
Abstract — In this paper, a scheme for estimating frequencies and damping factors of multidimensional nuclear magnetic resonance (NMR) data is presented. multidimensional NMR data can be modeled as the sum of several multidimensional damped sinusoids. The estimated frequencies and damping factors of multidimensional NMR data play important roles in determining protein structures. In this paper we present a highresolution subspace method for estimating the parameters of NMR data. Unlike other methods, this algorithm makes full use of the rankdeficiency and Hankel properties of the prediction matrix composed of NMR data. Hence, it can estimate the signal parameters under low signaltonoise ratio (SNR) by using a few data points. The effectiveness of the new algorithm is confirmed by computer simulations and it is tested by experimental data. Index Terms—Damped sinusoids, high resolution, multidimensional NMR, parameter estimation. I.
Minimax Localization of Structural Information in Large Noisy Matrices
"... We consider the problem of identifying a sparse set of relevant columns and rows in a large data matrix with highly corrupted entries. This problem of identifying groups from a collection of bipartite variables such as proteins and drugs, biological species and gene sequences, malware and signatures ..."
Abstract

Cited by 13 (2 self)
 Add to MetaCart
We consider the problem of identifying a sparse set of relevant columns and rows in a large data matrix with highly corrupted entries. This problem of identifying groups from a collection of bipartite variables such as proteins and drugs, biological species and gene sequences, malware and signatures, etc is commonly referred to as biclustering or coclustering. Despite its great practical relevance, and although several adhoc methods are available for biclustering, theoretical analysis of the problem is largely nonexistent. The problem we consider is also closely related to structured multiple hypothesis testing, an area of statistics that has recently witnessed a flurry of activity. We make the following contributions 1. We prove lower bounds on the minimum signal strength needed for successful recovery of a bicluster as a function of the noise variance, size of the matrix and bicluster of interest. 2. We show that a combinatorial procedure based on the scan statistic achieves this optimal limit. 3. We characterize the SNR required by several computationally tractable procedures for biclustering including elementwise thresholding, column/row average thresholding and a convex relaxation approach to sparse singular vector decomposition. 1
Hamilton And Jacobi Meet Again: Quaternions And The Eigenvalue Problem
 SIAM J. Matrix Anal. Appl
, 1993
"... . The algebra isomorphism between M 4 (R) and H\Omega\Gamma , where H is the algebra of quaternions, has unexpected computational payoff: it helps construct an orthogonal similarity that 2 \Theta 2 blockdiagonalizes a 4 \Theta 4 symmetric matrix. Replacing plane rotations with these more powerful 4 ..."
Abstract

Cited by 12 (2 self)
 Add to MetaCart
. The algebra isomorphism between M 4 (R) and H\Omega\Gamma , where H is the algebra of quaternions, has unexpected computational payoff: it helps construct an orthogonal similarity that 2 \Theta 2 blockdiagonalizes a 4 \Theta 4 symmetric matrix. Replacing plane rotations with these more powerful 4 \Theta 4 rotations leads to a quaternionJacobi method in which the `weight' of 4 elements (in a 2 \Theta 2 block) is transferred all at once onto the diagonal. Quadratic convergence sets in sooner, and the new method requires at least one fewer sweep than planeJacobi methods. An analogue of the sorting angle for plane rotations is developed for these 4 \Theta 4 rotations. Key words. eigenvalues, symmetric matrix, Jacobi method, quaternion, tensor product AMS subject classifications. 65F15, 15A18, 15A21, 15A69 1. Introduction. One hundred and fifty years ago, on 16 October 1843, W. R. Hamilton carved the equations defining the algebra of quaternions on the stones of Brougham Bridge, Dub...
Learning functions of few arbitrary linear parameters in high dimensions
 CoRR
"... Let us assume that f is a continuous function defined on the unit ball of R d, of the form f(x) = g(Ax), where A is a k ×d matrix and g is a function of k variables for k ≪ d. We are given a budget m ∈ N of possible point evaluations f(xi), i = 1,...,m, of f, which we are allowed to query in order ..."
Abstract

Cited by 11 (1 self)
 Add to MetaCart
Let us assume that f is a continuous function defined on the unit ball of R d, of the form f(x) = g(Ax), where A is a k ×d matrix and g is a function of k variables for k ≪ d. We are given a budget m ∈ N of possible point evaluations f(xi), i = 1,...,m, of f, which we are allowed to query in order to construct a uniform approximating function. Under certain smoothness and variation assumptions on the function g, and an arbitrary choice of the matrix A, we present in this paper 1. a sampling choice of the points {xi} drawn at random for each function approximation; 2. algorithms (Algorithm 1 and Algorithm 2) for computing the approximating function, whose complexity is at most polynomial in the dimension d and in the number m of points. Due to the arbitrariness of A, the choice of the sampling points will be according to suitable random distributions and our results hold with overwhelming probability. Our approach uses tools taken from the compressed sensing framework, recent Chernoff bounds for sums of positivesemidefinite matrices, and classical stability bounds for invariant subspaces of singular value decompositions. AMS subject classification (MSC 2010): 65D15, 03D32, 68Q30, 60B20, 60G50
Reconstruction of a lowrank matrix in the presence of Gaussian noise
 J. Mult. Anal
, 2013
"... In this paper we study the problem of reconstruction of a lowrank matrix observed with additive Gaussian noise. First we show that under mild assumptions (about the prior distribution of the signal matrix) we can restrict our attention to reconstruction methods that are based on the singular value ..."
Abstract

Cited by 10 (0 self)
 Add to MetaCart
(Show Context)
In this paper we study the problem of reconstruction of a lowrank matrix observed with additive Gaussian noise. First we show that under mild assumptions (about the prior distribution of the signal matrix) we can restrict our attention to reconstruction methods that are based on the singular value decomposition of the observed matrix and act only on its singular values (preserving the singular vectors). Then we determine the effect of noise on the SVD of lowrank matrices by building a connection between matrix reconstruction problem and spiked population model in random matrix theory. Based on this knowledge, we propose a new reconstruction method, called RMT, that is designed to reverse the effect of the noise on the singular values of the signal matrix and adjust for its effect on the singular vectors. With an extensive simulation study we show that the proposed method outperform even oracle versions of both soft and hard thresholding methods and closely matches the performance of a general oracle scheme. 1
Guaranteed matrix completion via nonconvex factorization
, 2014
"... Matrix factorization is a popular approach for largescale matrix completion and constitutes a basic component of many solutions for Netflix Prize competition. In this approach, the unknown lowrank matrix is expressed as the product of two much smaller matrices so that the lowrank property is auto ..."
Abstract

Cited by 10 (0 self)
 Add to MetaCart
Matrix factorization is a popular approach for largescale matrix completion and constitutes a basic component of many solutions for Netflix Prize competition. In this approach, the unknown lowrank matrix is expressed as the product of two much smaller matrices so that the lowrank property is automatically fulfilled. The resulting optimization problem, even with huge size, can be solved (to stationary points) very efficiently through standard optimization algorithms such as alternating minimization and stochastic gradient descent (SGD). However, due to the nonconvexity caused by the factorization model, there is a limited theoretical understanding of whether these algorithms will generate a good solution. In this paper, we establish a theoretical guarantee for the factorization based formulation to correctly recover the underlying lowrank matrix. In particular, we show that under similar conditions to those in previous works, many standard optimization algorithms converge to the global optima of the factorization based formulation, thus recovering the true lowrank matrix. To the best of our knowledge, our result is the first one that provides recovery guarantee for many standard algorithms such as gradient descent, SGD and block coordinate gradient descent. Our result also applies to alternating minimization, and a notable difference from previous studies on alternating minimization is that we do not need the resampling scheme (i.e. using independent samples in each iteration).