Citation Context

... on seminal developments in “compressed sensing” that determined conditions for when minimizing the ℓ1 norm of a vector over an affine space returns the sparsest vector in that space (see, e.g., [4], =-=[3]-=-, [1]). There is a strong parallelism between the sparse approximation and rank minimization settings. The rank of a diagonal matrix is equal to the number of non-zeros on the diagonal. Similarly, the...

Citation Context

...uilds on seminal developments in “compressed sensing” that determined conditions for when minimizing the ℓ1 norm of a vector over an affine space returns the sparsest vector in that space (see, e.g., =-=[4]-=-, [3], [1]). There is a strong parallelism between the sparse approximation and rank minimization settings. The rank of a diagonal matrix is equal to the number of non-zeros on the diagonal. Similarly...

Citation Context

...was sampled from the Gaussian ensemble with m rows and n2 columns. Then the nuclear norm minimization minimize subject to ‖X‖∗ A vec X = A vec Y0 was solved using the freely available software SeDuMi =-=[18]-=- using the semidefinite programming formulation described in [14]. On a 2.0 GHz Laptop, each semidefinite program could be solved in less than two minutes for 40 × 40 dimensional X. We declared Y0 to ...

Citation Context

...eminal developments in “compressed sensing” that determined conditions for when minimizing the ℓ1 norm of a vector over an affine space returns the sparsest vector in that space (see, e.g., [4], [3], =-=[1]-=-). There is a strong parallelism between the sparse approximation and rank minimization settings. The rank of a diagonal matrix is equal to the number of non-zeros on the diagonal. Similarly, the sum ...

Citation Context

...duction [2]. Rank minimization is also of interest to a broader optimization community in a variety of applications including inference with partial information [16] and embedding in Euclidean spaces =-=[11]-=-. In certain instances with special structure, the rank minimization problem can be solved via the singular value decomposition or can be reduced to the solution of a linear system [12], [13]. In gene...

Citation Context

...mpled from G(D). Then E‖G‖∗ = DEσi = 8 3π D3/2 + q(D) (3.13) where q(D)/D 3/2 = o(1). The constant in from of the D 3/2 comes from integrating √ λ against the Marčenko-Pastur distribution (see, e.g., =-=[17, 2]-=-): 1 2π ∫ 4 Secondly, a straightforward calculation reveals 0 σ(‖G‖∗) = √ 4 − t dt = 8 3π ≈ 0.85 . sup ‖G‖F = ‖H‖≤1 √ D . Plugging these values in with the appropriate dimensions completes the proof. ...

Citation Context

...riable have Gaussian tails. Theorem 3.5 Let x be a normally distributed random vector and let f be a function with Lipshitz constant L. Then ( P[|f(x) − E[f(x)]| ≥ t] ≤ 2 exp − t2 2L2 ) . (3.5) 9See =-=[15]-=- for a proof of this theorem with slightly weaker constants and several references for more complicated proofs that give rise to this concentration inequality. The following Lemma bounds the Lipshitz ...

Citation Context

...mpled from G(D). Then E‖G‖∗ = DEσi = 8 3π D3/2 + q(D) (3.13) where q(D)/D 3/2 = o(1). The constant in from of the D 3/2 comes from integrating √ λ against the Marčenko-Pastur distribution (see, e.g., =-=[17, 2]-=-): 1 2π ∫ 4 Secondly, a straightforward calculation reveals 0 σ(‖G‖∗) = √ 4 − t dt = 8 3π ≈ 0.85 . sup ‖G‖F = ‖H‖≤1 √ D . Plugging these values in with the appropriate dimensions completes the proof. ...

Citation Context

...re one minimizes the trace of a positive semidefinite decision variable instead of the rank (see, e.g., [2], [12]). A generalization of this heuristic to non-symmetric matrices introduced by Fazel in =-=[8]-=- minimizes the nuclear norm, *Center for the Mathematics of Information, California Institute of Technology, 1200 E California Blvd, Pasadena, CA brecht@ist.caltech.edu ** Electrical Engineering, Cali...

Citation Context

... Optimization problems involving constraints on the rank of matrices are pervasive in control applications, arising in the context of low-order controller design [7], [12], minimal realization theory =-=[9]-=-, and model reduction [2]. Rank minimization is also of interest to a broader optimization community in a variety of applications including inference with partial information [16] and embedding in Euc...

Citation Context

...oller design [9, 19], minimal realization theory [11], and model reduction [4]. In Machine Learning, problems in inference with partial information [23], multi-task learning [1],and manifold learning =-=[28]-=- have been formulated as rank minimization problems. Rank minimization also plays a key role in the study of embeddings of discrete metric spaces in Euclidean space [16]. In certain instances with spe...

Citation Context

...context of low-order controller design [9, 19], minimal realization theory [11], and model reduction [4]. In Machine Learning, problems in inference with partial information [23], multi-task learning =-=[1]-=-,and manifold learning [28] have been formulated as rank minimization problems. Rank minimization also plays a key role in the study of embeddings of discrete metric spaces in Euclidean space [16]. In...

Citation Context

...al realization theory [9], and model reduction [2]. Rank minimization is also of interest to a broader optimization community in a variety of applications including inference with partial information =-=[16]-=- and embedding in Euclidean spaces [11]. In certain instances with special structure, the rank minimization problem can be solved via the singular value decomposition or can be reduced to the solution...

Citation Context

...um of one Gaussian process is greater to that of another. Elementary proofs of both of these Theorems and several other Comparison Theorems can be found in §3.3 of [15]. Theorem 3.10 (Slepian’s Lemma =-=[24]-=-) Let X and Y by Gaussian random variables in RN such that { E[XiXj] ≤ E[YiYj] for all i ̸= j Then E[X 2 i ] = E[Yi] 2 for all i E[max i Yi] ≤ E[max i Xi] . ] 12Theorem 3.11 (Gordan [12, 13]) Let X =...

Citation Context

... available in cases that could also be solved by elementary linear algebra [13]. The first non-trivial sufficient conditions that guaranteed the success of the nuclear norm heuristic were provided in =-=[14]-=-. Focusing on the special case where one seeks the lowest rank matrix in an affine subspace, the authors provide a “restricted isometery” condition on the linear map defining the affine subspace which...

Citation Context

... were able to extend much of the analysis developed for the ℓ1 heuristic to provide guarantees for the nuclear norm heuristic. Building on a different collection of developments in compressed sensing =-=[5]-=-, [6], [17], we present a necessary and sufficient condition for the solution of the nuclear norm heuristic to coincide with the minimum rank solution in an affine space. The condition characterizes a...

Citation Context

...ptotic scenarios. I. INTRODUCTION Optimization problems involving constraints on the rank of matrices are pervasive in control applications, arising in the context of low-order controller design [7], =-=[12]-=-, minimal realization theory [9], and model reduction [2]. Rank minimization is also of interest to a broader optimization community in a variety of applications including inference with partial infor...

Citation Context

...-asymptotic scenarios. I. INTRODUCTION Optimization problems involving constraints on the rank of matrices are pervasive in control applications, arising in the context of low-order controller design =-=[7]-=-, [12], minimal realization theory [9], and model reduction [2]. Rank minimization is also of interest to a broader optimization community in a variety of applications including inference with partial...

Citation Context

... to extend much of the analysis developed for the ℓ1 heuristic to provide guarantees for the nuclear norm heuristic. Building on a different collection of developments in compressed sensing [5], [6], =-=[17]-=-, we present a necessary and sufficient condition for the solution of the nuclear norm heuristic to coincide with the minimum rank solution in an affine space. The condition characterizes a particular...

Citation Context

...volving constraints on the rank of matrices are pervasive in control applications, arising in the context of low-order controller design [7], [12], minimal realization theory [9], and model reduction =-=[2]-=-. Rank minimization is also of interest to a broader optimization community in a variety of applications including inference with partial information [16] and embedding in Euclidean spaces [11]. In ce...

Citation Context

... → Rm be a linear map, and let b ∈ Rm . The main optimization problem under study is minimize rank(X) (1.1) subject to A(X) = b . 2This problem is known to be NP-HARD and is also hard to approximate =-=[18]-=-. As mentioned above, a popular heuristic for this problem replaces the rank function with the sum of the singular values of the decision variable. Let σi(X) denote the i-th largest singular value of ...

Citation Context

...operators onto r-dimensional subspaces, we will have proved the Strong Bound. To proceed, we need to know the size of an ɛ-net. The following bound on such a net is due to Szarek. Theorem 3.7 (Szarek =-=[27]-=-) Consider the space of all projection operators on R n projecting onto r dimensional subspaces endowed with the metric d(P, P ′ ) = ‖P − P ′ ‖ Then there exists an ɛ-net in this metric space with car...

Citation Context

...n spaces [11]. In certain instances with special structure, the rank minimization problem can be solved via the singular value decomposition or can be reduced to the solution of a linear system [12], =-=[13]-=-. In general, however, minimizing the rank of a matrix subject to convex constraints is NPHARD. The best exact algorithms for this problem involve quantifier elimination and such solution methods requ...

Citation Context

...pian’s Lemma [24]) Let X and Y by Gaussian random variables in RN such that { E[XiXj] ≤ E[YiYj] for all i ̸= j Then E[X 2 i ] = E[Yi] 2 for all i E[max i Yi] ≤ E[max i Xi] . ] 12Theorem 3.11 (Gordan =-=[12, 13]-=-) Let X = (Xij) and Y = (Yij) be Gaussian random vectors in RN1×N2 such that ⎧ ⎪⎨ E[XijXik] ≤ E[YijYik] for all i, j, k Then E[XijXlk] ≥ E[YijYlk] ⎪⎩ E[X 2 ij ] = E[X2 ij ] E[min i max j for all i ̸= ...

Citation Context

...pian’s Lemma [24]) Let X and Y by Gaussian random variables in RN such that { E[XiXj] ≤ E[YiYj] for all i ̸= j Then E[X 2 i ] = E[Yi] 2 for all i E[max i Yi] ≤ E[max i Xi] . ] 12Theorem 3.11 (Gordan =-=[12, 13]-=-) Let X = (Xij) and Y = (Yij) be Gaussian random vectors in RN1×N2 such that ⎧ ⎪⎨ E[XijXik] ≤ E[YijYik] for all i, j, k Then E[XijXlk] ≥ E[YijYlk] ⎪⎩ E[X 2 ij ] = E[X2 ij ] E[min i max j for all i ̸= ...