### Noisy Tensor Completion via the Sum-of-Squares Hierarchy

, 2016

"... Abstract In the noisy tensor completion problem we observe m entries (whose location is chosen uniformly at random) from an unknown n 1 × n 2 × n 3 tensor T . We assume that T is entry-wise close to being rank r. Our goal is to fill in its missing entries using as few observations as possible. Let ..."

Abstract
- Add to MetaCart

(Show Context)
Abstract In the noisy tensor completion problem we observe m entries (whose location is chosen uniformly at random) from an unknown n 1 × n 2 × n 3 tensor T . We assume that T is entry-wise close to being rank r. Our goal is to fill in its missing entries using as few observations as possible. Let n = max(n 1 , n 2 , n 3 ). We show that if m = n 3/2 r then there is a polynomial time algorithm based on the sixth level of the sum-of-squares hierarchy for completing it. Our estimate agrees with almost all of T 's entries almost exactly and works even when our observations are corrupted by noise. This is also the first algorithm for tensor completion that works in the overcomplete case when r > n, and in fact it works all the way up to r = n 3/2− . Our proofs are short and simple and are based on establishing a new connection between noisy tensor completion (through the language of Rademacher complexity) and the task of refuting random constant satisfaction problems. This connection seems to have gone unnoticed even in the context of matrix completion. Furthermore, we use this connection to show matching lower bounds. Our main technical result is in characterizing the Rademacher complexity of the sequence of norms that arise in the sum-of-squares relaxations to the tensor nuclear norm. These results point to an interesting new direction: Can we explore computational vs. sample complexity tradeoffs through the sum-of-squares hierarchy?

### Generalized Higher-Order Orthogonal Iteration for Tensor Decomposition and Completion

"... Low-rank tensor estimation has been frequently applied in many real-world prob-lems. Despite successful applications, existing Schatten 1-norm minimization (SNM) methods may become very slow or even not applicable for large-scale problems. To address this difficulty, we therefore propose an efficien ..."

Abstract
- Add to MetaCart

(Show Context)
Low-rank tensor estimation has been frequently applied in many real-world prob-lems. Despite successful applications, existing Schatten 1-norm minimization (SNM) methods may become very slow or even not applicable for large-scale problems. To address this difficulty, we therefore propose an efficient and scal-able core tensor Schatten 1-norm minimization method for simultaneous tensor decomposition and completion, with a much lower computational complexity. We first induce the equivalence relation of Schatten 1-norm of a low-rank tensor and its core tensor. Then the Schatten 1-norm of the core tensor is used to replace that of the whole tensor, which leads to a much smaller-scale matrix SNM prob-lem. Finally, an efficient algorithm with a rank-increasing scheme is developed to solve the proposed problem with a convergence guarantee. Extensive experimen-tal results show that our method is usually more accurate than the state-of-the-art methods, and is orders of magnitude faster.

### New Ranks for Even-Order Tensors and Their Applications in Low-Rank Tensor Optimization

, 2015

"... In this paper, we propose three new tensor decompositions for even-order tensors correspond-ing respectively to the rank-one decompositions of some unfolded matrices. Consequently such new decompositions lead to three new notions of (even-order) tensor ranks, to be called the M-rank, the symmetric M ..."

Abstract
- Add to MetaCart

(Show Context)
In this paper, we propose three new tensor decompositions for even-order tensors correspond-ing respectively to the rank-one decompositions of some unfolded matrices. Consequently such new decompositions lead to three new notions of (even-order) tensor ranks, to be called the M-rank, the symmetric M-rank, and the strongly symmetric M-rank in this paper. We discuss the bounds between these new tensor ranks and the CP(CANDECOMP/PARAFAC)-rank and the symmetric CP-rank of an even-order tensor. In particular, we show: (1) these newly defined ranks actually coincide with each other if the even-order tensor in question is super-symmetric; (2) the CP-rank and symmetric CP-rank for a fourth-order tensor can be both lower and upper bounded (up to a constant factor) by the corresponding M-rank. Since the M-rank is much easi-er to compute than the CP-rank, we can replace the CP-rank by the M-rank in the low-CP-rank tensor recovery model. Numerical results on both synthetic data and real data from colored video completion and decomposition problems show that the M-rank is indeed an effective and easy computable approximation of the CP-rank in the context of low-rank tensor recovery.

### Rubik: Knowledge Guided Tensor Factorization and Completion for Health Data Analytics

"... Computational phenotyping is the process of converting het-erogeneous electronic health records (EHRs) into meaningful clinical concepts. Unsupervised phenotyping methods have the potential to leverage a vast amount of labeled EHR data for phenotype discovery. However, existing unsupervised phenotyp ..."

Abstract
- Add to MetaCart

(Show Context)
Computational phenotyping is the process of converting het-erogeneous electronic health records (EHRs) into meaningful clinical concepts. Unsupervised phenotyping methods have the potential to leverage a vast amount of labeled EHR data for phenotype discovery. However, existing unsupervised phenotyping methods do not incorporate current medical knowledge and cannot directly handle missing, or noisy data. We propose Rubik, a constrained non-negative tensor fac-torization and completion method for phenotyping. Rubik incorporates 1) guidance constraints to align with existing medical knowledge, and 2) pairwise constraints for obtain-ing distinct, non-overlapping phenotypes. Rubik also has built-in tensor completion that can significantly alleviate the impact of noisy and missing data. We utilize the Alternat-ing Direction Method of Multipliers (ADMM) framework to tensor factorization and completion, which can be easily scaled through parallel computing. We evaluate Rubik on two EHR datasets, one of which contains 647,118 records for 7,744 patients from an outpatient clinic, the other of which is a public dataset containing 1,018,614 CMS claims records for 472,645 patients. Our results show that Rubik can discover more meaningful and distinct phenotypes than the baselines. In particular, by using knowledge guidance constraints, Rubik can also discover sub-phenotypes for sev-eral major diseases. Rubik also runs around seven times faster than current state-of-the-art tensor methods. Finally, Rubik is scalable to large datasets containing millions of EHR records.

### ON HIGHER-ORDER SINGULAR VALUE DECOMPOSITION FROM INCOMPLETE DATA∗

"... Abstract. Higher-order singular value decomposition (HOSVD) is an efficient way for data reduction and also eliciting intrinsic structure of multi-dimensional array data. It has been used in many applications, and some of them involve incomplete data. To obtain HOSVD of the data with missing values, ..."

Abstract
- Add to MetaCart

Abstract. Higher-order singular value decomposition (HOSVD) is an efficient way for data reduction and also eliciting intrinsic structure of multi-dimensional array data. It has been used in many applications, and some of them involve incomplete data. To obtain HOSVD of the data with missing values, one can first impute the missing entries through a certain tensor completion method and then perform HOSVD to the reconstructed data. However, the two-step procedure can be inefficient and does not make reliable decomposition. In this paper, we formulate an incomplete HOSVD problem and combine the two steps into solving a single optimization problem, which simultaneously achieves imputation of missing values and also tensor decomposition. We also present two algorithms for solving the problem based on block coordinate update. Global convergence of both algorithms is shown under mild assumptions. The convergence of the second algorithm implies that of the popular higher-order orthogonality iteration (HOOI) method, and thus we, for the first time, give global convergence of HOOI. In addition, we compare the proposed methods to state-of-the-art ones for solving incomplete HOSVD and also low-rank tensor completion problems and demonstrate the superior performance of our methods over other compared ones. Furthermore, we apply them to face recognition and MRI image reconstruction to show their practical performance. Key words. multilinear data analysis, higher-order singular value decomposition (HOSVD), low-rank tensor completion, non-convex optimization, higher-order orthogonality iteration (HOOI), global convergence.

### OPTIMIZATION ON THE HIERARCHICAL TUCKER MANIFOLD- APPLICATIONS TO TENSOR COMPLETION

"... Abstract. In this work, we develop an optimization framework for problems whose solutions are well-approximated by Hierarchical Tucker (HT) tensors, an efficient structured tensor format based on recursive subspace factorizations. By exploiting the smooth manifold structure of these tensors, we cons ..."

Abstract
- Add to MetaCart

(Show Context)
Abstract. In this work, we develop an optimization framework for problems whose solutions are well-approximated by Hierarchical Tucker (HT) tensors, an efficient structured tensor format based on recursive subspace factorizations. By exploiting the smooth manifold structure of these tensors, we construct standard optimization algorithms such as Steepest Descent and Conjugate Gradient for completing tensors from missing entries. Our algorithmic framework is fast and scalable to large problem sizes as we do not require SVDs on the ambient tensor space, as required by other methods. Moreover, we exploit the structure of the Gramian matrices associated with the HT format to regularize our problem, reducing overfitting for high subsampling ratios. We also find that the organization of the tensor can have a major impact on completion from realistic seismic acquisition geometries. These samplings are far from idealized randomized samplings that are usually considered in the literature but are realizable in practical scenarios. Using these algorithms, we successfully interpolate large-scale seismic data sets and demonstrate the competitive computational scaling of our algorithms as the problem sizes grow. 1.

### s Time Multivariate time-series Collaborative filtering Movies Us

"... Towards better computation-statistics trade-off in tensor decomposition ..."

(Show Context)
### Noisy estimation of simultaneously structured models: Limitations of convex relaxation

"... Abstract — Models or signals exhibiting low dimensional behavior (e.g., sparse signals, low rank matrices) play an important role in signal processing and system identification. In this paper, we focus on models that have multiple structures simultaneously; e.g., matrices that are both low rank and ..."

Abstract
- Add to MetaCart

(Show Context)
Abstract — Models or signals exhibiting low dimensional behavior (e.g., sparse signals, low rank matrices) play an important role in signal processing and system identification. In this paper, we focus on models that have multiple structures simultaneously; e.g., matrices that are both low rank and sparse, arising in phase retrieval, quadratic compressed sensing, and cluster detection in social networks. We consider the estima-tion of such models from observations corrupted by additive Gaussian noise. We provide tight upper and lower bounds on the mean squared error (MSE) of a convex denoising program that uses a combination of regularizers to induce multiple structures. In the case of low rank and sparse matrices, we quantify the gap between the MSE of the convex program and the best achievable error, and we present a simple (nonconvex) thresholding algorithm that outperforms its convex counterpart and achieves almost optimal MSE. This paper extends prior work on a different but related problem: recovering simultaneously structured models from noiseless compressed measurements, where bounds on the num-ber of required measurements were given. The present work shows a similar fundamental limitation exists in a statistical denoising setting. Index Terms — simultaneously structured, low rank and sparse, denoising, estimation, compressed sensing I.

### Convex Relaxation for Low-Dimensional Representation: Phase Transitions and Limitations

, 2015

"... ..."

(Show Context)
### Generalized Higher-Order Orthogonal Iteration for Tensor Decomposition and Completion

"... Low-rank tensor estimation has been frequently applied in many real-world prob-lems. Despite successful applications, existing Schatten 1-norm minimization (SNM) methods may become very slow or even not applicable for large-scale problems. To address this difficulty, we therefore propose an efficien ..."

Abstract
- Add to MetaCart

(Show Context)
Low-rank tensor estimation has been frequently applied in many real-world prob-lems. Despite successful applications, existing Schatten 1-norm minimization (SNM) methods may become very slow or even not applicable for large-scale problems. To address this difficulty, we therefore propose an efficient and scal-able core tensor Schatten 1-norm minimization method for simultaneous tensor decomposition and completion, with a much lower computational complexity. We first induce the equivalence relation of Schatten 1-norm of a low-rank tensor and its core tensor. Then the Schatten 1-norm of the core tensor is used to replace that of the whole tensor, which leads to a much smaller-scale matrix SNM prob-lem. Finally, an efficient algorithm with a rank-increasing scheme is developed to solve the proposed problem with a convergence guarantee. Extensive experimen-tal results show that our method is usually more accurate than the state-of-the-art methods, and is orders of magnitude faster.