Citation Context

...FaCT can operate under a wide spectrum of settings, including plain matrix factorization, tensor factorization, as well as coupled decompositions. Thus, FlexiFaCT includes several recent methods [9], =-=[11]-=-, as special cases. 2. Scalability: FlexiFaCT scales very well both with the input size, as well as with the number of model parameters. 3. Proof of convergence: We prove that FlexiFaCT converges, eve...

Citation Context

...size, and is therefore feasible for large array analysis. For evaluation, the proposed approach is first examined on three small real-world datasets where InfTucker is feasible. Our model achieves higher prediction accuracy than InfTucker and other mulitlinear alternatives. Moreover, a simulation shows that our approach is able to capture the latent clusters in a tensor having nonlinear relationships. Finally, our approach is applied to analyze two real-world large arrays with billions of entries. The comparison with the state-of-the-art large scale tensor decomposition algorithm, GigaTensor (Kang et al., 2012), shows that our approach obtains significantly better predictive performance. 2 Background on Tensor Decomposition We first introduce the notations. A K-mode multiway array or tensor is denoted by M ∈ Rm1×...×mK , where the k-th mode has a dimension of mk, corresponding to mk nodes or objects. We use mi (i = (i1, . . . , iK)) to denote M’s entry at location i. Using the vectorization operation, we can stack all of M’s entries in a vector, vec(M), with size ∏K k=1mk by 1. In vec(M), the entry i = (i1, . . . , iK) of M is mapped to the entry at position j = iK + ∑K−1 i=1 (ik − 1) ∏K k+1mk. Give...

Citation Context

...algorithm avoids intermediate data explosion by ‘accumulating’ tensor-matrix operations, but it does not provision for efficient parallelization (the accumulation step must be performed serially). In =-=[10]-=-, a parallel algorithm that computes X(1)(C⊙B) with 5F NNZ flops using O(max(J +NNZ;K+NNZ)) intermediate memory is presented. The algorithm in [10] admits MapReduce implementation. Algorithm 1 Computi...

Citation Context

...pproach is to use tensor (multi-dimensional array) to model time evolving graphs by using the time as the 3rd dimension, and find correlations between dimensions using tensor decompositions (e.g. see =-=[24; 29]-=-). 5. Visualization and Understanding of Graphs. A graph forms a complicated object with many interaction between nodes. Visualization of graphs helps us better understand the structure and the intera...

Citation Context

...nts. The CP decomposition has found numerous applications in data mining [4, 18, 20], computational neuroscience [10, 21], and recently, in statistical learning for latent variable models [1, 30, 28, 6]. For latent variable modeling, these methods yield consistent estimates under mild conditions such as non-degeneracy and require only polynomial sample and computational complexity [1, 30, 28, 6]. Given the importance of tensor methods for large-scale machine learning, there has been an increasing interest in scaling up tensor decomposition algorithms to handle gigantic real-world data tensors [27, 24, 8, 16, 14, 2, 29]. However, the previous works fall short in many ways, as described subsequently. In this paper, we design and analyze efficient randomized tensor methods using ideas from sketching [23]. The idea is to maintain a low-dimensional sketch of an input tensor and then perform implicit tensor decomposition using existing methods such as tensor power updates, alternating least squares or online tensor updates. We obtain the fastest decomposition methods for both sparse and dense tensors. Our framework can easily handle modern machine learning applications with billions of training instances, and at ...

Citation Context

.../www.ieee.org/publications_standards/publications/rights/index.html for more information. LIAVAS AND SIDIROPOULOS: PARALLEL ALGORITHMS FOR CONSTRAINED TENSOR FACTORIZATION VIA ADMoM 5451 developed in =-=[21]-=-, which reported 100-fold scaling improvements relative to the prior art. The gist of [21] is to avoid the explicit computation of ‘blown-up’ intermediate matrix products in the ALS algorithm, particu...

Citation Context

...trate its usefulness in leveraging side-information provided in form of mode-network(s). 1 INTRODUCTION With the recent surge in multiway, multirelational, or “tensor” data sets (Nickel et al., 2011; =-=Kang et al., 2012-=-), learning algorithms that can extract useful knowledge from such data are becoming increasingly important. Tensor decomposition methods (Kolda and Bader, 2009) offer an attractive way to accomplish ...

Citation Context

...scale matrix factorization, very few works are devoted to large-scale tensors. Two ways have been recently considered in the literature. The first one consists in exploiting sparseness of tensors. In =-=[10]-=-, the GigaTensor algorithm is designed in order to minimize the number of floating point operations and to handle the size of intermediate data in order to overcome the intermediate data explosion pro...

Citation Context

... the underlying dynamics of human behavior still remains an important research challenge. Matrix/tensor factorization: There has been active research on matrix factorization [30, 31], tensor analysis =-=[4, 26, 13]-=-, tensor decompositions [15, 16, 9] and scalable methods [17, 1]. Here we focus on how to efficiently process the increments in tensor decomposition by matrix and tensor perturbation theory [25]. 3. P...