Results 1  10
of
18
FlexiFaCT: Scalable Flexible Factorization of Coupled Tensors on
"... Given multiple data sets of relational data that share a number of dimensions, how can we efficiently decompose our data into the latent factors? Factorization of a single matrix or tensor has attracted much attention, as, e.g., in the Netflix challenge, with users rating movies. However, we often h ..."
Abstract

Cited by 8 (1 self)
 Add to MetaCart
(Show Context)
Given multiple data sets of relational data that share a number of dimensions, how can we efficiently decompose our data into the latent factors? Factorization of a single matrix or tensor has attracted much attention, as, e.g., in the Netflix challenge, with users rating movies. However, we often have additional, side, information, like, e.g., demographic data about the users, in the Netflix example above. Incorporating the additional information leads to the coupled factorization problem. So far, it has been solved for relatively small datasets. We provide a distributed, scalable method for decomposing matrices, tensors, and coupled data sets through stochastic gradient descent on a variety of objective functions. We offer the following contributions: (1) Versatility: Our algorithm can perform matrix, tensor, and coupled factorization, with flexible objective functions including the Frobenius norm, Frobenius norm with an `1 induced sparsity, and nonnegative factorization. (2) Scalability: FlexiFaCT scales to unprecedented sizes in both the data and model, with up to billions of parameters. FlexiFaCT runs on standard Hadoop. (3) Convergence proofs showing that FlexiFaCT converges on the variety of objective functions, even with projections. 1
Memoryefficient parallel computation of tensor and matrix products for big tensor decomposition
 in Proceedings of the Asilomar Conference on Signals, Systems, and Computers
, 2014
"... Abstract—Lowrank tensor decomposition has many applications in signal processing and machine learning, and is becoming increasingly important for analyzing big data. A significant challenge is the computation of intermediate products which can be much larger than the final result of the computatio ..."
Abstract

Cited by 4 (3 self)
 Add to MetaCart
(Show Context)
Abstract—Lowrank tensor decomposition has many applications in signal processing and machine learning, and is becoming increasingly important for analyzing big data. A significant challenge is the computation of intermediate products which can be much larger than the final result of the computation, or even the original tensor. We propose a scheme that allows memoryefficient inplace updates of intermediate matrices. Motivated by recent advances in big tensor decomposition from multiple compressed replicas, we also consider the related problem of memoryefficient tensor compression. The resulting algorithms can be parallelized, and can exploit but do not require sparsity. I.
DISTRIBUTED LARGESCALE TENSOR DECOMPOSITION
"... Canonical Polyadic Decomposition (CPD), also known as PARAFAC, is a useful tool for tensor factorization. It has found application in several domains including signal processing and data mining. With the deluge of data faced in our societies, largescale matrix and tensor factorizations become a cru ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
(Show Context)
Canonical Polyadic Decomposition (CPD), also known as PARAFAC, is a useful tool for tensor factorization. It has found application in several domains including signal processing and data mining. With the deluge of data faced in our societies, largescale matrix and tensor factorizations become a crucial issue. Few works have been devoted to largescale tensor factorizations. In this paper, we introduce a fully distributed method to compute the CPD of a largescale data tensor across a network of machines with limited computation resources. The proposed approach is based on collaboration between the machines in the network across the three modes of the data tensor. Such a multimodal collaboration allows an essentially unique reconstruction of the factor matrices in an efficient way. We provide an analysis of the computation and communication cost of the proposed scheme and address the problem of minimizing communication costs while maximizing the use of available computation resources. Index Terms — Tensor decompositions, largescale data, distributed computation.
Parallel Algorithms for Constrained Tensor Factorization via Alternating Direction Method of Multipliers
, 2014
"... Abstract—Tensor factorization has proven useful in a wide range of applications, from sensor array processing to communications, speech and audio signal processing, and machine learning. With few recent exceptions, all tensor factorization algorithms were originally developed for centralized, inme ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
(Show Context)
Abstract—Tensor factorization has proven useful in a wide range of applications, from sensor array processing to communications, speech and audio signal processing, and machine learning. With few recent exceptions, all tensor factorization algorithms were originally developed for centralized, inmemory computation on a single machine; and the few that break away from this mold do not easily incorporate practically important constraints, such as nonnegativity. A new constrained tensor factorization framework is proposed in this paper, building upon the Alternating Direction Method of Multipliers (ADMoM). It is shown that this simplifies computations, bypassing the need to solve constrained optimization problems in each iteration; and it naturally leads to distributed algorithms suitable for parallel implementation. This opens the door for many emerging big dataenabled applications. The methodology is exemplified using nonnegativity as a baseline constraint, but the proposed framework can incorporate many other types of constraints. Numerical experiments are encouraging, indicating that ADMoMbased nonnegative tensor factorization (NTF) has high potential as an alternative to stateoftheart approaches. Index Terms—Tensor decomposition, PARAFACmodel, parallel algorithms.
Large Scale Tensor Decompositions: Algorithmic Developments and Applications
"... Tensor decompositions are increasingly gaining popularity in data science applications. Albeit extremely powerful tools, scalability to truly large datasets for such decomposition algorithms is still a challenging problem. In this paper, we provide an overview of recent algorithmic developments tow ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
(Show Context)
Tensor decompositions are increasingly gaining popularity in data science applications. Albeit extremely powerful tools, scalability to truly large datasets for such decomposition algorithms is still a challenging problem. In this paper, we provide an overview of recent algorithmic developments towards the direction of scaling tensor decompositions to big data. We present an exact Map/Reduce based algorithm, as well as an approximate, fully parallelizable algorithm that is sparsity promoting. In both cases, careful design and implementation is key, so that we achieve scalability and efficiency. We showcase the effectiveness of our methods, by providing a variety of real world applicationswhose volume previously rendered their analysis very hard, if not impossible where our algorithms were able to discover interesting patterns and anomalies. 1
Big Graph Mining: Algorithms and Discoveries
"... How do we find patterns and anomalies in very large graphs with billions of nodes and edges? How to mine such big graphs efficiently? Big graphs are everywhere, ranging from social networks and mobile call networks to biological networks and the World Wide Web. Mining big graphs leads to many intere ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
(Show Context)
How do we find patterns and anomalies in very large graphs with billions of nodes and edges? How to mine such big graphs efficiently? Big graphs are everywhere, ranging from social networks and mobile call networks to biological networks and the World Wide Web. Mining big graphs leads to many interesting applications including cyber security, fraud detection, Web search, recommendation, and many more. In this paper we describe Pegasus, a big graph mining system built on top of MapReduce, a modern distributed data processing platform. We introduce GIMV, an important primitive that Pegasus uses for its algorithms to analyze structures of large graphs. We also introduce HEigen, a large scale eigensolver which is also a part of Pegasus. Both GIMV and HEigen are highly optimized, achieving linear scale up on the number of machines and edges, and providing 9.2 × and 76 × faster performance than their naive counterparts, respectively. Using Pegasus, we analyze very large, real world graphs with billions of nodes and edges. Our findings include anomalous spikes in the connected component size distribution, the 7 degrees of separation in a Web graph, and anomalous adult advertisers in the whofollowswhom Twitter social network.
ZeroTruncated Poisson Tensor Factorization for Massive Binary Tensors
"... We present a scalable Bayesian model for lowrank factorization of massive tensors with binary observations. The proposed model has the following key properties: (1) in contrast to the models based on the logistic or probit likelihood, using a zerotruncated Poisson likelihood for binary data al ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
(Show Context)
We present a scalable Bayesian model for lowrank factorization of massive tensors with binary observations. The proposed model has the following key properties: (1) in contrast to the models based on the logistic or probit likelihood, using a zerotruncated Poisson likelihood for binary data allows our model to scale up in the number of ones in the tensor, which is especially appealing for massive but sparse binary tensors; (2) sideinformation in form of binary pairwise relationships (e.g., an adjacency network) between objects in any tensor mode can also be leveraged, which can be especially useful in “coldstart ” settings; and (3) the model admits simple Bayesian inference via batch, as well as online MCMC; the latter allows scaling up even for dense binary data (i.e., when the number of ones in the tensor/network is also massive). In addition, nonnegative factor matrices in our model provide easy interpretability, and the tensor rank can be inferred from the data. We evaluate our model on several largescale realworld binary tensors, achieving excellent computational scalability, and also demonstrate its usefulness in leveraging sideinformation provided in form of modenetwork(s).
TurboSMT: Accelerating Coupled Sparse MatrixTensor Factorizations by 200x
"... How can we correlate the neural activity in the human brain as it responds to typed words, with properties of these terms (like ’edible’, ’fits in hand’)? In short, we want to find latent variables, that jointly explain both the brain activity, as well as the behavioral responses. This is one of man ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
(Show Context)
How can we correlate the neural activity in the human brain as it responds to typed words, with properties of these terms (like ’edible’, ’fits in hand’)? In short, we want to find latent variables, that jointly explain both the brain activity, as well as the behavioral responses. This is one of many settings of the Coupled MatrixTensor Factorization (CMTF) problem. Can we accelerate any CMTF solver, so that it runs within a few minutes instead of tens of hours to a day, while maintaining good accuracy? We introduce TurboSMT, a metamethod capable of doing exactly that: it boosts the performance of any CMTF algorithm, by up to 200x, along with an up to 65 fold increase in sparsity, with comparable accuracy to the baseline. We apply TurboSMT to BrainQ, a dataset consisting of a (nouns, brain voxels, human subjects) tensor and a (nouns, properties) matrix, with coupling along the nouns dimension. TurboSMT is able to find meaningful latent variables, as well as to predict brain activity with competitive accuracy. 1
FEMA: Flexible Evolutionary Multifaceted Analysis for Dynamic Behavioral Pattern Discovery
"... Behavioral pattern discovery is increasingly being studied to understand human behavior and the discovered patterns can be used in many real world applications such as web search, recommender system and advertisement targeting. Traditional methods usually consider the behaviors as simple user and i ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
(Show Context)
Behavioral pattern discovery is increasingly being studied to understand human behavior and the discovered patterns can be used in many real world applications such as web search, recommender system and advertisement targeting. Traditional methods usually consider the behaviors as simple user and item connections, or represent them with a static model. In real world, however, human behaviors are actually complex and dynamic: they include correlations between user and multiple types of objects and also continuously evolve along time. These characteristics cause severe data sparsity and computational complexity problem, which pose great challenge to human behavioral analysis and prediction. In this paper, we propose a Flexible Evolutionary Multifaceted Analysis (FEMA) framework for both behavior prediction and pattern mining. FEMA utilizes a flexible and dynamic factorization scheme
SPLATT: Efficient and Parallel Sparse TensorMatrix Multiplication
"... Abstract—Multidimensional arrays, or tensors, are increasingly found in fields such as signal processing and recommender systems. Realworld tensors can be enormous in size and often very sparse. There is a need for efficient, highperformance tools capable of processing the massive sparse tensors ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract—Multidimensional arrays, or tensors, are increasingly found in fields such as signal processing and recommender systems. Realworld tensors can be enormous in size and often very sparse. There is a need for efficient, highperformance tools capable of processing the massive sparse tensors of today and the future. This paper introduces SPLATT, a C library with sharedmemory parallelism for threemode tensors. SPLATT contains algorithmic improvements over competing state of the art tools for sparse tensor factorization. SPLATT has a fast, parallel method of multiplying a matricized tensor by a KhatriRao product, which is a key kernel in tensor factorization methods. SPLATT uses a novel data structure that exploits the sparsity patterns of tensors. This data structure has a small memory footprint similar to competing methods and allows for the computational improvements featured in our work. We also present a method of finding cachefriendly reorderings and utilizing them with a novel form of cache tiling. To our knowledge, this is the first work to investigate reordering and cache tiling in this context. SPLATT averages almost 30× speedup compared to our baseline when using 16 threads and reaches over 80 × speedup on NELL2. KeywordsSparse tensors, PARAFAC, CANDECOMP, CPD, parallel I.