DMCA
Scalable tensor decompositions for multi-aspect data mining (2008)
| Venue: | In ICDM 2008: Proceedings of the 8th IEEE International Conference on Data Mining |
| Citations: | 63 - 2 self |
BibTeX
@INPROCEEDINGS{Kolda08scalabletensor,
author = {Tamara G. Kolda},
title = {Scalable tensor decompositions for multi-aspect data mining},
booktitle = {In ICDM 2008: Proceedings of the 8th IEEE International Conference on Data Mining},
year = {2008},
pages = {363--372}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Modern applications such as Internet traffic, telecommunication records, and large-scale social networks generate massive amounts of data with multiple aspects and high dimensionalities. Tensors (i.e., multi-way arrays) provide a natural representation for such data. Consequently, tensor decompositions such as Tucker become important tools for summarization and analysis. One major challenge is how to deal with highdimensional, sparse data. In other words, how do we compute decompositions of tensors where most of the entries of the tensor are zero. Specialized techniques are needed for computing the Tucker decompositions for sparse tensors because standard algorithms do not account for the sparsity of the data. As a result, a surprising phenomenon is observed by practitioners: Despite the fact that there is enough memory to store both the input tensors and the factorized output tensors, memory overflows occur during the tensor factorization process. To address this intermediate blowup problem, we propose Memory-Efficient Tucker (MET). Based on the available memory, MET adaptively selects the right execution strategy during the decomposition. We provide quantitative and qualitative evaluation of MET on real tensors. It achieves over 1000X space reduction without sacrificing speed; it also allows us to work with much larger tensors that were too big to handle before. Finally, we demonstrate a data mining case-study using MET. 1
Keyphrases
multi-aspect data mining scalable tensor decomposition output tensor space reduction right execution strategy natural representation multi-way array telecommunication record data mining case-study high dimensionality internet traffic modern application standard algorithm input tensor intermediate blowup problem large-scale social network qualitative evaluation massive amount tensor decomposition major challenge enough memory available memory memory overflow sparse data real tensor tensor factorization process multiple aspect surprising phenomenon important tool tucker decomposition memory-efficient tucker sparse tensor
