Results 1  10
of
10
Speedup Matrix Completion with Side Information: Application to MultiLabel Learning
"... In standard matrix completion theory, it is required to have at least O(n ln2 n) observed entries to perfectly recover a lowrank matrixM of size n × n, leading to a large number of observations when n is large. In many real tasks, side information in addition to the observed entries is often avai ..."
Abstract

Cited by 7 (2 self)
 Add to MetaCart
(Show Context)
In standard matrix completion theory, it is required to have at least O(n ln2 n) observed entries to perfectly recover a lowrank matrixM of size n × n, leading to a large number of observations when n is large. In many real tasks, side information in addition to the observed entries is often available. In this work, we develop a novel theory of matrix completion that explicitly explore the side information to reduce the requirement on the number of observed entries. We show that, under appropriate conditions, with the assistance of side information matrices, the number of observed entries needed for a perfect recovery of matrixM can be dramatically reduced to O(lnn). We demonstrate the effectiveness of the proposed approach for matrix completion in transductive incomplete multilabel learning. 1
Inferring Users ’ Preferences from Crowdsourced Pairwise Comparisons: A Matrix Completion Approach
"... Inferring user preferences over a set of items is an important problem that has found numerous applications. This work focuses on the scenario where the explicit feature representation of items is unavailable, a setup that is similar to collaborative filtering. In order to learn a user’s preference ..."
Abstract

Cited by 7 (0 self)
 Add to MetaCart
(Show Context)
Inferring user preferences over a set of items is an important problem that has found numerous applications. This work focuses on the scenario where the explicit feature representation of items is unavailable, a setup that is similar to collaborative filtering. In order to learn a user’s preferences from his/her response to only a small number of pairwise comparisons, we propose to leverage the pairwise comparisons made by many crowd users, a problem we refer to as crowdranking. The proposed crowdranking framework is based on the theory of matrix completion, and we present efficient algorithms for solving the related optimization problem. Our theoretical analysis shows that, on average, only O(r logm) pairwise queries are needed to accurately recover the ranking list of m items for the target user, where r is the rank of the unknown rating matrix, r m. Our empirical study with two realworld benchmark datasets for collaborative filtering and one crowdranking dataset we collected via Amazon Mechanical Turk shows the promising performance of the proposed algorithm compared to the stateoftheart approaches.
Semisupervised Clustering by Input Pattern Assisted Pairwise Similarity Matrix Completion
"... Many semisupervised clustering algorithms have been proposed to improve the clustering accuracy by effectively exploring the available side information that is usually in the form of pairwise constraints. However, there are two main shortcomings of the existing semisupervised clustering algorithms ..."
Abstract

Cited by 6 (2 self)
 Add to MetaCart
(Show Context)
Many semisupervised clustering algorithms have been proposed to improve the clustering accuracy by effectively exploring the available side information that is usually in the form of pairwise constraints. However, there are two main shortcomings of the existing semisupervised clustering algorithms. First, they have to deal with nonconvex optimization problems, leading to clustering results that are sensitive to the initialization. Second, none of these algorithms is equipped with theoretical guarantee regarding the clustering performance. We address these limitations by developing a framework for semisupervised clustering based on input pattern assisted matrix completion. The key idea is to cast clustering into a matrix completion problem, and solve it efficiently by exploiting the correlation between input patterns and cluster assignments. Our analysis shows that under appropriate conditions, only O(log n) pairwise constraints are needed to accurately recover the true cluster partition. We verify the effectiveness of the proposed algorithm by comparing it to the stateoftheart semisupervised clustering algorithms on several benchmark datasets. 1.
Online lowrank subspace clustering by basis dictionary pursuit. arXiv preprint arXiv:1503.08356,
, 2015
"... Abstract LowRank Representation (LRR) has been a significant method for segmenting data that are generated from a union of subspaces. It is also known that solving LRR is challenging in terms of time complexity and memory footprint, in that the size of the nuclear norm regularized matrix is nbyn ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
(Show Context)
Abstract LowRank Representation (LRR) has been a significant method for segmenting data that are generated from a union of subspaces. It is also known that solving LRR is challenging in terms of time complexity and memory footprint, in that the size of the nuclear norm regularized matrix is nbyn (where n is the number of samples). In this paper, we thereby develop a novel online implementation of LRR that reduces the memory cost from O(n 2 ) to O(pd), with p being the ambient dimension and d being some estimated rank (d < p ≪ n). We also establish the theoretical guarantee that the sequence of solutions produced by our algorithm converges to a stationary point of the expected loss function asymptotically. Extensive experiments on synthetic and realistic datasets further substantiate that our algorithm is fast, robust and memory efficient.
Stochastic Optimization for Kernel PCA∗
"... Kernel Principal Component Analysis (PCA) is a popular extension of PCA which is able to find nonlinear patterns from data. However, the application of kernel PCA to largescale problems remains a big challenge, due to its quadratic space complexity and cubic time complexity in the number of examp ..."
Abstract
 Add to MetaCart
Kernel Principal Component Analysis (PCA) is a popular extension of PCA which is able to find nonlinear patterns from data. However, the application of kernel PCA to largescale problems remains a big challenge, due to its quadratic space complexity and cubic time complexity in the number of examples. To address this limitation, we utilize techniques from stochastic optimization to solve kernel PCA with linear space and time complexities per iteration. Specifically, we formulate it as a stochastic composite optimization problem, where a nuclear norm regularizer is introduced to promote lowrankness, and then develop a simple algorithm based on stochastic proximal gradient descent. During the optimization process, the proposed algorithm always maintains a lowrank factorization of iterates that can be conveniently held in memory. Compared to previous iterative approaches, a remarkable property of our algorithm is that it is equipped with an explicit rate of convergence. Theoretical analysis shows that the solution of our algorithm converges to the optimal one at an O(1/T) rate, where T is the number of iterations.
unknown title
"... Nuclear norm regularization has been shown very promising for pursing a low rank solution for matrix variable in various machine learning problems. Many efforts have been devoted to develop efficient algorithms for solving the optimization problem in nuclear norm regularization. Solving the problem ..."
Abstract
 Add to MetaCart
(Show Context)
Nuclear norm regularization has been shown very promising for pursing a low rank solution for matrix variable in various machine learning problems. Many efforts have been devoted to develop efficient algorithms for solving the optimization problem in nuclear norm regularization. Solving the problem for largescale matrix variables, however, is still a challenging task since the complexity grows fast with the size of matrix variable. In this work, we propose a novel method called safe subspace screening (SSS), to improve the efficiency of the solver for nuclear norm regularized least squares problems. Motivated by the fact that the low rank solution can be represented by a few subspaces, the proposed method accurately discards a predominant percentage of inactive subspaces prior to solving the problem to reduce problem size. Consequently, a much smaller problem is required to solve, making it more efficient than optimizing the original problem. The proposed SSS is safe, in that its solution is identical to the solution from the solver. In addition, the proposed SSS can be used together with any existing nuclear norm solver since it is independent of the solver. We have evaluated the proposed SSS on several synthetic as well as real data sets. Extensive results show that the
4 Structured LowRank Matrix Factorization with Missing and Grossly Corrupted Observations
"... ar ..."
(Show Context)
Nuclear Norm Regularized Least Squares Optimization on Grassmannian Manifolds
"... This paper aims to address a class of nuclear norm regularized least square (NNLS) problems. By exploiting the underlying lowrank matrix manifold structure, the problem with nuclear norm regularization is cast to a Riemannian optimization problem over matrix manifolds. Compared with existing NNLS ..."
Abstract
 Add to MetaCart
(Show Context)
This paper aims to address a class of nuclear norm regularized least square (NNLS) problems. By exploiting the underlying lowrank matrix manifold structure, the problem with nuclear norm regularization is cast to a Riemannian optimization problem over matrix manifolds. Compared with existing NNLS algorithms involving singular value decomposition (SVD) of largescale matrices, our method achieves significant reduction in computational complexity. Moreover, the uniqueness of matrix factorization can be guaranteed by our Grassmannian manifold method. In our solution, we first introduce the bilateral factorization into the original NNLS problem and convert it into a Grassmannian optimization problem by using a linearized technique. Then the conjugate gradient procedure on the Grassmannian manifold is developed for our method with a guarantee of local convergence. Finally, our method can be extended to address the graph regularized problem. Experimental results verified both the efficiency and effectiveness of our method. 1
ORTHOGONAL RANKONE MATRIX PURSUIT FOR MATRIX COMPLETION
"... Abstract. In this paper, we propose an efficient and scalable low rank matrix completion algorithm. The key idea is to extend orthogonal matching pursuit method from the vector case to the matrix case. We further propose an economic version of our algorithm by introducing a novel weight updating rul ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract. In this paper, we propose an efficient and scalable low rank matrix completion algorithm. The key idea is to extend orthogonal matching pursuit method from the vector case to the matrix case. We further propose an economic version of our algorithm by introducing a novel weight updating rule to reduce the time and storage complexity. Both versions are computationally inexpensive for each matrix pursuit iteration, and find satisfactory results in a few iterations. Another advantage of our proposed algorithm is that it has only one tunable parameter, which is the rank. It is easy to understand and to use by the user. This becomes especially important in largescale learning problems. In addition, we rigorously show that both versions achieve a linear convergence rate, which is significantly better than the previous known results. We also empirically compare the proposed algorithms with several stateoftheart matrix completion algorithms on many realworld datasets, including the largescale recommendation dataset Netflix as well as the MovieLens datasets. Numerical results show that our proposed algorithm is more efficient than competing algorithms while achieving similar or better prediction performance. Key words. Low rank, singular value decomposition, rank minimization, matrix completion, matching pursuit 1. Introduction. Recently
Efficient LowRank Stochastic Gradient Descent Methods for Solving Semidefinite Programs
, 2014
"... We propose a lowrank stochastic gradient descent (LRSGD) method for solving a class of semidefinite programming (SDP) problems. LRSGD has clear computational advantages over the standard SGD peers as its iterative projection step (a SDP problem) can be solved in an efficient manner. Specifically ..."
Abstract
 Add to MetaCart
(Show Context)
We propose a lowrank stochastic gradient descent (LRSGD) method for solving a class of semidefinite programming (SDP) problems. LRSGD has clear computational advantages over the standard SGD peers as its iterative projection step (a SDP problem) can be solved in an efficient manner. Specifically, LRSGD constructs a lowrank stochastic gradient and computes an optimal solution to the projection step via analyzing the lowrank structure of its stochastic gradient. Moreover, our theoretical analysis shows the universal existence of arbitrary lowrank stochastic gradients which in turn validates the rationale of the LRSGD method. Since LRSGD is a SGD based method, it achieves the optimal convergence rates of the standard SGD methods. The presented experimental results demonstrate the efficiency and effectiveness of the LRSGD method.