#### DMCA

## Jacobi algorithm for the best low multilinear rank approximation of symmetric tensors (2013)

Venue: | SIAM JOURNAL ON MATRIX ANALYSIS AND APPLICATIONS |

Citations: | 5 - 1 self |

### Citations

1851 |
Independent component analysis, a new concept?”
- Comon
- 1994
(Show Context)
Citation Context ...69, 65F99 DOI. 10.1137/11085743X 1. Introduction. Higher-order tensors (three-way arrays) have been used as a tool in higher-order statistics [39, 36, 47, 38] and independent component analysis (ICA) =-=[13, 14, 19, 9]-=- for several decades. Other application areas include chemometrics, scientific computing, biomedical signal processing, image processing, and telecommunications. For an exhaustive list and references ... |

723 | Tensor Decompositions and Applications,
- Kolda, Bader
- 2009
(Show Context)
Citation Context ... decades. Other application areas include chemometrics, scientific computing, biomedical signal processing, image processing, and telecommunications. For an exhaustive list and references we refer to =-=[46, 34, 32, 8, 12]-=-. Let us first consider the general low multilinear rank approximation of thirdorder tensors. The problem consists of finding the best approximation of a given tensor A ∈ RI1×I2×I3 , subject to a cons... |

719 | Blind beamforming for non-Gaussian signals.
- Cardoso, Souloumiac
- 1993
(Show Context)
Citation Context ...literature. However, to the best of our knowledge, this direction has never been explored for finding the best low multilinear rank approximation of tensors. Jacobi-based algorithms in the literature =-=[9, 18, 17, 5, 6, 37]-=- are designed to solve problems in the framework of ICA or in relation to PARAFAC/CANDECOMP. The main purpose of the algorithms is simultaneously diagonalizing a set of matrices or approximately diago... |

611 |
Analysis of individual differences in multidimensional scaling via an N-way generalization of ”Eckart-Young” decomposition
- Carroll, Chang
- 1970
(Show Context)
Citation Context ...invariant to permutation of the indices. Symmetric tensors have been studied in [10] in relation to the parallel factor decomposition (PARAFAC) [24], also known as canonical decomposition (CANDECOMP) =-=[7]-=-. The goal there is to decompose a tensor into a (symmetric) sum of outer products of vectors (rank-1 terms). Symmetric tensors naturally appear, for example, when dealing with higher-order statistics... |

555 |
Foundation of the parafac procedure: Model and conditions for an explanatory multi-mode factor analysis
- Harshman
- 1970
(Show Context)
Citation Context ...metric (also called supersymmetric) tensors are tensors invariant to permutation of the indices. Symmetric tensors have been studied in [10] in relation to the parallel factor decomposition (PARAFAC) =-=[24]-=-, also known as canonical decomposition (CANDECOMP) [7]. The goal there is to decompose a tensor into a (symmetric) sum of outer products of vectors (rank-1 terms). Symmetric tensors naturally appear,... |

471 | A multilinear singular value decomposition
- Lathauwer, Moor, et al.
- 2000
(Show Context)
Citation Context ...ion of the best low multilinear rank approximation problem is not known. A generalization of the singular value decomposition (SVD) [23, sect. 2.5] called higher-order SVD (HOSVD) has been studied in =-=[15]-=-. A variation of this decomposition is know as the Tucker decomposition [49, 50]. In general, truncation of the HOSVD leads to a good but not necessarily to the best low multilinear rank approximation... |

428 |
Some mathematical notes on three-mode factor analysis
- Tucker
- 1966
(Show Context)
Citation Context ...eneralization of the singular value decomposition (SVD) [23, sect. 2.5] called higher-order SVD (HOSVD) has been studied in [15]. A variation of this decomposition is know as the Tucker decomposition =-=[49, 50]-=-. In general, truncation of the HOSVD leads to a good but not necessarily to the best low multilinear rank approximation. Recent iterative algorithms solving the problem include geometric Newton [21, ... |

295 | Optimization Algorithms on Matrix Manifolds. - Absil, Mahony, et al. - 2008 |

223 |
Tensor Methods in Statistics.
- McCullagh
- 1987
(Show Context)
Citation Context ...position, Jacobi rotation AMS subject classifications. 15A69, 65F99 DOI. 10.1137/11085743X 1. Introduction. Higher-order tensors (three-way arrays) have been used as a tool in higher-order statistics =-=[39, 36, 47, 38]-=- and independent component analysis (ICA) [13, 14, 19, 9] for several decades. Other application areas include chemometrics, scientific computing, biomedical signal processing, image processing, and t... |

192 | Jacobi angles for simultaneous diagonalization.
- Cardoso, Souloumiac
- 1996
(Show Context)
Citation Context ...literature. However, to the best of our knowledge, this direction has never been explored for finding the best low multilinear rank approximation of tensors. Jacobi-based algorithms in the literature =-=[9, 18, 17, 5, 6, 37]-=- are designed to solve problems in the framework of ICA or in relation to PARAFAC/CANDECOMP. The main purpose of the algorithms is simultaneously diagonalizing a set of matrices or approximately diago... |

133 | Principal component analysis of three-mode data by means of alternating least squares algorithms.
- PM, Leeuw
- 1980
(Show Context)
Citation Context ...@uclouvain.be, http://www.inma.ucl.ac.be/˜absil/). 651 652 MARIYA ISHTEVA, P.-A. ABSIL, AND PAUL VAN DOOREN tensors. The most widely used algorithm is still the one based on alternating least squares =-=[16, 34, 35, 3, 48]-=- because of its simplicity and its satisfying performance. We will refer to it as higher-order orthogonal iteration (HOOI). It is worth mentioning that the objective function associated with the probl... |

116 |
Multi-way Analysis.
- Smilde, Bro, et al.
- 2004
(Show Context)
Citation Context ...s already. Other application areas include chemometrics, scientific computing, biomedical signal processing, image processing and telecommunications. For an exhaustive list and references we refer to =-=[40, 30, 29, 7, 11]-=-. Let us first consider the general low multilinear rank approximation of thirdorder tensors. The problem consists of finding the best approximation of a given tensor A ∈ RI1×I2×I3 , subject to a cons... |

108 |
Higher-Order Spectra Analysis: A Nonlinear Signal Processing Framework, Prentice-Hall,
- Nikias, Petropulu
- 1993
(Show Context)
Citation Context ...position, Jacobi rotation AMS subject classifications. 15A69, 65F99 DOI. 10.1137/11085743X 1. Introduction. Higher-order tensors (three-way arrays) have been used as a tool in higher-order statistics =-=[39, 36, 47, 38]-=- and independent component analysis (ICA) [13, 14, 19, 9] for several decades. Other application areas include chemometrics, scientific computing, biomedical signal processing, image processing, and t... |

107 |
The expression of a tensor or a polyadic as a sum of products
- Hitchcock
- 1927
(Show Context)
Citation Context ...sists of finding the best approximation of a given tensor A ∈ RI1×I2×I3 , subject to a constraint on the multilinear rank of the approximation. The concept of multilinear rank was first introduced in =-=[25, 26]-=- and is simply a generalization of the row and column rank of matrices to higher-order tensors. We define the problem more precisely in the next section. A closed-form solution of the best low multili... |

105 | Computational Methods in Optimization. A Unified Approach, - Polak - 1971 |

104 | Generalized power method for sparse principal component analysis,” - Journee, Nesterov, et al. - 2010 |

99 | Symmetric tensors and symmetric tensor rank,”
- Comon, Golub, et al.
- 2008
(Show Context)
Citation Context ...we deal with symmetric tensors and symmetric approximations. Symmetric (also called supersymmetric) tensors are tensors invariant to permutation of the indices. Symmetric tensors have been studied in =-=[10]-=- in relation to the parallel factor decomposition (PARAFAC) [24], also known as canonical decomposition (CANDECOMP) [7]. The goal there is to decompose a tensor into a (symmetric) sum of outer product... |

84 |
Signal processing with higher-order spectra.
- Nikias, Mendel
- 1993
(Show Context)
Citation Context ...position, Jacobi rotation AMS subject classifications. 15A69, 65F99 DOI. 10.1137/11085743X 1. Introduction. Higher-order tensors (three-way arrays) have been used as a tool in higher-order statistics =-=[39, 36, 47, 38]-=- and independent component analysis (ICA) [13, 14, 19, 9] for several decades. Other application areas include chemometrics, scientific computing, biomedical signal processing, image processing, and t... |

82 | Nonnegative Matrix and Tensor Factorizations,
- Cichocki, Zdunek, et al.
- 2009
(Show Context)
Citation Context ... decades. Other application areas include chemometrics, scientific computing, biomedical signal processing, image processing, and telecommunications. For an exhaustive list and references we refer to =-=[46, 34, 32, 8, 12]-=-. Let us first consider the general low multilinear rank approximation of thirdorder tensors. The problem consists of finding the best approximation of a given tensor A ∈ RI1×I2×I3 , subject to a cons... |

76 | On the Best Rank-1 Approximation of Higher-Order Supersymmetric Tensors
- Kofidis, Regalia
- 2002
(Show Context)
Citation Context ...general case. Algorithms dedicated to the symmetric case are studied to a lesser extent. A symmetric version of HOOI for the special case of rank-1 tensors is mentioned in [16] and further studied in =-=[31, 42, 33]-=-. In [16] the special case of symmetric (2× 2× · · · × 2)-tensors and their rank-1 approximation is studied as well. An algorithm for the general symmetric case, based on the quasi-Newton method, is p... |

76 |
The extension of factor analysis to three-dimensional matrices.
- Tucker
- 1964
(Show Context)
Citation Context ...eneralization of the singular value decomposition (SVD) [23, sect. 2.5] called higher-order SVD (HOSVD) has been studied in [15]. A variation of this decomposition is know as the Tucker decomposition =-=[49, 50]-=-. In general, truncation of the HOSVD leads to a good but not necessarily to the best low multilinear rank approximation. Recent iterative algorithms solving the problem include geometric Newton [21, ... |

72 |
Multiple invariants and generalized rank of a p-way matrix or tensor.
- Hitchcock
- 1927
(Show Context)
Citation Context ...sists of finding the best approximation of a given tensor A ∈ RI1×I2×I3 , subject to a constraint on the multilinear rank of the approximation. The concept of multilinear rank was first introduced in =-=[25, 26]-=- and is simply a generalization of the row and column rank of matrices to higher-order tensors. We define the problem more precisely in the next section. A closed-form solution of the best low multili... |

59 |
Applied Multiway Data Analysis.
- Kroonenberg
- 2008
(Show Context)
Citation Context ... decades. Other application areas include chemometrics, scientific computing, biomedical signal processing, image processing, and telecommunications. For an exhaustive list and references we refer to =-=[46, 34, 32, 8, 12]-=-. Let us first consider the general low multilinear rank approximation of thirdorder tensors. The problem consists of finding the best approximation of a given tensor A ∈ RI1×I2×I3 , subject to a cons... |

55 | Computation of the canonical decomposition by means of a simultaneous generalized Schur decomposition.
- Lathauwer, Moor, et al.
- 2004
(Show Context)
Citation Context ...literature. However, to the best of our knowledge, this direction has never been explored for finding the best low multilinear rank approximation of tensors. Jacobi-based algorithms in the literature =-=[9, 18, 17, 5, 6, 37]-=- are designed to solve problems in the framework of ICA or in relation to PARAFAC/CANDECOMP. The main purpose of the algorithms is simultaneously diagonalizing a set of matrices or approximately diago... |

54 |
On the best rank-1 and rank-(R1,
- LATHAUWER, MOOR, et al.
- 2000
(Show Context)
Citation Context ...@uclouvain.be, http://www.inma.ucl.ac.be/˜absil/). 651 652 MARIYA ISHTEVA, P.-A. ABSIL, AND PAUL VAN DOOREN tensors. The most widely used algorithm is still the one based on alternating least squares =-=[16, 34, 35, 3, 48]-=- because of its simplicity and its satisfying performance. We will refer to it as higher-order orthogonal iteration (HOOI). It is worth mentioning that the objective function associated with the probl... |

41 | Shifted power method for computing tensor eigenpairs. arXiv:1007.1267v1 [math.NA],
- Kolda, Mayo
- 2010
(Show Context)
Citation Context ...general case. Algorithms dedicated to the symmetric case are studied to a lesser extent. A symmetric version of HOOI for the special case of rank-1 tensors is mentioned in [16] and further studied in =-=[31, 42, 33]-=-. In [16] the special case of symmetric (2× 2× · · · × 2)-tensors and their rank-1 approximation is studied as well. An algorithm for the general symmetric case, based on the quasi-Newton method, is p... |

37 | Dimensionality Reduction in Higher-Order Signal Processing and
- Lathauwer, Vandewalle
- 2004
(Show Context)
Citation Context ...69, 65F99 DOI. 10.1137/11085743X 1. Introduction. Higher-order tensors (three-way arrays) have been used as a tool in higher-order statistics [39, 36, 47, 38] and independent component analysis (ICA) =-=[13, 14, 19, 9]-=- for several decades. Other application areas include chemometrics, scientific computing, biomedical signal processing, image processing, and telecommunications. For an exhaustive list and references ... |

35 | A Newton-Grassmann method for computing the best multilinear rank-(r1, r2, r3) approximation of a tensor.
- Eldén, Savas
- 2007
(Show Context)
Citation Context ..., 50]. In general, truncation of the HOSVD leads to a good but not necessarily to the best low multilinear rank approximation. Recent iterative algorithms solving the problem include geometric Newton =-=[21, 29]-=-, quasi-Newton [45], trust-region [27], and particle swarm optimization [4] algorithms. In [44], a Krylov subspace algorithm is proposed for large sparse ∗Received by the editors December 1, 2011; acc... |

28 |
Improving the speed of multi-way algorithms
- Andersson, Bro
- 1998
(Show Context)
Citation Context ...@uclouvain.be, http://www.inma.ucl.ac.be/˜absil/). 651 652 MARIYA ISHTEVA, P.-A. ABSIL, AND PAUL VAN DOOREN tensors. The most widely used algorithm is still the one based on alternating least squares =-=[16, 34, 35, 3, 48]-=- because of its simplicity and its satisfying performance. We will refer to it as higher-order orthogonal iteration (HOOI). It is worth mentioning that the objective function associated with the probl... |

23 | Independent component analysis and (simultaneous) third-order tensor diagonalization - Lathauwer, Moor, et al. - 2001 |

20 | E.: Monotonic convergence of fixed-point algorithms for ICA.
- Regalia, Kofidis
- 2003
(Show Context)
Citation Context ... the one presented in [41]. The complexity of the two algorithms is comparable. The algorithm of [41] relates to steepest descent and can be interpreted as a generalized power method [30, sect. 3.4], =-=[43]-=-, whereas the proposed Jacobi algorithm is akin to a coordinate search method. Steepest descent tends to converge faster than coordinate search in terms of the number of iterations. However, the cost ... |

19 | An introduction to independent component analysis
- Lathauwer, Moor, et al.
(Show Context)
Citation Context ...69, 65F99 DOI. 10.1137/11085743X 1. Introduction. Higher-order tensors (three-way arrays) have been used as a tool in higher-order statistics [39, 36, 47, 38] and independent component analysis (ICA) =-=[13, 14, 19, 9]-=- for several decades. Other application areas include chemometrics, scientific computing, biomedical signal processing, image processing, and telecommunications. For an exhaustive list and references ... |

19 |
Tensor-based techniques for the blind separation of DSCDMA signals.
- Casting, Lathauwer
- 2007
(Show Context)
Citation Context ...ssifications. 15A69, 65F99 1. Introduction. Higher-order tensors (three-way arrays) have been used as a tool in higher-order statistics (HOS) [35, 32, 41, 34] and independent component analysis (ICA) =-=[12, 13, 18, 8]-=- for several decades already. Other application areas include chemometrics, scientific computing, biomedical signal processing, image processing and telecommunications. For an exhaustive list and refe... |

18 | Quasi-Newton methods on Grassmannians and multilinear approximations of tensors,
- Savas, Lim
- 2010
(Show Context)
Citation Context ...cation of the HOSVD leads to a good but not necessarily to the best low multilinear rank approximation. Recent iterative algorithms solving the problem include geometric Newton [21, 29], quasi-Newton =-=[45]-=-, trust-region [27], and particle swarm optimization [4] algorithms. In [44], a Krylov subspace algorithm is proposed for large sparse ∗Received by the editors December 1, 2011; accepted for publicati... |

12 |
rank one approximation of real symmetric tensors can be chosen symmetric, Front
- Best
- 2013
(Show Context)
Citation Context ...The solution above is not a saddle point but a (local or global) minimum of F in (2.1). Whether it is possible to construct similar examples for (super)symmetric tensors remains an open question; see =-=[22]-=-. However, the above example indicates that it is not trivial to assume that the best rank-(R,R,R) approximation of a symmetric tensor would be symmetric. Note also that in applications involving symm... |

12 |
Some additional results on principal components analysis of three-mode data by means of alternating least squares algorithms
- Berge, Leeuw, et al.
- 1987
(Show Context)
Citation Context |

11 |
The higher-order power method revisited: convergence proofs and effective initialization
- PA, Kofidis
(Show Context)
Citation Context ...f HOOI where the symmetry is preserved at each step has convergence problems in several cases. Fixes for this problem have been proposed in the literature for the case of rank-(1, 1, 1) approximation =-=[31, 42, 33]-=-. Our algorithm on the other hand has been designed for the general case of rank-(R,R,R) approximations. Another benefit of the proposed algorithm is its convergence behavior. We have proved that it c... |

9 | Accelerated line-search and trust-region methods. - Absil, Gallivan - 2009 |

9 |
First-order perturbation analysis of the best rank-(R1, R2, R3) approximation in multilinear algebra
- Lathauwer
(Show Context)
Citation Context ...metric algorithms. The convergence proof of the algorithm will be given in the next section. 662 MARIYA ISHTEVA, P.-A. ABSIL, AND PAUL VAN DOOREN 4.1. Partial symmetry. Consider the following example =-=[11]-=-: A = a ◦ b ◦ c+ b ◦ c ◦ a+ c ◦ a ◦ b, where ◦ stands for the outer product of vectors and a,b, c ∈ Rn have unit norm and are orthogonal to each other. We have A(i, j, k) = A(j, k, i) = A(k, i, j), i.... |

9 |
Huffel, Differential-geometric newton method for the best rank-(r1, r2, r3) approximation of tensors, Numerical Algorithms
- Ishteva, Lathauwer, et al.
(Show Context)
Citation Context ..., 50]. In general, truncation of the HOSVD leads to a good but not necessarily to the best low multilinear rank approximation. Recent iterative algorithms solving the problem include geometric Newton =-=[21, 29]-=-, quasi-Newton [45], trust-region [27], and particle swarm optimization [4] algorithms. In [44], a Krylov subspace algorithm is proposed for large sparse ∗Received by the editors December 1, 2011; acc... |

9 | A Jacobi-type method for computing orthogonal tensor decompositions
- Martin, Loan
(Show Context)
Citation Context |

8 |
A survey of tensor methods
- Lathauwer
- 2009
(Show Context)
Citation Context |

8 | Krylov subspace methods for tensor computations
- Savas, Elden
- 2009
(Show Context)
Citation Context ...tilinear rank approximation. Recent iterative algorithms solving the problem include geometric Newton [19, 27], quasi-Newton [39], trust-region [25] and particle swarm optimization [3] algorithms. In =-=[38]-=-, a Krylov subspace algorithm is proposed for large sparse tensors. The most widely used algorithm is still the one based on alternating least squares [15, 30, 31, 2, 42] because of its simplicity and... |

4 |
Tensor-based techniques for the blind separation
- Lathauwer, Castaing
(Show Context)
Citation Context |

4 | L.: Krylov-type methods for tensor computations
- Savas, Eldén
- 2013
(Show Context)
Citation Context ...ilinear rank approximation. Recent iterative algorithms solving the problem include geometric Newton [21, 29], quasi-Newton [45], trust-region [27], and particle swarm optimization [4] algorithms. In =-=[44]-=-, a Krylov subspace algorithm is proposed for large sparse ∗Received by the editors December 1, 2011; accepted for publication (in revised form) by T. G. Kolda February 28, 2013; published electronica... |

3 |
low multilinear rank approximation of higher-order tensors, based on the Riemannian trust-region scheme
- Ishteva, Absil, et al.
(Show Context)
Citation Context ... leads to a good but not necessarily to the best low multilinear rank approximation. Recent iterative algorithms solving the problem include geometric Newton [21, 29], quasi-Newton [45], trust-region =-=[27]-=-, and particle swarm optimization [4] algorithms. In [44], a Krylov subspace algorithm is proposed for large sparse ∗Received by the editors December 1, 2011; accepted for publication (in revised form... |

3 |
Monotonically convergent algorithms for symmetric tensor approximation. Linear Algebra and its
- Regalia
(Show Context)
Citation Context ... algorithm for the general symmetric case, based on the quasi-Newton method, is presented in [45]. Recently, an algorithm exploiting the gradient inequality of convex functionals has been proposed in =-=[41]-=-. We develop an algorithm for symmetric tensors, based on Jacobi rotations. The symmetry is preserved at each iteration. The main subproblem reduces to maximizing a polynomial of degree six (or of deg... |

2 | A modified particle swarm optimization algorithm for the best low multilinear rank approximation of higher-order tensors.
- Borckmans, Ishteva, et al.
- 2010
(Show Context)
Citation Context ...o the best low multilinear rank approximation. Recent iterative algorithms solving the problem include geometric Newton [21, 29], quasi-Newton [45], trust-region [27], and particle swarm optimization =-=[4]-=- algorithms. In [44], a Krylov subspace algorithm is proposed for large sparse ∗Received by the editors December 1, 2011; accepted for publication (in revised form) by T. G. Kolda February 28, 2013; p... |

2 |
J.M.F.: The assumption of proportional components when CANDECOMP is applied to symmetric matrices
- Dosse, Berge
(Show Context)
Citation Context ...009−0.1380 0.0728 0.1659 0.0612 −0.0322 −0.0735 ⎞ ⎠ , Â(:, :, 3) = ⎛ ⎝ 0.0017 −0.0009 −0.0020−0.3061 0.1614 0.3679 0.1356 −0.0715 −0.1630 ⎞ ⎠ . As can be seen, the partial symmetry has been lost. In =-=[20]-=- the authors consider another type of partial symmetry A(i, j, k) = A(j, i, k), i.e, symmetry with respect to the first and second mode of the tensor. They show that rank-1 CANDECOMP approximation usu... |

2 |
Tucker compression and local optima.
- Ishteva, Absil, et al.
- 2010
(Show Context)
Citation Context ...isfying performance. We will refer to it as higher-order orthogonal iteration (HOOI). It is worth mentioning that the objective function associated with the problem may have several stationary points =-=[28]-=- and none of the iterative algorithms is guaranteed to converge to the global optimum. Most of the algorithms, however, converge to local optima. In this paper, we deal with symmetric tensors and symm... |

2 | component analysis and (simultaneous) third-order tensor diagonalization - Independent |

1 |
compression and local optima
- Tucker
(Show Context)
Citation Context ...Louvain, Bâtiment Euler, Av. Georges Lemâıtre 4, B-1348 Louvain-la-Neuve, Belgium (http://www.inma.ucl.ac.be/~absil/, paul.vandooren@uclouvain.be). 1 2 M. ISHTEVA, P.-A. ABSIL, P. VAN DOOREN points =-=[26]-=- and none of the iterative algorithms is guaranteed to converge to the global optimum. Most of the algorithms however converge to local optima. In this paper, we deal with symmetric tensors and symmet... |