R. W. Brockett, Dynamical systems that sort lists, diagonalize matrices and solve linear programming problems, in Proceedings of the 27th IEEE Conference on Decis ion and Control, IEEE, (  Home/Search   Document Not in Database   Summary   Related Articles   Check

....point of the double bracket equations. We introduce di#erent types of optimality and prove local and global optimality results with respect to the Schatten p norms. 1. Introduction Double bracket flows were introduced simultaneously by Brockett in the context of control theory and optimization [Bro91] and by Chu and Driessel in connection with the computation of certain problems in numerical linear algebra [CD91] Let Sym(n) denote the set of real, n n symmetric matrices. A double bracket flow (DBF) is the matrix di#erential equation Y # = N , Y ] Y ] t where N Sym(n) and ....

....(1.3) is the key to practical computation of the solution of (1.2) whilst respecting its invariants [CIZ97, Ise02] The second feature of the DBF (1. 1) is that it is a gradient system, with a global Lyapunov function, therefore it is assured of convergence to a fixed point of the flow as t [Bro91]. More precisely, as shown in [BBR92] BFR90] it is a gradient The work of AMB was supported in part by the National Science Foundation. flow with respect to the so called normal metric on an appropriate adjoint orbit of a compact Lie group. This feature, of critical importance in applications, ....

[Article contains additional citation context not shown here]

R. W. Brockett, Dynamical systems that sort lists, diagonalize matrices, and solve linear programming problems, Linear Algebra Appl. 146 (1991), 79--91.

....able to perform MSA; however, in [26] is has been shown that with j 0 the learning equation results to be unstable, as the norm of each column vector of W tends to diverge with time. 3. 2 Chen Amari Lin (CAL) and Douglas Kung Amari (DKA) algorithms Following previous works of Brockett and Xu [3, 25], Chen et al. proposed to optimize a criterion function similar to (7) by means of a natural gradient based learning algorithm; the criterion may simply be the Rayleigh quotient. To explain their idea, it is worth defining the new matrix W = WjW 1 ] 2 R , where W 1 2 R ....

R.W. Brockett, Dynamical systems that sort lists, diagonalize matrices and solve linear programming problems, Linear Algebra and Its Applications, Vol. 146, pp. 79 -- 91, 1991

.... discussed in [23, 26] The notion and computational properties of the generalized gradient are throughly studied in nonsmooth analysis [9] In particular, tools for establishing stability and convergence properties of nonsmooth dynamical systems are presented in [3, 15, 27] Finally, we refer to [7, 17] for guidelines on how to design dynamical systems for optimization purposes, and to [4] for gradient descent flows in distributed computation in settings with fixed communication topologies. Recent years have witnessed a large research e#ort focused on motion planning and formation control ....

R. W. Brockett, Dynamical systems that sort lists, diagonalize matrices, and solve linear programming problems, Linear Algebra and its Applications, 146 (1991), pp. 79--91.

....real matrices with prescribed singular values. 1991 Mathematics Subject Classification. Primary 58F25, 65K10. Key words and phrases. reductive groups, gradient flows, orthogonal projections, double bracket equations, least squares optimizations. Typeset by A M S T E X 1 2 TIN YAU TAM Brockett [B1] independently studied Problem 1 and later studied Problem 1 and Problem 3 in the context of compact connected semisimple Lie groups. The main results in [CD] and [B1] for Problem 1 are the gradient flows of the functions Phi : O(n) R; k 7 1 2 kkx 0 k Gamma1 Gamma zk 2 ; and : O(x 0 ....

....orthogonal projections, double bracket equations, least squares optimizations. Typeset by A M S T E X 1 2 TIN YAU TAM Brockett [B1] independently studied Problem 1 and later studied Problem 1 and Problem 3 in the context of compact connected semisimple Lie groups. The main results in [CD] and [B1] for Problem 1 are the gradient flows of the functions Phi : O(n) R; k 7 1 2 kkx 0 k Gamma1 Gamma zk 2 ; and : O(x 0 ) R; x 7 1 2 kx Gamma zk 2 : The gradient flow of with respect to the normal metric (see the next section for the definition) on the tangent space of O(x 0 ....

[Article contains additional citation context not shown here]

R.W. Brockett, Dynamical system that sort lists, diagonalize matrices and solving least squares programming problems, Linear Algebra Appl. 146 (1991), 79--91.

....are established and discussed. Particularly, relations with decorrelation separation powerful theory developed by Moreau and Pesquet [13] are briefly analyzed. Then, relationships with Laheld and Cardoso [12] on Blind Source Separation and with the isospectral flow system theory by Brockett [2] are briefly investigated. 5.1 Relations with the Comon Moreau Pesquet contrast semicontrast theory The usefulness of the above approach has been experimentally proved in [7, 8] for solving PCA and ICA problems. Strictly speaking, in order to perform decorrelation a function U = Gamma Gamma ....

....i . It is worth to note that for system (14) 15) conditions (17) are fulfilled when the components of y are statistically independent, indeed in this case E[ rU )y T ] is diagonal, thus Phi is symmetric and Omega = 0. 5.3. Relations with the Brockett s isospectral flow system theory In [2] Brockett discusses a dynamical system that provides a very useful solution to some optimum searching problems, as Linear Programming Problems where equality as well as inequality constraints are involved. Here we try to interpret his system within the SOC class. Brockett s system may be ....

[Article contains additional citation context not shown here]

R.W. BROCKETT, Dynamical Systems that Sort Lists, Diagonalize Matrices and Solve Linear Programming Problems, Linear Algebra and Its Applications, Vol. 146, 1991, pp. 79 -- 91

....relies on orthogonal matrices, which led us to turn to Riemannian gradient flows in the special orthogonal group, that is the group of rotation matrices. Such flows have many different applications in the field of neural networks [DV93] signal processing [CAL98, BA99] or analog computation [Bro88, Won95]. To the best of our knowledge, they have not been applied to continuous function optimization. The paper is organized as follows. Section 2 introduces a general framework for statistical optimization of real functions. We give the definition of statistical criteria to be optimized and we consider ....

.... we express B as a trace B = 1 2 kzk 2 = 1 2 Tr zz t = 1 2 Tr M t (x m) x m) t M = 1 2 Tr (x m) x m) t M 2 M t ; where we have used Tr(RS) Tr(SR) We identify the last expression with the energy function from which Brockett derives isospectral matrix flows [Bro88]. The gradient of B at M can be computed from the difference B(N) B(M) for N 2 SO(n) in the neighborhood of M . Let A(n) be the space of n n antisymmetric matrices, that is matrices A such that A t = A. The matrix Me A , A 2 A(n) is in the neighborhood of M in SO(n) for A in the ....

R.W. Brockett. Dynamical systems that sort lists, diagonalize matrices and solve linear programming problems. In Proceedings of the 27th conference on decision and control, pages 799--803, Austin, Texas, December 1988.

....taken equal to P i y 4 i . It is worth to note that for system (3) 4) equilibrium conditions are fulfilled when the components of y are statistically independent, indeed in this case E[ rU )y T ] is diagonal, thus Phi is symmetric and G = 0. 4.2. Relationships with Brockett s theory In [3] Brockett discusses a dynamical system that provides a very elegant solution to some optimization questions, as Linear Programming Problems where equality as well as inequality constraints are involved. Here we try to interpret his system within the SOC class. Brockett s system may be represented ....

....W H QWN Delta : 11) Clearly, system (11) is of the form (10) where a control matrix Phi is assumed such that: ae Z t 0 Phi( d = GammaN W H QW Sigma ; and if aeoe = 1. Of course it should be G(0) 0. Here Sigma denotes whatever Hermite symmetric matrix in R p Thetap . In [3] it is shown that system (11) tries to maximize the objective function U (W ) tr(W H QWN ) under the constraint of orthonormality. For knowing details about the behavior of the system (11) readers please refer to [3] 4.3. Relationships with Barlow s sparse coding theory The usefulness of ....

[Article contains additional citation context not shown here]

R.W. BROCKETT, Dynamical Systems that Sort Lists, Diagonalize Matrices and Solve Linear programming Problems, Linear Algebra, Vol. 146, pp. 79 -- 91, 1991

.... decomposition, and minor component analysis (see [23] and references therein) Blind source separation by signal pre whitening [15, 14, 5] Denoising by sparse coding [20] Direction of arrival estimation [15] Best basis search (see [7] and references therein) Linear programming problem solving [4]; Optical character recognition by transformation invariant neuron [21] Fault detection [17] Moreover, the eigenvalue decomposition theorem or, more generally, the singular value decomposition theorem, are widely known mathematical tools that allow recasting any problem into a pair of ....

R.W. BROCKETT, Dynamical Systems that Sort Lists, Diagonalize Matrices and Solve Linear programming Problems, Linear Algebra and Its Applications, Vol. 146, pp. 79 -- 91, 1991

....number of applications. Examples include the motion of a lattice of particles under near neighbour exponential interaction (Toda flows) the interaction of two motions of a lattice (Kac van Moerbeke flows) applications to a range of problems in numerical algebra [8] and in linear programming [3] etc. The retention of the isospectral invariant under discretization is often of crucial importance: an extreme case is when specific isospectral flows (e.g. QR flows or double bracket flows [3, 8] are used to calculate eigenvalues of L 0 or to compute inverse eigenvalue problems [8] Let fLng ....

....Moerbeke flows) applications to a range of problems in numerical algebra [8] and in linear programming [3] etc. The retention of the isospectral invariant under discretization is often of crucial importance: an extreme case is when specific isospectral flows (e.g. QR flows or double bracket flows [3, 8]) are used to calculate eigenvalues of L 0 or to compute inverse eigenvalue problems [8] Let fLng n2Z be a sequence of approximants to the solution of (5:1) at the points fnhg n2Z and denote the eigenvalues of Ln by f n; g d =1 . In an appropriate ordering, isospectrality is tantamount ....

R.W. Brockett (1991). Dynamical systems that sort lists, diagonalize matrices and solve linear programming problems, Lin. Alg. Applcs 146, 79--91.

....of an arbitrary matrix is obtained. In this paper an algorithm for block diagonalizing a given matrix by orthogonal transformations is derived. This is achieved by a problem adapted discretization process of an equivalent dynamical system. The matrix flow is Brockett s double bracket flow [1] H = H; H; N ] choosing the matrix N appropriately with respect to the subspace separation problem. Essentially, after discretization, this results in a Jacobi type method which only works on the cross terms H 12 2 IR d Thetam Gammad (assuming a preparatory QRD) Therefore, only d(m ....

R. W. Brockett. Dynamical systems that sort lists, diagonalize matrices, and solve linear programming problems. Lin. Algebra & Applic., 146:79--91, 1991.

....and can be very complicated. A similar view of the process of computation as a flow from an initial condition to an attractor has been taken by a number of researchers. Brockett introduced a set of ODE s that perform various tasks such as sorting and solving linear programming problems [13]. In the comprehensive book of Helmke and Moore [14] one can find numerous other applications and references, among them, the state of the art in dynamical systems for linear programming. The Hopfield neural network is of particular interest because it is easily related to other NP complete ....

....systems. This philosophy can be applied to solve concrete problems in an efficient manner. In [9] we analyze the dynamical system for linear programming introduced by Faybusovich [47] and show that it can be interpreted as an optimal algorithm. This system is related to Brockett s bracket flows [13]. Numerous other problems solvable by dynamical systems are shown in the book of Helmke and Moore [14] Another line of work regarded the so called general purpose analog computer was dominated by Shannon [48] Pour el [49] and Rubel [50 52] Despite the similar title, that analog computer is ....

R. W. Brockett. Dynamical systems that sort lists, diagonalize matrices and solve linear programming problems. Linear Algebra and Its Applications, 146:79--91, 1991.

....sections learns the subspace spanned by the K principal components. In other words, such a fully symmetrical network can learn only a rotated basis of the principal component subspace. In order to extract true principal components, an asymmetry must be introduced in the learning algorithm ([10, 11, 2, 12]) Almost all known PCA algorithms related to our improved PSA algorithm can be presented in compact unified form as follows: P 0 = E[kxk 02 ]I K ; W 0 = I K x k = x k 0 x1 (optional deflation) y k = W k x k z k = P k01 y k P k = P k01 0 z k z T k = 1 z T k y k ) 1W k = P k n y ....

....[10] fl ij = 1 if i j, fl ij = 0 otherwise; 2. Oja s Stochastic Gradient Ascent (SGA) 2] fl ij = 1 if i = j, fl ij = 2 if i j, fl ij = 0 otherwise; 3. Oja s Ogawa Wangiviwattana Weighted Subspace Algorithm (WSA) 2] fl ij = i for i = 1; K; 4. Brocket s Subspace Algorithm (BSA) [11]: fl ij = K 1 0 j) K 1 0 i) for i; j = 1; K; Due to the limit of space our computer experiments results are limited only to the improved PSA algorithm (10) 5. Experimental results We have conducted our experiments on several standard natural images which were converted into ....

Brockett R.W., Dynamical systems that sort lists, diagonalize matrices and solve linear programming problems, Linear Algebra Applications, 146, 1991, 79-91.

....of matrix algebra problems. Feedforward and recurrent neural networks have been developed for solving systems of linear algebraic equations [2, 13, 14, 15, 17, 20] eigenvalue and eigenvector [3, 12] LU decomposition and Cholesky factorization [18] and a variety of other matrix algebra problems [1, 4, 21]. Specifically, nonlinear and linear recurrent neural networks [6, 10, 16, 17] have been developed for the inversion of square and full rank rectangular matrices. The results of these investigations have demonstrated the feasibility and potential of using neural networks for Moore Penrose ....

R. W. BROCKETT, Dynamical systems that sort lists, diagonalize matrices, and solve linear programming problems, Linear Algebra Appl., 146 (1991), pp. 79--91.

....output power Omega kyk 2 ff = P i Omega y 2 i ff = trace(A Sigma xx A 0 ) over A = fA : AA 0 = Ig. Observe that by our results in Section II.A, we already know that all such A are of the form CU 0 p with C orthogonal, and hence result in PCA analyzers. As proposed in Brockett [42], this goal can also be accomplished by constrained gradient ascent on the average output power. More generally, consider the weighted average power E(A) P i i Omega y 2 i ff . Notice that if 1 Delta Delta Delta p 0, then E(A) is maximized over A iff A = Sigmau 1 ; ....

....p ] 0 . By this choice for the weights, the rows of A are forced to be the first p eigenvectors of Sigma xx rather than an orthonormal set of linear combinations of them. Writing Theta = diag( 1 ; p ) we have E(A) htrace( Thetayy 0 )i = trace( ThetaA Sigma xx A 0 ) Brockett [42] shows that the gradient flow of E on A (with respect to the canonical Frobenius norm based metric on A) is given by A = ThetaA Sigma xx Gamma A Sigma xx A 0 ThetaA: 27) In fact, Brockett only considers the special case p = n, but the analysis for the case p n is virtually identical, ....

Brockett, R. W. (1991). Dynamical systems that sort lists, diagonalize matrices and solve linear programmingproblems. Linear Algebra ant its Applications, 146, 79--91.

....equations on an operator Lie algebra which take the socalled double bracket form dL doe = L; L; M ] The study of differential equations in the double bracket form on finite dimensional Lie algebras was initiated in [2] 44 CHAPTER III. SPECTRAL THEORY OF FOKKER PLANCK OPERATORS and [11] in connection with integrable gradient flows and numerical algorithms. It was shown, in particular, that such equations give rise to isospectral flows. In this chapter we present what seems to be a new framework in which double bracket equations appear. We show that the corresponding flows on an ....

....probability densities. For example, making the change of variable oe = e t , we obtain a particular case of the gradient flow of gaussians described by Nakamura in [32] We point out an interesting analogy between the results of Propositions 4 and 5 and the sorting algorithms described in [11]. If N is a real diagonal matrix with unrepeated eigenvalues, and if H(0) is a suitably chosen symmetric matrix, then the solution of the double bracket equation H = H; H; N ] approaches a diagonal matrix H(1) such that the diagonal elements of H(1) and N are similarly ordered; since H(1) is ....

[Article contains additional citation context not shown here]

R. W. Brockett, Dynamical systems that sort lists, diagonalize matrices, and solve linear programming problems, Linear Algebra Appl. 146 (1991), 79--91.

....an operator Lie algebra which take the so called double bracket form dL d# = L, L, M ] This leads to a simple expression for the flow of the corresponding eigenfunctions. The study of di#erential equations in the double bracket form on finite dimensional Lie algebras was initiated in [2] and [6] in connection with integrable gradient flows and numerical algorithms. It was shown, in particular, that such equations give rise to isospectral flows. In this paper we present what seems to be a new framework in which double bracket equations appear. The corresponding flows on an operator Lie ....

.... 0 the operator #(#) is unitary and the operators T (#) and N(#) are self adjoint, we arrive at the equation d# d# = T (1 2)#N #N# 1 T (1 2)# = #[T, N ] 33) which describes the evolution of the eigenbasis for T (#) induced by the flow (32) This is the same equation as the one obtained in [6] for the finite dimensional case. We point out an interesting analogy between the results of Propositions 10 and 11 and the sorting algorithms described in [6] If N is a real diagonal matrix with unrepeated eigenvalues, and if H(0) is a suitably chosen symmetric matrix, then the solution of the ....

[Article contains additional citation context not shown here]

R. W. Brockett, Dynamical systems that sort lists, diagonalize matrices, and solve linear programming problems, Linear Algebra Appl., 146 (1991), pp. 79--91.

No context found.

R. W. Brockett, Dynamical systems that sort lists, diagonalize matrices and solve linear programming problems, in Proceedings of the 27th IEEE Conference on Decis ion and Control, IEEE, (

No context found.

R. W. Brockett, Dynamical systems that sort lists, diagonalize matrices and solve linear programming problems, preprint, 1989.

No context found.

R. W. Brockett. Dynamical systems that sort lists, diagonalize matrices, and solve linear programming problems. Linear Algebra and its Applications, 146:79--91, 1991. 15

No context found.

R. W. Brockett. Dynamical systems that sort lists, diagonalize matrices, and solve linear programming problems. Linear Algebra and its Applications, 146:79--91, 1991.

No context found.

R. W. Brockett. Dynamical systems that sort lists, diagonalize matrices and solve linear programming problems. Lin. Alg. Applics , 146:79--91, 1991.

No context found.

R. W. Brockett. Dynamical systems that sort lists, diagonalize matrices and solve linear programming problems. Linear Algebra Appl., 146:79--91, 1991.

No context found.

Brockett, R.W.: Dynamical systems that sort lists, diagonalize matrices, and solve linear programming problems. Linear Algebra and its Applications 146 (1991) 79--91

No context found.

R. W. Brockett, Dynamical systems that sort lists, diagonalize matrices, and solve linear programming problems, Linear Algebra Appl. 146 (1991), 79--91.

No context found.

R.W. Brockett. Dynamical systems that sort lists. Linear Algebra and Its Applications, 146:79--91, 1991.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC