Baudet, G.M.: Asynchronous iterative methods for multiprocessors. Journal of the ACM 25 (1978) 226--244  Home/Search   Document Not in Database   Summary   Related Articles   Check

....[4] and the preliminary tests in [4] indicate that this method leads quickly to cheap solutions of limited accuracy. Due to the rapid development and increasing usage of parallel computers and distributed computing, it is now important to adapt the methods to the new architectures. Baudet s [1] experimental results on systems of linear equations show a considerable advantage for iterative methods on parallel computers with no synchronization. This leads to experimentation with totally asynchronous [2] block iterative methods for the solution of linear least squares problems. The block ....

Baudet, G. M.: Asynchronous Iterative Methods for Multiprocessors. Journal of the ACM. 25 (1978) 226--244

....so that the parallel computation can proceed in the correct order. Third the tasks should be able to deal with the failure of one or more of the tasks they communicate with in a manner consistent with the computation being performed. The approach we took combines two ideas: chaotic relaxation [10,11] and meshless methods[12] The first is to avoid having to synchronize across all the nodes between iterations. The second is to be able to adapt to faults without requiring a replacement node to fill in the hole in the grid. Together these two ideas form the basis for a naturally fault tolerant ....

G. M. Baudet, "Asynchronous Iterative Methods for Multiprocessors", Journal of the ACM, Volume 25, Issue 2, pp. 226-244, April 1978.

....Systems of equations. 75 IOAN LAZ AR that all processors have to wait at some synchronization point before proceeding to the next iteration. The asynchronous nonlinear multisplitting methods were considered in [1] and [12] i.e. methods where no synchronization barrier is present (see [5, 3, 7] for some general discussions on asynchronous methods) Bahi et al..l [1] studied asynchronous nonlinear multisplitting methods in a general context for nonlinear fixed point problems, while Szyld and Xu [12] studied these methods for problems of the form (1) and extended the study to the case of ....

....(10) s(k) min s i (k) We obtain immediately from (7) and (8) 11) s(k) k and lim s(k) #. Suppose that (7) 9) are satisfied, then we can define an increasing sequence l#N having the properties (12) 0#s(k)#k k0 (13) #s(k)#k k . L . The proofs given by Baudet [3] and El Tarazi [13] for general asynchronous iterations use the sequence defined above. This sequence says that the asynchronous iteration (6) updates all block components at least once at the steps k 0 , k 1 , If k l 1 k l = L for all l, we get a synchronous block Gauss Seidel ....

[Article contains additional citation context not shown here]

Gerard M. Baudet. Asynchronous iterative methods for multiprocessors. J. Association for Computing Machinery, 25:226--244, 1978.

.... of the chaotic iteration above (see also Section 3) Recent contributions to fixpoint theory provide efficient strategies for vector iteration, e.g. by using demand driven evaluation strategies (cf. VWL94, Jr94] The vector approach has been further generalized towards asynchronous iterations [Bau78, Cou77, UD89, Wei93], where f J i may use components of a choice of earlier vectors x j , with j i, of the iteration. Despite its power the vector iteration approach turns out too restrictive in two aspects. First, the functions involved in the fixpoint iteration may be such that they cannot be regarded as ....

G'erard Baudet. Asynchronous iterative methods for multiprocessors. Journal of the ACM, 25(2):226--244, April 1978.

....are synchronous in the sense that all processors have to wait at some synchronization point before proceeding to the next iteration. In this paper we study asynchronous nonlinear multisplitting methods for the solution of (1) i.e. methods where no synchronization barrier is present; see, e.g. [4], 6] 15] for some general discussion on asynchronous methods. We present here a framework which is different than that in the recent paper [1] Our paper deals with local convergence and, in a sense, is more specialized. In our special situation, our hypotheses are more general than those in ....

....the asynchronous nonlinear multisplitting method (ANM) we use a mathematical model describing the Computational Model 3.1. To that end, consider a counter k, which is updated every time a new vector is computed by some processor and let x , l = 1; L, be the initial guess; see, e.g. [4], 6] 15] 23] and the references given therein. Then we write l = T l i P L j=1 E j x r(j;k) if l 2 J k k Gamma1 l otherwise. 6) for k = 1; 2; where the sets J k and the sequence r(j; k) j = 1; L, k = 1; 2; satisfy the following standard conditions ....

G'erard M. Baudet. Asynchronous iterative methods for multiprocessors. Journal of the Association for Computing Machinery, 25:226--244, 1978.

....: ii) The set fi j 2 J i g is unbounded for all = 1; p. iii) lim i 1 r(k; i) 1 for all k = 1; p. The representation (3) together with (i) iii) can be found (sometimes with some minor variations) in most of the literature on asynchronous methods, including [1] [4], 6] 12] 14] 19] 21] 23] Chazan and Miranker [11] proved the following result, where, as before, we assume that T is nonnegative. Theorem 1. If ae(T ) 1 the asynchronous method (3) converges to the solution of (1) If ae(T ) 1, an initial vector x and a sequence fr(k; i)g, k = ....

G'erard M. Baudet. Asynchronous iterative methods for multiprocessors. Journal of the Association for Computing Machinery, 25:226--244, 1978.

....of a new iterate at the ith step. As is customary in the description and analysis of asynchronous algorithms, we assume that the subscripts r( i) and the sets J i satisfy the following conditions. They appear as classical conditions in convergence results for asynchronous iterations; see e.g. [5], 8] 14] 17] r( i) i for all = 1; 2; L; i = 1; 2; 21) lim r( i) 1 for all = 1; 2; L: 22) The set fi j 2 J i g is unbounded for all = 1; 2; L: 23) With this notation, the asynchronous counterpart of Algorithm 2 can be described by the ....

G. M. Baudet. Asynchronous iterative methods for multiprocessors. Journal of the Association for Computing Machinery, 25:226--244, 1978.

....Steihaug [4] and their preliminary tests indicate that this leads quickly to cheap solutions of limited accuracy. Due to the rapid development and increasing usage of parallel computers and distributed computing it has become important to adapt these methods to the new architectures. Baudet s [1] experimental results on systems of linear equations show a considerable advantage for iterative methods on parallel computers with no synchronization at all. This statement has led us to experiment with totally asynchronous [2] block iterative methods for the solution of linear least squares ....

.... attempt to solve the following Newton step equation [9] 0 A I A 0 0 Z 0 X A 0 Deltay Deltaz Gammar c Gammar b GammaX Ze oee A (5) where X =diag(x) Z =diag(z) r b = Ax Gamma b, r c = A y z Gamma c, is the duality gap, oe is an algorithm dependent parameter between [0,1], and e is the vector of all ones. Equation (5) can be reformulated eliminating Deltaz to give Z A c Gamma A e Ax Gamma b ; 6) Deltaz = X (oee Z Deltax) Gamma z; which is known as the augmented system. Let M = X Z) Then, from (6) we get AMA ....

G. M. Baudet, Asynchronous Iterative Methods for Multiprocessors, J. of the ACM 25, pp. 226--244 (1978).

....of the network problem solving process. In fact, additional mechanisms required to handle hardware, communication, and processing errors may be unnecessary with the FA C approach, since uncertainty resolving mechanisms are already a part of the distributed system s problemsolving structure [4, 17, 40]. In FA C distributed systems, it may be difficult to determine which alternative tasks are globally the most beneficial to perform without extensive inter node communication. This control uncertainty is due to differences between the natural distribution of control information among the nodes ....

....3] When this equilibrium is achieved, a coherent set of local views has been constructed. The MSYS problem solving technique is an example of a more general problem solving paradigm, called iterative refinement, that is contained in different forms in many types of problem solving systems [4, 50, 63, 64]. We feel that knowledge based AI approaches to problem solving provide a basis for the development of design methodologies for FA C distributed systems. The mechanisms used in these problem solving systems to resolve error from incorrect and incomplete data and knowledge can also be used to ....

[Article contains additional citation context not shown here]

Gerard M. Baudet. Asynchronous iterative methods for multiprocessors. Journal of the ACM, 25(2):226--244, April 1978.

....of the associative nets, that has been studied at the IEF [11, 10] It is an SIMD multiprocessor that implements asynchronous communication. It is based on the mesh topology. 3.5 Related works on asynchronism Several works concern the asynchronous computations for matrix like calculations. In [2], the author define the asynchronous iterations and shows that they give the same result as the synchronous one for the so called contractant operators of R n . These results have been extended in [21] for operators of S n , where S is either a finite or infinite set. Several experimentations ....

Baudet, G. M. Asynchronous iterative methods for multiprocessors. Journal of the ACM 25, 2 (April 1978), 226--244.

....the pioneering work by Chazan and Miranker [6] the condition ae(jH j) 1 is known as a necessary and sufficient condition for the convergence of asynchronous iterations. However, the precise mathematical model for the asynchronous iterations underlying the Chazan Miranker result (see [6] and [3] for a slight modification) is very general. Virtually any practical situation can in fact be modelled mathematically by assuming slightly more restricitive hypotheses than in the Chazan Miranker model, and this has indeed been done repeatedly in the literature on asynchronous iterations. Examples ....

....as a whole class of iterative methods derived from the total step method (3) One now allows that only certain components of the iterate are updated at a given time step and that more than just the previous iterate may be used in the updated process. The precise definition is as follows, see [3, 5]. 4 Definition 1 For k = 1; 2; let J k f1; ng and (s 1 (k) s n (k) 2 N n 0 be such that s i (k) k Gamma 1 for i = 1; n; k = 1; 2; 7) lim k 1 s i (k) 1 for i = 1; n; 8) for every i 2 f1; ng the set fk j i = J k g is ....

[Article contains additional citation context not shown here]

G'erard M. Baudet. Asynchronous iterative methods for multiprocessors. J. ACM, 25:226--244, 1978.

....of a new iterate at the ith step. As is customary in the description and analysis of asynchronous algorithms, we assume that the subscripts r(#, i) and the sets J i satisfy the following conditions. They appear as classical conditions in convergence results for asynchronous iterations; see e.g. [5], 8] 15] 20] r(#, i) i for all # =1, 2, L, i=1, 2, 21) lim i## r(#, i) # for all # =1, 2, L. 22) The set i # # J i is unbounded for all # =1, 2, L. 23) With this notation, the asynchronous counterpart of Algorithm 2 can be described by the following algorithm. ....

Baudet, G.M. (1978): Asynchronous iterative methods for multiprocessors. Journal of the Association for Computing Machinery 25, 226--244

....of inputs) Asynchronous iterations satisfying these conditions are called totally asynchronous [5] Such asynchronous iterations only exclude starving computations. There has been a considerable amount of work in the literature on the convergence conditions for totally asynchronous iterations [2, 4, 5, 6, 7, 9, 13, 14, 15, 18, 20, 21, 22, 23, 25, 26, 27]. The most general result can be stated as follows [5, 7, 27] x i t ( F i u i t ( 4 Proposition 1. Let X(k) be a sequence of sets such that . for all k. for all k, and furthermore all sequences z(k) such that z(k) X(k) for all k converge to x, where x is the unique ....

Baudet, G.M., "Asynchronous iterative methods for multiprocessors," Journal of the Association for Computing Machinery (JACM), 25(2), pp. 226-244, April 1978.

....iteration step by the next variables during the image scanning, is known to improve convergence speed, but is inherently sequential and directionally asymmetric. In between, and retaining advantages from both, stand the asynchronous updating modes that have been studied for relaxation algorithms [Bau78]. The idea is to allow at each iteration step the updating of a random subset of state variables from delayed values of other variables dating back to past iterations rather than the immediately preceding one. Thus it corresponds to a less constrained updating order. In fact, a local calculation ....

G. Baudet, "Asynchronous iterative methods for multiprocessors", Journal of the Association for Computing Machinery, Vol. 25, No. 2, April 1978, pp. 226-244.

....points: x # F (x) This is su#ciently general to include Newton s method. He provided theoretical convergence results for the case of contracting operators, namely operators F (x) with contracting Lipschitz matrices: F (x) F (y) # A x y for some nonnegative A such that #(A) 1. Baudet (1978) relaxed various theoretical hypotheses of the earlier papers and performed experiments on a 6 processor C.mmp at CMU; and so forth. Bersekas (1990) is a book length monograph on aynchronous methods. We do not need to contemplate full asynchrony to derive a latencytolerant benefit from such ....

Baudet (1978) Asynchronous Iterative Methods for Multiprocessors, J. of the ACM 25, pp. 226--244.

....n [i] if i = 2 J n F (X n ) A Omega X n Phi B if i 2 J n (3.3) Informally, Equation 3.3 reads as only selected processors (those of J n ) compute a new value using the last produced result of their direct ancestors. Such equations are known in the literature as asynchronous iterations (see [9, 31]) 16 3.4.4 The Fully Distributed Demon Under control of the fully distributed demon, activated processors do not necessaryly write their output registers within the same round. Thus Equation 3.3 does not hold in this context. Hence, a processor may compute its output value using its last read ....

....(X Dn [1] 1] X Dn [N ] N ] t Delta = A Omega (X Dn [1] 1] XDn [N ] N ] t Phi B if i 2 J n (3.4) 3.4. 5 Conditions for Convergence of Asynchronous Iterations Asynchronous iterations have been extensively studied for optimization purpose on parallel computers (see [9, 19, 23, 30, 31, 32]) Under particular conditions, asynchronous iterations (Equations 3.3 and 3.4) converge to the same result as synchronous iterations (Equation 3.2) while reducing data dependency. In [31] Uresin and Dubois give several sufficient conditions ensuring the convergence of asynchronous iterations. ....

G. M. Baudet. Asynchronous Iterative Methods for Multiprocessors. Journal of the ACM, Vol. 25, No. 2, pp. 226--244, April 1978. 24

....[4] and the preliminary tests in [4] indicate that this method leads quickly to cheap solutions of limited accuracy. Due to the rapid development and increasing usage of parallel computers and distributed computing, it is now important to adapt the methods to the new architectures. Baudet s [1] experimental results on systems of linear equations show a considerable advantage for iterative methods on parallel computers with no synchronization. This leads to experimentation with totally asynchronous [2] block iterative methods for the solution of linear least squares problems. The block ....

Baudet, G. M.: Asynchronous Iterative Methods for Multiprocessors. Journal of the ACM. 25 (1978) 226--244

....INTRODUCTION In this paper we use a block iterative method for solving sparse linear least squares problems. A general framework for this method is introduced by Dennis and Steihaug [4] and their preliminary tests indicate that this leads quickly to cheap solutions of limited accuracy. Baudet s [1] experimental results on systems of linear equations show a considerable advantage for iterative methods on parallel computers with no synchronization at all. His statement has led us to experiment with totally asynchronous [2] block iterative methods for the solution of linear least squares ....

G. M. Baudet, Asynchronous Iterative Methods for Multiprocessors, J. of the ACM 25, p. 226 (1978).

.... R n , the solution of F(x) x is obtained by a similar computational model, Daniel B. Szyld 5 where at step 3, the processor solves for x (or its approximation) in the equation F (x 1 ; x ; xL ) x . This model then relates to the work by many authors; see, e.g. [6], 10] 14] 26] 37] 42] 45] 47] 48] 56] 62] and the references given therein. See also further extensions in Section 6. 3 First Mathematical Models The first mathematical model analyzing the convergence of the asynchronous iteration dates from 1959. At that time there were no ....

....condition (ii 0 ) or equivalently (ii) some authors call this sequence admissible; see, e.g. 18] 20] In these papers, a sequence satisfying (iii 0 ) is called regulated. Condition (iii) is more general than (iii 0 ) since no uniform bound d is required, as pointed out in, e.g. [6], 11] 21] see also [60] for an analysis of a condition other than (iii 0 ) Most convergence results in this paper correspond to the more general assumption (iii) We should mention though that there are models where the additional assumption on uniformity is required; see, e.g. 11, Ch. ....

G'erard M. Baudet. Asynchronous iterative methods for multiprocessors. Journal of the Association for Computing Machinery, 25:226--244, 1978.

....on concurrent architectures, it would be worthwhile to investigate the competitive power of asynchronous algorithms. Imposing rather weak conditions on the distribution of the delays and the update order, the correctness of such algorithms has been proven for important classes of problems [19, 20, 23]. But if the delays take too large values, the convergence rates seem to be questionable. The most important (partially) synchronous iteration schemes are: ffl The Jacobi iteration scheme, defined by d(l; i; k) 1, 8l 2 N , 8i 2 I, 8k 2 K i . ffl The Gau Seidel iteration scheme, defined by the ....

G. M. Baudet, "Asynchronous Iterative Methods for Multiprocessors", J. ACM 25, 2 (Apr. 1978), pp. 226--244. 21

....In Ada access anomalies render a program erroneous, that is, they have undefined semantics. To illustrate an access anomaly, consider the small code sequence below, written in Fortran extended by a parallel doall: 1 An exception to this is chaotic relaxation [Bau]. doall i = 1,2 A[i] B[i] 1 . doall j = 1,2 C[i, j] A[i 1] B[j] endall . endall The dynamic parallel flow of control for this program is shown in Figure 1, where each separate execution stream is labeled by an identifier T i . Since location A[2] is written by execution stream ....

G.M. Baudet, "Asynchronous Iterative Methods for Multiprocessors", Journal of the ACM, Vol. 25, No. 2, April 1978, pp. 226-244.

....to allow for dynamic adjustment of the grid and temporary access of neighboring grid sections, the grid array should be allocated in shared memory. Moreover, because of the local nature of the computation, efficiency suffers greatly if shared writable memory is not cacheable. Chaotic relaxation [Ba78], in which consistency can be eased over several iterations, allows for even more cacheing. Option b) is very inefficient, because the operating system must intervene on each store to a shared variable and because some stores to shared memory may not require invalidation of other processors ....

Baudet, G. M., "Asynchronous Iterative Methods for Multiprocessors", Journal of the ACM, Vol. 25, No. 2, pp. 226-244, April 1978.

....consistency should be viewed as a necessary, but not sufficient, condition for a system to satisfy. We have used our formalism to prove that Slow memory, a highly non coherent memory proposed in [20] is sufficient for a large class of asynchronous iterative methods to solve fixed point problems [5, 8, 9]. Due to limitation on space we omit this proof and only summarize our formalism in section 2. We refer the interested reader to [16] The architecture community has also recognized the need to relax consistency constraints for shared memory multiprocessors, resulting in the proposals given in ....

Gerard M. Baudet. Asynchronous iterative methods for multiprocessors. J. ACM, 25(2):226-- 244, April 1978.

....consistency should be viewed as the weakest, but not the only, condition that a system must satisfy. We use our formalism to prove that Slow memory, a highly non coherent memory proposed in [HA90] is sufficient for a large class of asynchronous iterative methods to solve fixed point problems [Bau78, BT89, BT90] The architecture community has also recognized the need to relax consistency constraints for shared memory multiprocessors, resulting in the proposals given in [SD88, AH90, GLL 90] Systems implementing these weaker constraints have been described in [LLG 92, BR90] We have ....

Gerard M. Baudet. Asynchronous iterative methods for multiprocessors. JACM, 25(2):226--244, April 1978.

....are used in the current approximation. The second one indicates that as the computation proceeds, eventually one reads newer information for each of the components. The third one indicates that no component fails to be updated as time goes on. This mathematical model goes back at least to Baudet [11], although other authors had equivalent models; see the historical remarks in [61] Note that Definition 2.2 includes as special cases the classical synchronous successive approximation method (2) s i (k) k Gamma 1; I k = f1; mg) as well as block Gauss Seidel type methods (s i (k) ....

....k m 1 C A ; where the inequality in R m is componentwise [51] It can be shown quite easily that a P contraction with respect to x satisfies the assumption of Theorem 3.3 (w has to be taken as the Perron vector of a positive matrix sufficiently close to P ) We therefore have Corollary 3. 4 [11] Assume that each H k is a P contraction with respect to x with P independent of k. Then the asynchronous (non stationary) iterates x k from (5) converge to x , the unique common fixed point of all H k . The contraction conditions considered so far can be somewhat relaxed to ....

G'erard M. Baudet. Asynchronous iterative methods for multiprocessors. Journal of the Association for Computing Machinery, 25:226--244, 1978.

....may perform more than one update between communications, possibly using out of date data for the subsets from the other processors. Each processor must at times communicate its most up to date values for its subset to other processors. A formal description of the asynchronous iteration is given in [3] and is inspired by the definition of chaotic relaxations in [12] The definition we give here is very similar: for 1 i m and t = 1; 2; x i (t) x i (t Gamma 1) if i 62 J t Op i (x 1 (s 1 (t) xm (s m (t) if i 2 J t (3) where J t is a subset of f1; mg, and s i (t) ....

....here is very similar: for 1 i m and t = 1; 2; x i (t) x i (t Gamma 1) if i 62 J t Op i (x 1 (s 1 (t) xm (s m (t) if i 2 J t (3) where J t is a subset of f1; mg, and s i (t) is in N . We adopt three additional conditions for asynchronous iteration proposed in [3]: Condition 2.1 For 1 i m: i) s i (t) t for all t = 1; 2; ii) lim t 1 (s i (t) 1. iii) i occurs infinitely often in the sets J t , t = 1; 2; Condition (i) states that when a processor updates a component of the solution vector, it can only use previously computed ....

[Article contains additional citation context not shown here]

G. Baudet. Asynchronous Iterative Methods for Multiprocessors. Journal of the Association for Computing Machinery, 25:226--244, April 1978.

....and the inverses of the Jacobians can be computed in parallel and without synchronization. Moreover, it converges locally superlinearly. Key words: Asynchronous iterations, nonlinear equations, Newton s method, Schulz method. AMS(MOS) subject classification. 65J15. 1 Introduction In [1] Baudet introduced asynchronous iteration methods for nonlinear fixed point problems and the more general asynchronous iteration methods with memory. In the latter case, there is a given operator G : D m 0 ae IR nm D 0 ae IR n (1) for which a fixed point 2 D 0 ; e.g. G( ....

....if for all j 2 IN 0 there is a t j 2 IN ; such that J j [ J j 1 [ J j t j = f1; ng (8) holds. iv) J is regulated, if there is a number t 2 IN 0 ; such that for all j 2 IN 0 J j [ J j 1 [ J j t = f1; ng : 9) The following criterion on the operator G was introduced in [1]: Definition 1.3 Let G be an operator as in (1) Then G is said to be pseudocontracting on D 0 ; if there is a fixed point 2 D 0 of G and (i) D 0 is closed, ii) for any diagonal matrix P 2 L(IR n ) with P ii 2 f0; 1g 8 i 2 f1; ng it is P x ( I Gamma P ) y 2 D 0 8 x; y 2 D 0 , iii) ....

[Article contains additional citation context not shown here]

G.M. Baudet. Asynchronous iterative methods for multiprocessors . J. Assoc. Comput. Mach., 25(2): 226-244. 1978.

....pieces of x(0) For instance, the first piece of x(3) on processor 1 results from updating the first piece of x(0) only twice, and the first piece of x(3) on processor 3 results from updating the first piece of x(0) only once. A good formal description of the asynchronous iteration is given in [5] and is inspired by the definition of chaotic relaxations in [16] The definition we give here is very similar: 8i; t = 1; 2; x i (t) 8 : x i (t Gamma 1) if i 62 J t Op i (x 1 (s 1 (t) xm (s m (t) if i 2 J t ; 2.2.3) where J t is a subset of f1; mg, s i (t) is ....

....just states that a processor can sometimes update a component of the solution vector by applying the operator to some solution vector value. If J t = f1; mg, then each processor updates its piece of the solution vector at each iteration. In order to make this definition more useful, Baudet in [5] proposes the three additional conditions: Condition 2.2.1 Conditions for asynchronous iterations: i) s i (t) t for all t = 1; 2; ii) lim t 1 (s i (t) 1. iii) i occurs infinitely often in the sets J t , t = 1; 2; Condition (i) states that when a processor updates a component of ....

[Article contains additional citation context not shown here]

G. Baudet. Asynchronous Iterative Methods for Multiprocessors. Journal of the Association for Computing Machinery, 25:226--244, April 1978.

....method can be used to approximate the solution of (3) yielding the type of two stage methods which are considered in this paper; see [12] 16] and the references given therein. We point out that, since the number of inner iterations may vary from block to block, the convergence results in Baudet [1] or Chazan and Miranker [5] cannot be applied to our situation. In Sect. 2, we derive two new convergence results for block two stage iterative methods. We then investigate asynchronous variations of our two stage methods. These asynchronous methods arise naturally in parallel computations if one ....

....B in [4] and is in contrast to the totally asynchronous Algorithm 4.1 discussed in the next section. As is customary in the description and analysis of asynchronous algorithms, the iteration subscript is increased every time any (block) component of the iteration vector is computed; see e.g. [1], 3] 4] 5] 6] 18] and the references given therein. We note that as a consequence of this convention, the number of iterations in asynchronous algorithms cannot be compared directly with the number of iterations in synchronous ones. In a formal way, the sets J k f1; 2; Delta Delta ....

[Article contains additional citation context not shown here]

Baudet, G.M. (1978): Asynchronous iterative methods for multiprocessors. J. ACM 25, 226--244

.... psolu Application The psolu application solves the sparse linear system Ax = b using block chaotic relaxation [27] Block chaotic relaxation is a parallel iterative method developed to overcome the synchronization problems encountered when trying to parallelize traditional iterative methods [1, 10]. The block in block chaotic relaxation comes from the fact that the input matrix A is partitioned into blocks, which form the unit of parallelism. A static load balancing scheme uses the number of operations per block per iteration to estimate the processing time required for each block. Blocks ....

G. M. Baudet. Asynchronous iterative methods for multiprocessors. J. Assoc. Comput. Mach., 25:226-- 244, 1978.

....none of them. A call to isis accept events( enables pending updates from other processes to be applied to the local copy of shared memory and also for events like timeouts to be handled. 14 The potential performance advantage of asynchronous iterative methods was first established by Baudet in [Bau78] Epsilon = 0.0001 Desired average accuracy per component of x do f do f AbsoluteDiff = 0; for (i = MyLow; i MyHigh; i ) f NewXi = Gamma(b i P i Gamma1 j=1 a ij Theta x j P m j=i 1 a ij Theta x j ) a ii ; Use Read to read x j AbsoluteDiff = AbsoluteDiff ....

Gerard M. Baudet. Asynchronous iterative methods for multiprocessors. J. ACM, 25(2):226--244, April 1978.

....is f k ( x i ) if k 2 J i and x k i otherwise. Intuitively, at each step i the set J i denotes the indices k of the components which are updated. It is known that a fairness condition for the J i is mandatory. The vector approach has been further generalized towards asynchronous iterations [Bau78, Cou77, UD89, Wei93] where f J i may use components of a choice of earlier vectors x j , with j i, of the iteration. More precisely, x i 1 = df f J i (x 1 S 1 i ; x n S n i ) where S k i denotes the index in the iteration where the k th component in the i th step is ....

....than natural numbers. For this reason he gets as an extra condition that there may not be a limit ordinal between S k i and i. The limit condition reads F i fi S k i = fi for every limit ordinal fi. Uresin and Dubois [ UD89] present a comparable approach. Like Robert [Rob76] and Baudet [Bau78] they use metric spaces and limits. Wei [Wei93] avoids the notational difficulties arising from S k i by taking their effect into account in the definition of representation functions. Recent contributions to fixpoint theory provide efficient strategies for vector iteration, e.g. by using ....

G'erard Baudet. Asynchronous iterative methods for multiprocessors. Journal of the ACM, 25(2):226--244, April 1978.

....that the introduction of such asynchronous computations can substantially enhance the efficiency of parallel iterative methods [23, 8, 13] 36 Chapter 9 4. 1 Asynchronous Parallel Computations The convergence of asynchronous iterative methods have been studied by several researchers (e.g. [14, 43, 2, 6, 11, 71, 44, 72]) While totally asynchronous, or chaotic, algorithms allow arbitrarily large communication delays and differences in the frequency of computation of different processors, other algorithms, which are referred to as partially asynchronous, can not be established convergent unless an upper bound on ....

G. M. Baudet, Asynchronous iterative methods for multiprocessors, J. ACM, 25 (1978), pp. 226--244.

....in chapter 6 of [BT89] As an example we will use this method to find the solution to a linear system of equations, x = Ax b. Such an iteration will converge if 6 We will deal with the termination of the iteration in Chapter3. 7 In R i . 23 the spectral radius 8 of jAj, ae(jAj) 1 [Bau78] Application of this method to optimization problems, the shortest path problem, solution of differential equations and network flow problems is described in [BT89, BT90] In the next chapter we motivate and propose a system that provides programmers with a choice of non coherent behaviors that ....

....to solve a system of linear equations using an asynchronous iterative method [BT89] 4 . We also show how the operations of Mermera can be used to achieve barrier synchronization. 4 The potential performance advantage of asynchronous iterative methods was first established by Baudet in [Bau78] 29 Epsilon = 0.0001 Accuracy desired do f do f AbsoluteDiff = 0; for (i = MyLow; i MyHigh; i ) f NewXi = Gamma(b i P i Gamma1 j=1 a ij Theta x j P m j=i 1 a ij Theta x j ) a ii ; Use Read to read x j AbsoluteDiff = AbsoluteDiff abs(NewXi Read(XStartLoc ....

Gerard M. Baudet. Asynchronous iterative methods for multiprocessors. J. ACM, 25(2):226--244, April 1978.

....strictly alternate the tasks of computing and communicating the results, and reproduce the exact same sequence of iterations as in the sequential implementation since a new iteration is started by a processor only when all the results from other processors have been received. Asynchronous methods [1] decouple computation from communication and allow a new cycle to start even with obsolete information. Asynchronous iterations eliminate synchronization overhead, thus achieving smaller average time per iteration. However, in some cases such as monotone mappings [3] they have been shown to ....

....by asynchronism, we have chosen the most favorable execution environment for the asynchronous iterations. Optimal probabilities p 1 and p 2 were selected for each iteration matrix A by minimizing the spectral radius ae(A aug ) This minimization is approximate since p 1 and p 2 are discretized in [0,1]. This minimized spectral radius is used in the computation of the asynchronous speedup. It is possible (in fact, quite frequent) that the optimal asynchronous evolution corresponds to the synchronous one (p 1 = p 2 = 1) and that the speedup is 1. In the presentation of these results, we ....

Gerard M. Baudet. Asynchronous iterative methods for multiprocessors. Journal of the ACM, 25(2):226--244, April 1978.

....iterations arise naturally on MIMD type shared or local memory parallel computers if one is willing to trade minimal idle times of the individual processors for a less structured iterative process. The convergence theory for asynchronous iterations is fairly well developped by now (see [2 5,17]) In the present paper we will contribute to results on calculating enclosures via asynchronous iterations for special systems of equations. We start by giving some basic notations and definitions. For x; y 2 IR n we denote x y if x i y i ; i = 1; n. The order interval hx; yi with x; ....

Baudet, G.: Asynchronous iterative methods for multiprocessors, J. ACM 25 226--244 (1978)

....uniform, some requiring only local interaction among processing elements whereas others are inherently global in nature. A large subclass of such applications falls under the category of fixed point problems, a class that is amenable to parallel iterative methods, synchronous or asynchronous [Bau78, BT89]. These include dynamic programming, systems of linear equations, network flow problems, genetic algorithms, and ordinary differential equations, just to name a few. This paper deals with the issue of how to map such applications to large scale workstation clusters linked by local or wide area ....

....and asynchronous methods. As with classical iteration methods such as Jacobi and Gauss Seidel [Mar82] convergence may not be guaranteed, and when both converge, the asynchronous iterative method often converges faster than the synchronous one. For a comparative analysis of these two methods, see [Bau78, BT89]. Nevertheless, a large class of problems have been proven to be amenable to solution by asynchronous iterative methods [BT89] The importance of asynchronous methods lies in the elimination of the synchronization penalty which can be very high in large scale implementations. This enables them to ....

[Article contains additional citation context not shown here]

Gerard M. Baudet. Asynchronous iterative methods for multiprocessors. Journal of the Association of Computing Machinery, 25(2):226--244, 1978.

.... instance of the chaotic iteration above (see also Section 3) Recent contributions to fixpoint theory provide efficient strategies for vector iteration, e.g. by using demand driven evaluation strategies (cf. 22, 8] The vector approach has been further generalized towards asynchronous iterations [1, 3, 21, 23], where f J i may use components of a choice of earlier vectors x j , where j i , of the iteration. Despite its power the vector iteration approach turns out to be too restrictive in two aspects. First, the functions involved in the fixpoint iteration may be such that they cannot be regarded ....

G. Baudet. Asynchronous iterative methods for multiprocessors. Journal of the Association for Computing Machinery, 25(2):226--244, April 1978.

....of component functions and partial derivatives as the most complex task, our method will converge superlinearly, and under suitable conditions at least quadratically. Key words: Asynchronous iterations, Newton iterative methods. AMS(MOS) subject classification. 65J15. 1 Introduction Baudet [1] extended asynchronous iterations, introduced by Chazan and Miranker [4] for linear systems, to nonlinear systems. Further he introduced asynchronous iterations with memory, where the operator of iteration depends on more than one of each components of already computed iterations. In the ....

.... for matrices A k i (X; h) 2 L(IR n ) i = 1; m with m X j=1 jA k j (X; h)j v k (X; h) v ; 16) where the image and the definition area of the set of functionals f k j k 2 IK g is given by k : D m 0 Theta D h [0; then G is called weakly contracting on D 0 : In [1] Baudet introduced a somewhat stronger criterion, which he called contracting. El Tarazi assumed in [10] that a one point operator, independent on parameters, satisfies kG(x) Gamma k v kx Gamma k v ; 17) where k Delta k v is a weighted maximum norm, that is k z k v : max i ....

G.M. Baudet. Asynchronous iterative methods for multiprocessors . J. Assoc. Comput. Mach., 25(2): 226-244. 1978.

....possibly using out of date data for the pieces of the other processors, or not to perform any update at all. In addition, a processor can decide at any time to send its most up to date piece to some of the other processors. A good formal description of the asynchronous iteration is given in [2] and is inspired by the definition of chaotic relaxations in [11] The definition we give here is very similar: 8i; t = 1; 2; x i (t) 8 : x i (t Gamma 1) if i 62 J t Op i (x 1 (s 1 (t) xm (s m (t) if i 2 J t ; 3) where J t is a subset of f1; mg, and s i (t) is ....

.... in [11] The definition we give here is very similar: 8i; t = 1; 2; x i (t) 8 : x i (t Gamma 1) if i 62 J t Op i (x 1 (s 1 (t) xm (s m (t) if i 2 J t ; 3) where J t is a subset of f1; mg, and s i (t) is an integer for all i, and t = 1; 2; Baudet in [2] proposes the three additional conditions: Condition 2.1 Conditions for asynchronous iterations: i) s i (t) t for all t = 1; 2; ii) lim t 1 (s i (t) 1. iii) i occurs infinitely often in the sets J t , t = 1; 2; Condition (i) states that when a processor updates a component of the ....

[Article contains additional citation context not shown here]

G. Baudet. Asynchronous Iterative Methods for Multiprocessors. Journal of the Association for Computing Machinery, 25:226--244, April 1978.

....and Steihaug [4] and their preliminary tests indicate that this leads quickly to cheap solutions of limited accuracy. Due to the rapid development and increasing usage of parallel computers and distributed computing it has become important to adapt these methods to the new architectures. Baudet s [1] experimental results on systems of linear equations show a considerable advantage for iterative methods on parallel computers with no synchronization at all. This statement has led us to experiment with totally asynchronous [2] block iterative methods for the solution of linear least squares ....

.... Newton step equation [9] 0 0 A T I A 0 0 Z 0 X 1 A 0 Deltax Deltay Deltaz 1 A = 0 Gammar c Gammar b GammaX Ze oe e 1 A (5) where X =diag(x) Z =diag(z) r b = Ax Gamma b, r c = A T y z Gamma c, is the duality gap, oe is an algorithm dependent parameter between [0,1], and e is the vector of all ones. Equation (5) can be reformulated eliminating Deltaz to give X Gamma1 Z A T A Deltax Deltay = c Gamma A T y Gamma oe X Gamma1 e Ax Gamma b ; 6) Deltaz = X Gamma1 (oe e Z Deltax) Gamma z; which is known as the ....

G. M. Baudet, Asynchronous Iterative Methods for Multiprocessors, J. of the ACM 25, pp. 226--244 (1978).

....fixed point theorem, F has a unique fixed point x 2 R n . It has been proved in part (a) that F is contracting on R n . Thus, F is contracting in x 2 R n for the vector norm j. 5 Asynchronous Iterative Scheme Motivated by [7, 8, 9] Baudet proposed an asynchronous iterative scheme [12] in 1978. Unlike the chaotic iteration scheme which does not allow use of the values which was produced by an update s or more step earlier, the asynchronous iterative scheme has no restriction on the choice of the antecedent values used in the evaluation of an iterate. Furthermore, the operator ....

....one required by Miellou s model. 6 Asynchronous Fixed Point Algorithms In 1983, Bertsekas proposed an algorithmic model for distributed computation of fixed points. As indicated in [13] the computation model is similar to the models presented by Chazan and Miranker[7] Miellou [8, 9] and Baudet[12]. This model has been further refined by Bertsekas and Tsitsiklis in [1, 2] The asynchronous model in [1, 2] is defined as x i (t 1) f i (x 1 ( i 1 (t) x n ( i n (t) 8t 2 T i ; x i (t) 8t 62 T i : 6.1) where i j (t) are times satisfying 0 i j (t) t; 8t ....

[Article contains additional citation context not shown here]

G. M. Baudet, "Asynchronous iterative methods for multiprocessors," Journal of the Association for Computing Machinery, vol. 25, pp. 226-- 244, April 1978.

....is equivalent to the original one. 2 4 Concluding Remarks 4.1 Related Work As already mentioned in the introduction, the idea of chaotic iterations was originally used in numerical analysis. The concept goes back to the fifties and was successively generalized into the framework of Baudet [3] on which Cousot and Cousot [11] was based. Our notion of chaotic iterations on partial orders is derived from the last reference. A historical overview can be found in Cousot [10] Let us turn now to a review of the work on constraint propagation. We show how our results provide a uniform ....

G. M. Baudet. Asynchronous iterative methods for multiprocessors. Journal of the ACM, 25(2):226--244, 1978.

....to the linear system Ax = b, then it can be shown that M J (P ff ) has a dominant eigenvalue 1 (M J (P ff ) 1 Gamma 1=n) ff. Similarly 1 (M J (Q ff ) Gamma(1 Gamma 1=n) ff. For certain choices of and , and for even n, M J (R ; has a complex pair of dominant eigenvalues. In [1] numerical results are given for a linear system derived from the five point finite difference method for Laplace s equation on a rectangular grid. This system does not fit into any of the above categories as, although 1 (M J ) is real and positive, 2 (M J ) Gamma 1 (M J ) In this case, ....

....is used, although it is less marked. The matrices of Section 3 have 1 (M J ) real and negative, and so we expect that the iterates will oscillate and hence that convergence can be accelerated by using the asynchronous version of algorithm. In many cases this behaviour is observed. The results of [1] show that AJ requires more iterations than SJ for the linear system derived from Laplace s equation. We must assume that in this case either x(0) was such that C(t) tended to a positive constant value, or the communication computation ratio was high. 6 Conclusions The results of Section 3 show ....

Baudet, G.M. (1978) Asynchronous iterative methods for multiprocessors J. Assoc. Comp. Mach. 25, 226--244.

No context found.

Baudet, G.M.: Asynchronous iterative methods for multiprocessors. Journal of the ACM 25 (1978) 226--244

No context found.

Gerard M. Baudet. Asynchronous iterative methods for multiprocessors. Journal of the ACM, 25(2):226--244, April 1978.

No context found.

Gerard M. Baudet. Asynchronous iterative methods for multiprocessors. J. ACM, 25(2):226--244, April 1978.

No context found.

G.M. Baudet, "Asynchronous iterative methods for multiprocessors," J. ACM, vol. 25, no. 2, pp. 226-244, 1978.

No context found.

G. Baudet, Asynchronous iterative methods for multiprocessors, JACM, vol. 25, no. 2, pp. 226-244, 1978.

No context found.

G.M. Baudet. Asynchronous Iterative Methods for Multiprocessors. Journal of the ACM, 2:226--244, 1978.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC