S.H. Bokhari and H. Berryman "Complete Exchange on a Circuit Switched Mesh" Proc. IEEE Scalable High Performance Computing Conf. pp. 300-306, 1992.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....of future work. 2 Related Work Early work on collective communication focused on developing optimized algorithms for particular architectures, such as hypercube, mesh, or fat tree, with an emphasis on minimizing link contention, node contention, or the distance between communicating nodes [2 4, 14]. More recently, Dongarra et al. have developed automatically tuned collective communication algorithms [5, 19] Their approach consists of running tests to measure system parameters and then tuning their algorithms for those parameters. Researchers in Holland and at Argonne have optimized MPI ....

S. Bokhari and H. Berryman. Complete exchange on a circuit switched mesh. In Proceedings of the Scalable High Performance Computing Conference, pages 300{ 306, 1992.

....Valiant has identified the h relation personalized communication as the basis for organizing communication between two consecutive major computation steps. Previous parallel algorithms for personalized communication (typically for a hypercube, e.g. 28, 40, 37, 12, 13, 10, 1] a mesh, e.g. [24, 39, 29, 14, 25], or other circuit switched network machines, e.g. 34, 19, 32, 38] tend to be network or machine dependent, and thus not efficient when ported to current parallel machines. In this paper, we introduce a novel deterministic algorithm that is shown to be both efficient and scalable across a number ....

....As far as we can tell, this algorithm was first reported (in Japanese) by Take ( 40] for the hypercube network topology. Later, several variations of this algorithm were developed (still dependent upon network topology) such as the Optimal Circuit Switched, Hypercube, or Mesh Algorithm ([38, 10, 25, 37, 12, 13, 14, 1, 32, 23, 34, 19, 24]) the Pairwise Exchange (PEX) algorithm ( 43, 41, 42] and the general Linear Permutation algorithm ( 45] For our comparison, we consider the standard algorithm consisting of p steps, such that during step i, 0 i p Gamma 1) processor j sends data labelled for processor k = i Phi j ....

S.H. Bokhari and H. Berryman. Complete Exchange on a Circuit Switched Mesh. In Proceedings of Scalable High Performance Computing Conference, pages 300--306, Williamsburg, VA, April 1992.

....in parallel computing. In this communication pattern, every processor communicates a block of distinct data to every other processor in the system [2,3,5,6] Many scientific applications require the all to all personalized exchange communication pattern. Several studies by Bokhari and Berryman [1], Sunder et al. 16] and Tseng et al. 19] have produced algorithms using message combining in 2 d x 2 d meshes or tori. These algorithms incur an O(2 ) execution time due to message start ups and O(23d) time due to message transmissions. Recently, Suh and Yalamanchili [13] proposed algorithms ....

S. H. Bokhari and H. Berryman, Complete Exchange on a Circuit Switched Mesh, Scalable High Performance Computing Conference, pages 300-306, 1992.

....ultimately reduces the broadcasting time. The analysis brings up closed formulation that yields the termination time of the algorithm. The fourth contribution of this thesis (Chapter 5) is a combined total exchange algorithm based on a combination of the direct [109, 120] and standard exchange [71, 24] algorithms. This ensures a better termination time than that which can be achieved by either of the two algorithms. Also, known algorithms [20, 40] for scattering and all to all broadcasting have been adapted to the network. In Chapter 6, I present the frequency and distributions of receive ....

....As the first contribution of this chapter, the analysis of the broadcasting algorithm includes a closed formulation that yields the termination time. Secondly, I contribute by proposing a combined total exchange algorithm based on a combination of the direct [109, 120] and standard exchange [71, 24] algorithms. This ensures a better termination time than what can be achieved by either of the two algorithms. Meanwhile, known algorithms for scattering and all to all broadcasting from the literature [40, 21] have been adapted to the network. 5.1 Introduction Communication operations may be ....

[Article contains additional citation context not shown here]

S. H. Bokhari and H. Berryman, "Complete Exchange on a Circuit Switched Mesh", Proceedings of the 1992 Scalable High Performance Computing Conference, April 1992, pp. 300-306.

....sends a distinct message of the same size to each processor. This operation is used in parallel fast Fourier transform, matrix transpose, and some parallel database join operations [4] The problem of all to all personalized communication on hypercubes and meshes has been studied extensively [1, 2, 3, 6]. But the number of algorithms for rings and tori are relatively few. One straightforward algorithm given in [4] for a ring of p processors completes the operation in a total time of . In the first step of that algorithm, each processor sends half of its messages in a packet in one direction and ....

S. H. Bokhari, H. Berryman. Complete Exchange on a Circuit Switched Mesh. Proc. 1992 Scalable High Performance Computing Conference, 1992, pp. 300-306.

....In a direct algorithm (e.g. 14, 15] for meshes) the complete exchange is performed in several contention free phases. In each phase, some of the messages are delivered directly from source nodes to their destination nodes using disjoint paths. On the contrary, in an indirect solution (e.g. [2, 8, 16] for meshes) intermediate nodes may be used to buffer, re arrange, and forward the messages. In meshes, the indirect solutions tend to be more efficient than the direct ones (see [8, 16] for a comparison) This complete exchange problem has been extensively studied for meshes [2, 8, 14, 15, 16] ....

....(e.g. 2, 8, 16] for meshes) intermediate nodes may be used to buffer, re arrange, and forward the messages. In meshes, the indirect solutions tend to be more efficient than the direct ones (see [8, 16] for a comparison) This complete exchange problem has been extensively studied for meshes [2, 8, 14, 15, 16]. However, these algorithms when applied to tori can not take advantage of wraparound interconnections to implement complete exchange with reduced latency. This leads to a challenge whether complete exchange algorithms for tori are possible which can use their bisection bandwidth effectively. In ....

[Article contains additional citation context not shown here]

S. H. Bokhari and H. Berryman. Complete exchange on a circuit switched mesh. In Scalable High Performance Computing Conf., pages 300--306, 1992.

....in a submesh and send them as a single message to one processor Q within that submesh. Q then forwards the data blocks towards their respective destinations in subsequent phases. This combining or indirect approach typically takes a divide and conquer strategy. For example, Bokhari and Berryman [1] described two algorithms that use this approach called Binary Exchange and Quadrant Exchange. Binary Exchange recursively divides the mesh into halves while Quadrant Exchange organizes the mesh in quadrants. The latter has been shown to be faster than the former. Sundar et al. 6] proposed ....

S. H. Bokhari, H. Berryman. Complete Exchange on a Circuit Switched Mesh. Scalable High Performance Computing Conference 1992. pp.300-306.

....in a submesh and send them as a single message to one processor Q within that submesh. Q then forwards the data blocks towards their respective destinations in subsequent 1 phases. This combining or indirect approach typically takes a divide and conquer strategy. For example, Bokhari and Berryman [9] developed two algorithms that use this approach called Binary Exchange and Quadrant Exchange. Binary Exchange recursively divides the mesh into halves while Quadrant Exchange organizes the mesh in quadrants. The latter has been shown to be faster than the former. Sundar et al. 24] proposed ....

....has been extensively studied on various topologies such as hypercubes [14, 6, 7, 5, 1, 19, 25] and tori [23] Solutions have also been formulated for linear arrays and rings [10] and other architectures [16, 18] with circuit switching. Several studies have focussed on the 2D mesh architecture [9, 21, 27, 26, 24]. 29 Another class of studies has aimed at an efficient solution to complete exchange on specific parallel computers [28, 6, 19] Many papers have analyzed the problem in the context of specific applications such as parallel sorting [2, 1, 15, 22] Efficient implementation of complete exchange ....

Shahid H. Bokhari and H. Berryman. Complete exchange on a circuit switched mesh. Proceedings of the Scalable High Performance Computing Conference, pages 300--306, 1992.

....in a submesh and send them as a single message to one processor Q within that submesh. Q then forwards the data blocks towards their respective destinations in subsequent phases. This combining or indirect approach typically takes a divide and conquer strategy. For example, Bokhari and Berryman [1] described two algorithms that use this approach called Binary Exchange and Quadrant Exchange. Binary Exchange recursively divides the mesh into halves while Quadrant Exchange organizes the mesh in quadrants. The latter is shown to be faster than the former. Sundar et al. 6] proposed another ....

S. H. Bokhari, H. Berryman. Complete Exchange on a Circuit Switched Mesh. Scalable High Performance Computing Conference 1992. pp.300-306. 19

....research issue. In order to apply CALMANT, we have designed a CC cube algorithm for the Complete Exchange problem that is described in the next section. The resulting algorithm has been compared with a wide range of proposals specially tuned for the solution of the Complete Exchange: Binary [1], Quadrant [1] Modified Quadrant [14] Store and Forward (SAF) 14] Direct [11] Cyclic [12] and Hybrid [13] methods. Some of these proposals are addressed to 2 dimensional meshes and some others to c dimensional meshes. The comparison is based on analytical models for the communication time and ....

....issue. In order to apply CALMANT, we have designed a CC cube algorithm for the Complete Exchange problem that is described in the next section. The resulting algorithm has been compared with a wide range of proposals specially tuned for the solution of the Complete Exchange: Binary [1] Quadrant [1], Modified Quadrant [14] Store and Forward (SAF) 14] Direct [11] Cyclic [12] and Hybrid [13] methods. Some of these proposals are addressed to 2 dimensional meshes and some others to c dimensional meshes. The comparison is based on analytical models for the communication time and the results ....

Bokhari S.H. and Berryman H. "Complete Exchange on a Circuit Switched Mesh". Proc. IEEE Scalable High Performance Computing Conf., pp. 300-306, 1992.

....the reduction to one and the scatter. These routines are defined as collective communications because all nodes either participate as contributors, receivers or both. Unlike previous work on such routines, however, this thesis considers the entire range of vector lengths and partition sizes [2,5,6,19,20,21,22]. Specifically, the typical approaches thus far are either optimal in the case of short or long vectors, but rarely for both. But even a consideration of both extremes is insufficient [4] In the critical range of medium length 3 vectors, neither short nor long vector approaches are the most ....

....time. Mesh based implementations must contend with the much more restricted bisection bandwidth of the mesh architecture. As a result, a myriad of methods have been proposed, each of which have certain relative message latency, data transmission time and data rearrangement cost advantages [6,10,20]. Many of these techniques require power of two meshes or make poor use of excess network bandwidth (a fatal flaw as will be shown below) Thus, this thesis presents a generalization of the best of these techniques using the primitives previously outlined in Sections 5 and 6. The interleaved ....

S. H. Bokhari and H. Berryman. "Complete Exchange on a Circuit-Switched Mesh." Proceedings of the Scalable High Performance Computing Conference, pg. 300-306, Apr. 1992.

....29] Traditionally, wormhole routed systems have supported only point to point (unicast) message passing mechanism. This mechanism allows a message to have exactly one destination. This leads to collective communication operations being implemented as multiple phases of unicast message exchange [3, 4, 22, 33, 35]. Each of these phases encounters a communication start up cost which is quite high on current systems (5.0 40.0 microsec) Thus, these operations are getting implemented on current systems with considerable high latency. Since collective communication operations primarily fall into critical paths ....

S. H. Bokhari and H. Berryman. Complete Exchange on a Circuit-Switched Mesh. In Proceedings of the Scalable High Performance Computing Conference, pages 300--306, 1992.

....takes the network topology into account and tries to minimize both the number of communication steps and the conflicts within the network so to fully exploit the bandwidth available at the hardware level. Efforts in this direction have been essentially devoted to all to all collective operations [3, 10, 22] where conflicts can seriously degrade performance. Nevertheless the results attained have given rise to a more general approach suitable to other collective primitives as well [4] This approach produces, in general, non portable algorithms, and it has been applied almost exclusively to mesh ....

S.H. Bokhari and H. Berryman, "Complete Exchange on a Circuit Switched Mesh", Procs. of Scalable high Performance Computing Conf., 1992, pp. 300--306.

....identi ed above. Applications of complete exchange include matrix algorithms, fast Fourier transformation (FFT) graph algorithms, and data redistribution in HPF [13] It can also be used to evaluate the quality of an interconnection network. Pervious work for complete exchange can be found in [3, 9, 23, 27, 28, 29] for meshes, and [6, 10, 25, 26, 31, 32] for tori. Here the torus network is considered, which architecture has been adopted by commercial machines such as Cray T3D T3E. The switching model under consideration is wormhole routing, which has been widely used in existing machines such as Caltech ....

....a worm of m bytes is typically modeled as ts mtx [21] The former cost is termed as startup time, and the latter the transmission time. 2 of data sent in complete exchange is fairly large (See the comparison in Section 6) We comment that the complete exchange algorithms developed for meshes [3, 9, 23, 27, 28, 29] may be directly applied on tori. However, such algorithms may fail in using the additional bandwidth provided by tori (a torus has twice the bisection bandwidth that of a mesh of the same size) and thus are inherently slower than good torus algorithms, as has been observed by [32] In this ....

S. H. Bokhari and H. Berryman. Complete exchange on a circuit switched mesh. In Proceedings of Scalable High Performance Computing Conference, 1992.

....methods are good when t s is large and or the problem size is small. In addition, indirect methods must pay the cost of data movements in memory when combining and forwarding data. This cost does not exist in direct methods. Relevant indirect algorithms are: Binary exchange and Quadrant exchange [BoBe92], Modified Quadrant exchange and SAF exchange [TaSe94] and Cyclic exchange [SJPS94] all of them for meshes, and the Diagonal Propagation exchange [TsGP95] and Fragopoulou and Akl exchange 1 [FrAk95] for tori. Finally, hybrid algorithms use an indirect approach in some of the steps of the ....

S.H. Bokhari and H. Berryman, Complete Exchange on a Circuit Switched Mesh, in proceedings of the IEEE Scalable High Performance Computing Conf., 1992, pp. 300-306.

....specifically for broadcasting, offers better performance. We are not aware of any previous one port algorithms for the scattering and gossiping patterns in either meshes or toruses with cut through routing. The multi scattering pattern has been intensively studied for meshes in the one port model [4, 24, 27, 28, 29, 30]. Since these algorithms do not use wraparound links, most of them are quite complicated compared to the solution presented in this paper. The performance of our algorithm is better than all of these algorithms for short messages and is also better than the algorithms in [4, 27, 29, 30] for long ....

....[4, 24, 27, 28, 29, 30] Since these algorithms do not use wraparound links, most of them are quite complicated compared to the solution presented in this paper. The performance of our algorithm is better than all of these algorithms for short messages and is also better than the algorithms in [4, 27, 29, 30] for long messages. The performance of our algorithm is approximately the same as the algorithms in [24, 28] for long messages. There are also several algorithms for toroidal meshes [11, 13, 31, 32, 33] By using virtual channels, our algorithm offers better performance than any of these ....

S. H. Bokhari and H. Berryman. Complete exchange on a circuit switched mesh. Proc. 1992 IEEE Scalable High Performance Computing Conference, 1992, 300--306.

....MB1 Fk MB2 Fk 85 60 33 1369 80 63 22703 100 83 88633 110 93 d M N k 1 ( 1 k log 1 ( 0 1 2 7 1 2 7 FIGURE 4. Sequential tree algorithm N 2 log N 1 ( d N 2 log 0 4 6 7 5 2 3 1 (m 1 ) m 4 ,m 5 ,m 6 ,m 7 ) m 6 ,m 7 ) m 7 ) m 5 ) m 2 ,m 3 ) m 3 ) 1] 2] 2] [3] [3] 3] 3] Time 0 1 2 3 4 5 6 Nodes 0 6 2 0 4 7 3 1 5 7 9 10 8 4 0 FIGURE 5. a) Binomial tree algorithm, b) its implementation on OK N with d = 1 d d d d d d d (b) a) 11 communications and the length of messages at each step for multinode broadcasting on an 8 node message based ....

....Fk MB2 Fk 85 60 33 1369 80 63 22703 100 83 88633 110 93 d M N k 1 ( 1 k log 1 ( 0 1 2 7 1 2 7 FIGURE 4. Sequential tree algorithm N 2 log N 1 ( d N 2 log 0 4 6 7 5 2 3 1 (m 1 ) m 4 ,m 5 ,m 6 ,m 7 ) m 6 ,m 7 ) m 7 ) m 5 ) m 2 ,m 3 ) m 3 ) 1] 2] 2] 3] [3] [3] 3] Time 0 1 2 3 4 5 6 Nodes 0 6 2 0 4 7 3 1 5 7 9 10 8 4 0 FIGURE 5. a) Binomial tree algorithm, b) its implementation on OK N with d = 1 d d d d d d d (b) a) 11 communications and the length of messages at each step for multinode broadcasting on an 8 node message based ....

[Article contains additional citation context not shown here]

S. H. Bokhari and H. Berryman, "Complete Exchange on a Circuit Switched Mesh," Proceedings of the 1992 Scalable High Performance Computing Conference, Apr. 1992, pp. 300-306

....on the IBM SP machine at Southampton 4 Related work There has been considerable progress in the development of scalable all to all (complete exchange) algorithms for architectures of limited connectivity hypercube, 2 D meshes. A number of algorithms have been proposed, e.g. Quadrant Exchange [BB92] and Cyclic Exchange [SJPS94] that aim to reduce link contention by combining, sequencing and routing individual messages in different ways. In those algorithms messages are not transmitted directly from sender to receiver, but are forwarded by intermediate nodes where they are ....

S.H. Bokhari and H. Berryman. Complete exchange on a circuit switched mesh. In Proceedings of the Scalable High Performance Computing Conference, pages 300--306. IEEE Computer Society Press, 1992.

....power of two, which is a valid assumption for those architectures. The mesh architecture introduces different problems because of high contention and the fact that the user can allocate a mesh size which need not be a power of two and may even be an odd number (e.g. 5 Theta5) Bokhari and Berryman [3] describe two algorithms for a circuit switched mesh, which assume that the number of processors is a power of two. In this paper, we discuss algorithms for both power of two and non power of two meshes. We have developed analytical models to estimate the performance of the algorithms. We present ....

Bokhari, S., and Berryman, H., "Complete Exchange on a Circuit Switched Mesh", Proc. of Scalable High Perf. Computing Conf., 1992, pp. 300--306.

....Each processor sends different messages to different processors. This operation is used in parallel fast Fourier transform, matrix transpose, and some parallel database join operations [4] The problem of all to all personalized communication on hypercubes and meshes has been studied extensively [1, 2, 3, 6]. But the number of algorithms for rings and tori are relatively few. One straightforward algorithm given in [4] for a ring of p processors completes the operation in a total time of . In the first step of that algorithm, each processor sends half of its messages in a packet in one direction and ....

....time to rearrange the messages between phases. The algorithm proposed here is faster than the one in [5] Table 3: Comparison of Algorithms for All to All Personalized Communication Topology Algorithm Type Transmission Time Start up Time 2D Mesh Scott [3] Direct 2D Mesh Bokhari Berryman [6] Indirect 2D Mesh Sundar, et al. 2] Indirect 2D Torus Tseng, et al. 5] Indirect 2D Torus This Paper Direct t s t w mt w t s 1 4 p pmt w 1 4 p pt s 3 4 p p 1 ( mt w 3 p 1 ( t s 1 2 p pmt w pt s 1 4 p p 4 ( mt w 1 2 p 4 ( t s 1 8 ....

S. H. Bokhari, H. Berryman. "Complete Exchange on a Circuit Switched Mesh." In Scalable High Performance Computing Conference, pages 300-306, 1992.

....with several other processors sending distinct array elements to each processor. Several methods for performing the complete exchange a special case of the all to many communication in which each processor communicates with every other processor have been presented in the literature [2, 3, 10, 14, 24, 25, 26]. A multi phase approach for complete exchange on hypercube connected multicomputers was presented in [2] The underlying idea of the multi phase approach is to combine data to be sent to a set of processors into one message and indirectly send it through this set of processors; each processor ....

....reduces the number of message startups required for the complete exchange but incurs an additional data transmission cost as a data element has to be packed, transmitted and unpacked multiple times. Similar combining or store and forward schemes for two dimensional meshes were presented in [3, 10, 25, 26]. The issue of scheduling the irregular all to many personalized communication for array redistribution has received relatively little attention. Static and run time scheduling techniques to reduce node and link contention while performing general irregular all to many personalized communication ....

S. Bokhari and H. Berryman. Complete exchange on a circuit switched mesh. In Proc. of Scalable High Performance Computing Conference, pages 300--306, May 1992.

....of the rest of the nodes. Applications of complete exchange include matrix algorithms, fast Fourier transformation (FFT) graph algorithms, and data distribution in HPF. It can also be used to evaluate the quality of an interconnection network. Pervious work for complete exchange can be found in [1, 3, 6, 8, 9, 10] for meshes, and [2, 4, 7, 11, 12] for tori. Here the torus network is considered, which architecture has been adopted by commercial machines such as Cray T3D T3E. The switching model under consideration is wormhole routing, which has been widely used in existing machines. Works related to such ....

S. H. Bokhari and H. Berryman. Complete exchange on a circuit switched mesh. In Proceedings of Scalable High Performance Computing Conference, 1992.

....identified above. Applications of complete exchange include matrix algorithms, fast Fourier transformation (FFT) graph algorithms, and data redistribution in HPF [12] It can also be used to evaluate the quality of an interconnection network. Pervious work for complete exchange can be found in [3, 9, 22, 26, 27, 28] for meshes, and [6, 10, 24, 25, 30, 31] for tori. Here the torus network is considered, which architecture has been adopted by commercial machines such as Cray T3D T3E. The switching model under consideration is wormhole routing, which has been widely used in existing machines such as Caltech ....

.... in [24, 25] However, the constant associated with the transmission time is relatively high and the effect of this is significant as the amount of data sent in complete exchange is fairly large (See the comparison in Section 6) We comment that the complete exchange algorithms developed for meshes [3, 9, 22, 26, 27, 28] may be directly applied on tori. However, such algorithms may fail in using the additional bandwidth provided by tori (a torus has twice the bisection bandwidth that of a mesh of the same size) and thus are inherently slower than good torus algorithms, as has been observed by [31] In this paper, ....

S. H. Bokhari and H. Berryman. Complete exchange on a circuit switched mesh. In Proceedings of Scalable High Performance Computing Conference, 1992.

No context found.

S.H. Bokhari and H. Berryman "Complete Exchange on a Circuit Switched Mesh" Proc. IEEE Scalable High Performance Computing Conf. pp. 300-306, 1992.

No context found.

S. H. Bokhari and S. Berryman. Complete exchange on a circuit switched mesh. In Proc. Scalable High Performance Computing Conf., pages 300--306, 1992.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC