Wormhole routing has emerged as the most widely used switching technique in massively parallel computers. We present here a detailed survey of various techniques for enhancing the performance and reliability of the wormhole routing schemes in directly connected networks. We start with an overview of the direct network topologies and a comparison of various switching techniques. Next, the characteristics of wormhole routing mechanism are described in detail along with the theory behind deadlock-free routing. The performance of routing algorithms depends on the selection of path between the source and the destination, the network traffic, and the router design. The routing algorithms are implemented in the router chips. We outline the router characteristics and describe the functionality of various elements of the router. Depending on the usage of paths between the source and the destination, the routing algorithms are classified as deterministic, fully adaptive, and partially adaptive. ...

Deadlock-free routing algorithms have been developed recently without fully understanding the frequency and characteristics of deadlocks. Using a simulator capable of true deadlock detection, we measure a network's susceptibility to deadlock due to various design parameters. The effects of bidirectionality, routing adaptivity, virtual channels, buffer size and node degree on deadlock formation are studied. In the process, we provide insight into the frequency and characteristics of deadlocks and the relationship between routing flexibility, blocked messages, resource dependencies and the degree of correlation needed to form deadlock. 1 Introduction Interconnection network routing algorithms aim to minimize message blocking by efficiently utilizing network virtual channel and physical channel resources while ensuring deadlock freedom. Routing approaches to accomplish this can be based on avoiding deadlock or on recovering from deadlock. The main distinction between these two approaches...

This paper proposes a novel concept of multidestination message passing mechanism for wormhole k-ary n-cube networks. Similar to the familiar car-pool concept, this mechanism allows data to be delivered to or picked-up from multiple nodes with a single message-passing step. Such messages can propagate along any valid path in a wormhole network conforming to the underlying base routing scheme (deterministic or adaptive). The mechanism requires very little modification to the existing routers and can have unicast message-passing as a subset operation. A base-routing-conformed-path (BRCP) model to implement this mechanism is presented. The model is illustrated for several commonly-used routing schemes and the associated deadlock-freedom properties are analyzed. Using this model a set of new algorithms are proposed and evaluated to implement broadcasting and multicasting. It is shown that these algorithms can considerably reduce the latency of these collective communication operations co...

We present an adaptive deadlock-free routing algorithm which decomposes a given network into two virtual interconnection networks, V IN 1 and V IN 2 . V IN 1 supports deterministic deadlockfree routing, and V IN 2 supports fully-adaptive routing. Whenever a channel in V IN 1 or V IN 2 is available, it can be used to route a message. Each node is identified to be in one of three states: safe, unsafe, and faulty. The unsafe state is used for deadlock-free routing, and an unsafe node can still send and receive messages. When nodes become faulty/unsafe, some channels in V IN 2 around the faulty/unsafe nodes are used as the detours of those channels in V IN 1 passing through the faulty/unsafe nodes, i.e., the adaptability in V IN 2 is transformed to support fault-tolerant deadlock-free routing. Using information on the state of each node's neighbors, we have developed an adaptive fault-tolerant deadlock-free routing scheme for n-dimensional meshes and hypercubes with only two virtual channe...

. Networks of workstations are rapidly emerging as a costeffective alternative to parallel computers. Switch-based interconnects with irregular topologies allow the wiring flexibility, scalability and incremental expansion capability required in this environment. The irregularity also makes routing and deadlock avoidance on such systems quite complicated. Current proposals avoid deadlock by removing cyclic dependencies between channels. As a consequence, many messages are routed following non-minimal paths, increasing latency and wasting resources. In this paper, we propose a general methodology for the design of adaptive routing algorithms for networks with irregular topology. These routing algorithms allow messages to follow minimal paths in most cases, reducing message latency and increasing network throughput. The methodology is based on the application of the theory of deadlock avoidance proposed in [14], which increases routing flexibility by allowing cyclic dependencies between ...

: Directory-based distributed shared memory (DSM) systems have drawn high interests in parallel computing research and industry in recent years. Current generation systems are limited to using point-to-point messages for cache invalidation requests and associated acknowledgments. Such an approach incurs a large number of control messages, heavy network traffic, and high occupancy at home nodes. In this paper we present a new approach to reduce cache invalidation overheads. We introduce a set of multidestination-based reservation and gather worms to distribute invalidation requests and collect acknowledgments. Six different grouping schemes are investigated in generating such multidestination worms in order to implement fully-mapped directory-based cache coherence on networks using deterministic (e-cube) or adaptive (turn-model) wormhole routing. These schemes are evaluated for different applications and 2D mesh configurations through execution-driven simulation. The interplay among ov...

A theory for the design of deadlock-free adaptive routing algorithms for wormhole networks has been proposed in [11, 14]. This theory supplies the sufficient conditions for an adaptive routing algorithm to be deadlock-free, even when there are cyclic dependencies between channels. Also, two design methodologies have been proposed. Multicast communication refers to the delivery of the same message from one source node to an arbitrary number of destination nodes. Two multicast wormhole routing methods have been presented in [22] for multicomputers with 2D-mesh and hypercube topologies. This paper develops the theoretical background for the design of deadlock-free adaptive multicast routing algorithms for wormhole networks. Some basic definitions and two theorems are proposed, developing conditions to verify that an adaptive multicast routing algorithm is deadlock-free, even when there are cyclic dependencies between channels. As an example, the multicast routing algorithms presented in [...

In this paper, we propose a new router architecture that supports wormhole switching and circuit switching concurrently. This architecture has been designed to take advantage of temporal communication locality. This can be done by establishing a circuit between nodes that are going to communicate frequently. Messages using those circuits face no contention. By combining circuit switching, pre-established physical circuits and wave pipelining across channels and switches, it is possible to increase network bandwidth considerably, also reducing latency for communications that use pre-established physical circuits. This router architecture also allows to reduce the overhead of the software messaging layer in multicomputers by offering a better hardware support. Preliminary performance evaluation results show a drastic reduction in latency and increment in throughput when messages are long enough, even if circuits are established for a single transmission and locality is not exploited. 1 M...

Network performance in tightly-coupled multiprocessors typically degrades rapidly beyond network saturation. Consequently, designers must keep a network below its saturation point by reducing the load on the network. Congestion control via source throttling---a common technique to reduce the network load---prevents new packets from entering the network in the presence of congestion. Unfortunately, prior schemes to implement source throttling either lack vital global information about the network to make the correct decision (whether to throttle or not) or depend on specific network parameters, network topology, or communication patterns. This paper presents a global-knowledge-based, selftuned, congestion control technique that prevents saturation at high loads across different network configurations and communication patterns. Our design is composed of two key components. First, we use global information about a network to obtain a timely estimate of network congestion. We compare thi...

Networks of workstations (NOWs) are being considered as a cost-effective alternative to parallel computers. Many NOWs are arranged as a switch-based network with irregular topology, which makes routing and deadlock avoidance quite complicated. Current proposals use the up*/down* routing algorithm to remove cyclic dependencies between channels and avoid deadlock. However, routing is considerably restricted and most messages must follow non-minimal paths, increasing latency and wasting resources. In this paper, we propose a new methodology to compute deadlock-free routing tables for NOWs. The methodology tries to minimize the limitations of the current proposals in order to improve network performance. It is based on generating an underlying acyclic connected graph from the network graph and assigning a sequence number to each switch, which is used to remove cyclic dependencies. Evaluation results show that the routing algorithm based on the new methodology increases throughput by a factor of up to 2 in large networks, also reducing latency significantly.