We evaluate the effect of processor speed, network characteristics, and software overhead on the performance of release-consistent software distributed shared memory. We examine five different protocols for implementing release consistency: eager update, eager invalidate, lazy update, lazy invalidate, and a new protocol called lazy hybrid. This lazy hybrid protocol combines the benefits of both lazy update and lazy invalidate. Our simulations indicate that with the processors and networks that are becoming available, coarse-grained applications such as Jacobi and TSP perform well, more or less independent of the protocol used. Medium-grained applications, such as Water, can achieve good performance, but the choice of protocol is critical. For sixteen processors, the best protocol, lazy hybrid, performed more than three times better than the worst, the eager update. Fine-grained applications such as Cholesky achieve little speedup regardless of the protocol used because of the frequency of synchronization operations and the high latency involved. While the use of relaxed memory models, lazy implementations, and multiple-writer protocols has reduced the impact of false sharing, synchronization latency remains a serious problem for software distributed shared memory systems. These results suggest that future work on software DSMs should concentrate on reducing the amount ofsynchronization or its effect.

We are developing Munin, a system that allows programs written for shared memory multiprocessors to be executed efficiently on distributed memory machines. Munin attempts to overcome the architectural limitations of shared memory machines, while maintaining their advantages in terms of ease of programming. Our system is unique in its use of loosely coherent memory, based on the partial order specified by a shared memory parallel program, and in its use of type-specific memory coherence. Instead of a single memory coherence mechanism for all shared data objects, Munin employs several different mechanisms, each appropriate for a different class of shared data object. These type-specific mechanisms are part of a runtime system that accepts hints from the user or the compiler to determine the coherence mechanism to be used for each object. This paper focuses on the design and use of Munin's memory coherence mechanisms, and compares our approach to previous work in this area.

Distributed memory multiprocessing offers a cost-effective and scalable solution for a large class of scientific and numeric applications. Unfortunately, the performance of current distributed memory programming environments suffers because the frequency of communication between processors can exceed that required to ensure a correctly functioning program. Midway is a shared memory parallel programming system which addresses the problem of excessive communication in a distributed memory multiprocessor. Midway programs are written using a conventional MIMD-style programming model executing within a single globally shared memory. Local memories on each processor cache recently used data to counter the effects of network latency. Midway is based on a new model of memory consistency called entry consistency. Entry consistency exploits the relationship between synchronization objects and the data which they protect. Updates to shared data are communicated between processors only when not ...

Distributed shared memory (DSM) is a software abstraction of shared memory on a distributed memory machine. One of the key problems in building an e cient DSM system is to reduce the amount ofcommunication needed to keep the distributed memories consistent. In this paper we presentfour techniques for doing so: (1) software release consistency � (2) multiple consistency protocols � (3) write-shared protocols � and (4) an update-with-timeout mechanism. These techniques have been implemented in the Munin DSM system. We compare the performance of seven application programs on Munin, rst to their performance when implemented using message passing and then to their performance when running on a conventional DSM system that does not embody the above techniques. On a 16-processor cluster of workstations, Munin's performance is within 5 % of message passing for four out of the seven applications. For the other three, performance is within 29 % to 33%. Detailed analysis of two of these three applications indicates that the addition of a function shipping capability would bring their performance within 7 % of the message passing performance. Compared to a conventional DSM system, Munin achieves performance improvements ranging from a few to several hundred percent, depending on the application.

A distributed shared memory (DSM) system allows shared memory parallel programs to be executed on distributed memory multiprocessors. The challenge in building a DSM system is to achieve good performance over a wide range of shared memory programs without requiring extensive modifications to the source code. The performance challenge translates into reducing the amount of communication performed by the DSM system to that performed by an equivalent message passing program. This thesis describes four novel techniques for reducing the communication overhead of DSM, including: (i) the use of software release consistency, (ii) support for multiple consistency protocols, (iii) a multiple writer protocol, and (iv) an update timeout mechanism. Release consistency allows modifications of shared data to be handled via a delayed update queue, which masks network latencies. Providing multiple cons...

Shared memory is a simple yet powerful paradigm for structuring systems. Recently, there has been an interest in extending this paradigm to non-shared memory architectures as well. For example, the virtual address spaces for all objects in a distributed object-based system could be viewed as constituting a global distributed shared memory. We propose a set of primitives for managing distributed shared memory. We present an implementation of these primitives in the context of an object-based operating system as well as on top of Unix. The purpose of this paper is to present a set of mechanisms for DSM and an implementation of these mechanisms. All the resources of the system are viewed as potentially shared objects. The name space of these objects constitute a distributed shared memory. The objects are composed of segments, where a segment is a logical entity that has attributes such as read-only, and read-write. There is a concept of ownership and the node where a segment is created (the owner node) is responsible for guaranteeing the consistency of the segment. The distributed shared memory controller (DSMC) to be described next is the entity that provides the mechanisms for managing these segments.

Software distributed shared memory (DSM) is a software abstraction of shared memory on a distributed memory machine. The key problem in building an e cient DSM system is to reduce the amount of communication needed to keep the distributed memories consistent. The Munin DSM system incorporates a number of novel techniques for doing so, including the use of multiple consistency protocols and support for multiple concurrent writer protocols. Due to these, and other, features, Munin is able to achieve high performance on a variety of numerical applications This paper contains a detailed description of the design and implementation of the Munin prototype, with special emphasis given to its novel write shared protocol. Furthermore, it describes a number of lessons that we learned from our experience with the prototype implementation that are relevant to the implementation of future DSMs.

This paper addresses the architectural dependencies in the design of the system and evaluates performance of the implementation. The new version, MIRAGE + , performs well compared to Mirage even though eight times the amount of data is sent on each page fault because of the larger page size used in the implementation. We show that performance of systems with a large page size to network packet size can be dramatically improved on conventional hardware by applying three well-known techniques: packet blasting, compression, and running at interrupt level

So much has already been written about everything that you can't nd out anything about it. | James Thurber, Lanterns and Lances (1961) Loosely-coupled distributed systems haveevolved using message passing as the main paradigm for sharing information. Other paradigms used in loosely-coupled distributed systems, such as rpc, are usually implemented on top of an underlying message-passing system. On the other hand, in tightly-coupled architectures, such asmulti-processor machines, the paradigm is usually based on shared memory with its attractively simple programming model. The shared-memory paradigm has recently been extended for use in more loosely-coupled architectures and is known as distributed shared memory (dsm [153, 178,58]) in this context. This chapter discusses some of the issues involved in the design and implementation of such adsm in loosely-coupled distributed systems and brie y discusses related work in other elds. In dsm systems, processes share data transparently across node boundaries � data faulting, location, and movement are handled by thedsm system. Among other things, this allows parallel programs designed to use the shared-memory abstraction to execute without modi cation on a

This paper describes the evolution of a distributed shared memory (DSM) system, Mirage, from its original implementation on VAX computers to its current implementation on modern high-end personal computers. Mirage provides a form of shared memory that is network transparent in a loosely coupled environment. The system hides network boundaries for processes that are accessing shared memory and is upward compatible with the System V UNIX 1 interface. This paper addresses the architectural dependencies in the design of the system and evaluates performance of the implementation. Mirage + performance is similar to Mirage, but the communication bottleneck has become more severe because of the larger page size used in the implementation. We show how this problem can be resolved on conventional hardware at little additional expense by using compression techniques. 1 UNIX is a Registered Trademark of AT&T. Contents 1 Introduction 1 1.1 Reasons for Porting to a New Platform : : : : : : ...