Data Type. A collection of operations, each with a name and a signature defining the number and types of its arguments. application domain A domain whose purpose is to execute an application program. binding An association of a name with some value; in IDC, the local data structures for invoking an operation on an interface. class A set of objects sharing the same implementation. An object is an instance of exactly one class. closure The concrete realisation of an interface. A record containing two pointers, one to an operation table and the other to a state record. concrete type A data type whose structure is explicit. constructor An operation on an interface which causes the creation of an object, and returns the interfaces exported by the object. context A collection of bindings of names to values. context slot A data structure used to hold processor execution state within a domain. ix domain The entity which is activated by the kernel scheduler. Domains can be thought of a...

Modern switched networks such as ATM and Myrinet enable low-latency, high-bandwidth communication. This performance has not been realized by current applications, because of the high processing overheads imposed by existing communications software. These overheads are usually not hidden with large packets; most network traffic is small. We have developed Fast Sockets, a local-area communication layer that utilizes a high-performance protocol and exports the Berkeley Sockets programming interface. Fast Sockets realizes round-trip transfer times of 60 microseconds and maximum transfer bandwidth of 33 MB/second between two UltraSPARC 1s connected by a Myrinet network. Fast Sockets obtains performance by collapsing protocol layers, using simple buffer management strategies, and utilizing knowledge of packet destinations for direct transfer into user buffers. Using receive posting, we make the Sockets API a single-copy communications layer and enable regular Sockets programs to exploit the performance of modern networks. Fast Sockets transparently reverts to standard TCP/IP protocols for wide-area communication.

Plexus is a networking architecture that allows applications to achieve high performance with customized protocols. Application-specific protocols are written in a typesafe language and installed dynamically into the operating system kernel. Because these protocols execute within the kernel, they can access the network interface and other operating system services with low overhead. Protocols implemented with Plexus outperform equivalent protocols implemented on conventional monolithic systems. Plexus runs in the context of the SPIN extensible operating system. 1 Introduction This paper describes the design and implementation of Plexus, a protocol architecture that allows arbitrary applications to define application-specific protocols. Plexus allows protocol processing to be tailored using application-level knowledge, thus providing the framework for supporting new protocols [CSZ92], and implementing optimizations to existing protocols such as integrated layer processing and applica...

Arsenic is a Gigabit Ethernet NIC which exports an extended interface to the operating system and user applications. Unlike conventional adaptots, it implements some of the protection and multiplexing functions traditionally performed by the operating system. This enables applications to be given direct access to their own 'virtual interface', allowing them to send and receive packets without operating system interaction. Packet filters

We are building a very high performance 1.2 Gb/s ATM network interface chip called the APIC (ATM Port Interconnect Controller). In addition to borrowing useful ideas from a number of research and commercial prototypes, the APIC design embraces several innovative features, and integrates all of these pieces into a coherent whole. This paper describes some of the novel ideas that have been incorporated in the APIC design with a view to improving the bandwidth and latency seen by end-applications. Among the techniques described, Protected DMA and Protected I/O were designed to allow applications to queue data for transmission or reception directly from user-space, effectively bypassing the kernel. This argues for moving the entire protocol stack including the interface device driver into user-space, thereby yielding better latency and throughput performance than kernel-resident implementations. Pool DMA when used with Packet Splitting, is a technique that can be used to build true zero-copy kernel-resident protocol stack implementations, using a page-remapping technique. Orchestrated Interrupts and Interrupt Demultiplexing are mechanisms that are used to reduce the frequency of interrupts issued by the APIC; the interrupt service time itself is significantly reduced through the use of a hardware Notification Queue which is used to report the occurrence of events in the APIC to software.

Application-specific safe message handlers (ASHs) are designed to provide applications with hardware-level network performance. ASHs are user-written code fragments that safely and efficiently execute in the kernel in response to message arrival. ASHs can direct message transfers (thereby eliminating copies) and send messages (thereby reducing send-response latency). In addition, the ASH system provides support for dynamic integrated layer processing (thereby eliminating duplicate message traversals) and dynamic protocol composition (thereby supporting modularity). ASHs provide this high degree of flexibility while still providing network performance as good as, or (if they exploit application-specific knowledge) even better than, hard-wired in-kernel implementations. A combination of user-level microbenchmarks and end-to-end system measurements using TCP demonstrate the benefits of the ASH system.

In traditional operating systems, modifying the network protocol code is a tedious and error-prone task, largely because the networking stack resides in the kernel. For this reason, among others, many have proposed moving the networking stack to user-level. Unfortunately, implementations of this design have never entered widespread use due to the impractical requirements they place on the user: either the kernel or applications must be modified; or code cannot be moved seamlessly between the user-level and kernel stacks. In this paper, we present Alpine, a user-level networking infrastructure free from these drawbacks. Alpine supports a FreeBSD networking stack on top of a Unix operating system. It is freely available as source code. In this paper, we discuss the challenges we faced in virtualizing the FreeBSD networking stack without compromising on kernel, networking stack, and application compatibility. We then show how Alpine is effective at easing the burden of debugging and testing protocol modifications or new network protocols. In our experience, Alpine can reduce the overhead of modifying a protocol from hours to minutes. 1

Integrated Layer Processing (ILP) is an implementation concept which "permit[s] the implementor the option of performing all the [data] manipulation steps in one or two integrated processing loops " [1]. To estimate the achievable benefits of ILP, a file transfer application with an encryption function on top of a user-level TCP has been implemented and the performance of the application in terms of throughput and packet processing times has been measured. The results show that it is possible to obtain performance benefits by integrating marshalling, encryption and TCP checksum calculation. They also show that the benefits are smaller than in simple experiments, where ILP effects have not been evaluated in a complete protocol environment. Simulations of memory access and cache hit rate show that the main benefit of ILP is reduced memory accesses rather than an improved cache hit rate. The results further show that data manipulation characteristics may significantly influence the cache behavior and the achievable performance gain of ILP. 1

This paper describes how Xok /ExOS's kernel mechanisms and library operating system organization achieve this flexibility, and retrospectively shares our experiences and lessons learned (both positive and negative). It also describes how we used this flexibility to build and specialize three network data services: the Cheetah HTTP server, the webswamp Web benchmarking tool, and an application-level TCP forwarder. Overall measurements show large performance improvements relative to similar services built on conventional interfaces, in each case reaching the maximum possible end-to-end performance for the experimental platform. For example, Cheetah provides factor of 2--4 increases in throughput compared to highly tuned socket-based implementations and factor of 3--8 increases compared to conventional systems. Webswamp can offer loads that are two to eight times heavier. The TCP forwarder provides 50--300% higher throughput while also providing end-to-end TCP semantics that cannot be achieved with POSIX sockets. With more detailed measurements and profiling, these overall performance improvements are also broken down and attributed to the specific specializations described, providing server writers with insights into where to focus their optimization efforts

The Amoeba group communication system has two unique aspects: (1) it uses a sequencer-based protocol with negative acknowledgements for achieving a total order on all group messages; and (2) users choose the degree of fault tolerance they desire. This paper reports on our design decisions in retrospect, the performance of the Amoeba group system, and our experiences using the system. We conclude that sequencer-based group protocols achieve high performance (comparable to Amoeba's fast remote procedure call implementation), that the scalability of our sequencer-based protocols is limited by message processing time, and that the flexibility and modularity of user-level implementations of protocols is likely to outweigh the potential performance loss.