G. Buzzard, D. Jacobson, M. Mackey, S. Marovich, and J. Wilkes. "An Implementation of the Hamlyn Sender-managed Interface Architecture." In Proceedings of the Second Symposium on Operating Systems Design and Implementation (OSDI '96), Seattle, WA, Oct. 1996.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....registers and buffer queues in the descriptor. This enables RDMA, which allows the network adapter to reduce copy overhead by accessing application buffers directly. The combination of user level network access and copy avoidance has a lengthy heritage in research systems spanning two decades [2, 4, 6, 33, 38]. The experiments in Section 7 quantify the improvement in access overhead that DAFS gains from RDMA and transport offload on direct access NICs. 2.3 User Level File Systems In addition to overhead reduction, the DAFS protocol leverages user level networking to enable the network file system ....

G. Buzzard, D. Jacobson, S. Marovich, and J. Wilkes. An Implementation of the Hamlyn Sender-Managed Interface Architecture. In Proc. of Second Symposium on Operating Systems Design and Implementation, October 1996.

....manage ment, together with a sufficiently reliable network hardware such as the Myrinet, eliminates receive buffer overruns and avoids the need for message buffering, ac knowledgements and re transmission protocols. Several other systems have also explored the advantages of using this approach [6, 8]. The semantics of MPI, however, specify a model of stream oriented commu nication. There are two challenges in trying to translate remote memory semantics into the MP stream oriented model. First, we must separate the transfer of data from the transfer of control messages. Second, we must ....

....based on fast remote memory technology. Several recent research projects have explored the benefits of using pro grammable network interfaces provided by current gigabit networks [3] These ben efits include lower message overhead possible when interfaces are directly accessible at user level [6, 9, 13], lower large message latency possible when interfaces frag ment and pipelining data transfers between host memory and the network [2, 7, 14] higher throughput possible when fragmentation and pipeline are adaptive to message size [10, 12] and lower overheads possible when interfaces implement ....

[Article contains additional citation context not shown here]

Greg Buzzard, David Jacobson, Milon Mackey, Scott Marovitch, and John Wilkes (HP Labs). An implementation of the Hamlyn sender-managed interface architecture. In 2rid USENIX Symposium on Operating System Design and Implementation (OSDI), pages 245-60, October 1996.

....using memory mapped access to the device and bypassing the Kostas Magoutis is with the Division of Engineering and Applied Sciences, Harvard University. E mail: magoutis eecs.harvard.edu . kernel. Research on system area networks with user level networking and RDMA capabilities [4] 5] [6], 7] has motivated the design of distributed systems based on an RDMA paradigm. Recent commercially available SAN interconnects such as InfiniBand [8] and protocols such as VI [9] o#er advanced RDMA capabilities for data transfer and atomic operations. Network interfaces for these interconnects ....

G. Buzzard, D. Jacobson, S. Marovich, and J. Wilkes, "An Implementation of the Hamlyn Sender-Managed Interface Architecture, " in Proceedings of the Second Symposium on Operating Systems Design and Implementation, October 1996.

....hardware implementation. 2 Background 2.1 Virtual Interface Architecture VIA is the result of an effort to standardise academic research into user level networks. The design is most strongly influenced by the U Net[6] project, but a wide variety of other user level networks have been proposed[7, 8, 9]. The VIA network model has since been adopted for the Infiniband switched fabric interconnect, which has wide and powerful industry support, and hence is likely to be widely adopted. The VIA Specification[10] defines the network interface s architectural model, including the division of ....

Greg Buzzard, David Jacobson, Milon Mackey, Scott Marovich, and John Wilkes. An implementation of the Hamlyn sender-managed interface architecture. In 2nd Symposium on Operating Systems Design and Implementation, pages 245-- 259, October 1996.

....network in one cabinet (a System Area Network or SAN) may be made to be sufficiently reliable that protocol level fault tolerance becomes unattractive compared to a combination of link level fault tolerance (e.g. via ECC) and simple end to end fault detection. Other researchers take the same view [9, 24]. MPP manufacturers have found it feasible to build reliable networks for machines with several hundreds of nodes [68] Placing the network in one cabinet avoids most of the practical causes of failures, e.g. those due to unpluggings, independent power supply failures, cable damage and ....

....placed close to the processor. Anticipating continued system integration, we place our N on the processor cache bus. The CN paper showed how to partly compensate for a more distant N by exploiting standard cache coherence support [55] Memory based interfaces, Figure 8 lb in multicomputers [6, 9, 65, 68, 71] and workstations [16, 18, 75, 76] provide easy protection for multiprogramming if the N also demultiplexes messages into per process buffers. Automatic hardware buffering also deals well with sinking bursts of messages and provides the lowest overhead (by avoiding the processors) when messages ....

[Article contains additional citation context not shown here]

Greg Buzzard, David Jacobson, Milon Mackey, Scott Marovich, and John Wilkes. An Implementation of the Hamlyn Sender-Managed Interface Architecture. In Proceedings of the Second Symposium on Operating System Design and Implementation, pages 245-259, 1996.

....messages destined for other programs, and from spoofing messages such that they appear to originate from other sources. Existing user level networking efforts have addressed connection oriented networks such as ATM [1] 2] and special purpose System Area Networks (SANs) 3] 4] 5] [6]. Establishing a virtual circuit between two endpoints provides a convenient means of securely identifying the source and destination applications of transmitted data. User level interfaces then need only ensure that applications are restricted to sending and receiving on virtual circuits that ....

G. Buzzard, D. Jacobson, M. Mackey, S. Mavorich, and J. Wilkes, "An implementation of the Hamlyn sender-managed interface architecture, " in 2nd Symposium on Operating Systems Design and Implementation (OSDI '96). Oct. 1996, pp. 245-259, USENIX.

....hardware implementation. 2 Background 2.1 Virtual Interface Architecture VIA is the result of an e ort to standardise academic research into user level networks. The design is most strongly in uenced by the U Net[6] project, but a wide variety of other user level networks have been proposed[7, 8, 9]. The VIA network model has since been adopted for the In niband switched fabric interconnect, which has wide and powerful industry support, and hence is likely to be widely adopted. The VIA Speci cation[10] de nes the network interface s architectural model, including the division of ....

Greg Buzzard, David Jacobson, Milon Mackey, Scott Marovich, and John Wilkes. An implementation of the Hamlyn sendermanaged interface architecture. In 2nd Symposium on Operating Systems Design and Implementation, pages 245-259, October 1996. 9

....and management. Consequently, demanding applications such as database management systems prefer to take total responsibility [67] In the analogous case of the network interface, there is also much interest in allowing applications to access devices directly and bear responsibility for management [6, 46, 72]. An important recent approach to application customization in operating systems can also be applied to storage systems. This is the use of mechanisms that allow applications to help operating 10 systems make critical policy decisions [3, 22] For example, applications and storage systems can ....

BUZZARD, G., JACOBSON, D., MACKEY, M., MAROWCH, S., AND WLKES, J. An implementation of the Hamlyn sender-managed interface architecture. In Proceedings of the 1996.

....and employed unused processor opcodes to implement remote memory operations [12] Fast Messages [9] allow 12 direct user level access to the network interface, but do not support simultaneous use by multiple applications. The HP Hamlyn network implements user level sends and receives in hardware [1]. ParaStation [13] provides unprotected userlevel access to the network interface. With Active Messages [7] each message contains the address of a user level handler that is executed upon message arrival with the message body as an argument. This allows the programmer and compiler to overlap ....

G. Buzzard, D. Jacobson, M. Mackey, S. Marovich, and J. Wilkes. An Implementation of the Hamlyn SenderManaged Interface Architecture. In Proceedings of the Proceedings of the Second Symposium on Operating System Design and Implementation, pp. 245-259, 1996.

....mechanism is required. Surprisingly, minimal messaging has often proven an elusive target. For example, in many designs that support minimal messaging, there are often restrictions placed to the size or location of the message buffers that can be directly accessed by the NI [BLA 94,GCP96,BJM 96,Hor95] While the application can incorporate such message buff ers within its data structures, in practice the complexity and cost of managing them can be significant and they may end up being used as intermediate buffers. We can trace the cause of such limitations to the address translation ....

....Active Messages, are powerful enough to support minimal messaging. The sender specifies the source and destination addresses as offsets in message segments for every message. In theory, you can define message segments that cover the entire application address space. How ever, in current Hamlyn [BJM 96,Wi192] and Active Messages [Mar94,LC95,CLMY96] implementations the address translation structures are not there or are limited to their reach. For example, the latest prototype Hamlyn implementation [BJM 96] is built on hardware identical to ours (Myrinet) yet message buffers must be pinned in ....

[Article contains additional citation context not shown here]

Greg Buzzard, David Jacobson, Milon Mackey, Scott Marovich, and John Wilkes. An Implementation of the Hamlyn Sender-Managed Interface Architecture. In Second USENIX Symposium on Operating Systems Design and Implementation, October 1996. Seattle, WA.

....registers and bu#er queues in the descriptor. This enables RDMA, which allows the network adapter to reduce copy overhead by accessing application bu#ers directly. The combination of user level network access and copy avoidance has a lengthy heritage in research systems spanning two decades [2, 4, 6, 33, 38]. The experiments in Section 7 quantify the improvement in access overhead that DAFS gains from RDMA and transport o#oad on direct access NICs. 2.3 User Level File Systems In addition to overhead reduction, the DAFS protocol leverages user level networking to enable the network file system ....

G. Buzzard, D. Jacobson, S. Marovich, and J. Wilkes. An Implementation of the Hamlyn Sender-Managed Interface Architecture. In Proc. of Second Symposium on Operating Systems Design and Implementation, October 1996.

....accesses involve only memory and interconnection network transfers, they proceed at high speed. For example, while typical disk latency is around 10 ms, typical network latency is around 1 ms (for small data transfers) Modern interconnection networks provide latency as low as a few microseconds [5, 7, 14, 17, 20, 1]. Thus, Network RamDisks may result in significant performance improvements over magnetic disks, especially when application performance depends on latency. Besides performance, the Network RamDisk offers a high level of data reliability using either conventional methods such as data replication, ....

G. Buzzard, D. Jacobson, M. Mackey, S. Marovich, and J. Wilkes. An Implementation of the Hamlyn Sender-Managed Interface Architecture. In Second Symposium on Operating System Design and Implementation, pages 245--259, October 1996.

....65 cycles if it causes a cache miss on the message arrival notification structure, which is likely on the first poll, and 34 cycles on subsequent polls as long the notification remains in cache. 7 Related Work Many groups are currently researching efficient messaging for workstations. The Hamlyn[4] effort at Hewlett Packard Labs is similar in many respects to the Avalanche approach. It utilizes a sender based protocol, HP Runway based workstations, and the Myrinet interconnect. As their interface, however, they use a LaNai positioned on an I O bus rather than the system bus. They report a ....

Buzzard, G., D.Jacobson, Mackey, M., Marovich, S., and Wilkes, J. An implementation of the Hamlyn sender-managed interface architecture. In Proceedings of the Second Symposium on Operating System Design and Implementation (October 1996).

....messages destined for other programs, and from spoofing messages such that they appear to originate from other sources. Existing user level networking efforts have addressed connection oriented networks such as ATM [1] 2] and specialpurpose System Area Networks (SANs) 3] 4] 5] [6]. Establishing a virtual circuit between two end points provides a IEEE INFOCOM 2001 2 convenient means of securely identifying the source and destination applications of transmitted data. User level interfaces then need only ensure that applications are restricted to sending and receiving on ....

G. Buzzard, D. Jacobson, M. Mackey, S. Mavorich, and J. Wilkes, "An implementation of the Hamlyn sender-managed interface architecture," in 2nd Symposium on Operating Systems Design and Implementation (OSDI '96). Oct. 1996, pp. 245--259, USENIX.

.... Modern operating systems are now supporting efficient, lightweight communication mechanisms such as shared memory based intranode communication channels [1] highly optimized kernel IPC paths [10, 21] and new inter node communication models such as active messages [33] and sender based protocols [4, 32]. As Clark and Tennenhouse predicted in 1990 [7] these improvements in low level communication systems have moved the performance bottlenecks for distributed applications out of the network and operating system layers and into the applications themselves. Recent work by Schmidt et al. 18, 28] ....

G. Buzzard, D. Jacobson, M. Mackey, S. Marovich, and J. Wilkes. An implementation of the Hamlyn sender-managed interface architecture. In Proceedings of the Second Symposium on Operating Systems Design and Implementation, pages 245--259, Seattle, WA, Oct. 1996. USENIX Association.

....and employed unused processor opcodes to implement remote memory operations [13] Fast Messages [10] allow direct user level access to the network interface, but do not support simultaneous use by multiple applications. The HP Hamlyn network implements user level sends and receives in hardware [2]. ParaStation [14] provides unprotected user level access to the network interface. With Active Messages [8] each message contains the address of a user level handler that is executed upon message arrival with the message body as an argument. This allows the programmer and compiler to overlap ....

G. Buzzard, D. Jacobson, M. Mackey, S. Marovich, and J. Wilkes. An Implementation of the Hamlyn Sender-Managed Interface Architecture. In Proceedings of the Second Symposium on Operating System Design and Implementation, pp. 245-259, 1996.

....protocol can be supported only under a particular type of automaticreceive DMA; all other interfaces use a buffered protocol. The Hamlyn network 41 PIO DMA Automatic Buffered Proc. initiated Unbuffered Receive interfaces Figure 2. 3: Taxonomy of the receive interface interface from HP Labs [56, 57, 58] supports automatic, unbuffered DMA. The authors terminology for this property is sender managed communication. Hamlyn is also one of the few network interface architectures that supports a network that delivers packets out of order. The principal advantage of an unbuffered protocol is the ....

....ffl Protocol support: buffered, unbuffered. Table 2. 1 describes the attributes of eight different network interfaces: University of Washington Meerkat 1 [35] Intel Paragon, Princeton SHRIMP [42] CMU iWarp [23] Stanford DASH [45] Thinking Machines CM 5 [14] MIT MDP [22] and HP Labs Hamlyn [57, 58]. Some entries appear more than once because there are multiple attributes to many of these network interfaces. For instance, iWarp contains both systolic communication and DMA. These systems were chosen as representative examples of the wide span of possibilities that are available to the network ....

Greg Buzzard, David Jacobson, Milton Mackey, Scott Marovich and John Wilkes. An implementation of the Hamlyn sender-managed interface architecture. Proc. of the 2nd Symposium on Operating Systems Design and Implementation, Seattle WA, October 1996.

....design for multicomputer environments presents unique opportunities and cost performance tradeoffs. In dedicated multicomputers, network interface designs are typically tightly coupled with the processor, and as such can exploit attributes of such a tight coupling. For example, the Hamlyn [30] network interface performs sender controlled data transfer, in which the sender controls the memory location at the receiver where data will be deposited. Similarly, the network interface for the SHRIMP multicomputer [18] realizes direct writes to remote memory via virtual memory mapped ....

G. Buzzard, D. Jacobson, M. Mackey, S. Marovich, and J. Wilkes, "An implementation of the Hamlyn sender-managed interface architecture," in Proc. USENIX Symp. on Operating Systems Design and Implementation, pp. 245--259, October 1996.

....target commodity workstation clusters. Several recent research projects have explored the benefits of using programmable network interfaces provided by current gigabit networks [6, 3] These benefits include the lower message overhead possible when interfaces are directly accessible at user level [7, 10, 21], the lower largemessage latency possible when interfaces fragment and pipeline data transfers between host memory and the network [4, 8, 22] the higher throughput possible when fragmentation and pipeline are adaptive to message size [18, 20] and the lower overheads possible when interfaces ....

.... possible when interfaces fragment and pipeline data transfers between host memory and the network [4, 8, 22] the higher throughput possible when fragmentation and pipeline are adaptive to message size [18, 20] and the lower overheads possible when interfaces implement sender based flow control [7]. Our work differs from each of these projects in that we focus on functionality that is higher level than the network protocol layer. The SPINE system from the University of Washington provides a safe and extensible environment for programming network processors [11, 12] In contrast, our work ....

G. Buzzard, D. Jacobson, M. Mackey, S. Marovitch, and J. W. H. Labs). An implementation of the Hamlyn sendermanaged interface architecture. In 2nd USENIX Symposium on Operating System Design and Implementation (OSDI), pages 245--60, October 1996.

....network messages are hints that can be safely dropped at some cost in performance, and probabilistic flow control is built into the distributed paging algorithm. 2. 5 Related Work The basic structure of Trapeze is derived from our study of AM, FM, and other fast messaging systems, e.g. Hamlyn [4], U net [1] SHRIMP [2] and others. These systems are designed to minimize latency of small messages by eliminating operating system overheads and network protocol processing. Some of them include bulk data transfer facilities optimized for high bandwidth. How6 ever, none of them identify low ....

G. Buzzard, D. Jacobson, M. Mackey, S. Marovich, and J. Wilkes. An implementation of the Hamlyn sender-managed interface architecture. In Proceedings of the Second Symposium on Operating Systems Design and Implementation. USENIX Association, October 1996.

....and network adapter architectures have focused on improving communication performance over high speed interconnects such as ATM, FDDI, HIPPI, and Myrinet. The main interest is in building faster and lighter messaging primitives or distributed shared memory systems for clusters of workstations [3, 4, 13, 19]. While we are also interested in achieving better communication performance, our focus is on identifying an application NI interface that allows efficient implementations of applicationspecific mechanisms for state sharing across the cluster. The DVCM abstraction is our solution to this ....

G. Buzzard, D. Jacobson, M. Mackey, S. Marovich, and J. Wilkes. An implementation of the Hamlyn sendermanaged interface architecture. Proceedings of the 2nd Symposium on Operating Systems Design and Implementations, Oct. 1996.

....pages provides a very ecient means of requesting 115 the interface to send a message, but does require a very large physical address space, and may potentially double the TLB entry requirement of a process. 6.4. 2 Hamlyn Like the SHRIMP project, the Hewlett Packard Hamlyn project [Wilkes92, Buzzard96] developed an application accessible network interface to enable workstations to be connected together to form a multicomputer. The current version of the interface allows HP PA RISC workstations to be connected via their highspeed graphics bus to a Myricom Myrinet interconnect [Seitz95] ....

....busywait if required. Alternatively, they may be unblocked or sent a signal by the operating system demultiplexing the interrupt. Hamlyn provides a very exible high performance communication mechanism, but does require a relatively complex hardware implementation. The Hamlyn application library [Buzzard96] does support stream and datagram protocols, but this is not the interface s natural mode of operation. The sender directed communication makes it very well suited for message passing between closely co operating processes. 6.4.3 Osiris The OSIRIS interface [Davie93, Druschel94] was developed ....

Greg Buzzard, David Jacobson, Milon Mackey, Scott Marovich, and John Wilkes. An implementation of the Hamlyn sender-managed interface architecture. In Proceedings of the 2nd Symposium on Operating Systems Design and Implementation, October 1996. (pp 116, 117)

....data between regions without copying. These optimizations are useful also for network I O with operating system bypass or with noncopy semantics. I. Introduction The emergence of gigabit networks has motivated considerable recent work on copy avoidance in network I O [5] 16] 1] 7] 11] [4], 2] Because networks can now have point to point bandwidth greater than the main memory copy bandwidth of many end hosts [2] data copying by the operating system can easily become a bottleneck. The avoidance of such bottleneck can significantly improve communications performance. However, ....

G. Buzzard, D. Jacobson, M. Mackey, S. Marovitch and J. Wilkes. "An implementation of the Hamlyn sender-managed interface architecture", in Proc. OSDI'96, USENIX, Oct. 1996, pp. 245-259.

No context found.

G. Buzzard, D. Jacobson, M. Mackey, S. Marovich, and J. Wilkes. "An Implementation of the Hamlyn Sender-managed Interface Architecture." In Proceedings of the Second Symposium on Operating Systems Design and Implementation (OSDI '96), Seattle, WA, Oct. 1996.

No context found.

. G. Buzzard, D. Jacobson, M. Mackey, S. Marovich, J. Wilkes, "An Implementation of the Hamlyn SenderManaged Interface Architecture",inProc. of 2nd USENIX OSDI Symposium, pp. 245-259, Seattle, WA, October 1996.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC