Results 1 - 10
of
18
Scaling Memcached at Facebook
- In NSDI
, 2013
"... Abstract: Memcached is a well known, simple, inmemory caching solution. This paper describes how Facebook leverages memcached as a building block to construct and scale a distributed key-value store that supports the world’s largest social network. Our system handles billions of requests per second ..."
Abstract
-
Cited by 83 (1 self)
- Add to MetaCart
(Show Context)
Abstract: Memcached is a well known, simple, inmemory caching solution. This paper describes how Facebook leverages memcached as a building block to construct and scale a distributed key-value store that supports the world’s largest social network. Our system handles billions of requests per second and holds trillions of items to deliver a rich experience for over a billion users around the world. 1
Using Lightweight Modeling To Understand Chord
"... Correctness of the Chord ring-maintenance protocol would mean that the protocol can eventually repair all disruptions in the ring structure, given ample time and no further disruptions while it is working. In other words, it is “eventual reachability. ” Under the same assumptions about failure behav ..."
Abstract
-
Cited by 16 (4 self)
- Add to MetaCart
(Show Context)
Correctness of the Chord ring-maintenance protocol would mean that the protocol can eventually repair all disruptions in the ring structure, given ample time and no further disruptions while it is working. In other words, it is “eventual reachability. ” Under the same assumptions about failure behavior as made in the Chord papers, no published version of Chord is correct. This result is based on modeling the protocol in Alloy and analyzing it with the Alloy Analyzer. By combining the right selection of pseudocode and textual hints from several papers, and fixing flaws revealed by analysis, it is possible to get a version that may be correct. The paper also discusses the significance of these results, describes briefly how Alloy is used to model and reason about Chord, and compares Alloy analysis to model-checking.
Low-Latency Multi-Datacenter Databases using Replicated Commits
"... Web service providers have been using NoSQL datastores to provide scalability and availability for globally distributed data at the cost of sacrificing transactional guarantees. Recently, major web service providers like Google have moved towards building storage systems that provide ACID transactio ..."
Abstract
-
Cited by 10 (0 self)
- Add to MetaCart
(Show Context)
Web service providers have been using NoSQL datastores to provide scalability and availability for globally distributed data at the cost of sacrificing transactional guarantees. Recently, major web service providers like Google have moved towards building storage systems that provide ACID transactional guarantees for globally distributed data. For example, the newly published system, Spanner, uses Two-Phase Commit and Two-Phase Locking to provide atomicity and isolation for globally distributed data, running on top of Paxos to provide fault-tolerant log replication. We show in this paper that it is possible to provide the same ACID transactional guarantees for multi-datacenter databases with fewer crossdatacenter communication trips, compared to replicated logging, by using a more efficient architecutre. Instead of replicating the transactional log, we replicate the commit operation itself, by running Two-Phase Commit multiple times in different datacenters, and use Paxos to reach consensus among datacenters as to whether the transaction should commit. Doing so not only replaces several inter-datacenter communication trips with intra-datacenter communication trips, but also allows us to integrate atomic commitment and isolation protocols with consistent replication protocols so as to further reduce the number of cross-datacenter communication trips needed for consistent replication; for example, by eliminating the need for an election phase in Paxos. 1.
Harmony: Towards automated self-adaptive consistency in cloud storage
- in: 2012 IEEE International Conference on Cluster Computing (CLUSTER’12
, 2012
"... Abstract—In just a few years cloud computing has become a very popular paradigm and a business success story, with storage being one of the key features. To achieve high data availability, cloud storage services rely on replication. In this context, one major challenge is data consistency. In contra ..."
Abstract
-
Cited by 8 (5 self)
- Add to MetaCart
(Show Context)
Abstract—In just a few years cloud computing has become a very popular paradigm and a business success story, with storage being one of the key features. To achieve high data availability, cloud storage services rely on replication. In this context, one major challenge is data consistency. In contrast to traditional approaches that are mostly based on strong consis-tency, many cloud storage services opt for weaker consistency models in order to achieve better availability and performance. This comes at the cost of a high probability of stale data being read, as the replicas involved in the reads may not always have the most recent write. In this paper, we propose a novel approach, named Harmony, which adaptively tunes the consistency level at run-time according to the application requirements. The key idea behind Harmony is an intelligent estimation model of stale reads, allowing to elastically scale up or down the number of replicas involved in read operations to maintain a low (possibly zero) tolerable fraction of stale reads. As a result, Harmony can meet the desired consistency of the applications while achieving good performance. We have implemented Harmony and performed extensive evaluations with the Cassandra cloud storage on Grid’5000 testbed and on Amazon EC2. The results show that Harmony can achieve good performance without exceeding the tolerated number of stale reads. For instance, in contrast to the static eventual consistency used in Cassandra, Harmony reduces the stale data being read by almost 80 % while adding only minimal latency. Meanwhile, it improves the throughput of the system by 45% while maintaining the desired consistency requirements of the applications when compared to the strong consistency model in Cassandra. Keywords-consistency; replications; data stale; Cassandra; cloud; self-adaptive I.
Building global and scalable systems with atomic multicast,” Middleware
, 2014
"... The rise of worldwide Internet-scale services demands large distributed systems. Indeed, when handling several millions of users, it is common to operate thousands of servers spread across the globe. Here, replication plays a central role, as it contributes to improve the user experience by hiding f ..."
Abstract
-
Cited by 3 (1 self)
- Add to MetaCart
(Show Context)
The rise of worldwide Internet-scale services demands large distributed systems. Indeed, when handling several millions of users, it is common to operate thousands of servers spread across the globe. Here, replication plays a central role, as it contributes to improve the user experience by hiding failures and by providing acceptable latency. In this paper, we claim that atomic multicast, with strong and well-defined proper-ties, is the appropriate abstraction to efficiently design and implement globally scalable distributed systems. We sub-stantiate our claim with the design of two modern online services atop atomic multicast, a strongly consistent key-value store and a distributed log. In addition to presenting the design of these services, we experimentally assess their performance in a geographically distributed deployment. 1.
Message Futures: Fast Commitment of Transactions in Multi-datacenter Environments
"... Geo-replication of large Internet services is increasingly deployed for better data locality and fault tolerance. Maintaining consistency across datacenters is expensive and requires wide-area communication. This renders current solutions to either settle for weaker forms of consistency or suffer fr ..."
Abstract
-
Cited by 2 (1 self)
- Add to MetaCart
(Show Context)
Geo-replication of large Internet services is increasingly deployed for better data locality and fault tolerance. Maintaining consistency across datacenters is expensive and requires wide-area communication. This renders current solutions to either settle for weaker forms of consistency or suffer from large delays. In this work we present Message Futures, a strongly consistent concurrency control manager with low commit latency to ensure mutual consistency of replicas across datacenters. By judicial message passing of relevant information and at opportune time intervals, Message Futures can also enforce different priority levels of access, where each datacenter experiences a commit latency relative to its priority. In fact, in many common cases, transactions can be committed locally without the need for any communication. An experimental evaluation of Message Futures on a geo-replicated multi-datacenter setting is presented. We show that Message Futures achieves a commit latency around one RTT (Round-Trip Time) for datacenters with identical priority, and a latency comparable to committing locally for high priority datacenters.
Scalable State-Machine Replication
"... Many current online services have stringent availability and performance requirements. High availabil-ity entails tolerating component failures and is typically accomplished with replication. For many online services, “high performance ” essentially means the ability to serve an arbitrarily large lo ..."
Abstract
-
Cited by 2 (0 self)
- Add to MetaCart
(Show Context)
Many current online services have stringent availability and performance requirements. High availabil-ity entails tolerating component failures and is typically accomplished with replication. For many online services, “high performance ” essentially means the ability to serve an arbitrarily large load, something that can be achieved if the service can scale throughput when provided with additional resources (e.g., processors). In light of the requirements of modern online services, combining high availability and high performance without sacrificing consistency is a challenging endeavor of utmost importance. Replication has been extensively studied by the research community. Early work in the database com-munity dates back to the late 1970s and early 1980s, addressing both theoretical and practical concerns (e.g., one-copy serializability [1], primary copy replication [10]). In the distributed systems community, foundational work on object replication (i.e., non-transactional behavior) dates back to the late 1970s (e.g., state machine replication [7]). Although the topic is well-established, general-purpose replication techniques are mostly used for high availability, not performance—in fact, a replicated service is usu-ally outperformed by its single-server implementation. Replication for performance typically sacrifices consistency: the behavior of the replicated service does not correspond to the intuitive behavior of a single-server service. Although some services can settle for weak guarantees (e.g., [4]), weak consistency
How to Make Chord Correct (Using a Stable Base)
"... Abstract. The Chord distributed hash table (DHT) is well-known and frequently used to implement peer-to-peer systems. Chord peers find other peers, and access their data, through a ring-shaped pointer struc-ture in a large identifier space. Despite claims of proven correctness, i.e., eventual reacha ..."
Abstract
-
Cited by 1 (0 self)
- Add to MetaCart
(Show Context)
Abstract. The Chord distributed hash table (DHT) is well-known and frequently used to implement peer-to-peer systems. Chord peers find other peers, and access their data, through a ring-shaped pointer struc-ture in a large identifier space. Despite claims of proven correctness, i.e., eventual reachability, formal modeling has shown that the Chord ring-maintenance protocol is not correct under its original operating as-sumptions [25]. It has not, however, discovered whether Chord could be made correct with reasonable operating assumptions. The principle con-tribution of this paper is to provide the first specification of a correct version of Chord, using the assumption that there is a small “stable base ” of permanent members. The paper presents a simple, sufficient, and arguably necessary inductive invariant, and a partially automated proof of correctness. 1
Signature redacted Certified by...
"... This dissertation addresses two challenges relating to in-memory storage systems. The first challenge is storing and retrieving data at a rate close to the capabilities of the underlying memory system, particularly in the face of parallel accesses from multiple cores. We present Masstree, a high per ..."
Abstract
- Add to MetaCart
(Show Context)
This dissertation addresses two challenges relating to in-memory storage systems. The first challenge is storing and retrieving data at a rate close to the capabilities of the underlying memory system, particularly in the face of parallel accesses from multiple cores. We present Masstree, a high performance in-memory key-value store that runs on a single multi-core server. Masstree is derived from a concurrent B+tree. It provides lock-free reads for good multi-core performance, which requires special care to avoid writes interfering with concurrent reads. To reduce time spent waiting for memory for workloads with long common key prefixes, Masstree arranges a set of B+trees into a Trie. Masstree uses software prefetch to further hide DRAM latency. Several optimizations improve con-currency. Masstree achieves millions of queries per second on a 16-core server, which is more than 30x as fast as MongoDB [6] or VoltDB [17]. The second challenge is replicating storage for fault-tolerance without being limited by slow writes to stable disk storage. Lazy VSR is a quorum-based replication protocol that is fast and can recover from simultaneous crashes of all the replicas as long as a majority
