Results 1 -
3 of
3
Filo: consolidated consensus as a cloud service
"... Abstract Consensus is at the core of many production-grade distributed systems. Given the prevalence of these systems, it is important to offer consensus as a cloud service. To match the multi-tenant requirements of the cloud, consensus as a service must provide performance guarantees, and prevent ..."
Abstract
- Add to MetaCart
(Show Context)
Abstract Consensus is at the core of many production-grade distributed systems. Given the prevalence of these systems, it is important to offer consensus as a cloud service. To match the multi-tenant requirements of the cloud, consensus as a service must provide performance guarantees, and prevent aggressive tenants from disrupting the others. Fulfilling this goal is not trivial without overprovisioning and under-utilizing resources. We present Filo, the first system to provide consensus as a multi-tenant cloud service with throughput guarantees and efficient utilization of cloud resources. Tenants request an SLA by specifying their target throughput and degree of fault-tolerance. Filo then efficiently consolidates tenants on a shared set of servers using a novel placement algorithm that respects constraints imposed by the consensus problem. To respond to the load variations at runtime, Filo proposes a novel distributed controller that piggybacks on the consensus protocol to coordinate resource allocations across the servers and distribute the unused capacity fairly. Using a real testbed and simulations, we show that our placement algorithm is efficient at consolidating tenants, and while obtaining comparable efficiency and fairness, our distributed controller is ∼ 5x faster than the centralized baseline approach.
Ridge: high-throughput, low-latency atomic multicast
, 2015
"... It has been shown that the highest throughput for broadcasting messages in a point-to-point network is achieved with a ring topology. Although several ring-based group communication protocols have benefited from this observation, broadcasting mes-sages along a ring overlay may lead to high latencies ..."
Abstract
- Add to MetaCart
(Show Context)
It has been shown that the highest throughput for broadcasting messages in a point-to-point network is achieved with a ring topology. Although several ring-based group communication protocols have benefited from this observation, broadcasting mes-sages along a ring overlay may lead to high latencies: In a system with n processes, at least n − 1 communication steps are necessary for all processes to deliver a message. In this work, we argue that it is possible to reach optimal throughput without resort-ing to a ring topology (or to ip-multicast, typically unavailable inwide-area networks). This can be done by routing messages through different paths, while carefully using the available bandwidth at each process, resulting in a significantly lower latency for every message (potentially a single communication step). Based on this idea, we pro-pose Ridge, a Paxos-based atomic multicast protocol where each message is initially forwarded to a single destination, the distributor, whose responsibility is to propagate the message to all other destinations. To utilize all bandwidth available in the sys-tem, processes alternate in the role of distributor. By doing this, themaximum system throughput matches that of ring-based protocols, with a latency that is not signific-antly dependent on the size of the system. Finally, we show that Ridge can also deliver messages optimistically, with even lower latency.
Consensus-Oriented Parallelization: How to Earn Your First Million – To appear in Middleware ’15 –
"... Consensus protocols employed in Byzantine fault-tolerant systems are notoriously compute intensive. Unfortunately, the traditional approach to execute instances of such pro-tocols in a pipelined fashion is not well suited for mod-ern multi-core processors and fundamentally restricts the overall perf ..."
Abstract
- Add to MetaCart
(Show Context)
Consensus protocols employed in Byzantine fault-tolerant systems are notoriously compute intensive. Unfortunately, the traditional approach to execute instances of such pro-tocols in a pipelined fashion is not well suited for mod-ern multi-core processors and fundamentally restricts the overall performance of systems based on them. To solve this problem, we present the consensus-oriented paralleliza-tion (COP) scheme, which disentangles consecutive consen-sus instances and executes them in parallel by independent pipelines; or to put it in the terminology of our main target, today’s processors: COP is the introduction of superscalar-ity to the field of consensus protocols. In doing so, COP achieves 2.4 million operations per second on commodity server hardware, a factor of 6 compared to a contemporary pipelined approach measured on the same code base and a factor of over 20 compared to the highest throughput num-bers published for such systems so far. More important, however, is: COP provides up to 3 times as much through-put on a single core than its competitors and it can make use of additional cores where other approaches are confined by the slowest stage in their pipeline. This enables Byzan-tine fault tolerance for the emerging market of extremely demanding transactional systems and gives more room for conventional deployments to increase their quality of service.
