We describe a technique for writing distributed applications which manage themselves over one or more utility computing infrastructures: by dynamically acquiring new computational resources, deploying themselves on these resources, and releasing others when no longer required. Unlike prior work, such management functionality is closely integrated with the application, allowing greater freedom in application-specific policies and faster response to failures and other changes in the environment without requiring any external management system. We address the programming complexity of this approach by applying constraint logic programming, and describe Rhizoma, an experimental runtime to explore these ideas. We present early experience of deploying

Current cloud services are deployed on wellprovisioned and centrally controlled infrastructures. However, there are several classes of services for which the current cloud model may not fit well: some do not need strong performance guarantees, the pricing may be too expensive for some, and some may be constrained by the data movement costs to the cloud. To satisfy the requirements of such services, we propose the idea of using distributed voluntary resources—those donated by end-user hosts—to form nebulas: more dispersed, lessmanaged clouds. We first discuss the requirements of cloud services and the challenges in meeting these requirements in such voluntary clouds. We then present some possible solutions to these challenges and also discuss opportunities for further improvements to make nebulas a viable cloud paradigm. 1

Resource discovery enables applications deployed in heterogeneous large-scale distributed systems to find resources that meet their execution requirements. In particular, most applications need resource requirements to be satisfied simultaneously for multiple resources (such as CPU, memory and network bandwidth). Due to the inherent dynamism in many large-scale systems caused by factors such as load variations, network congestion, and churn, providing statistical guarantees on such resource requirements is important to avoid application failures and overheads. However, existing resource discovery techniques either provide statistical guarantees only for individual resources, or take a static or memoryless approach to meeting resource requirements along multiple dimensions. In this paper, we present HiDRA, a resource discovery technique providing statistical guarantees for resource requirements spanning multiple dimensions simultaneously. Our technique takes advantage of the multivariate normal distribution for the probabilistic modeling of resource capacity over multiple dimensions. Through analysis of PlanetLab traces, we show that HiDRA performs nearly as well as a fully-informed algorithm, showing better precision and having recall within 3 % of such an algorithm. We have also deployed HiDRA on a 307-machine PlanetLab testbed, and our live experiments on this testbed demonstrate that HiDRA is a feasible, low-overhead approach to statistical resource discovery in a distributed system. 1

Content Distribution Networks (CDNs) have gained considerable attention in the past few years. As such, there is need for developing frameworks for carrying out CDN simulations. In this paper, we present a modeling and simulation framework for CDNs, called CDNsim. CDNsim has been designated to provide a realistic simulation for CDNs, simulating the surrogate servers, the TCP/IP protocol and the main CDN functions. The main advantages of this tool are its high performance, its extensibility and its user interface which is used to configure its parameters. CDNsim provides an automated environment for conducting experiments and extracting client, server and network statistics. The purpose of CDNsim is to be used as a testbed for CDN evaluation and experimentation. This is quite useful both for the research community (to experiment with new CDN data management techniques) and for CDN developers (to evaluate profits on prior certain CDN installations).

Abstract: Network testbeds such as Emulab and the Open Network Laboratory use virtualization to enable users to define end user virtual networks within a shared substrate. This involves mapping users ' virtual network nodes onto distinct substrate components and mapping virtual network links onto substrate paths. The mappings guarantee that different users ' activities can not interfere with one another. The problem of mapping virtual networks onto a shared substrate is a variant of the general graph embedding problem, long known to be NP-hard. In this paper, we focus on a more general version of the problem that supports advance scheduling of virtual network mappings. We experimentally study the performance of heuristic testbed schedulers in the context of the Open Network Laboratory. Our algorithms incorporate Mixed Integer Programs to optimally solve key subproblems, are fast enough to respond to reservation requests in under one second, and rarely reject requests needlessly.

Abstract—Operations work continuously to overcome the degrading effects of operational "entropy". Historically, groups have consolidated operations, for instance into data centers. But, consolidated operations does not accommodate the variation in heterogenous and widely distributed systems. For example, PlanetLab is a global deployment for developing and evaluating distributed, network services under real-world conditions and is composed of more than 700 online machines across 400 sites in more than 35 countries with more than 550 distinct hardware configurations (more than 75%). And, with so many different systems in use, software updates are not fully tested until deployed. To address these challenges, PlanetLab uses two strategies: automate common tasks to relieve central operators, and share the overhead of infrastructure management with remote operators. To do this we developed MyOps, a companion process to PlanetLab that automates the most common operations of monitoring, log-collection and analysis, event notification, and problem escalation to reach issue resolution. Using automation and notification, we find that simple incentives increased the likelihood of problem resolution by at least 15 % compared to notices alone and up to 34 % when there are no notices at all. I.

Over the past decade, Peer-to-Peer (P2P) file-sharing and streaming systems have evolved as a cheap and effective technology in distributing content to users. Guaranteeing a level of performance in P2P systems is, therefore, of utmost importance. However, P2P file-sharing and streaming applications suffer from a fundamental problem of unfairness, where many users have a tendency to free-ride by contributing little or no upload bandwidth while consuming much download bandwidth. By taking away an unfair share of resources, free-riders deteriorate the quality of service experienced by other users, by causing slower download times in P2P file-sharing networks and higher stream updates ’ miss rates in P2P streaming networks. Previous attempts at addressing fair bandwidth allocation in P2P, such as BitTorrent-like systems, suffer from slow peer discovery, inaccurate predictions of neighboring peers ’ bandwidth allocations, under-utilization of bandwidth, and complex parameter tuning. We present FairTorrent, a new deficit-based distributed algorithm that accurately rewards peers in accordance with their contribution in a file-sharing P2P system. In a

Distributed Hash Tables (DHTs) are scalable, self-organizing, and adaptive to underlying topology changes, thus being a promising infrastructure for hosting large-scale distributed applications. The ever-wider use of DHT infrastructures has found more and more applications that require support for range queries. Recently, a number of DHT-based range query schemes have been proposed. However, most of them suffer from high query delay or imbalanced load distribution. To address these problems, in this paper we first present an efficient indexing structure called Balanced Kautz (BK) tree that uniformly maps the m-dimensional data space onto DHT nodes, and then propose a BK tree-based range query scheme called ERQ that processes range queries in a parallel fashion and guarantees to return the results in a bounded delay. In a DHT with N nodes, ERQ can answer any range of query in less than log N(2log log N +1) hops in a load-balanced manner, irrespective of the queried range, the whole space size, or the number of queried attributes. The effectiveness of our proposals is demonstrated through experiments.

PlanetLab is a global deployment of more than 1000 registered machines for developing and evaluating distributed, network services under real-world conditions[4], [5]. The PlanetLab platform is based on MyPLC 1: an open-source, software

New technologies have recently emerged to challenge the very nature of computing: multicore processors, virtualized operating systems and networks, and data-center clouds. One can view these technologies as forming levels within a new, global computingplatform. We aim toopen a newarea of research, called multiplicity computing, that takes a radically differentapproachtotheengineeringofapplications for this platform. Unlike other efforts, which are largely focused on innovations within specific levels, multiplicity computing embraces the platform as a virtually unlimited space of essentially redundant resources. This space is formed as a whole from the cross product of resources available at each level in the platform, offering a“multiplicity”of end-to-end resources. We seek to discover fundamentally new ways of exploiting the combinatorial multiplicity of computational, communication, and storage resources to obtain scalable applications exhibiting improved quality, dependability, and security that are both predictable and measurable.