Results 1 -
7 of
7
Deconstructing Amazon EC2 Spot Instance Pricing
, 2011
"... Cloud providers possessing large quantities of spare capacity must either incentivize clients to purchase it or suffer losses. Amazon is the first cloud provider to address this challenge, by allowing clients to bid on spare capacity and by granting resources to bidders while their bids exceed a pe ..."
Abstract
-
Cited by 44 (1 self)
- Add to MetaCart
Cloud providers possessing large quantities of spare capacity must either incentivize clients to purchase it or suffer losses. Amazon is the first cloud provider to address this challenge, by allowing clients to bid on spare capacity and by granting resources to bidders while their bids exceed a periodically changing spot price. Amazon publicizes the spot price but does not disclose how it is determined. By analyzing the spot price histories of Amazon’s EC2 cloud, we reverse engineer how prices are set and construct a model that generates prices consistent with existing price traces. We find that prices are usually not market-driven as sometimes previously assumed. Rather, they are typically generated at random from within a tight price interval via a dynamic hidden reserve price. Our model could help clients make informed bids, cloud providers design profitable systems, and researchers design pricing algorithms.
Reliable Provisioning of Spot Instances for Compute-intensive Applications
"... Abstract—Cloud computing providers are now offering their unused resources for leasing in the spot market, which has been considered the first step towards a full-fledged market economy for computational resources. Spot instances are virtual machines (VMs) available at lower prices than their standa ..."
Abstract
-
Cited by 11 (1 self)
- Add to MetaCart
(Show Context)
Abstract—Cloud computing providers are now offering their unused resources for leasing in the spot market, which has been considered the first step towards a full-fledged market economy for computational resources. Spot instances are virtual machines (VMs) available at lower prices than their standard on-demand counterparts. These VMs will run for as long as the current price is lower than the maximum bid price users are willing to pay per hour. Spot instances have been increasingly used for executing compute-intensive applications. In spite of an apparent economical advantage, due to an intermittent nature of biddable resources, application execution times may be prolonged or they may not finish at all. This paper proposes a resource allocation strategy that addresses the problem of running compute-intensive jobs on a pool of intermittent virtual machines, while also aiming to run applications in a fast and economical way. To mitigate potential unavailability periods, a multifaceted faultaware resource provisioning policy is proposed. Our solution employs price and runtime estimation mechanisms, as well as three fault-tolerance techniques, namely checkpointing, task duplication and migration. We evaluate our strategies using tracedriven simulations, which take as input real price variation traces, as well as an application trace from the Parallel Workload Archive. Our results demonstrate the effectiveness of executing applications on spot instances, respecting QoS constraints, despite occasional failures. Index Terms—cloud computing; spot market; scheduling; fault-tolerance; I.
CAP 3: A Cloud Auto-Provisioning Framework for Parallel Processing Using On-demand and Spot Instances
"... Abstract—Cloud computing has drawn increasing attention from the scientific computing community due to its ease of use, elasticity, and relatively low cost. Because a high-performance computing (HPC) application is usually resource demanding, without careful planning, it can incur a high monetary ex ..."
Abstract
-
Cited by 3 (1 self)
- Add to MetaCart
(Show Context)
Abstract—Cloud computing has drawn increasing attention from the scientific computing community due to its ease of use, elasticity, and relatively low cost. Because a high-performance computing (HPC) application is usually resource demanding, without careful planning, it can incur a high monetary expense even in Cloud. We design a tool called CAP 3 (Cloud Auto-Provisioning framework for Parallel Processing) to help a user minimize the expense of running an HPC application in Cloud, while meeting the user-specified job deadline. Given an HPC application, CAP 3 automatically profiles the application, builds a model to predict its performance, and infers a proper cluster size that can finish the job within its deadline while minimizing the total cost. To further reduce the cost, CAP 3 intelligently chooses the Cloud’s reliable on-demand instances or low-cost spot instances, depending on whether the remaining time is tight in meeting the application’s deadline. Experiments on Amazon EC2 show that the execution strategy given by CAP 3 is costeffective, by choosing a proper cluster size and a proper instance type (on-demand or spot). Index Terms—Cloud computing; provisioning; virtual cluster; parallel scientific application; spot instance I.
Fault-Tolerant Workflow Scheduling Using Spot Instances on Clouds
"... Scientific workflows are used to model applications of high throughput computation and complex large scale data analysis. In recent years, Cloud computing is fast evolving as the target platform for such applications among researchers. Furthermore, new pricing models have been pioneered by Cloud pro ..."
Abstract
-
Cited by 1 (0 self)
- Add to MetaCart
(Show Context)
Scientific workflows are used to model applications of high throughput computation and complex large scale data analysis. In recent years, Cloud computing is fast evolving as the target platform for such applications among researchers. Furthermore, new pricing models have been pioneered by Cloud providers that allow users to provision resources and to use them in an efficient manner with significant cost reductions. In this paper, we propose a scheduling algorithm that schedules tasks on Cloud resources using two different pricing models (spot and on-demand instances) to reduce the cost of execution whilst meeting the workflow deadline. The proposed algorithm is fault tolerant against the premature termination of spot instances and also robust against performance variations of Cloud resources. Experimental results demonstrate that our heuristic reduces up to 70 % execution cost as against using only on-demand instances.
Contents lists available at SciVerse ScienceDirect Future Generation Computer Systems
"... journal homepage: www.elsevier.com/locate/fgcs Characterizing spot price dynamics in public cloud environments ..."
Abstract
- Add to MetaCart
(Show Context)
journal homepage: www.elsevier.com/locate/fgcs Characterizing spot price dynamics in public cloud environments
Characterizing Spot Price Dynamics in Public Cloud Environments
"... The surge in demand for utilizing public Cloud resources has introduced many trade-offs between price, performance and recently reliability. Ama-zon’s Spot Instances (SIs) create a competitive bidding option for the public Cloud users at lower prices without providing reliability on services. It is ..."
Abstract
- Add to MetaCart
(Show Context)
The surge in demand for utilizing public Cloud resources has introduced many trade-offs between price, performance and recently reliability. Ama-zon’s Spot Instances (SIs) create a competitive bidding option for the public Cloud users at lower prices without providing reliability on services. It is gen-erally believed that SIs reduce monetary cost to the Cloud users, however it appears from the literature that their characteristics have not been explored and reported. We believe that characterization of SIs is fundamental in the design of stochastic scheduling algorithms and fault tolerant mechanisms in public Cloud environments for spot market. In this paper, we have done a comprehensive analysis of SIs based on one year price history in four data centers of Amazon’s EC2. For this purpose, we have analyzed all different types of SIs in terms of spot price and the inter-price time (time between price changes) and determined the time dynamics for spot price in hour-in-day and day-of-week. Moreover, we have proposed a statistical model that fits well these two data series. The results reveal that we are able to model spot price dynamics as well as the inter-price time of each SI by the mixture of Gaus-sians distribution with three or four components. The proposed model is validated through extensive simulations, which demonstrate that our model
Performance Analysis of HPC Applications in the Cloud
"... The scalability of High Performance Computing (HPC) applications depends heavily on the efficient support of network communications in virtualized en-vironments. However, Infrastructure as a Service (IaaS) providers are more focused on deploying systems with higher computational power intercon-necte ..."
Abstract
- Add to MetaCart
(Show Context)
The scalability of High Performance Computing (HPC) applications depends heavily on the efficient support of network communications in virtualized en-vironments. However, Infrastructure as a Service (IaaS) providers are more focused on deploying systems with higher computational power intercon-nected via high-speed networks rather than improving the scalability of the communication middleware. This paper analyzes the main performance bot-tlenecks in HPC applications scalability on Amazon EC2 Cluster Compute platform: (1) evaluating the communication performance on shared memory and a virtualized 10 Gigabit Ethernet network; (2) assessing the scalability of representative HPC codes, the NAS Parallel Benchmarks, using an im-portant number of cores, up to 512; (3) analyzing the new cluster instances (CC2), both in terms of single instance performance, scalability and cost-efficiency of its use; (4) suggesting techniques for reducing the impact of the virtualization overhead in the scalability of communication-intensive HPC codes, such as the direct access of the Virtual Machine to the network and reducing the number of processes per instance; and (5) proposing the com-bination of message-passing with multithreading as the most scalable and cost-effective option for running HPC applications on Amazon EC2 Cluster Compute platform.
