Citation Context

...al of these systems is to provide large-scale, flexible and secure resource sharing among dynamic collections of individuals, institutions, and resources, also referred as virtual organizations (VOs) =-=[4]-=-. In such settings, users from multiple administrative domains pool available resources to harness their aggregate power, to benefit from the increased computing power and the diversity of these resou...

Citation Context

...rformance studies by means of DiPerF [72] on PlanetLab for a pretty well know distributed lookup service. The service chosen for testing was the PAST application, built on top of the PASTRY substrate =-=[42]-=-. The chosen setup was very similar to the one used for DI-GRUBER: the same PlanetLab nodes (aroundThe design, usage, and performance of GRUBER 123 Table 5 DI-GRUBER accuracy function of the exchange...

Citation Context

...level services as well as any Grid services. A VO is a group of participants who seek to share resources for some common purpose. From the perspective of a single site in an environment such as Grid3 =-=[2, 69]-=-, a VO corresponds to either one or several users, depending on local uSLAs. However, the problem is more complex than a cluster fair-share allocation problem, because each VO has different allocation...

Citation Context

...e OSG/Grid3 [2] environment for raw resource brokering for various workloads. Also, we tested its performance for higher-level (Grid) service brokering on an ad-hoc Grid testbed deployed on PlanetLab =-=[70]-=-. 4.1 Testing Environment The first testing environment, OSG/Grid3, uses Euryale [65] as a system to run jobs over Grid3/OSG [2]. Euryale uses Condor-G [30] (and Globus’ WSGRAM [38]) to submit and to ...

Citation Context

...ddresses issues regarding how uSLAs can be stored, retrieved, and disseminated efficiently in these types of distributed environments and has been implemented by means of both the Grid Services (OGSI =-=[37]-=-) and Web Services (WS [38]) versions of the Globus Toolkit (GT3, respectively, GT4). The main elements of GRUBER are: None of the above systems fully address the problem of controlled resource manage...

Citation Context

...from 7:00 to 16:00 on date X for any remote user from Grid3” [24]. 2.2 Related Work Current solutions for controlling resource access in large scale distributed systems focus mainly on access control =-=[25, 26, 40]-=-, but other groups have started pursuing various paths for controlled resource sharing [5, 6, 27–32]. Finer access control mechanisms focus on enabling resource providers in expressing additional cond...

Citation Context

...d an interface for monitoring agreements at runtime. For this syntax, we use a schema that includes from the WS-Agreement specification support for resource monitoring metrics and goal specifications =-=[31, 43, 52]-=-. The resource monitoring metrics describe how various utilizations must be measured or how these quantities should be collected from an underlying monitoring system. A goal specification provides sup...

Citation Context

... users measure the distance to, and the masses of, clusters of galaxies in the SDSS data set; applications are composed again of many components, but in this case they have input/ output dependencies =-=[18]-=- that can be represented using direct acyclic graphs (DAGs). The iVDGL VO performs protein sequence comparisons at increasingly larger scales (various size workflows in which a single BLAST job has an...

Citation Context

...n ad-hoc Grid testbed deployed on PlanetLab [70]. 4.1 Testing Environment The first testing environment, OSG/Grid3, uses Euryale [65] as a system to run jobs over Grid3/OSG [2]. Euryale uses Condor-G =-=[30]-=- (and Globus’ WSGRAM [38]) to submit and to monitor jobs at sites. It takes a late binding approach in assigning jobs to sites, meaning that site placement decisions are made immediately prior to runn...

Citation Context

...1.2.1 OSG/Grid3 Scenario OSG/Grid3 comprises tens of institutions and hundreds to thousands of individual investigators that collectively control thousands of computers and associated storage systems =-=[2, 14]-=-. Each individual investigator and institution participates in, and contributes resources to multiple collaborative projects that vary in scale and formality. Figure 1 depicts a graphical representati...

Citation Context

.... All communications client-agreement provider are done through a specific API, and the underlying language for messages is RSL, with only one QoS quantitative parameter per reservation request. SNAP =-=[6]-=- tries to overcome previous resource managements by providing a generic framework instead of considering specialized classes of resources. The generalized framework maps resource interactions onto a w...

Citation Context

... such cooperative environments. Important challenges for uSLA-based resource management can arise in practice from: the lack of automated mechanisms for uSLA discovery, publication, or interpretation =-=[34]-=- to the complexity of the uSLA operations to be performed (to satisfy the requirements [35, 36]) of many resources and users. Additionally, the increased scale of such distributed systems calls for mi...

Citation Context

... GRUBER algorithms address differently these two resource types, conditioned mainly by the local site managers in each case (Condor [53], PBS [54], etc. for low-level resources vs. ARESRAN [24], SAML =-=[55]-=-, etc for services). 3.2.1 CPU Brokering GRUBER decides which consumers are best for a request from a CPU availability point of view by implementing the following logic: & & If there are fewer waiting...

Citation Context

...us scenarios. The paper ends with acquired lessons during this work and our conclusions. 2 Background Information and Related Work The thread shared by most Grid environments is cooperative computing =-=[3, 49]-=-. The goal of these systems is to provide large-scale, flexible and secure resource sharing among dynamic collections of individuals, institutions, and resources, also referred as virtual organization...

Citation Context

...nt Service Role Management, and the Strategic Agreement Management. Cremona focuses on advance reservations, automated SLA negotiation and verification, as well as advanced agreement management. GARA =-=[29]-=- represents modular and extensible system architecture for resource reservations to support end-to-end applications QoS. It offers a single interface for reserving different types of resources (networ...

Citation Context

...ssume that the cluster RM is able to enforce by itself the desired usage policies, which are provided by means of our S-POP module. Examples of such cluster RMs are Condor [58], Portable Batch System =-=[59]-=-, and Load Sharing Facility [60], widely used on Grid3 [2]. The S-POP’s main functions are to optionally provide and to translate to/from the RM local usage policies, and to monitor the actual resourc...

Citation Context

... control. In this case, we assume that the cluster RM is able to enforce by itself the desired usage policies, which are provided by means of our S-POP module. Examples of such cluster RMs are Condor =-=[58]-=-, Portable Batch System [59], and Load Sharing Facility [60], widely used on Grid3 [2]. The S-POP’s main functions are to optionally provide and to translate to/from the RM local usage policies, and t...

Citation Context

...from 7:00 to 16:00 on date X for any remote user from Grid3” [24]. 2.2 Related Work Current solutions for controlling resource access in large scale distributed systems focus mainly on access control =-=[25, 26, 40]-=-, but other groups have started pursuing various paths for controlled resource sharing [5, 6, 27–32]. Finer access control mechanisms focus on enabling resource providers in expressing additional cond...

Citation Context

...ed. Such relationships can be designed by bi-lateral rules that are driven by internal policies that govern any institution. Cremona is a such project developed at IBM as a part of the ETTK framework =-=[31]-=-. It is an implementation of the WS-Agreement specification and its architecture separates multiple layers of agreement management, orthogonal to the agreement management functions: the Agreement Prot...

Citation Context

...uring job execution, and (f) collating and presenting results. The broker supports a declarative and dynamic parametric programming model for creating Grid applications [33]. Service level agreements =-=[5, 21]-=- focus on establishing consumer-provider relationships concerning how resources must be consumed. Such relationships can be designed by bi-lateral rules that are driven by internal policies that gover...

Citation Context

...ources and services from independent utility providers that specialize in providing those services (see Fig. 3). Service examples include scheduling prediction services, monitoring services (MonALISA =-=[22]-=-), or community authorization services (e.g., DOE certificate authority [23]). In this scenario, the service provider requires that uSLAs express the amount of resources or services a consumer is enti...

Citation Context

...convincing the reader that even though DIGRUBER’s transaction throughput seems low compared to ‘other transaction processing systems,’ we have performed further performance studies by means of DiPerF =-=[72]-=- on PlanetLab for a pretty well know distributed lookup service. The service chosen for testing was the PAST application, built on top of the PASTRY substrate [42]. The chosen setup was very similar t...

Citation Context

...deals with re-planning. The postscript file transfers output files to the collection area, registers produced files, checks on successful job execution, and updates file popularity. DAS-2 environment =-=[66]-=-, a wide-area distributed computer of 200 Dual Pentium-III computer nodes, represents a second example where GRUBER can be used with success. The environment is built out of clusters of workstations, ...

Citation Context

...n certain resource without any further restrictions once access is granted. A policy based scheduling framework for Grid-enabled resource allocations is under development at the University of Florida =-=[32]-=-. This framework provides scheduling strategies that (a) control the request assignment to Grid resources by adjusting resource usage accounts or request priorities; (b) manage efficiently resources a...

Citation Context

...llows multiple decision points to coexist and cooperate in real-time. DI-GRUBER targets to provide a scalable management service with the same functionalities as GRUBER but in a distributed approach. =-=[64]-=- It is a two layer resource brokering service (see Fig. 7), capable of working over large Grids, extending GRUBER with support for multiple scheduling decision points that cooperate by periodically ex...

Citation Context

...e tested its performance for higher-level (Grid) service brokering on an ad-hoc Grid testbed deployed on PlanetLab [70]. 4.1 Testing Environment The first testing environment, OSG/Grid3, uses Euryale =-=[65]-=- as a system to run jobs over Grid3/OSG [2]. Euryale uses Condor-G [30] (and Globus’ WSGRAM [38]) to submit and to monitor jobs at sites. It takes a late binding approach in assigning jobs to sites, m...

Citation Context

...el agreements (uSLAs) Abbreviations uSLA VO RM 1 Introduction usage Service Level Agreement Virtual Organization Resource Manager GRUBER is an infrastructure for usage service level agreements (uSLAs =-=[1]-=-) specification, management and enforcement in any distributed environment in general; our implementation has been successfully DO09060; No of Pages100 C.L. Dumitrescu, et al. deployed and used in Gr...

Citation Context

...uring job execution, and (f) collating and presenting results. The broker supports a declarative and dynamic parametric programming model for creating Grid applications [33]. Service level agreements =-=[5, 21]-=- focus on establishing consumer-provider relationships concerning how resources must be consumed. Such relationships can be designed by bi-lateral rules that are driven by internal policies that gover...

Citation Context

...xt 7. return S 3.2.3 Service (Higher-level Resource) Brokering For service brokering, GRUBER uses an internal representation based on a variable time-slot representation, as introduced by Wolf et al. =-=[57]-=-, and each uSLA has assigned its own time intervals. This structure allows for an unlimited number of uSLAs and unlimited or unknown periods of time intervals. The processing logic we propose and eval...

Citation Context

...ssumed a static environment, and hence did not offer flexibility for dynamic environments. The DI-GRUBER implementation evolved to support dynamic environments through the use of the WS-Index Service =-=[39]-=- provided with GT4; our solution uses the functionalities offered by the WSIndex Service for service registering and querying. In our implementation, each DI-GRUBER decision point registers with a pre...

Citation Context

... arise in practice from: the lack of automated mechanisms for uSLA discovery, publication, or interpretation [34] to the complexity of the uSLA operations to be performed (to satisfy the requirements =-=[35, 36]-=-) of many resources and users. Additionally, the increased scale of such distributed systems calls for minimizing the need for human supervision and for automating as many management tasks as possible...

Citation Context

...isk, and network) and services (e.g., any Grid-enabled service). The GRUBER algorithms address differently these two resource types, conditioned mainly by the local site managers in each case (Condor =-=[53]-=-, PBS [54], etc. for low-level resources vs. ARESRAN [24], SAML [55], etc for services). 3.2.1 CPU Brokering GRUBER decides which consumers are best for a request from a CPU availability point of view...

Citation Context

... arise in practice from: the lack of automated mechanisms for uSLA discovery, publication, or interpretation [34] to the complexity of the uSLA operations to be performed (to satisfy the requirements =-=[35, 36]-=-) of many resources and users. Additionally, the increased scale of such distributed systems calls for minimizing the need for human supervision and for automating as many management tasks as possible...

Citation Context

...he authorization schemes implemented by the GT4, the so called Policy Decision Point (PDP). GT4 allows a chain of PDPs to be configured internally, with each PDP evaluating to an independent decision =-=[63]-=-. The authorization engine of the framework then uses a policy combination algorithm to combine the decisions returned by114 C.L. Dumitrescu, et al. Fig. 5 Correlations MP, SC, and AP Fig. 6 ARESRAN ...

Citation Context

...225 21,362 20,434 Speedup 67.32 60.22 63.12 51.77 4.2 Low-level Resource Brokering Example on OSG/Grid3 In Table 1 are captured the results achieved by running a BLAST workload composed of 1×500 jobs =-=[71]-=- over Grid3. Here, in the ideal case, the values are: Comp = 100, Util = 25.00, Delay = 3,600, Time = 3,000, and Speedup = 150, while the metrics were defined by Dumitrescu et al. [71]. In this case G...

Citation Context

...level services as well as any Grid services. A VO is a group of participants who seek to share resources for some common purpose. From the perspective of a single site in an environment such as Grid3 =-=[2, 69]-=-, a VO corresponds to either one or several users, depending on local uSLAs. However, the problem is more complex than a cluster fair-share allocation problem, because each VO has different allocation...

Citation Context

...m local or remote data source during job execution, and (f) collating and presenting results. The broker supports a declarative and dynamic parametric programming model for creating Grid applications =-=[33]-=-. Service level agreements [5, 21] focus on establishing consumer-provider relationships concerning how resources must be consumed. Such relationships can be designed by bi-lateral rules that are driv...

Citation Context

...d an interface for monitoring agreements at runtime. For this syntax, we use a schema that includes from the WS-Agreement specification support for resource monitoring metrics and goal specifications =-=[31, 43, 52]-=-. The resource monitoring metrics describe how various utilizations must be measured or how these quantities should be collected from an underlying monitoring system. A goal specification provides sup...

Citation Context

...or controlled resource provisioning. In such cases, mini-markets are built for resource brokeringThe design, usage, and performance of GRUBER 103 Fig. 3 Service outsourcing scenario and provisioning =-=[27]-=-. The Grid Service Broker, a part of the GridBus Project, mediates access to distributed resources by (a) discovering suitable data sources for a given analysis scenario, (b) suitable computational re...

Citation Context

...raphical representation of the OSG/ Grid3 sites. In this environment, several VOs exist that are composed of users with various common interests and applications. The most common ones are the USATLAS =-=[15]-=-, Sloan Digital Sky Survey (SDSS) [16] and iVDGL [17] VOs. USATLAS VO users simulate the collisions of protons on protons at 14 TeV for the LHC experiment; applications are composed of hundreds of emb...

Citation Context

...id3 sites. In this environment, several VOs exist that are composed of users with various common interests and applications. The most common ones are the USATLAS [15], Sloan Digital Sky Survey (SDSS) =-=[16]-=- and iVDGL [17] VOs. USATLAS VO users simulate the collisions of protons on protons at 14 TeV for the LHC experiment; applications are composed of hundreds of embarrassingly parallel programs with lar...

Citation Context

...his environment, several VOs exist that are composed of users with various common interests and applications. The most common ones are the USATLAS [15], Sloan Digital Sky Survey (SDSS) [16] and iVDGL =-=[17]-=- VOs. USATLAS VO users simulate the collisions of protons on protons at 14 TeV for the LHC experiment; applications are composed of hundreds of embarrassingly parallel programs with large input/output...

Citation Context

...mple, before important conferences we have observed that Grid utilization increases and job higher contention occurs, while during holidays most of the time resources are free for long time intervals =-=[19]-=-. These observations presented by Iosup et al. [19] motivate our introduction of a uSLA that ensures “whenever there is no contention users can use a certain amount of resources, while when contention...

Citation Context

...ovide 1000 requests for service A from 7:00 to 16:00 for 1 month for any remote user from Grid3” or “I accept 1,000 requests for service A from 7:00 to 16:00 on date X for any remote user from Grid3” =-=[24]-=-. 2.2 Related Work Current solutions for controlling resource access in large scale distributed systems focus mainly on access control [25, 26, 40], but other groups have started pursuing various path...

Citation Context

...e to enforce by itself the desired usage policies, which are provided by means of our S-POP module. Examples of such cluster RMs are Condor [58], Portable Batch System [59], and Load Sharing Facility =-=[60]-=-, widely used on Grid3 [2]. The S-POP’s main functions are to optionally provide and to translate to/from the RM local usage policies, and to monitor the actual resource utilization. An advantage of t...

Citation Context

...e uSLA syntax/semantic described above. SC descriptions are collected from the site RM configurations, filtered and, after translation, published through a specific monitoring system, e.g., VOGanglia =-=[61]-=-, MonALISA [22, 34] or GRUBERSiteMonitor [43, 62]. We have proposed three levels of description for the statement “site X gives ATLAS 30% over a month:” & & & MP: a description of a site manager’s pol...

Citation Context

...scriptions are collected from the site RM configurations, filtered and, after translation, published through a specific monitoring system, e.g., VOGanglia [61], MonALISA [22, 34] or GRUBERSiteMonitor =-=[43, 62]-=-. We have proposed three levels of description for the statement “site X gives ATLAS 30% over a month:” & & & MP: a description of a site manager’s policy for the site, e.g., MP (VOs) = “give ATLAS 30...

Citation Context

...one for wide-area communication, whereas Myrinet, a popular multi-Gigabit LAN, is used for local communication. The Grid scheduling technologies used in this environment are the Sun Grid Engine (SGE) =-=[67]-=- and KOALA [66]. KOALA has been designed and implemented by the PDS group in Delft in the context of the Virtual Lab for e-Science (VL-e) research project. The main feature of KOALA is its support for...

Citation Context

...ow uSLAs can be stored, retrieved, and disseminated efficiently in these types of distributed environments and has been implemented by means of both the Grid Services (OGSI [37]) and Web Services (WS =-=[38]-=-) versions of the Globus Toolkit (GT3, respectively, GT4). The main elements of GRUBER are: None of the above systems fully address the problem of controlled resource management in distributed environ...