Citation Context

...ce managers (i.e. PBSs[5], Condor [48], etc). General-purpose components such assGARA (general architecture for advanced reservation) [17],sand specialized resource management services such as Falkons=-=[40]-=- (although strictly speaking, Falkon also provides servicessbeyond the fabric layer).sFigure 2: Grid Protocol ArchitecturesThe connectivity layer defines core communication andsauthentication protocol...

Citation Context

...pReduce [5] is an example for this category. 3) Keeping data size modest, but increasing the number of tasks moves us into the loosely coupled applications involving many tasks (yellow); Swift/Falkon =-=[6, 7]-=- and Pegasus/DAGMan [8] are examples of this category. 4) Finally, the combination of both many tasks and large datasets moves us into the data-intensive many-task computing category (green); examples...

Citation Context

...ust coincide with some other event, such as a class [11, 38], and in applications expressible as a workflow of independent tasks that can be executed more efficiently by multilevel scheduling methods =-=[29, 36, 16]-=-. The need for best-effort resource allocation arises in many contexts and is supported by batch schedulers. Although batch schedulers can and do include support for advance reservations, this mechani...

Citation Context

...rained parallelism using workflow frameworks such as Kepler [16] and Taverna [17], where the individual tasks could themselves be parallel applications written in MPI. Software systems such as Falkon =-=[18]-=-, SWARM [19], and DAGMan [20] can be used to schedule applications which comprise of a collection of a large number of individual sub tasks. Once these applications are developed, in the traditional a...

Citation Context

...nt can look up the status information by relying on ZHT. We performed several synthetic benchmark experiments to evaluate the performance of MATRIX, and how it compares to the state-of-the-art Falkon =-=[53]-=- light-weight task execution framework (see Figure 18). The workload consisted of 100K tasks of various lengths, ranging from 0 seconds (NO-OP) to 1, 2, 4, and 8 seconds. It might be difficult to comp...

Citation Context

...ited knowledge of workloads, which may involve many different applications. In our exploration of these issues, we build upon previous work on Falkon, a Fast and Light-weight tasK executiON framework =-=[4, 12]-=-, which provides for dynamic acquisition and release of resources (“workers”) and the dispatch of analysis tasks to those workers. We describe Falkon data caching extensions that enable (in their curr...

Citation Context

...atform supporting flexible runtime policies for large-scale execution of multiple BOTs. First, we unify the grids by establishing an overlay of execution clients, a technique termed overlay computing =-=[2, 10, 26, 34]-=-. While widely used for eliminating long queuing delays and aggregating multiple grids, the existing technologies fall short in grids with strict firewall policies and private networks. Our implementa...

Citation Context

... data locality for general computing frameworks that are not constrained by the map-reduce model to allow data-aware job scheduling with batch schedulers (e.g. PBS [16], SGE [17], Condor [18], Falkon =-=[19]-=-). We believe that future HEC systems should be designed with nonvolatile memory (e.g. solid state memory [20], phase change memory [21]) on every compute node (Figure 7); every compute node would act...

Citation Context

...ications because it enables the application to allocate a resource once and use it to execute many tasks, which reduces the total scheduling overhead and can dramatically improve workflow performance =-=[28, 45, 46, 41]-=-. Although in clusters and grids, pilot job systems, such as Condor glide-ins [13] aim to simulate resource provisioning, they face limitations imposed by the target systems, for example the maximum w...