Results 1 - 10
of
46
Condor and the Grid
"... Since 1984, the Condor project has helped ordinary users to do extraordinary computing. Today, the project continues to explore the social and technical problems of cooperative computing on scales ranging from the desktop to the world-wide computational grid. In this chapter, we provide the history ..."
Abstract
-
Cited by 227 (37 self)
- Add to MetaCart
(Show Context)
Since 1984, the Condor project has helped ordinary users to do extraordinary computing. Today, the project continues to explore the social and technical problems of cooperative computing on scales ranging from the desktop to the world-wide computational grid. In this chapter, we provide the history and philosophy of the Condor project and describe how it has interacted with other projects and evolved along with the field of distributed computing. We outline the core components of the Condor system and describe how the technology of computing must reflect the sociology of communities. Throughout, we reflect on the lessons of experience and chart the course travelled by research ideas as they grow into production systems.
The Kangaroo Approach to Data Movement on the Grid
, 2001
"... Access to remote data is one of the principal challenges of grid computing. While performing I/O, grid applications must be prepared for server crashes, performance variations, and exhausted resources. To achieve high throughput in such a hostile environment, applications need a resilient service th ..."
Abstract
-
Cited by 102 (20 self)
- Add to MetaCart
Access to remote data is one of the principal challenges of grid computing. While performing I/O, grid applications must be prepared for server crashes, performance variations, and exhausted resources. To achieve high throughput in such a hostile environment, applications need a resilient service that moves data while hiding errors and latencies. We illustrate this idea with Kangaroo, a simple data movement system that makes opportunistic use of disks and networks to keep applications running. We demonstrate that Kangaroo can achieve better end-to-end performance than traditional data movement techniques, even though its individual components do not achieve high performance.
Parrot: Transparent User-Level Middleware for Data Intensive Computing
- In Workshop on Adaptive Grid Middleware
, 2003
"... Distributed computing continues to be an alphabet-soup of services and protocols for managing computation and storage. To live in this environment, applications require middleware that can transparently adapt standard interfaces to new distributed systems; such software is known as an interposition ..."
Abstract
-
Cited by 50 (23 self)
- Add to MetaCart
(Show Context)
Distributed computing continues to be an alphabet-soup of services and protocols for managing computation and storage. To live in this environment, applications require middleware that can transparently adapt standard interfaces to new distributed systems; such software is known as an interposition agent. In this paper, we present several lessons learned about interposition agents via a progressive study of design possibilities. Although performance is an important concern, we pay special attention to less tangible issues such as portability, reliability, and compatibility. We begin with a comparison of seven methods of interposition, focusing on one method, the debugger trap, that requires special techniques to achieve acceptable performance on popular operating systems. Using this method, we implement a complete interposition agent, Parrot, that splices existing remote I/O systems into the namespace of standard applications. The primary design problem of Parrot is the mapping of fixed application semantics into the semantics of the available I/O systems. We offer a detailed discussion of how errors and other unexpected conditions must be carefully managed in order to keep this mapping intact. We conclude with a evaluation of the performance of the I/O protocols employed by Parrot, and use an Andrew-like benchmark to demonstrate that semantic differences have consequences in performance. 1.
Pipeline and Batch Sharing in Grid Workloads
- In Proceedings of High-Performance Distributed Computing (HPDC-12
, 2003
"... We present a study of six batch-pipelined scientific workloads that are candidates for execution on computational grids. Whereas other studies focus on the behavior of single applications, this study characterizes workloads composed of pipelines of sequential processes that use file storage for comm ..."
Abstract
-
Cited by 39 (12 self)
- Add to MetaCart
We present a study of six batch-pipelined scientific workloads that are candidates for execution on computational grids. Whereas other studies focus on the behavior of single applications, this study characterizes workloads composed of pipelines of sequential processes that use file storage for communication and also share significant data across a batch. This study includes measurements of the memory, CPU, and I/O requirements of individual components as well as analyses of I/O sharing within complete batches. We conclude with a discussion of the ramifications of these workloads for end-to-end scalability and overall system design.
Gathering at the well: Creating communities for grid I/O
- In Proceedings of Supercomputing 2001
, 2001
"... Grid applications have demanding I/O needs. Schedulers must bring jobs and data in close proximity in order to satisfy throughput, scalability, and policy requirements. Most systems accomplish this by making either jobs or data mobile. We propose a system that allows jobs and data to meet by binding ..."
Abstract
-
Cited by 37 (7 self)
- Add to MetaCart
(Show Context)
Grid applications have demanding I/O needs. Schedulers must bring jobs and data in close proximity in order to satisfy throughput, scalability, and policy requirements. Most systems accomplish this by making either jobs or data mobile. We propose a system that allows jobs and data to meet by binding execution and storage sites together into I/O communities which then participate in the wide-area system. The relationships between participants in a community may be expressed by the ClassAd framework. Extensions to the framework allow community members to express indirect relations. We demonstrate our implementation of I/O communities by improving the performance of a key high-energy physics simulation on an international distributed system. 1.
A Java-based Composition and Manipulation Framework for Computational Grids
- In: Proceedings of the CCGrid2002
, 2002
"... Increasingly complex problems in many areas of today's scientific research demand new magnitudes of compute power. Top of the line supercomputers can merely provide for a fraction of the resources required by many computationally intensive projects currently undertaken at research institutions ..."
Abstract
-
Cited by 32 (5 self)
- Add to MetaCart
Increasingly complex problems in many areas of today's scientific research demand new magnitudes of compute power. Top of the line supercomputers can merely provide for a fraction of the resources required by many computationally intensive projects currently undertaken at research institutions around the world. On the other hand, a vast amount of personal computers and workstations, many more powerful than a supercomputer was 10 years ago, sit idle or utilize their resources to generate screensaver images while their user is away.
CODO: Firewall Traversal by Cooperative On-Demand Opening, 14th IEEE Symposium on High Performance Distributed Computing, (HPDC14), Research Triangle Park, July 2005 http://www.cs.wisc.edu/ sschang/papers/CODO-hpdc.pdf [D-Grid]. M.Meier, E.Gruenter, R.Nie
- Grid Computing, 2004. Proceedings. Fifth IEEE/ACM International Workshop on Grid Computing, ISBN 0-7695-2256-4
"... Firewalls and network address translators (NATs) cause significant connectivity problems together with their benefits. Many ideas to solve these problems have been explored both in academia and in industry. Yet, no single system solves the problem entirely. Considering diverse and even conflicting u ..."
Abstract
-
Cited by 20 (2 self)
- Add to MetaCart
(Show Context)
Firewalls and network address translators (NATs) cause significant connectivity problems together with their benefits. Many ideas to solve these problems have been explored both in academia and in industry. Yet, no single system solves the problem entirely. Considering diverse and even conflicting use cases and requirements from organizations, we propose an integrated approach that provides a suite of mechanisms and allows communicating peers to choose the best available mechanism. As an important step toward the final goal, we categorize previous efforts and briefly analyze each category in terms of use cases supported, security impacts, performance, and so forth. We then introduce a new firewall traversal system, called CODO, that solves the connectivity problem more securely than other systems in its category. 1.
TESLA: A Transparent, Extensible Session-Layer Architecture for End-to-end Network Services
- In Proc. of the Fourth USENIX Symposium on Internet Technologies and Systems (USITS
, 2003
"... This paper describes TESLA, a transparent and extensible framework allowing session-layer services to be developed using a high-level flow-based abstraction. TESLA services can be deployed transparently using dynamic library interposition and can be composed by chaining event handlers in a graph str ..."
Abstract
-
Cited by 18 (0 self)
- Add to MetaCart
(Show Context)
This paper describes TESLA, a transparent and extensible framework allowing session-layer services to be developed using a high-level flow-based abstraction. TESLA services can be deployed transparently using dynamic library interposition and can be composed by chaining event handlers in a graph structure. We show how TESLA can be used to implement several session-layer services including encryption, SOCKS, application-controlled routing, flow migration, and traffic rate shaping, all with acceptably low performance degradation
Supporting Secure Ad-hoc User Collaboration in Grid Environments
, 2002
"... We envision that many usage scenarios involving computational grids will be based on small, dynamic working groups for which the ability to establish transient collaboration with little or no intervention from resource administrators is a key requirement. Current grid security mechanisms support ind ..."
Abstract
-
Cited by 17 (1 self)
- Add to MetaCart
(Show Context)
We envision that many usage scenarios involving computational grids will be based on small, dynamic working groups for which the ability to establish transient collaboration with little or no intervention from resource administrators is a key requirement. Current grid security mechanisms support individual users who are members of well-defined virtual organizations. Recent research seeks to provide manageable grid security services for self-regulating, stable communities. Our prior work with component-based systems for grid computation demonstrated a need to support spontaneous, limited, short-lived collaborations. Such collaborations most often rely on shared or delegated fine grained access privileges to data and executable files as well as to grid compute resources. The mechanisms we are developing focus on the management and the enforcement of fine grained access rights. Our solution employs standard attribute certificates to bind rights to users (or their surrogates) and enables the high level management of such fine grained privileges which may be freely delegated, traded, and combined. Enforcement is provided by POSIX operating systems extensions that extend standard file permissions and regulate resource usage through access control lists. These extensions are available for common platforms and fully support legacy services.
Parrot: An Application Environment for Data-Intensive Computing
- Journal of Parallel and Distributed Computing Practices
, 2005
"... Distributed computing continues to be an alphabet-soup of services and protocols for managing computation and storage. To live in this environment, applications require middleware that can transparently adapt standard interfaces to new distributed systems; such middleware is known as an interpositio ..."
Abstract
-
Cited by 16 (4 self)
- Add to MetaCart
Distributed computing continues to be an alphabet-soup of services and protocols for managing computation and storage. To live in this environment, applications require middleware that can transparently adapt standard interfaces to new distributed systems; such middleware is known as an interposition agent. In this paper, we present several lessons learned about interposition agents via a progressive study of design possibilities. Although performance is an important concern, we pay special attention to less tangible issues such as portability, reliability, and compatibility. We begin with a comparison of seven methods of interposition and select one method, the debugger trap, that is the slowest but also the most reliable. Using this method, we implement a complete interposition agent, Parrot, that splices existing remote I/O systems into the namespace of standard applications. The primary design problem of Parrot is the mapping of fixed application semantics into the semantics of the available I/O systems. We o#er a detailed discussion of how errors and other unexpected conditions must be carefully managed in order to keep this mapping intact. We conclude with a evaluation of the performance of the I/O protocols employed by Parrot, and use an Andrewlike benchmark to demonstrate that semantic di#erences have consequences in performance.
