Large scientific collaborations are moving towards service oriented architectures for implementation and deployment of globally distributed systems. Clarens is a high performance, easy to deploy Web Service framework that supports the construction of such globally distributed systems. This paper discusses some of the core functionality of Clarens that the authors believe is important for building distributed systems based on Web Services that support scientific analysis. 1.

The ultimate goal of grid technologies is to materialize the vision of grids as virtual supercomputers of unprecedented power, through utilization of geographically disperse distributively owned resources. Despite the overwhelming success of grids in running pleasantly parallel tasks, there still exists a large set of demanding applications considered the exclusive prerogative of real supercomputers. A few examples include complex systems and weather simulations, computational fluid dynamics and other tightly coupled parallel applications. These rely on a static execution environment with predictable performance, provided through efficient co-allocation of a large number of reliable homogeneously interconnected resources. In this paper, we describe a novel quasiopportunistic supercomputer system that enables execution of demanding parallel applications in grids through identification and implementation of the set of key technologies required to bridge the gap between grids and supercomputers. These technologies include an economic incentive-based framework for establishing and maintaining grid-wise allocation agreements; a co-allocation subsystem that is integrated with the economic framework and enhanced by communication topology-aware allocation mechanisms; a fault tolerant message passing library that hides the failures of the underlying resources; and data pre-staging orchestration.

We describe the NSF-funded UltraLight project. The project’s goal is to meet the data-intensive computing challenges of the next generation of particle physics experiments with a comprehensive, network-focused agenda. In particular we argue that instead of treating the network traditionally, as a static, unchanging and unmanaged set of inter-computer links, we instead will use it as a dynamic, configurable, and closely monitored resource, managed end-toend, to construct a next-generation global system able to meet the data processing, distribution, access and analysis needs of the high energy physics (HEP) community. While the initial UltraLight implementation and services architecture is being developed to serve HEP, we expect many of UltraLight’s developments in the areas of networking, monitoring, management, and collaborative research, to be applicable to many fields of data intensive e-science. In this paper we give an overview of, and motivation for the UltraLight project, and provide early results within different working areas of the project.