B. Allcock, I. Foster, V. Nefedova, A. Chevrenak, E. Deelman, C. Kesselman, et al. HighPerformance Remote Access to Climate Simulation Data: A Challenge Problem for Data Grid Technologies. In proceedings of SC' 01.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....(SRB) 4] Data cataloging and access to relational and object oriented databases is provided. Replica optimization is not provided yet but SRB might profit from approaches like ours. The most related work with respect to replica access optimization can be found in the Earth Since Grid (EDG) [3] project where preliminary replica selection has been demonstrated using the Network Weather Service (NWS) 23] Rather than providing an integrated replication framework with consistency checks etc. GridFTP transfer sources and destinations are selected via NWS. Within the Grid and High Energy ....

Bill Allcock, Ian Foster, Veronika Nefedov, Ann Chervenak, Ewa Deelman, Carl Kesselman, Jason Lee, Alex Sim, Arie Shoshani, Bob Drach, and Dean Williams. High-Performance Remote Access to Climate Simulation Data: A Challenge Problem for Data Grid Technologies. In Supercomputing 2001.

....implemented a high performance service for use by Grid schedulers. The organization of the system is discussed and performance results are presented. 1 Introduction In a Computational Grid setting, the Information Service (IS) is a key component. Experience has shown that schedulers for the Grid [1, 4, 5, 12, 13] require high performance and timely delivery of IS data particularly performance data. If the IS is slow, the scheduler itself will be slow, thereby negatively impacting the user s perception of application performance. Further, it is critical that the IS be accessible in as open a fashion ....

B. Allcock, I. Foster, V. Nefedova, A. Chervenak, E. Deelman, C. Kesselman, J. Leigh, A. Sim, and A. Shoshani. High-performance remote access to climate simulation data: A challenge problem for data grid technologies. In Proceedings of SC01, 2001. http://www.globus.org/research/papers/ sc01ewa_esg_chervenak_final.pdf.

.... challenge problems in areas such as biophysics, chemistry, biology, scientific instrumentation [4] drug design [23] tomography [35] high energy physics [38] data mining, financial analysis, nuclear simulations, material science, chemical engineering, environmental studies, climate modeling [37], weather prediction, molecular biology, neuroscience brain activity analysis [66] structural analysis, mechanical CAD CAM and astrophysics. In the past, applications were developed as monolithic entities. A monolithic application is typically the same as a single executable program that does ....

B. Allcock, I. Foster, V. Nefedova, A. Chervenak, E. Deelman, C. Kesselman, J. Lee, A. Sim, A. Shoshani, B. Drach and D. Williams, High-Performance Remote Access to Climate Simulation Data: A Challenge Problem for Data Grid Technologies, Proceedings of SC2001.

....can be costly, both in terms of wasted resource and lost time while the probe takes place. Performance monitoring tools such as the Network Weather Service (NWS) 25, 30] address the first problem by making the predictions explicitly using statistical techniques. Application level schedulers [2, 5, 19, 23] have been able to use these predictions (and measures of prediction error) to achieve good execution performance levels in a variety of Grid settings, despite fluctuating resource performance. However, in making these forecasts, current NWS methodologies (described in [29, 30] share the second ....

....TCP IP throughput, are unfortunately best forecast from a history of such responses. Moreover, the dynamics of the network are frequently changing, making old measurements increasingly less statistically valuable as time passes. As such, instrumenting Grid applications or tools (e.g. GridFTP [2]) and using the observations from the instrumentation may yield dramatic inaccuracies if the time between application runs is significant. In this work, we describe a new multivariate forecasting technique that enables the NWS to automatically correlate monitor data from different sources, and to ....

B. Allcock, I. Foster, V. Nefedova, A. Chervenak, E. Deelman, C. Kesselman, J. Lee, A. Sim, A. Shoshani, B. Drach, and D. Williams. Highperformance remote access to climate simulation data. In Proc. SC2001.

....servers to supply data streams fast enough to enable the execution of various multi user, real time applications in an Internet environment. Bethel et al. 12] show how DPSS is used for building Visapult, a prototype and framework for remote visualization of large datasets. Allcock et al. [3] point out that the data grid infrastructure will need to service thousands of users efficiently, and also highlight that the management of data and replicas is also an important aspect of grid aware applications. In some sense, our approach is complementary to these efforts because it also ....

B.Allcock,I.Foster,V.Nefedova,A.Chervenak,E.Deelman, C. Kesselman, J. Lee, A. Sim, A. Shoshani, B. Drach, and D. Williams. High-performance remote access to climate simulation data: A challenge problem for data grid technologies. In Proceedings of the 2001.

....operation of distributed servers to supply data streams fast enough to enable various multi user, real time applications in an Internet environment. Bethel et al. 10] show how DPSS is used for building Visapult, a prototype and framework for remote visualization of large datasets. Allcock et al. [3] points out that the data grid infrastructure will need to service thousands of users efficiently, and also highlight the point that the management of data and replicas is also an important aspect Application Server subquery ubq results subquery subq eUl Figure 2. The suite of ....

B. Allcock, I. Foster, V. Nefedova, A. Chervenak, E. Deelman, C. Kesselman, J. Lee, A. Sim, A. Shoshani, B. Drach, and D. Williams. High-performance remote access to climate simulation data: A challenge problem for data grid technologies. In Proceedings of the 2001.

....can be costly, both in terms of wasted resource and lost time while the probe takes place. Performance monitoring tools such as the Network Weather Service (NWS) 23, 28] address the first problem by making the predictions explicitly using statistical techniques. Application level schedulers [2, 4, 16, 20] have been able to use these predictions (and measures of prediction error) to achieve good execution performance levels in a variety of Grid settings, despite fluctuating resource performance. However, to make these forecasts, current NWS methodologies (described in [27, 28] share the second ....

....TCP IP throughput, are unfortunately best forecast from a history of such responses. Moreover, the dynamics of the network are frequently changing, making old measurements increasingly less statistically valuable as time passes. As such, instrumenting Grid applications or tools (e.g. GridFTP [2]) and using the observations from the instrumentation may yield dramatic inaccuracies if the time between application runs is significant. In this work, we describe a new multivariate forecasting technique that enables the NWS to automatically correlate monitor data from different sources, and to ....

B. Allcock, I. Foster, V. Nefedova, A. Chervenak, E. Deelman, C. Kesselman, J. Lee, A. Sim, A. Shoshani, B. Drach, and D. Williams. Highperformance remote access to climate simulation data. In Proc. SC2001.

.... can be used to solve grand challenge problems in areas such as biophysics, chemistry, biology, drug design [17] tomography [28] high energy physics [31] data mining, financial analysis, nuclear simulations, material science, chemical engineering, environmental studies, climate modeling [30], weather prediction, molecular biology, structural analysis, mechanical CAD CAM, astrophysics, scientific instrumentation, and so on. 5. Conclusions and Future Trends There are currently a large number of projects and diverse range of existing and emerging Grid developmental approaches being ....

B. Allcock et. al., High-Performance Remote Access to Climate Simulation Data: A Challenge Problem for Data Grid Technologies, IEEE Supercomputing (SC 2001), Denver, USA.

No context found.

B. Allcock, I. Foster, V. Nefedova, A. Chervenak, E. Deelman, C. Kesselman, J. Leigh, A. Sim, A. Shoshani, B. Drach, D. Williams, "High-Performance Remote Access to Climate Simulation Data: A Challenge Problem for Data Grid Technologies," presented at SC2001.

....values logged are sustained measures through the transfer. The end to end GridFTP bandwidth is obtained by the formula: BW = Filesize TransferTime 2. 1 Instrumenting GridFTP GridFTP is part of the Globus Toolkit [11] and is widely used as a secure, high performance data transfer protocol [2, 5, 7, 11, 15, 27]. It extends standard FTP with several features needed in Grid environments, such as security on control and data channels, multiple data channels for parallel transfers, partial file transfers, and third party transfers [1] GridFTP essential consists of two modules, the control module and the ....

W. Allcock, I. Foster, V. Nefedova, A. Chevrenak, E. Deelman, C. Kesselman, A. Sim, A. Shoshani, B. Drach, D. Williams, High-Performance Remote Access to Climate Simulation Data: A Challenge Problem for Data Grid Technologies, To be published in the Proceedings of Supercomputing (SC'01), 2001.

....Experience The NASA Information Power Grid is perhaps the most extensive deployment of the MDS 2 Grid Information Service, and has done extensive measurements of that service. Recent simulations of US airspace traffic were conducted across the three sites of this grid [11] The Earth Systems Grid [12] brings together shared resources and climate scientists from ANL, NCAR, ORNL, and LBL. It is progressing past the demonstration stage, developing software and infrastructure to tackle important problems in the modeling of climate processes. Within the CMS experiment, an application called GDMP ....

Allcock, B., et al., High-Performance Remote Access to Climate Simulation Data: A Challenge Problem for Data Grid Technologies. Supercomputing

....4 describes the simulation framework and the results. Finally, we present brief conclusions and future directions in Section 5. 2. Replication in Data Grids Globus is a community based, open architecture, opensource set of services and software libraries that support Grids and Grid applications [1, 2]. The Globus Toolkit [5] provides middleware services for grid computing environments. There are four main components of Globus: the Grid Security Infrastructure, the Globus Resource Management architecture, the Globus Information Management, and the Data Management architecture. This latter ....

W. Allcock, I. Foster, V. Nefedova, A. Chervenak, E. Deelman, C. Kesselman, J. Lee, A. Sim, A. Shoshani, B. Drach, D. Williams. "High-Performance Remote Access to Climate Simulation Data: A Challenge Problem for Data Grid Technologies." Proceedings of SC 2001.

....(PPDG) 3] and the Earth Science Grid (ESG) 5] to perform grid file replication functions. The HRM was also used in a demo for SuperComputing 2000 as part of an infrastructure to get files from multiple locations for an Earth Science Grid application (ESG) This was described in a recent paper [9]. We are now evaluating several cache replacement policies to be used by DRMs, by both conducting simulations and setting up real testbeds. 4. The implementation of the analysis scenario The analysis scenario described in Figure 1 was implemented as part of a demo during the Supercomputing ....

B. Allcock, A. Chervenak, E. Deelman, R. Drach, I. Foster, C. Kesselman, J. Lee, V. Nefedova, A. Sim, A. Shoshani, D. Williams, High-Performance Remote Access to Climate Simulation Data: A Challenge Problem for Data Grid Technologies, Proceedings of Supercomputing Conference (2001).

....our system could be extended to perform file transfer probes at regular intervals for the sake of gathering data about the performance, and not for transferring useful data, but we do not consider that approach here. Figure 1: ISI ANL GridFTP end to end bandwidth and NWS probe bandwidth GridFTP [3] is part of the Globus Toolkit TM [14] and is widely used as a secure, high performance data transfer protocol [3, 6, 10, 18, 32] It extends standard FTP implementations with several features needed in Grid environments, such as security on control and data channels, multiple data channels for ....

....the performance, and not for transferring useful data, but we do not consider that approach here. Figure 1: ISI ANL GridFTP end to end bandwidth and NWS probe bandwidth GridFTP [3] is part of the Globus Toolkit TM [14] and is widely used as a secure, high performance data transfer protocol [3, 6, 10, 18, 32]. It extends standard FTP implementations with several features needed in Grid environments, such as security on control and data channels, multiple data channels for parallel transfers, partial file transfers, and third party transfers. GridFTP consists of two modules: the control, or server, ....

W. Allcock, I. Foster, V. Nefedova, A. Chevrenak, E. Deelman, C. Kesselman, A. Sim, A. Shoshani, B. Drach, and D. Williams, HighPerformance Remote Access to Climate Simulation Data: A Challenge Problem for Data Grid Technologies, Proceedings of Supercomputing (SC'01), November, 2001.

....components, involving experiments coupling end to end resources for interactive visual exploration of large data in representative distributed environments. 1. Introduction Grid environments [21] are increasingly being used by applications such as particle physics [22] climate modeling [3], or astrophysics [4] These applications make use of unique, high end supercomputers and storage systems and produce large multi dimensional data sets. This type of work is increasingly being performed in collaborative efforts between geographically distributed scientists and organizations, ....

B. Allcock, I. Foster, V. Nefedova, A. Chervenak, E. Deelman, C. Kesselman, A. Sim, A. Shoshani, B. Drach, and D. Williams. High-performance remote access to climate simulation data: A challenge problem for data grid technologies. Submitted for publication, 2001.

....all) HEP persistent objects can be treated as read only after initial creation. 2.2 Related Data Grid Projects Data Grid concepts are being explored in a number of projects worldwide. In the U.S. the Earth System Grid (ESG) is applying Data Grid technologies to the management of climate data [Allc01b], while the Particle Physics Data Grid (PPDG) PPDG01] and Grid Physics Network (GriPhyN) GriP01] projects are both working in the HEP domain. Efforts have started within PPDG as well as EU DataGrid to use the GDMP code base. Since GDMP is and has been a mutual effort of EU DataGrid and PPDG, ....

W. Allcock, I. Foster, V. Nefedova, A. Chervenak, E. Deelman, C. Kesselman, J. Lee, A. Sim, A. Shoshani, B. Tierney, B. Drach, D. Williams, High-Performance Remote Access to Climate Simulation Data: A Challenge Problem for Data Grid Technologies, Preprint, Argonne National Laboratory, 2001.

No context found.

B. Allcock, I. Foster, V. Nefedova, A. Chevrenak, E. Deelman, C. Kesselman, et al. HighPerformance Remote Access to Climate Simulation Data: A Challenge Problem for Data Grid Technologies. In proceedings of SC' 01.

No context found.

B. Allcock, I. Foster, V. Nefedova, A. Chervenak, E. Deelman, C. Kesselman, J. Leigh, A. Sim, A. Shoshani, B. Drach, and D. Williams, High-performance remote access to climate simulation data: A challenge problem for data grid technologies, In Proc. of the SuperComputing Conference, November 2001.

No context found.

Allcock B, Foster I, Nefedova V, Chervenak A, Deelman E, Kesselman C, Lee J, Sim A, Shoshani A, Drach B, Williams D. High-performance remote access to climate simulation data: A challenge problem for data Grid technologies. Proceedings of SC2001 Conference, Denver, CO, November 2001.

No context found.

B. Allcock, I. Foster, V. Nefedova, A. Chervenak, E. Deelman, C. Kesselman, J. Lee, A. Sim, A. Shoshani, B. Drach, and D. Williams, "High-performance remote access to climate simulation data: a challenge problem for data grid technologies," in Proceedings of the 2001.

No context found.

B. Allcock, I. Foster, V. Nefedova, A. Chervenak, E. Deelman, C. Kesselman, J. Leigh, A. Sim, and A. Shoshani. High-performance remote access to climate simulation data: A challenge problem for data grid technologies. In Proceedings of IEEE SC'01 Conference on High-performance Computing, 2001. http://www.globus.org/research/papers/ sc01ewa_esg_chervenak_final.pdf.

No context found.

B. Allcock, I. Foster, V. Nefedova, A. Chervenak, E. Deelman, C. Kesselman, J. Leigh, A. Sim, and A. Shoshani. High-performance remote access to climate simulation data: A challenge problem for data grid technologies. In Proceedings of IEEE SC'01 Conference on High-performance Computing, 2001.

No context found.

W. Allcock, I. Foster, V. Nefedova, A. Chevrenak, E. Deelman, C. Kesselman, A. Sim, A. Shoshani, B. Drach, and D. Williams, High-Performance Remote Access to Climate Simulation Data: A Challenge Problem for Data Grid Technologies, Proceedings of Supercomputing (SC'01), November, 2001.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC