J. F. Karpovich, J. C. French and A. S. Grimshaw, High Performance Access to Radio Astronomy Data: A Case Study, Proc. 7th Inter. Conference on Scientific and Statistical Database Management, Charlottesville, VA, Sep. 1994, 240-249.  Home/Search   Document Details and Download   Summary   Related Articles   Check

.... (e.g. Ptool [3] is its design approach in which the data server is optimized for typical operations in HEP data analysis and designed to be easily integrated into a parallel computing system [6] The Aswan is similar in many ways to recently proposed special purpose le systems (e.g. ELFS [4, 5] and disk directed I O [8] However, the Aswan does not replace existing le systems, instead it is designed to sit on top of existing data sources ( le systems, tape robots, DBMSs, etc. In addition, the Aswan is purposefully designed to be simple to avoid delays on the critical path to data ....

J. Karpovich, J. French, A. Grimshaw. High Performance Access to Radio Astronomy Data: A Case Study. Proc. 7th Intl. Conf. Scientic and Statistical Database Management. pp. 240-249, 1994.

....chunk of data and found that the output of a single solution file required from 4 to 26 times as long as an iteration of their flow solver. Database researchers have emphasized providing efficient data organization on disk. 2] designed a specialized database system for particle physics codes; [5] used a PLOP file structure for the array storage of radio astronomy applications. Parallel i o researchers have emphasized the design of parallel file systems or parallel i o libraries suitable for large scale scientific applications. Most of these works applied the collective i o approach to ....

J. F. Karpovich, J. C. French and A. S. Grimshaw, High Performance Access to Radio Astronomy Data: A Case Study, Conf. on Scientific and Stat. Database Management, 1994.

.... stores (e.g. Ptool [3] is its design approach in which the data server is optimized for typical operations in HEP data analysis and designed to be easily integrated into a parallel computing system [9] ASWAN is similar in many ways to recently proposed special purpose le systems (e.g. ELFS [4, 5] and disk directed I O [11] However, the server does not replace existing le systems, instead it is designed to sit on top of existing data sources ( le systems, tape robots, DBMSs, etc. In addition, ASWAN is purposefully designed to be simple to avoid delays on the critical path to data ....

J.F. Karpovich, J.C. French, A.S. Grimshaw, High Performance Access to Radio Astronomy Data: A Case Study. Proc. 7th Intl. Conf. Scientic and Statistical Database Management pg 240-249 1994.

....order will lead to disastrous performance when access patterns are different from storage patterns (e.g. row major versus column major) Most of these works have been applied to real applications. For example, Bell88] designed a specialized database system for particle physics codes; Karpovich94] used a PLOP file structure for the array storage of radio astronomy applications at the NRAO; Seamons94a, Seamons94b] proposed natural chunking (also used in this paper) for storing the multidimensional array of CFD applications. The first two applications are read intensive and perform ....

J. F. Karpovich, J. C. French and A. S. Grimshaw, High Performance Access to Radio Astronomy Data: A Case Study, Conference on Scientific and Statistical Database Management, 1994.

....their run (2400 Bytes MFlop avg. while others only do I O at the beginning or end (14 Bytes MFlop avg) But I O is bursty, so larger bandwidths are suggested. The applications are parallel programs running on Intel Delta, nCUBE 1, or nCUBE 2; and are in C, FORTRAN, or both. ffl [KGF93, KFG94] They store a sparse, multidimensional radio astronomy data set as a set of tagged data values, i.e. as a set of tuples, each with several keys and a data value. They use a PLOP format to partition each dimension into slices, so that each intersection of the slices forms a bucket. They decide ....

John F. Karpovich, James C. French, and Andrew S. Grimshaw. High performance access to radio astronomy data: A case study. In Proceedings of the 7th International Working Conference on Scientific and Statistical Database Management, September 1994. Also available as Univ. of Virginia TR CS-94-25.

....and has less CPU and message passing overhead. In experiments with reading and writing one and two dimensional matrices, disk directed I O was as much as 18 times faster than traditional caching in some access patterns, and was never slower [Kot94] An interesting application. Karpovich et al. KFG94] describe the problem of storing and retrieving radio astronomy data sets. The read mostly data set is large and multi dimensional: each data point represents an astronomical reading at some time at some frequency on some instrument pointed at some region of the sky. Needless to say the data set ....

John F. Karpovich, James C. French, and Andrew S. Grimshaw. High performance access to radio astronomy data: A case study. In Proceedings of the 7th International Working Conference on Scientific and Statistical Database Management, September 1994. Also available as Univ. of Virginia TR CS-94-25.

.... [3] is its design approach in which the data server is optimized for typical operations in HEP data analysis and designed to be easily integrated into a parallel computing system [6] This data server is similar in many ways to recently proposed special purpose file systems (exemplified by ELFS [4, 5] and diskdirected I O [8] However, the server does not replace existing file systems, instead it is designed sit on top of existing data sources (file systems, tape robots, DBMSs, etc. In addition, the data server is purposefully designed to be simple to avoid delays on the critical path to ....

.... [3] is its design approach in which the data server is optimized for typical operations in HEP data analysis and designed to be easily integrated into a parallel computing system [6] This data server is similar in many ways to recently proposed special purpose file systems (exemplified by ELFS [4, 5] and diskdirected I O [8] However, the server does not replace existing file systems, instead it is designed sit on top of existing data sources (file systems, tape robots, DBMSs, etc. In addition, the data server is purposefully designed to be simple to avoid delays on the critical path to ....

J.F. Karpovich, J.C. French, A.S. Grimshaw, High Performance Access to Radio Astronomy Data: A Case Study. Proc. 7th Intl. Conf. Scientific and Statistical Database Management pg 240-249 1994.

.... in one fork and real data in another (e.g. a compression library similar to that described in [SW95] could store compressed data chunks in one fork and directory information in another) An example of using forks for both data and metadata may be found in data files like those described in [KFG94] The style of FITS file described in this study contained records with 6 keys, describing the frequency domain, the antenna, and the time the data was collected. The data portion of each record contained a pair of data elements, one for each of two polarizations, and each data element contains ....

John F. Karpovich, James C. French, and Andrew S. Grimshaw. High performance access to radio astronomy data: A case study. In Proceedings of the 7th International Working Conference on Scientific and Statistical Database Management, pages 240--249, September 1994. Also available as UVA TR CS-94-25.

....on those files. A library that was capable of handling many kinds of queries and FITS files is a perfect example of the type of domain specific library we expect to be implemented on Galley. 6. 1 FITS at NRAO One specific example of how FITS files are used in practice is described in [KGF93, KFG94] This type of FITS file contains records with 6 keys, describing the frequency domain (U; V; W ) the baseline, and the time the data was collected. The baseline is a single number that indicates which antenna or combination of antennas generated that record. The data portion of each record ....

....data set, where the subvolumes may be defined along one or more of the axes. For example, a user may want to examine all the records within a given time range, and sorted along the U axis. Previous work on these files has focused on increasing locality along several dimensions simultaneously. In [KFG94] the authors examine studied the effectiveness of Piecewise Linear Order Preserving Hashing (PLOP) files at reducing the amount of time required to perform common queries, by increasing certain kinds of locality within the files. While locality can also improve performance in parallel file ....

John F. Karpovich, James C. French, and Andrew S. Grimshaw. High performance access to radio astronomy data: A case study. In Proceedings of the 7th International Working Conference on Scientific and Statistical Database Management, pages 240--249, September 1994. Also available as UVA TR CS-94-25.

....compute processor memories and the disks. Compared to a traditional system with caches at the I O processors, this strategy optimizes the disk accesses, uses less memory (no cache at the I O processors) and has less CPU and message passing overhead. An interesting application. Karpovich et al. [10] describe the problem of storing and retrieving radioastronomy data sets. The read mostly data set is large and multi dimensional: each data point represents an astronomical reading at some time at some frequency on some instrument pointed at some region of the sky. Needless to say the data set ....

John F. Karpovich, James C. French, and Andrew S. Grimshaw, "High performance access to radio astronomy data: A case study", in Proceedings of the 7th International Working Conference on Scientific and Statistical Database Management, Sept. 1994, Also available as Univ. of Virginia TR CS-94-25.

....for which I O performance is critical and applications for which I O performance is not important. Examples of the latter include programs being debugged, programs that are compute bound or communication bound or programs that run for relatively short periods. Acharya et al.[1] Karpovich et al. [22] and Smirni et al.[42] have looked at individual I O intensive applications to determine what is needed to make them run fast. Many of the conclusions reached by these studies are useful for designing I O systems of parallel machines. However, these studies were limited by the small number of ....

J. Karpovich, J. French, and A. Grimshaw. High-performance access to radio astronomy data: A case study. In Proceedings of the 7th International Working Conference on Scientific and Statistical Database Management, pages 240--9, Sept 1994.

....queries on those files. A library that was capable of handling many kinds of queries and FITS files is a perfect example of the type of domain specific library we expect to be implemented on Galley. 7.1. 1 FITS at NRAO One specific example of how FITS files are used in practice is described in [KFG94, KGF93] This type of FITS file contains records with 6 keys, describing the frequency domain (U; V; W ) the baseline, and the time the data was collected. The baseline is a single number that indicates which antenna or combination of antennas generated that record. The data portion of each ....

....data set, where the sub volumes may be defined along one or more of the axes. For example, a user may want to examine all the records within a given time range, sorted along the U axis. Previous work on these files has focused on increasing locality along several dimensions simultaneously. In [KFG94, KGF93] the authors examine studied the effectiveness of Piecewise Linear Order Preserving Hashing (PLOP) files at reducing the amount of time required to perform common queries, by increasing certain kinds of locality within the files. While locality can also improve performance in parallel ....

John F. Karpovich, James C. French, and Andrew S. Grimshaw. High performance access to radio astronomy data: A case study. In Proceedings of the 7th International Working Conference on Scientific and Statistical Database Management, pages 240-- 249, September 1994. Also available as UVA TR CS-94-25.

.... [3] is its design approach in which the data server is optimized for typical operations in HEP data analysis and designed to be easily integrated into a parallel computing system [6] Our data server is similar in many ways to recently proposed specialpurpose file systems (exemplified by ELFS [4, 5] and disk directed I O [8] However, our server does not replace existing file systems, instead it is designed sit on top of existing data sources (file systems, tape robots, DBMSs, etc. In addition, our data server is purposefully simple to avoid delays on the critical path to data access. ....

J.F. Karpovich, J.C. French, A.S. Grimshaw, High Performance Access to Radio Astronomy Data: A Case Study. Proc. 7th Intl. Conf. Scientific and Statistical Database Management pg 240-249 1994.

No context found.

J. F. Karpovich, J. C. French and A. S. Grimshaw, High Performance Access to Radio Astronomy Data: A Case Study, Proc. 7th Inter. Conference on Scientific and Statistical Database Management, Charlottesville, VA, Sep. 1994, 240-249.

No context found.

J. F. Karpovich, J. C. French, and A. S. Grimshaw, "High performance access to radio astronomy data: A case study," in Proceedings of the 7th International Working Conference on Scientific and Statistical Database Management, Sept. 1994.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC