UniTree User Guide. Release 2.0, UniTree Software, Inc., 1998. 10  Home/Search   Document Not in Database   Summary   Related Articles   Check

....coupled with local file systems, i.e. using local disks as data storage. As data intensive applications, especially large scale scientific applications may have very large storage space requirement, people have to employ large storage system such as tertiary storage systems (HPSS[13, 20] UniTree[40]) and large database systems (Oracle) In addition, these large storage resources may no long tightly coupled with local compute resource: they could be distributed over wide area network. A major concern of accessing these storage resources is performance. I O (remote I O) evaluation and ....

UniTree User Guide, Release 2.0. UniTree Software, Inc., 1998. 11

....that can harness parallel I O capabilities of current multiprocessor architectures. In addition to that, the data consistence and integrity semantics provided by almost all DBMS put an added obstacle to high performance. Finally, although hierarchical storage management systems (e.g. [38]) are effective in large scale data transfers between storage devices in different levels of a storage hierarchy, they also, like parallel file systems and DBMS, lack application specific access pattern information, and consequently, their I O access strategies and optimizations are targeted at ....

....access pattern of a dataset, the MDMS can, for example, advise the HSS to perform collective I O [16] or prefetching [19] for this dataset. ffl It stores information about the pending access patterns. It utilizes this information in taking some global data movement decisions (e.g. file migration [38, 14] and prestaging [38, 14] possibly involving datasets from multiple applications. ffl It keeps meta data for specifying access history and trail of navigation. This information can then be utilized in selecting appropriate optimization policies in successive runs. These queries are performed ....

[Article contains additional citation context not shown here]

UniTree User Guide, Release 2.0. UniTree Software, Inc., 1998. 28

....optimizations found in MPI IO. But it is now typical for modern scientific simulations to adopt tertiary archival as permanent storage for holding huge amounts of datasets generated by parallel processors. These tertiary storage systems such as High Performance Storage System (HPSS) 3] Unitree [10] etc. usually have disks and tapes to form a storage hierarchy inside the systems which is one level below the magnetic disks. The characteristics of this storage hierarchy are that the data stored are large and the data access time is very slow: the access time of a typical I O call for a ....

....has to establish its own I O connection, data sieving is slightly worse than collective I O in performance, but compared to the naive approach, we observed significant performance improvements (Figure 7) 4. 3 Subfile In many tape based storage systems, the access granularity is a whole file [10]. Consequently, even if the program tries to access only a section of the tape resident file, the entire file must be transferred from the tape to the upper level storage media (e.g. magnetic disk) This can result in poor I O performance for many access patterns. Subfile [6] is a technique that ....

UniTree User Guide, Release 2.0. UniTree Software, Inc., 1998. 14

....disks as data storage. As the rate that data is produced increases and the amount significantly exceeds that of disk capacity, large scale scientific applications have to turn to other large storage resources. These storage resources include tertiary storage systems such as HPSS [12,18] UniTree [40] and large database systems such as Oracle. With the shift from typical computing centers to a more distributed computing enviornment, these additional large storage resources may no longer be locally available. These storage archives, due to nature and complexity are most likely available at a ....

UniTree User Guide, Release 2.0 (UniTree Software, Inc., 1998.

....that can harness parallel I O capabilities of current multiprocessor architectures. In addition to that, the data consistence and integrity semantics provided by almost all DBMS put an added obstacle to high performance. Finally, although hierarchical storage management systems (e.g. [39]) are effective in large scale data transfers between storage devices in different levels of a storage hierarchy, they also, like parallel file systems and DBMS, lack application specific access pattern information, and consequently, their I O access strategies and optimizations are targeted at ....

....access pattern of a dataset, the MDMS can, for example, advise the HSS to perform collective I O [17] or prefetching [20] for this dataset. ffl It stores information about the pending access patterns. It utilizes this information in taking some global data movement decisions (e.g. file migration [39, 13] and prestaging [39, 13] possibly involving datasets from multiple applications. ffl It keeps meta data for specifying access history and trail of navigation. This information can then be utilized in selecting appropriate optimization policies in successive runs. Overall, the MDMS keeps vital ....

[Article contains additional citation context not shown here]

UniTree User Guide. Release 2.0, UniTree Software, Inc., 1998. 22

....optimizations found in MPI IO. But it is now typical for modern scientific simulations to adopt tertiary archival as permanent storage for holding huge amounts of datasets generated by parallel processors. These tertiary storage systems such as High Performance Storage System (HPSS) 3] Unitree [10] etc. usually have disks and tapes to form a storage hierarchy inside the systems which is one level below the magnetic disks. The characteristics of this storage hierarchy are that the data stored are large and the data access time is very slow: the access time of a typical I O call for a ....

....has to establish its own I O connection, data sieving is slightly worse than collective I O in performance, but compared to the naive approach, we observed significant performance improvements (Figure 7) 4. 3 Subfile In many tape based storage systems, the access granularity is a whole file [10]. Consequently, even if the program tries to access only a section of the tape resident file, the entire file must be transferred from the tape to the upper level storage media (e.g. magnetic disk) This can result in poor I O performance for many access patterns. Subfile [6] is a technique that ....

UniTree User Guide, Release 2.0. UniTree Software, Inc., 1998. 14

....found in MPI IO. But it is now typical for modern scientific simulations to adopt tertiary archival as permanent storage for holding huge amounts of datasets that are generated by parallel processors. These tertiary storage systems such as High Performance Storage System (HPSS) 4] Unitree [11] etc. usually have disks and tapes to form a storage hierarchy inside the systems which is one level below the magnetic disks. The characteristics of this storage hierarchy are that the data stored are large and the data access time is very slow: the access time of a typical I O call for a ....

....has to establish its own I O connection, data sieving is slightly worse than collective I O in performance, but compared to the naive approach, we observed significant performance improvements (Figure 8) 4. 3 Subfile In many tape based storage systems, the access granularity is a whole file [11]. Consequently, even if the program tries to access only a section of the tape resident file, the entire file must be transferred from the tape to the upper level storage media (e.g. magnetic disk) This can result in poor I O performance for many access patterns. Subfile [7] is a technique that ....

UniTree User Guide. Release 2.0, UniTree Software, Inc., 1998. 16

....that can harness parallel I O capabilities of current multiprocessor architectures. In addition to that, the data consistence and integrity semantics provided by almost all DBMS put an added obstacle to high performance. Finally, although hierarchical storage management systems (e.g. [36]) are effective in large scale data transfers between storage devices in different levels of a storage hierarchy, they also, like parallel file systems and DBMS, lack application specific access pattern information, and consequently, their I O access strategies and optimizations are targeted at ....

....access pattern of a dataset, the MDMS can, for example, advise the HSS to perform collective I O [15] or prefetching [18] for this dataset. ffl It stores information about the pending access patterns. It utilizes this information in taking some global data movement decisions (e.g. file migration [36, 13] and prestaging [36, 13] possibly involving datasets from multiple applications. ffl It keeps meta data for specifying access history and trail of navigation. This information can then be utilized in selecting appropriate optimization policies in successive runs. Overall, the MDMS keeps vital ....

[Article contains additional citation context not shown here]

UniTree User Guide, Release 2.0. UniTree Software, Inc., 1998.

No context found.

UniTree User Guide. Release 2.0, UniTree Software, Inc., 1998. 10

No context found.

UniTree User Guide. Release 2.0, UniTree Software, Inc., 1998. 14

No context found.

UniTree User Guide. Release 2.0, UniTree Software, Inc., 1998. 7

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC