M. H. Nodine and J. S. Vitter. Paradigms for optimal sorting with multiple disks. In Proc. of the 26th Hawaii Int. Conf. on Systems Sciences, 1993.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....provided matching upper and lower I O bounds for several problems. These bounds apply to the PDM model. The lower bound for sorting states that the worst case number of I O s required for sorting is Theta( N BD log M B N B ) 1 [2, 39] Several EM algorithms exist for sorting, including [1, 2, 3, 29, 30, 41, 42, 31]. Surprisingly, it turns out that performing a permutation requires Theta(minf N D ; N BD logM B N B g) I Os [2, 39] while the same can be performed in linear time in the RAM model. Similarly, the worst case number of I Os required to transpose a p Theta q matrix from row major order ....

Nodine, M. H., and Vitter, J. S. Paradigms for optimal sorting with multiple disks. In Proc. of the 26th Hawaii Int. Conf. on Systems Sciences (1993).

....N = 10 10 to N = 10 12 . An increasingly popular approach to further increase the throughput of the I O system is to use a number of disks in parallel [63, 65, 133] Several authors have considered an extension of the above model with a parameter D denoting the number of disks in the system [19, 98, 99, 100, 133]. In the parallel disk model [133] one can read or write one block from each of the D disks simultaneously in one I O. The number of disks D range up to 10 2 in current disk arrays. Our I O model corresponds to the one shown in Figure 1.2, where we only count the number of blocks of B elements ....

....the modified algorithm still needs O(N log B n) I Os to sort the segments. In this paper we focus our attention on the single disk model. As described in Section 1.1, striping can be used to implement our algorithms on parallel disk systems with D 1. Additionally, techniques from [98] and [100] can be used to extend many of our results to parallel disk systems. In the conference version of this paper we conjectured that all our results could be improved by the optimal factor of D on parallel disk systems with D disks, but it is still an open problem whether the required merges can be ....

[Article contains additional citation context not shown here]

M. H. Nodine and J. S. Vitter. Paradigms for optimal sorting with multiple disks. In Proc. of the 26th Hawaii Int. Conf. on Systems Sciences, 1993.

....remarks. 2 Existing Systems and System Proposals In recent years a number of parallel file systems for high performance computer systems have been built or proposed [CFP, Corb, DiS, Kotb, KSU, PGK, Pie] At the same time, a number of I O optimal computation paradigms have been proposed [Cora, CoW, GTV, NoV, ViS]. Unfortunately, existing I O systems tend not to adequately support the functionality required in order to implement I O optimal algorithms. In particular, many file systems do not support independent head movement on different disks or striping a file by putting complete blocks on individual ....

....which are summarized in Table 2. Distribution is a process whereby data is read sequentially from a single input stream, and each item is written to an output stream determined by a user defined distribution object. It can be very useful in sorting and related problems where elements are ordered [GTV, NoV, ViS]. Merging is essentially the reverse of distribution; it takes a number of ordered input streams and merges them into a single ordered output stream. All items from a single input stream remain in the same order in the output stream, although the different input streams may be interleaved in ....

[Article contains additional citation context not shown here]

M. H. Nodine and J. S. Vitter, "Paradigms for Optimal Sorting with Multiple Disks," Proc. 5th Annual ACM Symp. on Parallel Algorithms and Architectures (1993).

....for single processors, and for the PRAM. For non PRAM interconnection networks such as hypercube and cube connected cycles, only the randomized algorithms of [88] are simultaneously optimal in both I O and computation. A summary of randomized and deterministic EM sorting techniques is presented in [68]. Goodrich, Tsay, Vengroff and Vitter [55] sketched EM algorithms for a large number of problems in computational geometry, and proposed several useful paradigms, such as distribution sweeping and batch filtering for designing efficient EM algorithms (see Section 1.3.2 for more information) They ....

....At the time of writing, the buffer tree and related applications are exceptions. Barve, Grove and Vitter [15] examined the practicality of EM sorting techniques and proposed a randomized sorting technique that is I O optimal, yet simpler in practice than previous EM sorting methods, summarized in [68]. ViC (Colvin and Cormen [26] is language for coding data parallel programs with out of core (external) variables. The ViC compiler provides the necessary input output calls to allow a user program to manipulate huge data objects without explicitly coding the I O requests. The compiler makes ....

M. H. Nodine and J. S. Vitter. Paradigms for optimal sorting with multiple disks. In Proc. of the 26th Hawaii Int. Conf. on Systems Sciences, 1993.

....by an extensive experimental study. 1.1 Previous Related Work As mentioned above, most of the previous work on I O efficient computation is theoretical. Early work on algorithms for parallel disk systems concentrates largely on fundamental problems such as sorting, matrix multiplication, and FFT [1, 24, 31]. The main focus of this early work is therefore directed at problems that involve permutation at a basic level. Indeed, just the problem of implementing various classes of permutation has been a central theme in external memory I O research [1, 12, 13, 15, 31] More recently, external memory ....

M. H. Nodine and J. S. Vitter. Paradigms for optimal sorting with multiple disks. In Proc. of the 26th Hawaii Int. Conf. on Systems Sciences, January 1993.

....N = 10 10 to N = 10 12 . An increasingly popular approach to further increase the throughput of the I O system is to use a number of disks in parallel [63, 65, 133] Several authors have considered an extension of the above model with a parameter D denoting the number of disks in the system [19, 98, 99, 100, 133]. In the parallel disk model [133] one can read or write one block from each of the D disks simultaneously in one I O. The number of disks D range up to 10 2 in current disk arrays. Our I O model corresponds to the one shown in Figure 1.2, where we only count the number of blocks of B elements ....

....the modified algorithm still needs O(N log B n) I Os to sort the segments. In this paper we focus our attention on the single disk model. As described in Section 1.1, striping can be used to implement our algorithms on parallel disk systems with D 1. Additionally, techniques from [98] and [100] can be used to extend many of our results to parallel disk systems. In the conference version of this paper we conjectured that all our results could be improved by the optimal factor of D on parallel disk systems with D disks, but it is still an open problem whether the required merges can be ....

[Article contains additional citation context not shown here]

M. H. Nodine and J. S. Vitter. Paradigms for optimal sorting with multiple disks. In Proc. of the 26th Hawaii Int. Conf. on Systems Sciences, 1993.

....and M and B to respectively denote the numbers of cells fitting in the main memory and in a disk block. Each I O operation reads or writes one disk block. Previous Related Work We first briefly review the work on I O techniques. In addition to early work on sorting and scientific computing [2, 28, 44], recently various researchers have been investigating external memory algorithms for graphs [1, 12] and for computational geometry [1, 3, 5, 6, 7, 10, 18, 21, 29, 38, 43] As mentioned before, most of the results are theoretical, and yet the experiments of Chiang [11] Vengroff and Vitter [42] ....

M. H. Nodine and J. S. Vitter. Paradigms for optimal sorting with multiple disks. In Proc. of the 26th Hawaii Int. Conf. on Systems Sciences, January 1993.

....to reduce the I O bottleneck are called external memory algorithms. In recent years various researchers have been investigating external memory algorithms for graphs [10] and for computational geometry [2, 4, 5, 8, 13, 16, 23, 31, 34] in addition to early work on sorting and scientific computing [1, 21, 35]. Although most of the results are theoretical, the experiments of Chiang [9] and of Vengroff and Vitter [33] on some of these techniques show that they result in significant improvements over traditional algorithms in practice. Also, Teller et al. 32] describe a system to compute radiosity ....

M. H. Nodine and J. S. Vitter. Paradigms for optimal sorting with multiple disks. In Proc. of the 26th Hawaii Int. Conf. on Systems Sciences, January 1993.

....by an extensive experimental study. Previous Related Work As mentioned above, most of the previous work on I O efficient computation is theoretical. Early work on algorithms for parallel disk systems concentrates largely on fundamental problems such as sorting, matrix multiplication, and FFT [1, 28, 38]. The main focus of this early work is therefore directed at problems that involve permutation at a basic level. Indeed, just the problem of implementing various classes of permutation has been a central theme in external memory I O research [1, 14, 15, 17, 38] Also, a general connection between ....

M. H. Nodine and J. S. Vitter. Paradigms for optimal sorting with multiple disks. In Proc. of the 26th Hawaii Int. Conf. on Systems Sciences, 1993.

....N = 10 10 to N = 10 12 . An increasingly popular approach to further increase the throughput of the I O system is to use a number of disks in parallel [50, 51, 97] Several authors have considered an extension of the above model with a parameter D denoting the number of disks in the system [21, 73, 71, 72, 97]. In the parallel disk model [97] one can read or write one block from each of the D disks simultaneously in one I O. The number of disks D range up to 10 2 in current disk arrays. The parallel disk model corresponds to the one shown in Figure 2, where we only count the number of blocks of B ....

M. H. Nodine and J. S. Vitter. Paradigms for optimal sorting with multiple disks. In Proc. of the 26th Hawaii Int. Conf. on Systems Sciences, 1993.

.... introduce the following shorthand notation to represent the I O complexity of each of these primitives: scan(x) x DB ; which represents the number of I Os needed to read x items, and sort(x) x DB log M=B x B ; which is proportional to the optimal number of I Os needed to sort x items [87]. 4.1.2 Previous Work Early work on external memory algorithms concentrated largely on fundamental problems such as sorting, matrix multiplication, and FFT [2, 87, 122] The main focus of this early work was therefore directed at problems that involved permutation at a basic level. Indeed, just ....

....needed to read x items, and sort(x) x DB log M=B x B ; which is proportional to the optimal number of I Os needed to sort x items [87] 4.1. 2 Previous Work Early work on external memory algorithms concentrated largely on fundamental problems such as sorting, matrix multiplication, and FFT [2, 87, 122]. The main focus of this early work was therefore directed at problems that involved permutation at a basic level. Indeed, just the problem of implementing various classes of permutation has been a central theme in external memory I O research [2, 36, 37, 38, 122] More recently, external memory ....

M. H. Nodine and J. S. Vitter. Paradigms for optimal sorting with multiple disks. In Proc. of the 26th Hawaii Int. Conf. on Systems Sciences, January 1993.

No context found.

M. H. Nodine and J. S. Vitter. Paradigms for optimal sorting with multiple disks. In Proc. of the 26th Hawaii Int. Conf. on Systems Sciences, Jan. 1993.

....TPIE implements a moderately sized set of highlevel access methods. The access methods were chosen based on their paradigmatic importance in the design of I O efficient algorithms. Using these access methods, we can implement I O efficient algorithms for many problems, including sorting [2, 16, 17, 23], permuting [9, 10, 11, 23] computational geometry problems [3, 4, 12, 24] graph theoretic problems [8] and scientific problems [9, 14, 23] Because such a large number of problems can be solved using a relatively small number of paradigms, it is important that the access method ....

....lg(M=2B) in (2) is larger than the denominator lg(M=2DB) in (1) by an additive term of lg D, the leading constant factor in (2) is larger than that of (1) by a multiplicative factor of k. A number of distribution sort algorithms exist that use independent disks in a theoretically optimal way [16, 17, 23, 1], with values of k ranging from approximately 3 to 20. Before implementing an external sort on parallel disks, it is useful to examine the circumstances under which the I O complexity (2) for using the disks independently is less than the I O complexity (1) with striping. If we neglect the ....

M. H. Nodine and J. S. Vitter. Paradigms for optimal sorting with multiple disks. In Proc. of the 26th Hawaii Int. Conf. on Systems Sciences, Jan. 1993.

.... the I O complexity of each of these primitives: scan(x) x DB ; which represents the number of I Os needed to read x items striped across the disks, and sort(x) x DB log M=B x B ; which is proportional to the optimal number of I Os needed to sort x items striped across the disk [19]. 1.2 Previous Work. Early work in externalmemory algorithms for parallel disk systems concentrated largely on fundamental problems such as sorting, matrix multiplication, and FFT [1, 19, 26] The main focus of this early work was therefore directed at problems that involved permutation at a ....

....M=B x B ; which is proportional to the optimal number of I Os needed to sort x items striped across the disk [19] 1.2 Previous Work. Early work in externalmemory algorithms for parallel disk systems concentrated largely on fundamental problems such as sorting, matrix multiplication, and FFT [1, 19, 26]. The main focus of this early work was therefore directed at problems that involved permutation at a basic level. Indeed, just the problem of implementing various classes of permutation has been a central theme in external memory I O research [1, 6, 7, 8, 26] More recently, external memory ....

[Article contains additional citation context not shown here]

M. H. Nodine and J. S. Vitter. Paradigms for optimal sorting with multiple disks. In Proc. of the 26th Hawaii Int. Conf. on Systems Sciences, Jan. 1993.

....TPIE implements a moderately sized set of high level access methods. The access methods were chosen based on their paradigmatic importance in the design of I O efficient algorithms. Using these access methods, we can implement I O efficient algorithms for many problems, including sorting [2, 15, 16, 22], permuting [8, 9, 10, 22] computational geometry problems [3, 4, 11, 23] graph theoretic problems [7] and scientific problems [8, 13, 22] Because such a large number of problems can be solved using a relatively small number of paradigms, it is important that the access method implementations ....

....are distributed over the D disks. Although the the denominator in (2) is larger than the denominator in (1) by an additive term of lg D, the leading constant factor in (2) is larger than that of (1) by a multiplicative factor of k. A number of independent disk distribution sort algorithms exist [1, 15, 16, 22], with values of k ranging from approximately 3 to 20. Before implementing an external sort on parallel disks, it is useful to examine the circumstances 3 For a recent example, see [3] under which the I O complexity (2) for using the disks independently is less than the I O complexity (1) with ....

M. H. Nodine and J. S. Vitter. Paradigms for optimal sorting with multiple disks. In Proc. of the 26th Hawaii Int. Conf. on Systems Sciences, Jan. 1993.

No context found.

M. H. Nodine and J. S. Vitter. Paradigms for optimal sorting with multiple disks. In Proc. of the 26th Hawaii Int. Conf. on Systems Sciences, 1993.

No context found.

M. H. Nodine and J. S. Vitter. Paradigms for optimal sorting with multiple disks. In Proc.26th Hawaii Internat. Conf. on Systems Sciences, 1993.

No context found.

M.H. Nodine and J.S. Vitter. Paradigms for Optimal Sorting with Multiple Disks. Proc. Hawaii International Conference on System Sciences 1:50-59, 1993.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC