Results 1 
7 of
7
External Memory Algorithms and Data Structures
, 1998
"... Data sets in large applications are often too massive to fit completely inside the computer's internal memory. The resulting input/output communication (or I/O) between fast internal memory and slower external memory (such as disks) can be a major performance bottleneck. In this paper, we surve ..."
Abstract

Cited by 360 (23 self)
 Add to MetaCart
Data sets in large applications are often too massive to fit completely inside the computer's internal memory. The resulting input/output communication (or I/O) between fast internal memory and slower external memory (such as disks) can be a major performance bottleneck. In this paper, we survey the state of the art in the design and analysis of external memory algorithms and data structures (which are sometimes referred to as "EM" or "I/O" or "outofcore" algorithms and data structures). EM algorithms and data structures are often designed and analyzed using the parallel disk model (PDM). The three machineindependent measures of performance in PDM are the number of I/O operations, the CPU time, and the amount of disk space. PDM allows for multiple disks (or disk arrays) and parallel CPUs, and it can be generalized to handle tertiary storage and hierarchical memory. We discuss several important paradigms for how to solve batched and online problems efficiently in external memory. Programming tools and environments are available for simplifying the programming task. The TPIE system (Transparent Parallel I/O programming Environment) is both easy to use and efficient in terms of execution speed. We report on some experiments using TPIE in the domain of spatial databases. The newly developed EM algorithms and data structures that incorporate the paradigms we discuss are significantly faster than methods currently used in practice.
Algorithms and Data Structures for External Memory
, 2006
"... Data sets in large applications are often too massive to fit completely inside the computer's internal memory. The resulting input/output communication (or I/O) between fast internal memory and slower external memory (such as disks) can be a major performance bottleneck. Algorithms and Data Str ..."
Abstract

Cited by 84 (6 self)
 Add to MetaCart
(Show Context)
Data sets in large applications are often too massive to fit completely inside the computer's internal memory. The resulting input/output communication (or I/O) between fast internal memory and slower external memory (such as disks) can be a major performance bottleneck. Algorithms and Data Structures for External Memory surveys the state of the art in the design and analysis of external memory (or EM) algorithms and data structures, where the goal is to exploit locality in order to reduce the I/O costs. A variety of EM paradigms are considered for solving batched and online problems efficiently in external memory. Algorithms and Data Structures for External Memory describes several useful paradigms for the design and implementation of efficient EM algorithms and data structures. The problem domains considered include sorting, permuting, FFT, scientific computing, computational geometry, graphs, databases, geographic information systems, and text and string processing. Algorithms and Data Structures for External Memory is an invaluable reference for anybody interested in, or conducting research in the design, analysis, and implementation of algorithms and data structures. This book is originally published as
A Theoretical Framework for MemoryAdaptive Algorithms
 In IEEE Symposium on Foundations of Computer Science
, 1999
"... External Memory algorithms play a key role in database management systems and large scale processing systems. External memory algorithms are typically tuned for efficient performance given a fixed, statically allocated amount of internal memory. However, with the advent of realtime database system ..."
Abstract

Cited by 13 (1 self)
 Add to MetaCart
(Show Context)
External Memory algorithms play a key role in database management systems and large scale processing systems. External memory algorithms are typically tuned for efficient performance given a fixed, statically allocated amount of internal memory. However, with the advent of realtime database system and database systems based upon administratively defined goals, algorithms must increasingly be able to adapt in an online manner when the amount of internal memory allocated to them changes dynamically and unpredictably. In this paper, we present a theoretical and applicable framework for memoryadaptive algorithms (or simply MA algorithms). We define the competitive worstcase notion of what it means for an MA algorithm to be dynamically optimal and prove fundamental lower bounds on the performance of MA algorithms for problems such as sorting, standard matrix multiplication, and several related problems. Our main tool for proving dynamic optimality is the notion of resource consumption, wh...
Parallel Algorithms in External Memory
, 2000
"... External memory (EM) algorithms are designed for computational problems in which the size of the internal memory of the computer is only a small fraction of the problem size. The Parallel Disk Model (PDM) of Vitter and Shriver is widely used to discriminate between external memory algorithms on the ..."
Abstract

Cited by 5 (1 self)
 Add to MetaCart
External memory (EM) algorithms are designed for computational problems in which the size of the internal memory of the computer is only a small fraction of the problem size. The Parallel Disk Model (PDM) of Vitter and Shriver is widely used to discriminate between external memory algorithms on the basis of input/output (I/O) complexity. Parallel algorithms are designed to efficiently utilize the computing power of multiple processing units, interconnected by a communication mechanism. A popular model for developing and analyzing parallel algorithms is the Bulk Synchronous Parallel (BSP) model due to Valiant. In this work we develop simulation techniques, both randomized and deterministic, which produce efficient EM algorithms from efficient algorithms developed under BSPlike parallel computing models. Our techniques can accommodate one or multiple processors on the EM target machine, each with one or more disks, and they also adapt to the disk blocking factor of the target machine. ...
Cacheadaptive algorithms
 In Proceedings of the TwentyFifth Annual ACMSIAM Symposium on Discrete Algorithms (SODA
, 2014
"... We introduce the cacheadaptive model, which generalizes the externalmemory model to apply to environments in which the amount of memory available to an algorithm can fluctuate. The cacheadaptive model applies to operating systems, databases, and other systems where the allocation of memory to pro ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
(Show Context)
We introduce the cacheadaptive model, which generalizes the externalmemory model to apply to environments in which the amount of memory available to an algorithm can fluctuate. The cacheadaptive model applies to operating systems, databases, and other systems where the allocation of memory to processes changes over time. We prove that if an optimal cacheoblivious algorithm has a particular recursive structure, then it is also an optimal cacheadaptive algorithm. Cacheoblivious algorithms having this form include FloydWarshall all pairs shortest paths, naı̈ve recursive matrix multiplication, matrix transpose, and Gaussian elimination. While the cacheoblivious sorting algorithm Lazy Funnel Sort does not have this recursive structure, we prove that it is nonetheless optimally cacheadaptive. We also establish that if a cacheoblivious algorithm is optimal on “square ” (wellbehaved) memory profiles then, given resource augmentation it is optimal on all memory profiles. We give paging algorithms for the case where the cache size changes dynamically. We prove that LRU with 4memory and 4speed augmentation is competitive with optimal. Moreover, Belady’s algorithm remains optimal even when the cache size changes. Cacheobliviousness is distinct from cacheadaptivity. We exhibit a cacheoblivious algorithm that is not cacheadaptive and a cacheadaptive algorithm for a problem having no optimal cacheoblivious solution.
Acquisitions et services bibliographiques
, 1999
"... copies of this thesis in microfom, paper or electronic formats. The author retains ownership of the copyright in this thesis. Neither the thesis nor substantial extracts fiom it may be printed or othenirise reproduced without the author's permission. L'auteur a accordé une licence non excl ..."
Abstract
 Add to MetaCart
(Show Context)
copies of this thesis in microfom, paper or electronic formats. The author retains ownership of the copyright in this thesis. Neither the thesis nor substantial extracts fiom it may be printed or othenirise reproduced without the author's permission. L'auteur a accordé une licence non exclusive permettant à la Bibliothèque nationale du Canada de reproduire, prêter, distribuer ou vendre des copies de cette thèse sous la fome de microfiche/fïlxn, de reproduction sur papier ou sur format