Caching and prefetching are important mechanisms for speeding up access time to data on secondary storage. Recent work in competitive online algorithms has uncovered several promising new algorithms for caching. In this paper, we apply a form of the competitive philosophy for the first time to the problem of prefetching to develop an optimal universal prefetcher in terms of fault ratio, with particular applications to large-scale databases and hypertext systems. Our algorithms for prefetching are novel in that they are based on data compression techniques that are both theoretically optimal and good in practice. Intuitively, in order to compress data effectively, you have to be able to predict future data well, and thus good data compressors should be able to predict well for purposes of prefetching. We show for powerful models such as Markov sources and mth order Markov sources that the page fault rates incurred by our prefetching algorithms are optimal in the limit for almost all sequences of page accesses.
|
771
|
Information Theory and Reliable Communication
– Gallager
- 1968
|
|
637
|
Estinwtion of Dependences Based on Empirical Data
– Vapnik
- 1982
|
|
555
|
Modeling for text compression
– Bell, Witten, et al.
- 1989
|
|
536
|
Learnability and the Vapnik-Chervonenkis Dimension
– Blumer, Ehrenfeucht, et al.
- 1989
|
|
523
|
Arithmetic coding for data compression
– Witten, Neal, et al.
- 1987
|
|
481
|
Compression of individual sequences via variable rate coding
– Ziv, Lempel
- 1978
|
|
455
|
Design and evaluation of a compiler algorithm for prefetching
– Mowry, Lam, et al.
- 1992
|
|
250
|
Data compression using adaptive coding and partial string matching
– Cleary, Witten
- 1984
|
|
224
|
Practical prefetching via data compression
– Curewitz, Krishnan, et al.
- 1993
|
|
173
|
Efficient distribution-free learning of probabilistic concepts
– Kearns, Schapire
- 1994
|
|
151
|
Reducing memory latency via nonblocking and prefetching caches
– Chen, Baer
- 1992
|
|
148
|
Arithmetic coding
– Langdon
- 1979
|
|
143
|
A Second Course
– Karlin, Taylor
- 1981
|
|
131
|
On the complexity of finite sequences
– Lempel, Ziv
- 1976
|
|
116
|
Fido: A cache that learns to fetch
– Palmer, Zdonik
- 1991
|
|
115
|
Universal prediction of individual sequences
– Feder, Merhav, et al.
- 1992
|
|
114
|
Competitive paging with locality of reference
– Borodin, Irani, et al.
- 1995
|
|
111
|
A strongly competitive randomized paging algorithm
– McGeoch, Sleator
- 1989
|
|
94
|
Large Deviations for Performance Analysis
– Shwartz, Weiss
- 1995
|
|
89
|
An analog of the minimax theorem for vector payoffs
– Blackwell
- 1956
|
|
79
|
Strongly Competitive Algorithms for Paging With Locality of Reference
– Irani, Karlin, et al.
- 1992
|
|
71
|
On the computational complexity of approximating distributions by probabilistic automata
– Abe, Warmuth
- 1992
|
|
65
|
Software Support for Speculative Loads
– Rogers, Li
- 1992
|
|
64
|
Markov paging
– Karlin, Phillips, et al.
|
|
48
|
A Status Report on Research in Transparent Informed Prefetching
– Patterson, Gibson, et al.
- 1993
|
|
47
|
Occam's razor
– Blumer, Ehrenfeucht, et al.
- 1987
|
|
44
|
On the necessity of Occam algorithms
– Pitt, Board
- 1990
|
|
33
|
Optimal Prediction for Prefetching in the Worst Case
– Krishnan, Vitter
- 1994
|
|
33
|
Discrete sequence prediction and its applications
– Laird, Saul
- 1994
|
|
30
|
Analysis of arithmetic coding for data compression
– Howard, Vitter
- 1992
|
|
27
|
A note on Ziv–Lempel model for compressing individual sequences
– Langdon
- 1983
|
|
23
|
On competitive algorithms for paging problems
– Fiat, Karp, et al.
- 1991
|
|
22
|
Tarjan. "Amortized efficiency of list update and paging rules
– Sleator, E
- 1985
|
|
14
|
A theory of productivity in the creative process
– Brady
- 1986
|
|
14
|
Approximation to Bayes Risk in Repeated Plays," Contributions to the Theory
– Hannan
- 1957
|
|
8
|
Large Deviations for Coding Markov Chains and Gibbs Random Fields
– Amit, Miller
- 1990
|
|
8
|
Adaptive Prefetching for Disk Buffers
– Salem
- 1991
|
|
8
|
Compound Bayes Predictors with Apparent Markov Structure
– Cover, Shenhar
- 1977
|
|
7
|
Large deviations, hypothesis testing, and source coding for finite Markov sources
– Natarajan
- 1985
|
|
7
|
Lempel "Compression of Individual Sequences Via Variable-Rate Coding
– Ziv, A
- 1978
|
|
5
|
An Analysis of Prepaging," Computing 22
– Trivedi
- 1979
|
|
1
|
Competitive Paging Algorithms," Carnegie-Mellon University, CS-- 88--196
– Fiat, Karp, et al.
- 1988
|
|
1
|
patent pending, preliminarily approved
– Vitter, Curewitz
- 1994
|
|
