K. Brown, M. Carey, M. Livny, Managing memory to meet multiclass workload response time goals, Proceedings of the International Conference on Very Large Databases (VLDB'93), 1993, pp. 328--341.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....workload, respectively. The author concludes that the primary performance issue in a mixed workload is the handling of I O demands on the data disks, and suggests that a priority scheme is required in the database system as a whole to balance the two types of workloads. Brown, Carey and DeWitt [Brown92, Brown93] discuss the allocation of memory as the critical resource in a mixed workload environment. They introduce a system with multiple workload classes, each with varying response time goals that are specified to the memory allocator. They show that a modified memory manager is able to successfully ....

Brown, K., Carey, M. and Livny, M. "Managing Memory to Meet Multiclass Workload Response Time Goals" VLDB, August 1993.

....The author concludes that the primary performance issue in a mixed workload is the handling of I O demands on the data disks, and suggests that a priority scheme is required in the database system as a whole to balance the two types of workloads. 9.6. 2 Memory Allocation Brown, Carey and DeWitt [Brown92, Brown93] discuss the allocation of memory as the critical resource in a mixed workload environment. They introduce a system with multiple workload classes, each with varying response time goals that are specified to the memory allocator. They show that a modified memory manager is able to successfully ....

Brown, K., Carey, M. and Livny, M. "Managing Memory to Meet Multiclass Workload Response Time Goals" VLDB, August 1993.

....workload, respectively. The author concludes that the primary performance issue in a mixed workload is the handling of I O demands on the data disks, and suggests that a priority scheme is required in the database system as a whole to balance the two types of workloads. Brown, Carey and DeWitt [Brown92, Brown93] discuss the allocation of memory as the critical resource in a mixed workload environment. They introduce a system with multiple workload classes, each with varying response time goals that are specified to the memory allocator. They show that a modified memory manager is able to successfully ....

Brown, K., Carey, M. and Livny, M. "Managing Memory to Meet Multiclass Workload Response Time Goals" VLDB, August 1993.

....of this work was done while the author was visiting I.N.R.I.A. in Sophia Antipolis, France. Supported in part by Army Research Office MURI grant DAAH04 96 1 0013 and by National Science Foundation research grant CCR 9522047. and database systems based upon administratively defined goals [5, 4] allow higher priority computations or queries to steal resources such as internal memory from ongoing external memory computations. Traditional external memory algorithms exhibit drastic performance degradation (because of thrashing like behavior) when they are subjected to sudden fluctuations in ....

K. P. Brown, M. J. Carey, and M. Livny. Managing memory to meet multiclass workload response time goals. Proc. 19th Int. Conf. on Very Large Data Bases, August 1993.

....implementation information is not readily available. Clearly, if automated goal driven performance tuning for database management systems is to become a reality, comprehensive algorithms need to be developed and evaluated. The goal oriented memory and MPL management algorithms presented in [Brown 93a] Brown 94] and [Brown 95] represent a step in the direction of goal oriented DBMS resource allocation. These papers form the basis for this thesis and will be presented in Chapters 4 and 5. 24 Chapter 3 Simulation Model sim.u.la.tion n sim y la sh n 1 : the act or process of simulating ....

.... anonymous IBMer on why IBM softwares uses so much memory In this chapter, a disk buffer memory controller algorithm called Class Fencing is presented. First, Section 4. 1 reviews two previous goal oriented disk buffer memory allocation algorithms, Dynamic Tuning [Chung 94] and Fragment Fencing [Brown 93a] highlighting both their features and their limitations. Section 4.2 then presents the Class Fencing algorithm. Class Fencing is based on a concept called hit rate concavity, which allows it to be more responsive, stable, and robust (as compared to the previous algorithms) while remaining ....

[Article contains additional citation context not shown here]

K. Brown, M. Carey, M. Livny, "Managing Memory to Meet Multiclass Workload Response Time Goals," Proc. 19th Int'l VLDB Conf, Dublin, Ireland, August 1993.

....measurements from our implementation. We assume that the aggregation is being performed directly on a base relation stored on disks as in the example query. The parameters of the study are listed in Table 3 unless otherwise specified. These parameters are similar to those in previous studies e.g. BCL93] The CPU 52 speed is chosen to reflect the characteristics of the current generation of commercially available microprocessors. The I O rate was as observed on the SUN disk on the SUN SparcServer20 51. We model a high speed, high bandwidth network as in commercial multiprocessors like IBM ....

Kurt P. Brown, Michael J. Carey, and Miron Livny. Managing Memory to Meet Multiclass Workload Response Time Goals. In Proc. of 19th VLDB Conf., pages 328--341, 1993.

....tuning problem of buffer pool sizes for buffer pool requests originating from transactions. These are both read and write requests and are mostly independent of each other. The tuning algorithm dynamically reacts to changes in transaction arrival rates and buffer pool usage patterns. Brown et al. [3, 4] address the problem of goal oriented buffer management for queries and transactions belonging to performance classes. Their approach differs from ours in various ways: we use the performance index (defined below) as a common yardstick to measure relative performance of all classes and they use ....

K.P. Brown, M.J. Carey, and M. Livny. Managing memory to meet multiclass workload response time goals. In TR1146, U. of Wisconsin, Madison, also in Proc. of the 19th Int'l VLDB Conference, August 1993.

....memory management issues continue to be the focus of the database research community. Numerous studies have been devoted to static memory allocation and buffer management [Chou85, Sacc86, Corn89, Falo91, O Neil93] A very few recent papers address techniques that adapt to a dynamic environment [Zell90, Pang93, Brow92, Brow93, Meht93b]. A hash join algorithm requires a significant amount of memory to keep a hash table for efficient execution. This amount varies from the square root of the size of the inner relation to the full inner relation size. The hash table must be held in memory for the entire period of join execution. ....

....[Meht93] Interquery buffering refers to sharing buffer frames that different jobs may use for storing their own data at different times. In inter query buffering, several jobs use a common buffer pool without demarcating bounds between separate jobs. Inter query buffering was investigated in [Brow92, Brow93, Brow94] for a mixed workload of hash join queries and short read transactions. While [Brow92] was a preliminary study comparing several policies of dividing memory between queries and transactions, two other papers ( Brow93, Brow94] proposed the fragment fencing algorithm for managing a multiclass ....

[Article contains additional citation context not shown here]

Brown, K., Carey, M., Livny, M., "Managing Memory to Meet Multiclass Workload Response Time Goals", Proc. 19th Int'l VLDB Conf., Dublin, Ireland, Aug. 1993.

....different: once a page is decoded, it is not used again (except if we support a rewind feature) Thus, page replacement policies such as LRU and FIFO, which aim to maximize the hit rate are not applicable to MEDIC. While a number of studies have addressed real time transaction and IO scheduling [4, 5, 6, 8, 21], to our knowledge no work has dealt with memory disk cache management for multimedia clients. The work that is most relevant is reported in [25] That work proposes a Priority Memory Management (PMM) algorithm that minimizes the number of missed deadlines by adapting both the multiprogramming ....

K. Brown, M. Carey, and M. Livny. Managing memory to meet multiclass workload response time goals. Proc. VLDB Conference, pages 328--41, 1993.

....of bottlenecks. The relatively static nature of data placement decisions also increases the need for an efficient placement algorithm. Other resources, like processors and memory can be re allocated at run time, allowing for the design of dynamic policies that can adapt to workload transients [Brow93, Meht93, Rahm93, Brow94]. However, data placement can be changed only by an expensive reorganization of the relations in the database. All these factors make data placement an extremely important issue in highperformance SN systems. In order to exploit I O parallelism in a parallel SN database system, tuples belonging to ....

.... see [Ghan90, Hua90, Ghan92, Falo93] While the data placement strategy presented in [Cope88] also decided the placement of relations in memory, we feel that caching of relations in memory is better managed by a dynamic memory management policy, like the fragment fencing algorithm presented in [Brow93]. Such policies can adapt to runtime changes while a static data placement strategy cannot. Therefore, data placement in this paper refers only to the placement of relations on disks. Although data placement has been studied extensively in the past and various algorithms have been proposed, there ....

[Article contains additional citation context not shown here]

Brown, K., Carey, M., and Livny, M., "Managing Memory to Meet Multiclass Workload Response Time Goals",<F3.17e+05> Proc. VLDB<F3.733e+05> Conf, Dublin, Ireland, August 1993.

....measurements from our implementation. We assume that the aggregation is being performed directly on a base relation stored on disks as in the example query. The parameters of the study are listed in Table 1 unless otherwise specified. These parameters are similar to those in previous studies e.g. BCL93] The CPU speed is chosen to reflect the characteristics of the current generation of commercially available microprocessors. The I O rate was as observed on the SUN disk on the SUN SparcServer20 51. We model both high speed, high bandwidth network as in commercial multiprocessors like IBM SP 2 ....

K. P. Brown, M. J. Carey, and M. Livny. Managing Memory to Meet Multiclass Workload Response Time Goals. In Proc. of 19th VLDB Conf., 1993.

....measurements from our implementation. We assume that the aggregation is being performed directly on a base relation stored on disks as in the example query. The parameters of the study are listed in Table 1 unless otherwise specified. These parameters are similar to those in previous studies e.g. BCL93] The CPU speed and network speed were chosen to reflect the characteristics of the current generation of 1 In the TPC D benchmark 13 out of 15 queries with aggregates have GROUP BY. Symbol Description Values N number of processors 32 Mips MIPS of the processor 15 R size of relation 400 MB ....

....for the Analytical Models commercially available multiprocessors (e.g. the Intel Paragon) The I O rate was as observed on the Maxtor disk on the Paragon. The software parameters are based on instruction counts taken from the Gamma prototype and are similar to those in previous studies e.g. BCL93] In the following we assume that aggregation on a node is done by hashing. In the remainder of this section we discuss previously proposed approaches to parallel aggregation and two new approaches that we have not seen discussed in the literature. 2.1 Traditional Approach local aggregation ....

Kurt P. Brown, Michael J. Carey, and Miron Livny. Managing Memory to Meet Multiclass Workload Response Time Goals. In Proc. of 19th VLDB Conf., pages 328--341, 1993.

....at the query scheduling level as well. We have considered only workloads involving mixes of queries in this thesis; RTDBS workloads are likely to contain transactions as well as queries. Thus, it would be useful to combine long term data buffering techniques, such as those proposed in [Brow93], with PAQS in order to provide a truly complete memory manager for RTDBSs. The concurrent execution of long running queries and short transactions also raises concurrency control issues that need to be resolved. Finally, PAQS typically takes 5 to 6 iterations to decide on the best MPL setting and ....

K.P. Brown, M.J. Carey, M. Livny, "Managing Memory to Meet Multiclass Workload Response Time Goals", Proc. of the 19th Int. Conf. on Very Large Data Bases, August 1993.

....used in this study models a centralized system composed of one CPU and its memory buffer pool, four SCSI disks and a set of external terminals from which scans are submitted. Our simulator is based on the simulator of a parallel database machine that was used in a number of studies, e.g. in [Mehta93a, Shatdal93, Brown93, Mehta93b, Brown94], and is written in the CSIM C process oriented simulation language ( Schwetman90] The terminals model the external workload source for the system. For closed end experiments, each terminal submits a stream of scans, one at a time, waiting for a response to each before sending the next one. ....

....performance goals. Although we do not consider such a complex workload, we assume a lower memory allocation priority for scans and limit the I O size to one 8 KB page, allocating only two memory pages per scan in the buffer pool. We follow with this setting a number of database performance studies [Brown93, Mehta93b, Brown94]. Relations are of the same length and fully declustered across disks into partitions, one partition per disk. Each partition is stored on a disk in contiguous blocks. A scan processes file partitions sequentially one after another. We denote by diskMPL the number of scans executing concurrently ....

K. Brown, M. Carey, M. Livny. Managing Memory to Meet Multiclass Workload Response Time Goals. In Proc. 19th Int'l VLDB Conf., Dublin, Ireland (1993), pp. 328-341.

.... lesser extent, adaptive approaches have been applied to selected problems in the area of database and transaction pro 386 Gerhard Weikum et al. cessing systems, especially for transaction routing in distributed data sharing systems (e.g. 26] 61] 65] 98] and for memory management (e.g. [9], 11] Unfortunately, the mathematical apparatus of the classical control theory cannot be applied easily to problems of this sort. The reason is that control theoretic models usually relate input and output parameters in terms of differential equations or difference equations. For many tuning ....

....out one or more processes when a certain level of memory overcommitment is reached. This method effectively controls the multiprogramming level of the system. Similar forms of hot set management and corresponding scheduling policies have been discussed from a database specific viewpoint, too [9], 11] 12] 25] 50] 56] 69] 95] The problem of multi user memory management becomes significantly harder, however, when the hots sets of concurrent transactions are overlapping, that is, when we observe both intra and inter transaction locality. On the other hand, dealing with ....

[Article contains additional citation context not shown here]

K. P. Brown, M. J. Carey and M. Livny. Managing Memory to Meet Multiclass Workload Response Time Goals. International Conference on Very Large Data Bases, Dublin (1993).

....of some issues that we believe must be addressed for a complete solution to the problem. 2 A First Step: Goal Oriented Disk Buffer Management In this section we briefly summarize our initial results in the area of goal oriented disk buffer management; a complete description can be found in [Brown 93] We have developed an algorithm called fragment fencing that can be attached to existing buffer allocation and replacement mechanisms and that acts to prevent those mechanisms from making decisions that may jeopardize performance goals. Goals are provided to the algorithm in the form of average ....

....the observed access frequencies of each fragment 2 (those with higher access frequencies are favored for memory residency) and a best guess as to the response time improvement that will result when the fragment s memory residency is increased. The details of this process are discussed in [Brown 93] Target residencies for each fragment are enforced by modifying the existing (base) replacement policy to avoid stealing a page if that would bring the number of memory resident pages below the target for a fragment. Enforcing target residencies thus provides a passive way to fence off ....

[Article contains additional citation context not shown here]

K. Brown, M. Carey, M. Livny, "Managing Memory to Meet Multiclass Workload Response Time Goals," to appear Proc. 19th Int'l VLDB Conf, Dublin, Ireland, August 1993.

....(DBMS) are faced with increasingly demanding performance objectives. These objectives could be time constraint requirements, as in real time database systems [SIGM88, Abbo88, Huan89, Hari90, Kort90, Kim91, RTS92] or administration defined performance goals as in goal oriented database systems [Ferg93, Brow93]. Traditional first come first serve or round robin scheduling policies are no longer adequate to meet such objectives; a DBMS has to prioritize transactions that are competing for system resources according to the system objectives and the resource requirements of the transactions. With priority ....

K.P. Brown, M.J. Carey, M. Livny, "Managing Memory to Meet Multiclass Workload Response Time Goals", Proc. of the 19th Int. Conf. on Very Large Data Bases, August 1993.

....order to derive efficient processing plans for relational queries. Furthermore, simulation is currently used extensively for studying parallel execution strategies for complex queries [CLYY92] and for evaluating strategies for handling complex multiuser workloads in centralized database systems [BCL93, MD93] Most simulation studies of parallel database systems have not explored truly large systems (e.g. 100 s of nodes) due to the prohibitive costs of simulating such systems in detail. If accurate analytical models could be developed, such scheduling strategies could be evaluated This work ....

....algorithm. Our model is based on the approximate mean value analysis, an approach that has proven accurate in modeling parallel architectures [VLZ88, WE90, CS91] Specific system effects are captured by modifying the response time equations. Validation of the model against the simulator used in [BCL93, MD93] shows that the model yields accurate results. The remainder of the paper is organized as follows: Section 2 describes the hybrid hash join algorithm. Section 3 describes the analytical model for the hybrid hash join. The results of the validation of the model and several other experiments, ....

[Article contains additional citation context not shown here]

K. P. Brown, M. J. Carey, and M. Livny. "Managing Memory to Meet Multiclass Workload Response Time Goals". In Proc. of the 19th VLDB Conf., Dublin, Ireland, August 1993.

....routing, CPU and disk scheduling, memory management, data placement, processor allocation, query optimization, etc. Each of these could be driven by performance objectives. Recently, techniques have been proposed for goal oriented transaction routing [Ferg 93] and goal oriented buffer management [Brown 93a] This work was partially supported by the IBM Corporation through a Research Initiation Grant. However, a complete solution to the problem of automatically satisfying multiclass performance goals must employ more than one mechanism; each class can have different resource consumption ....

....that the algorithms do not have to maintain specific response times very accurately. Rather, they need only determine the correct relative response times when comparing between different routing possibilities. Another approach to achieving per class performance goals, called fragment fencing [Brown 93a] uses disk buffer allocation to explicitly manage buffer hit rates. Fragment fencing maintains per class statistics on database reference frequencies and observed hit rates, and uses them to determine a minimum number of memory resident pages for each fragment of the database. Fragments are ....

[Article contains additional citation context not shown here]

K. Brown, M. Carey, M. Livny, "Managing Memory to Meet Multiclass Workload Response Time Goals," Proc. 19th Int'l VLDB Conf, Dublin, Ireland, August 1993.

....inherent in real world database management systems and workloads, accurately predicting the disk buffer allocation required to achieve a particular response time is extremely difficult. Therefore, the general approach common to all goal oriented resource allocation work [Pierce 83, Ferg 93, Brown 93, Brown 94, Chung 94] is the notion of feedback coupled with best guess estimation. The idea is to observe the actual response times of a class relative to its response time goal, and to use the difference between the two as an input to an estimator that will adjust resource allocation knobs in ....

....process of observing, estimating, and adjusting knobs is repeated continuously at regular intervals. The length of these intervals can be expressed as a predefined number of transaction completions, and should be chosen to strike a good balance between responsiveness and statistical stability [Brown 93] 1.2 Criteria for Success How successfully a class meets its response time goal is not the only criteria with which to judge a goal oriented resource allocation algorithm. In our view, the following criteria must be satisfied by any goal oriented resource allocation algorithm before it can be ....

[Article contains additional citation context not shown here]

K. Brown, M. Carey, M. Livny, "Managing Memory to Meet Multiclass Workload Response Time Goals," Proc. 19th Int'l VLDB Conf, Dublin, Ireland, August 1993.

....1. Introduction Database management systems (DBMS) are faced with increasingly demanding performance objectives. These objectives include time constraints, as in real time database systems [SIGM88, RTS92] and administratively defined performance goals, as in goal oriented database systems [Ferg93, Brow93]. Traditional DBMS scheduling policies are no longer adequate to meet such objectives; a DBMS has to prioritize transactions that are competing for system resources according to the system wide objectives and the resource requirements of the transactions. A consequence of priority scheduling is ....

K.P. Brown, M.J. Carey, M. Livny, "Managing Memory to Meet Multiclass Workload Response Time Goals", Proc. of the 19th Int. Conf. on Very Large Data Bases, August 1993.

No context found.

K. Brown, M. Carey, M. Livny, Managing memory to meet multiclass workload response time goals, Proceedings of the International Conference on Very Large Databases (VLDB'93), 1993, pp. 328--341.

No context found.

K. P. Brown, M. J. Carey, and M. Livny. Managing memory to meet multiclass workload response time goals. In Proceedings of Very Large Database Conference, pages 328--341, 1993.

No context found.

Kurt P. Brown, Michael J. Carey, Miron Livny. Managing Memory to Meet Multiclass Workload Response Time Goals, VLDB 1993: 328341.

No context found.

K. P. Brown, M. J. Carey, and M. Livny. Managing memory to meet multiclass workload response time goals. In Proceedings of Very Large Database Conference, pages 328--341, 1993.

No context found.

K. Brown, M. Carey, M. Livny, Managing memory to meet multiclass workload response time goals, Proceedings of the International Conference on Very Large Databases (VLDB'93), 1993, pp. 328--341.

No context found.

K. P. Brown, M. J. Carey, and M. Livny. "Managing Memory to Meet Multiclass Workload Response Time Goals". In Proceedings of the 19 th International VLDB Conference, 1993. Also available as Technical Report No. 1146, University of Wisconsin.

No context found.

K.P. Brown, M.J. Carey, and M. Livny, Managing memory to meet multiclass workload response time goal. In Proc. Of 19th VLDB Conf., Brighton, England, September 1993, 328-341.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC