H. Lu, M.-C. Shan, K.-L. Tan, Optimization of multi-way join queries for parallel execution, Proc. of the 17th Conference on Very Large Data Bases, Barcelona, Spain, September 1991.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....costs) linear tree that does not contain cartesian products. Subsequently, it is remarked in [KBZ86] that the restriction to linear trees may not be a good choice for parallel systems. However, the space of possible join trees is very large if restriction to linear trees is dropped [LVZ93] In [LST91, SwG88] partially heuristic algorithms are proposed that aim at limiting the time spent on searching the space of possible query trees for the cheapest one. SHV92] proposes to parallelize this search. In these papers, the cost formula used evaluates the total costs of a query tree, not taking the ....

H. Lu, M. C. Shan & K. L. Tan, "Optimization of multiway join queries for parallel execution," in Proc 17th VLDB Conf, Barcelona, Spain, September 3-6, 1991, 54960.

....exploited to improve the quality of the results and to reduce the optimization overhead. In that aspect, our technique is innovative, since previous studies on combinatorial query optimization were based on sequential algorithms. Our technique processes bushy query trees. In models like [1] 7] [18], one bushy tree is created by a constructive algorithm, which is intended to find the optimal way of performing the joins. In models like [35] 34] 12] 9] 10] an initial tree is created using some mechanism (possibly an augmentation heuristic) and a randomized algorithm is applied on it ....

H. Lu, M.-C. Shan, and K.-L. Tan, "Optimization of Multi-Way Join Queries for Parallel Execution," Proc. Int'l Conf. Very Large Databases, pp. 549--560, Barcelona, Spain, 1991.

....for multiprocessor systems. However, they in general discuss how to arrange a processing tree such that a balanced workload and the maximum parallelism can be achieved, a very different goal from ours. Their processing trees include a left deep tree, a right deep tree, and a bushy tree [25] [20], 1] 9] 23] 29] 30] A left deep tree refers to a tree where all the right children must be leaf nodes, while, in a right deep tree, all the left children are leaf nodes. A tree that is neither a left deep tree nor a right deep tree is a bushy tree. In a uniprocessor environment, however, ....

H. Lu, M.C. Shan, and K.L. Tan, Optimization of Multi-Way Join Queries for Parallel Execution, Proc. Int'l Conf. Very Large Data Bases, 1991.

....Many search strategies (dynamic programming, deterministic approaches, stochastic techniques [13] have been proposed to find a good parallel execution plan. They all suffer form the drawback, that they limit the search space considerably (e.g. only linear trees in [13] no pipeline parallelism in [26], only left deep trees in [21] Much effort was spent on the different manner of parallel execution according to the processing tree shape [17, 27, 28] e.g. linear or bushy tree) However resource allocation, including the computation of the degree of intra parallelism has not yet deeply ....

....plans both the implementation method and the access paths must be determined. In order to simplify implementation issues, we only considered hash join algorithm and only provide seqscan accessing methods. Anyway this congestion do not affect the basics of our approach. Both synchronized bushy [26] and left linear tree shapes (see section 2) are considered. 9 3) Parallelization and Mapper: Optimal allocation of the physical resources (see section 4.2) 8 It is clear that considering all permutations means to examine n candidate plans (for a linear tree shape; n is the join number) which ....

[Article contains additional citation context not shown here]

Lu H. Shan M.-C. and Tan K.-L. Optimization of Multi-Way Join Queries for Parallel Execution. In Proceedings of the International Conference on Very Large Databases, pages 549--560, Barcelone, Spain, September 1991.

....bushy trees for parallel query processing. A combination of analytical and experimental results was given in [16] 31] to shed some light on the complexity of choosing left deep and bushy trees. An integrated approach dealing with both intra operator and inter operator parallelism was presented in [21], where a greedy scheme taking various join methods and their corresponding costs into consideration was proposed. A two step approach to deal with join sequence scheduling and processor allocation for parallel query processing was devised in [6] Several query plans in processing multi join ....

H. Lu, M.-C. Shan, and K.-L. Tan. Optimization of Multi-Way Join Queries for Parallel Execution. Proceedings of the 17th International Conference on Very Large Data Bases, pages 549--560, September 1991.

....at the cost of searching a larger design space. A combination of analytical and experimental results was given in [20] to shed some light on the complexity of choosing left deep and bushy trees. An integrated approach dealing with both intra operator and inter operator parallelism was presented in [25], where a greedy scheme taking various join methods and their corresponding costs into consideration was proposed. In [10] the resource contention for parallel query execution is taken into consideration to incorporate the sources and deterrents of parallelism in the traditional execution space ....

H. Lu, M.-C. Shan, and K.-L. Tan. Optimization of Multi-Way Join Queries for Parallel Execution. Proceedings of the 17th International Conference on Very Large Data Bases, pages 549--560, September 1991.

....right deep, section 6 describes the heuristic approach, section 7 discusses the evaluation parameters, and the experimental design, section 8 has results and discussions, and section 9 has conclusions and future work. 2 Related Work Work in parallel query optimization [18] 1] 2] 8] 7] 10] [15] [20] 23] 6] 26] 22] has focussed on minimizing the total execution time (elapsed time) for a query. It is difficult, if not impossible, to directly compare the different techniques proposed in literature. This is because these methods have been developed for different architectures, assuming ....

....models, using different cost models, and different evaluation criterion. There still seems to be no consensus on the right model for any of the above domains. There is almost a consensus on one issue though, namely the representation of an execution plan. Most of the proposed methods [6] 23] 1] [15] [8] 18] represent an execution plan using an annotated tree. The nodes of the tree correspond to joins in [23] 15] while they correspond to finer granularity operations like hash, probe, sort, merge, etc. in [6] 18] 8] 1] This is mainly dependent on the granularity at which the different ....

[Article contains additional citation context not shown here]

H. Lu, M.C. Shan, and K.L. Tan. Optimization of multi-way join queries for parallel execution. Proceedings of the 17th International Conference on Very Large Data Bases, September 1991.

....[SELI79, MACK86] and continues to be done [LIPT90] no good cost models exist for the parallel environment. To quote DeWitt and Gray, While the necessary optimizer technology exists, accurate cost models have not been developed, let al..one validated. More work is needed in this area [DeWI90] [LU91] proposed a cost model in which response time of a query plan is considered. Not having pipelining between successive levels of the plan tree simplifies its calculation considerably. Query execution is modeled in a data flow manner in [WILS91] and a flow rate based expression for the average ....

....the problem, e.g. in a distributed memory architecture processor memory assignment are equivalent since each goes with the other. Disk allocation, primarily for 2 intermediate results, will usually be guided by the data declustering approach being used. Processor assignment has been considered in [LU91, HONG92, GANG92, SRIV93, TUREK92, CHEN92b, HUA93], while memory assignment has been considered in [SRIV93 and ZIAN93] Since memory continues to be a critical resource for the performance of database operations even in the parallel environment [SCHN90] a careful modeling of contention on it is important. A first step towards memory allocation ....

H. Lu, M.-C. Shan and K.-L. Tan, "Optimization of Multi-Way Join Queries for Parallel Execution", in Proceedings of the 17th VLDB Conference, Barcelona, Spain, August 1991.

....submitted queries, each of which may be complicated to one degree or another How many processors should be allotted to each independent query 1. 1 Preliminaries The literature on the subject of parallelizing queries is certainly in its infancy, with the focus so far on intraquery scheduling [7, 8, 9, 10]. In this paper we shall devise and evaluate a number of algorithms designed to handle both intra query and inter query scheduling. In fact, the winning algorithm for question 2 will be seen to be hierarchical in nature. In the first phase, a good quality precedence based schedule is created for ....

H. Lu, M.-C. Shan, and K.-L. Tan. Optimization of Multi-Way Join Queries for Parallel Execution. In Proceedings of the 17th International Conference on Very Large Data Bases, Barcelona, Spain, pages 549--560, September 1991.

....R 3 R 4 R 1 R 2 R 3 R 4 R 5 R (a) left deep tree (c) bushy tree (b) right deep tree 5 R 4 R 3 R 2 R 1 R Figure 1: Illustration of different query trees. bushy trees. An integrated approach dealing with both intra operator and inter operator parallelism was presented in [20], where a greedy scheme taking various join methods and their corresponding costs into consideration was proposed. As an extension to [12] an algorithm handling processor scheduling in a bushy tree was proposed in [11] where the inter operator parallelism is achieved by properly selecting ....

....the compile time 2 . The join sequence numbers 1 Note that in dealing with a linear execution tree, one usually has only two joining relations residing in memory at a time, thus limiting the applicability of hash filters to the joining attribute. 2 Various heuristics, such as those in [7] and [20], can be applied to build a bushy execution tree. Note that assigning sequence numbers to joins while building a bushy tree involves little overhead. 3 R 1 A B a 1 b 1 a 2 b 1 a 2 b 2 a 2 b 4 a 3 b 4 a 4 b 7 a 4 b 9 HFR1 (B) h(b i ) set 0 0 1 1 2 1 3 0 4 1 R 2 B C b 1 c 2 b 2 ....

H. Lu, M.-C. Shan, and K.-L. Tan. Optimization of Multi-Way Join Queries for Parallel Execution. Proceedings of the 17th International Conference on Very Large Data Bases, pages 549--560, September 1991.

....plays an important role in query optimization for parallel execution. In this paper we will discuss some of the ways in which queries can be optimized for parallel execution. We will look at various cost models, search algorithms and methods of generating query execution plans (QEPs) GANG92] [LU91], resource allocation techniques [GAR96] NIC93] TAN93] and ways of dealing with loadimbalance [LuTAN94] We will focus on some of the major issues associated with parallel databases and how well these algorithms address them. Search Strategies for Finding Optimal QEPs Optimization strategies ....

....to generate optimal plans since it is difficult to see how an optimal plan can be constructed without considering preemptable resources such as CPU, disks and the communication network. Modeling a QEP is the first step in searching for an optimal one. The method used in most approaches ( GAR96] [LU91], LAN94] GANG92] is to model QEPs as 4 annotated join trees. The level of granularity is thus reflected in the nodes of the join tree, which are base relations at the leaves and components of join operations in internal nodes. Figure 1. A three way join of R1, R2 and R3 Figure 2. The join ....

[Article contains additional citation context not shown here]

H. Lu, M-C. Shan, K-L Tan, "Optimization of Multi-Way Join Queries for Parallel Execution" Proceedings of the 17th International Conference on Very Large Databases, Barcelona, September 1991

....to perform local query optimization. There may be different operations provided by each participating database. Object oriented query processing techniques should also be integrated [KGBW90] At the interoperation layer, we have an intermediate representation of the query, which is derived from [LST91] This is a tree based representation, in which the root represents the result, and the leaves the collections of instances (i.e. classes and relations) described in export schemas. The query translator processes this tree to obtain a more efficient query tree. This optimized query tree ....

H. Lu, M.-C. Shan, and K.-L. Tan. Optimization of Multi--Way Join Queries for Parallel Execution. In Proceedings of the Seventh International Conference on Very Large Data Bases, 1991, pp. 549--560.

....within a multiprocessor system, which is termed interquery parallelism [48] It can be seen that to exploit the third form of parallelism, one has to resort to the results derived for inter operator parallelism within a query. During the past few years some light has been shed on this issue [13, 21, 22, 37, 42, 46]. As an effort toward this trend, the objective of this paper is to study and improve the execution of multi join queries, and devise efficient schemes to exploit inter operator parallelism to minimize the query execution time in a multiprocessor based database system 1 . Note that different ....

H. Lu, M.-C. Shan, and K.-L. Tan. Optimization of Multi-Way Join Queries for Parallel Execution. Proceedings of the 17th International Conference on Very Large Data Bases, pages 549--560, September 1991.

....of bushy trees for parallel query processing. A combination of analytical and experimental results was given in [14] to shed some light on the complexity of choosing left deep and bushy trees. An integrated approach dealing with both intra operator and inter operator parallelism was presented in [16], where a greedy scheme taking various join methods and their corresponding costs into consideration was proposed. A heuristic approach that deals with a query plan tree for effective resource assignments in a bottom up manner was presented in [23] A twostep approach to handle join sequence ....

H. Lu, M.-C. Shan, and K.-L. Tan. Optimization of Multi-Way Join Queries for Parallel Execution. Proceedings of the 17th International Conference on Very Large Data Bases, pages 549--560, September 1991.

.... les strategies d optimisation a deux phases ( SH88, HM95b] et la plupart des SGBD paralleles commerciaux (Sybase, Informix, DB2) determinent d abord un scenario d execution sequentiel optimal (phase 1) puis le parallelise (phase 2) A l inverse, les strategies d optimisation a une phase [HM94, LST91, STY93, LVZZ94] travaillent directement dans l espace des scenarios d execution paralleles. Si les strat egies a deux phases s averent pertinentes dans le cadre de plate formes cibles a memoire partagee [SH88] il est de plus en plus largement admis que les techniques a une phase permettent de ....

H. Lu, M.-C. Shan, and K.-L. Tan. Optimization of Multi-Way Join Queries for Parallel Execution. In Proceedings of the InternationalConference on Very Large Databases, pages 549--560, Barcelone, Spain, September 1991.

....to answer the first question. System issues associated with providing intra query parallelism are addressed in [11] 12] Several researchers have considered the problem of optimizing the 2 1 2 3 4 5 6 7 8 9 10 11 12 Figure 1: A Sample Query Precedence Tree execution of a single parallel query [13, 14, 15, 16, 17, 18, 19, 20]. Even so, only a few of these [15, 16, 17] fully consider the first question, namely how many processors should be allotted to each phase of the parallel query. A somewhat similar question is addressed in [18] but the emphasis is on the allotment of memory rather than processors. Furthermore, ....

.... intra query parallelism are addressed in [11] 12] Several researchers have considered the problem of optimizing the 2 1 2 3 4 5 6 7 8 9 10 11 12 Figure 1: A Sample Query Precedence Tree execution of a single parallel query [13, 14, 15, 16, 17, 18, 19, 20] Even so, only a few of these [15, 16, 17] fully consider the first question, namely how many processors should be allotted to each phase of the parallel query. A somewhat similar question is addressed in [18] but the emphasis is on the allotment of memory rather than processors. Furthermore, none of these references attempts to answer ....

H. Lu, M.-C. Shan, and K.-L. Tan. Optimization of Multi-Way Join Queries for Parallel Execution. In Proceedings of the 17th International Conference on Very Large Data Bases, Barcelona, Spain, pages 549--560, September 1991. 26

....of bushy trees for parallel query processing. A combination of analytical and experimental results was given in [13] to shed some light on the complexity of choosing left deep and bushy trees. An integrated approach dealing with both intra operator and inter operator parallelism was presented in [17], where a greedy scheme taking various join methods and their corresponding costs into consideration was proposed. As an extension to [11] an algorithm handling processor scheduling in a bushy tree was proposed in [10] where the inter operator parallelism is achieved by properly selecting ....

....performance study in Section 4 is a multiprocessor system with distributed memories and shared disks. Barring any tuple placement skew, the scheme developed in this paper is applicable to the shared nothing architecture where each disk is accessi 2 Various heuristics, such as those in [6] and [17], can be applied to build a bushy execution tree. Note that assigning sequence numbers to joins while building a bushy tree involves little overhead. ble only by a single node. To facilitate our presentation and performance evaluation, the join method on which we shall demonstrate the application ....

H. Lu, M.-C. Shan, and K.-L. Tan. Optimization of Multi-Way Join Queries for Parallel Execution. Proceedings of the 17th International Conference on Very Large Data Bases, pages 549--560, September 1991.

....bushy trees for parallel query processing. A combination of analytical and experimental results was given in [14] to shed some light on the 122 complexity of choosing left deep and bushy trees. An integrated approach dealing with both intra operator and interoperator parallelism was presented in [20], where a greedy scheme taking various join methods and their corresponding costs into consideration was proposed. As an extension to [12] an algorithm handling processor scheduling in a bushy tree was proposed in [11] where the inter operator parallelism is achieved by properly selecting ....

....via simulation. It is experimentally shown that the application of HFs is in general a very powerful means to improve the execution of multi join queries, and the improvement becomes more prominent as the number of relations in a query increases. 2 Various heuristics, such as those in [7] and [20], can be applied to build a bushy execution tree. Note that assigning sequence numbers to joins while building a bushy tree involves little overhead 123 The rest of this paper is organized as follows. Preliminaries are given in Sect. 2. The effect of HFs and the proposed scheme are presented in ....

Lu H, Shan M-C, Tan K-L (1991) Optimization of multi-way join queries for parallel execution. Proc 17th Int Conf Very Large Data Bases, September, pp 549--560

....from our analytical cost model while varying several parameter values in section 6. Finally, we discuss our conclusions in section 7. 2 Previous work Because the join is the most frequently used and the most time consuming operation in a relational database system, there have been a lot of works [2, 3, 4, 7, 8, 9] dealing with parallel join algorithms in order to develop efficient join algorithms under several multiprocessor architectures. Those works can be roughly divided into two groups; one group of works deals with the basic parallel join methods and their performance, and another group of works ....

H. Lu, M. Shan, and K. Tan. Optimization of Multi-Way Join Queries for Parallel Execution. In Proceedings of the 17th International Conference on Very Large Data Bases, pages 549--560, 1991.

....To the best of our knowledge, scheduling theory does not provide algorithms that handle communication costs. Scheduling pipelines constitutes a nontrivial generalization of the classical problem of multiprocessor scheduling [GLLK79] Prior work in parallel query optimization [Hon92a, SE93, LCRY93, LST91, SYT93, SYG92, TWPY92, WFA92] ignored the communication overhead of exploiting parallelism. This was sometimes justified by restricting to situations where communication overhead is low: shared memory architectures with omission of pipelined parallelism. Brute force algorithms are impractical for ....

H. Lu, M-C. Shan, and K-L. Tan. Optimization of Multi-Way Join Queries for Parallel Execution. In Proceedings of the Seventeenth International Conference on Very Large Data Bases, Barcelona, Spain, September 1991.

....(intra operation) of relational and specialized (transitive closure, external closure) operators. This method is integrated in the conception process of SQL queries optimizer in a parallel environment. 2. Parallelization of operations and propagation 2. 1 Principle Parallelization of relational [11, 21, 23, 27] and specialized [8, 28] operations consists in going through the query graph so that the cost evaluator [17] may determine the following for each operation: 1) the best algorithm in terms of response time (for example nested loop join, hashing join, sort merge join, 2) the optimal number of ....

H. Lu et al.: Optimization of Multi-Way Join Queries for Parallel Execution. In: Proc. of the 17th VLDB Conf.. Barcelona, September 1991, pp. 549-560.

....of previous joins, in a pipelined way, producing a left deep join tree. In [15] the spanning tree of a query graph is used to find a linear order. An observation related to these heuristics is that the linear join orders produced are not necessarily the best candidates for parallel execution [20]. Restricting the schedules to synchronized query execution plans as in [20] however, may result in missing an optimal solution which often happens to be an asynchronous bushy plan. A linear order to be used in multidatabase query optimization, therefore, should allow the optimization algorithms ....

....In [15] the spanning tree of a query graph is used to find a linear order. An observation related to these heuristics is that the linear join orders produced are not necessarily the best candidates for parallel execution [20] Restricting the schedules to synchronized query execution plans as in [20], however, may result in missing an optimal solution which often happens to be an asynchronous bushy plan. A linear order to be used in multidatabase query optimization, therefore, should allow the optimization algorithms to consider promising asynchronous bushy inners so that optimal or near ....

H. Lu, M. Shan, K. Tan. "Optimization of Multi-way Join Queries for Parallel Execution", In Proc. of the 17th Intl. Conf. on Very Large Data Bases, pp. 549-560, Barcelona, September, 1991.

....the pipeline delay problem. These problems stem from the fixed association between data, operators and processors. In shared disk [Pir90] there is more flexibility since all processors have equal access to disks. Thus, intraoperator parallelism does not require static relation partitioning [Lu91]. Inter operator parallelism is also less constrained since any processor can be allocated to any operator. Load balancing for independent parallelism is addressed in [Hsi94] while only pipeline parallelism is considered in [Lo93] Shared memory offers even more flexibility since all processors ....

H. Lu, M.-C. Shan, K.-L. Tan, "Optimization of Multi-Way Join Queries for Parallel Execution ". Int. Conf. on VLDB, Barcelona, September 1991.

No context found.

H. Lu, M.-C. Shan, K.-L. Tan, Optimization of multi-way join queries for parallel execution, Proc. of the 17th Conference on Very Large Data Bases, Barcelona, Spain, September 1991.

No context found.

H. Lu, M.-C. Shan, and K.-L. Tan. Optimization of Multi-Way Join Queries for Parallel Execution. Proceedings of the 17th International Conference on Very Large Data Bases, pages 549--560, September 1991.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC