M. Spiliopoulou, M. Hatzopoulos, and Y. Contronis. Parallel Optimization of Large Join Queries with Set Operators and Aggregates in a Parallel Environment Supporting Pipeline. IEEE Transactions on Knowledge and Data Engineering, 8(3):429--445, June 1996.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....utilized in sequential query optimization performs almost an exhaustive search over the search space. Even, if special implementation techniques are used to reject obvious costly execution planning, this technique becomes soon intractable in the parallel context, as pointed out by several authors [9, 10, 11]. Therefore several works propose heuristic based optimization methods [12, 13, 14, 15] These methods work well if low cost partitions of the search space are accessed. However, as the navigation is more or less randomly, locally advantageous moves might be accepted allowing no access to global ....

....plan can be found. The use of randomized search strategies for parallel query optimization was first examined by Lanzelotte et al. 16, 9] based on experiences in sequential optimization [17] The presented framework only considered intra operator parallelism. Later work like Spilipoulou et al. [10] studied the effectiveness of those strategies for inter operator parallelism. However, this approach suffers from the fact that it does not consider resource limitations. In this context we propose to study randomized search strategies not only from the point of view of its generating solution ....

[Article contains additional citation context not shown here]

M. Spiliopoulou, M. Hatzopoulos, and Y. Contronis. Parallel Optimization of Large Join Queries with Set Operators and Aggregates in a Parallel Environment Supporting Pipeline. IEEE Transactions on Knowledge and Data Engineering, 8(3):429--445, June 1996.

....the search to orderings allowing no inter operator parallelism. However, the exploitation of inter operator parallelism has been shown to be very effective in the case of high performance parallel machines. Thus latter works managed the search complexity by applying randomized search or heuristics [8,9]. These methods work well if low cost partitions of the search space are accessed. However, as the navigation is more or less randomly, locally advantageous moves might be accepted allowing no access to global low cost strategies later on. This is not acceptable for complex database applications. ....

....to utilize the available resources inefficiently. Optimizers support for interoperator parallelism in these prototypes is yet weak. In order to manage the operator ordering problem for the GBT space, randomized search or greedy heuristics are proposed. Randomized search has mainly been used in [8,13]. The performance of this method is hard to predicate as the local transformations are chosen randomly out of the set of possible ones. If GBT space has to be searched, it could not be guaranteed that for the most cases the search ends in a low cost partition of the search space. More recently, ....

[Article contains additional citation context not shown here]

M. Spiliopoulou, M. Hatzopoulos, and Y. Contronis. Parallel Optimization of Large Join Queries with Set Operators and Aggregates in a Parallel Environment Supporting Pipeline. IEEE Transactions on Knowledge and Data Engineering, 8(3):429--445, June 1996.

....of different operators from the 5 same execution schedule is modelled, taking into account pipelined and partitioned execution. The approach is applied to a particular system (DB2 Parallel Edition on an IBM SP2 architecture) but no comparison results are reported. M. Spiliopoulou et al. [30,28,29] study the problem of estimating execution time for queries composed of multiple pipelined operators scheduled to run in a parallel database system. The intention is to incorporate the execution time prediction mechanism into a generic optimiser for parallel query processing. The developed cost ....

M. Spiliopoulou, M. Hatzopoulos and Y. Cotronis, "Parallel optimization of large join queries with set operators and aggregates in a parallel environment supporting pipeline", IEEE Trans. On Knowledge and Data Engineering, vol. 8, no.3, pp.429445, June 1996.

....tuples, la taille des tuples entre 1 et 1.000 octets, le nombre d attributs entre 2 et 26, le ratio (nombre de valeurs uniques) cardinalite de la relation) entre 0 et 1. Les caracteristiques proposees sont donc assez proches des bancs de tests proposes par Ioannidis [IK90] puis Spiliopoulou [SHC96] On notera cependant que la dispersion de la cardinalite des relations est, a l instar du banc de test TPC D, plus importante afin de tenir compte des caracteristiques propres aux applications d aide a la decision. L architecture consideree est un systeme sans partage de 8, 32 et 64 processeurs. ....

....l impact, la pertinence, par rapport aux arbres denses classiques, des arbres denses serialises (ADS) vis a vis de la recherche de scenarios d execution optimaux. En guise de reference, une recherche de scenarios denses classiques, menee a l aide de l algorithme Iterative Improvement (II) SHC96] a tout d abord ete effectuee 200 fois en ce qui concerne les requetes a 6, 11 et 15 jointures et 50 fois en ce qui concerne les requetes a 25 jointures. Dans une deuxieme serie d experimentations, nous avons integre les ADS dans l espace de recherche de l optimiseur en vue de determiner leur ....

M. Spiliopoulou, M. Hatzopoulos, and Y. Contronis. Parallel Optimization of Large Join Queries with Set Operators and Aggregates in a Parallel Environment Supporting Pipeline. IEEE Transactions on Knowledge and Data Engineering, 8(3), June 1996.

....attention it deserves. A number of conventional relational query processing approaches have been applied to or extended for answering OLAP queries. Some of this work has concentrated on efficiently performing GROUP BY [8, 9, 20] aggregation [10, 23, 33, 30, 50, 68, 69] join or range queries [32, 60, 64], or supporting incomplete query answers [6, 29, 66] Several approaches have been proposed for supporting the SQL CUBE operator, including [2, 17, 23, 42, 53, 58] Yet another facet of query processing that has received attention in the literature is that of efficiency. Fast query evaluation is ....

M. Spiliopoulou, M. Hatzopoulos, and Y. Cotronis. Parallel optimization of large join queries with set operators and aggregates in a parallel environment supporting pipeline. IEEE TKDE, 8(3):429--45, June 1996.

....complex queries, which can grow up to 25 joins for relation size of some Gigabytes [2] Parallelism opens new perspective to handle those challenges, but increases the complexity of query optimization. This render inadequate the currently used algorithms for finding an optimal execution scenario [3]. In this context we use randomized search algorithms to present a low cost and effective solution to this problem. In detail, a parallel query optimizer has first to determine the join ordering of those queries [1] This implies to fix the degree of inter operation parallelism, i.e. the number of ....

....until a low cost scenario could be found. The use of randomized search strategies in parallel query optimization was first examined by Lanzelotte et al. 7] based on experiences in sequential optimization [11] The presented framework only considered intra operator parallelism. Spilipoulou et al. [3] then study the effectiveness of those strategy for inter operation parallelism. However, this approach suffers from the fact that it do not consider resource limitations and thus generate parallelism without load balance. Thus only a subset of effective inter operation parallelism is considered. ....

[Article contains additional citation context not shown here]

M. Spiliopoulou, M. Hatzopoulos, and Y. Contronis. Parallel Optimization of Large Join Queries with Set Operators and Aggregates in a Parallel Environment Supporting Pipeline. IEEE Transactions on Knowledge and Data Engineering, 8(3), June 1996.

....the optimization of join queries. Large join queries, occuring in decision support systems, knowledge bases and advanced database applications, including CAD CAM, are of particular interest, and many researchers have proposed techniques for their efficient optimization [SG88, IK91, LVZ93, LOY94, SHC96] The solutions space for parallel join queries is too large to be scanned exhaustively. Techniques based on combinatorial optimization are recently being tested on parallel architectures [LVZ93, LOY94, SHC96] after having been studied extensively for sequential queries [SG88, IK91, SMK93] ....

....have proposed techniques for their efficient optimization [SG88, IK91, LVZ93, LOY94, SHC96] The solutions space for parallel join queries is too large to be scanned exhaustively. Techniques based on combinatorial optimization are recently being tested on parallel architectures [LVZ93, LOY94, SHC96] after having been studied extensively for sequential queries [SG88, IK91, SMK93] Despite their promising results, their effectiveness is affected by the shape of the solutions space: Iterative improvement might be inefficient in a space containing high plateaus and local minima at various ....

[Article contains additional citation context not shown here]

Myra Spiliopoulou, Michalis Hatzopoulos, and Yannis Cotronis. Parallel optimization of large join queries with set operators and aggregates in a parallel environment supporting pipeline. IEEE Trans. on Knowledge and Data Engineering, 1996. To appear.

....time of a query schedule becomes more realistic. However, the new parameters expressing the influence of latency, multitasking and network load on query cost, increase the search space to be scanned by the optimizer. We therefore envisage optimization techniques based on randomized algorithms [IK91, LVZ93, SHC95] or genetic algorithms [MMS94] for the exploration of the search space of our model. In the next section, we introduce our base terminology, specify the problem of schedule cost computation and present our schedule execution model. In section 4, we present a mechanism computing the execution ....

Myra Spiliopoulou, Michalis Hatzopoulos, and Yannis Cotronis. Parallel optimization of large join queries with set operators and aggregates in a parallel environment supporting pipeline. IEEE Trans. on Knowledge and Data Engineering, 1995. To appear.

....support; their efficient processing is constantly gaining in importance. For the optimization of large join queries, combinatorial optimization techniques with polynomially increasing optimization time have been studied in [ Swami and Gupta, 1988; Ioannidis and Kang, 1990; Lanzelotte et al. 1993; Spiliopoulou et al. 1996 ] etc. However, their efficiency depends on the shape of the cost function, which can vary considerably [ Bennett et al. 1991 ] Consequently, methodologies based on evolutional computation gain in interest, since they are more robust towards varying cost function shapes. Genetic algorithms for ....

....written to disk, from which it is retrieved by the parent. The second cost model, CM 2 , also incorporates pipeline: the output of a node is not written to disk but sent immediately to its parent, which is executed simultaneously with its children. Those cost models are analyzed in [ Stillger and Spiliopoulou, 1996 ] 4 Current Status Our GP model is part of a larger model for parallel query optimization. For small join queries, the search space is scanned exhaustively [ Spiliopoulou et al. 1993 ] For larger queries, two variations of iterative improvement have been implemented [ Spiliopoulou et al. ....

[Article contains additional citation context not shown here]

Myra Spiliopoulou, Michalis Hatzopoulos, and Yannis Cotronis. Parallel optimization of large join queries with set operators and aggregates in a parallel environment supporting pipeline. IEEE Trans. on Knowledge and Data Engineering, 1996. To appear.

....Moreover, query execution plans with different resource demands are not directly comparable. This implies the need of a special metric and increases considerably the search space of query optimizers generating alternative query execution plans to find the optimal solution, as those described in [9, 10, 12, 13, 21]. In this study, we present a cost model for the estimation of I O and communication cost in a parallel shareddisk database system. We adopt the bushy tree representation to exploit both bushy parallelism and pipelining. Our approach to the exploitation of parallelism stays between the two ....

....metric. We analyze the operator cost by considering various algorithms and studying the behaviour of each one in pipelined execution. We then combine the cost components to compute the cost of the entire query execution plan. Although the cost model is used for an optimizer processing bushy trees [21], it can also be applied to any other tree type, such as right deep trees. In the next section we describe the architecture, storage structures and algorithms considered, and the cost parameters describing them. In sections 3 and 4 we present the cost formulae for isolated and pipelined processes, ....

[Article contains additional citation context not shown here]

M. Spiliopoulou, M. Hatzopoulos, and Y. Cotronis. Parallel optimization of large join queries with set operators and aggregates in a parallel environment supporting pipeline. IEEE Trans. on Knowledge and Data Engineering, 1995. To appear.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC