W. Hasan and R. Motwani. Coloring away communication in parallel query optimization. In VLDB, 1995.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....and inter process communication. The partitioner firstly identifies whether an operator requires its input data to be partitioned by a specific at tribute when executed on multiple processors (for example, so that the potentially matching tuples from the operands of a join can be compared [10]) Sec ondly, it checks whether data repartitioning is required, i.e. whether data needs to be exchanged among the processors, for example for joining or for submitting to an operation call on a specific machine. The exchanges are placed immediately below the operators that require the data to ....

W. Hasan and R. Motwahl. Coloring away communication in parallel query optimization. In Proceedings of the 21th VLDB Conference, 1995.

....and inter process communication. The partitioner firstly identifies whether an operator requires its input data to be partitioned by a specific at tribute when executed on multiple processors (for example, so that the potentially matching tuples from the operands of a join can be compared [10]) Sec ondly, it checks whether data repartitioning is required, i.e. whether data needs to be exchanged among the processors, for example for joining or for submitting to an operation call on a specific machine. The exchanges are placed immediately below the operators that require the data to ....

W. Hasan and R. Motwahl. Coloring away communication in parallel query optimization. In Proceedings of the 21th VLDB Conference, 1995.

....and inter process communication. The partitioner rstly identi es whether an operator requires its input data to be partitioned by a speci c attribute when executed on multiple processors (for example, so that the potentially matching tuples from the operands of a join can be compared [10]) Secondly, it checks whether data repartitioning is required, i.e. whether data needs to be exchanged among the processors, for example for joining or for submitting to an operation call on a speci c machine. The exchanges are placed immediately below the operators that require the data to be ....

W. Hasan and R. Motwani. Coloring away communication in parallel query optimization. In Proceedings of the 21th VLDB Conference, 1995.

....the architectures experimented with. Then, we explain how the whole query can be executed on smart disk system. We also explain the notion of operation bundling. 4. 1 Individual Database Operations Query optimization and processing in distributed environments has been studied by many researchers [19, 20, 28, 23, 37]. Many of the algorithms we have used in this work are adopted from the algorithms developed for distributed systems. We had to simplify some of the algorithms. But, these simplifications do not invalidate our comparisons, because we use the same assumptions and similar algorithms for both the ....

W. Hasan and R. Motwani. Coloring Away Communication in Parallel Query Optimization. In Proc. 21st Conference on Very Large Databases (VLDB'95), 1995.

....operations to execute the whole query. We introduce the notion of operation bundling and explain the protocol we devised for reducing the communication. 3. 1 Individual Database Operations Query optimization and processing in distributed environments has been studied by many researchers [20, 21, 28, 23, 38]. Many of the algorithms we have used in this work are adopted from the algorithms developed for distributed systems. We had to simplify some of the algorithms. But, these simplifications do not invalidate our comparisons, because we use the same assumptions and similar algorithms for both the ....

W. Hasan, R. Motwani. Coloring Away Communication in Parallel Query Optimization. In Proc. 21st Conference on Very Large Databases (VLDB'95), 1995.

....the architectures experimented with. Then, we explain how the whole query can be executed on smart disk system. We also explain the notion of operation bundling. 4.1. Individual Database Operations Query optimization and processing in distributed environments has been studied by many researchers [19, 20, 28, 23, 37]. Many of the algorithms we have used in this work are adopted from the algorithms developed for distributed systems. We had to simplify some of the algorithms. But, these simplifications do not invalidate our comparisons, because we use the same assumptions and similar algorithms for both the ....

W. Hasan and R. Motwani. Coloring Away Communication in Parallel Query Optimization. In Proc. 21st Conference on Very Large Databases (VLDB'95), 1995.

....for partitioning a query plan into a number of subplans that can be executed in parallel using some type of parallelism is by placing two data dependent operators in different nodes of the parallel machine if data repartitioning is required for the proper execution of the consumer operator. [18] distinguishes attribute sensitive operators and attribute insensitive operators. An attribute sensitive operator, when executed on multiple processors, requires its input data to be partitioned by a distinguished attribute, whereas an attribute insensitive operator does not. An exchange operator ....

W. Hasan and R. Motwani. Coloring away communication in parallel query optimization. In Proceedings of the 21st VLDB Conference, 1995.

....of the query plans into sub plans which are then executed in parallel on di erent nodes of the parallel processing environment. The granularity of this decomposition varies and can be as ne as parallelizing single operator as studied for example in [SD89, SD90, WFA95] but is often chosen coarser [HM95, CHM95, GI96, GI97] 3. Architecture 3 These approaches have in common that they require communication between single nodes for shipping or exchanging partial results. This causes network contention and synchronization e ects where nodes have to wait for others to complete their tasks rst. As a ....

....query plans into sub plans which are then executed in parallel on di erent nodes of the parallel processing environment. The granularity of this decomposition varies and can be as ne as parallelizing single operator as studied for example in [SD89, SD90, WFA95] but is often chosen coarser [HM95, CHM95, GI96, GI97] 3. Architecture 3 These approaches have in common that they require communication between single nodes for shipping or exchanging partial results. This causes network contention and synchronization e ects where nodes have to wait for others to complete their tasks rst. As a ....

W. Hasan and R. Motwani. Coloring Away Communication in Parallel Query Optimization. In Proc. of the Int'l. Conf. on Very Large Data Bases, pages 239-250, Zurich, Switzerland, September 1995.

....the plan into different sets of operators. Figure 3 illustrates two alternative ways of inserting the parallelism related operator in the plan shown in figure 2(b) There are many possible ways of partitioning a query plan and, therefore, many possible placements for the Exchange operators. [12] distinguishes attribute sensitive operators and attribute insensitive operators. An attribute sensitive operator is an operator partitionable only for partitionings that use a distinguished attribute. On the other hand, an attribute insensitive operator is partitionable for all partitionings ....

W. Hasan and R. Motwani. Coloring away communication in parallel query optimization. In Proceedings of the 21th VLDB Conference, pages 239--250, 1995.

....sequential query evaluation plan (QEP) The latter deals with mapping a sequential QEP to a parallel execution environment. The final result is a parallel query execution plan (cf. Fig. 1) Much research has been devoted to achieve the best possible parallelization of a given sequential plan [8, 9, 13, 3, 10]. A common approach is to incorporate many features of the target architecture in the cost model, e.g. communication costs or hardware description. Based on this information a static parallel schedule is derived [8, 4] However, from a validation point of view increasing the number features ....

W. Hasan and R. Motwani. Coloring away communication in parallel query optimization. In Proc. Int'l. Conf. on Very Large Data Bases, Zurich, Switzerland, September 1995.

....work very effectively. In terms of memory consumption, the RD appears to be better than the FP. Among the different query tree shapes, the most competitive seems to be the bushy tree since it allows for more effective parallelization. A different approach in PDS query processing is discussed in [38] where the problem is decomposed 42 into two phases: Join Ordering and Query Rewrite (JOQR) and Parallelization. The rationale of this approach resembles that followed in the compilation of programming languages where the problem is fragmented into several distinct phases in order to deal ....

....the ordering operations and selects the method to compute each operation (for example, the method to compute the joins) In JOQR, an important issue is the choice of the partitioning attributes in the query tree so that the total sum of communication and computation costs is minimized. In [38], this problem is reduced to a query tree coloring problem. Here, the partitioning attributes are regarded as colors and the repartitioning cost is saved when adjacent operators have the same color. Subsequently, the costs function considers communication and computation costs, access methods ....

W. Hasan and R. Motwani. Coloring Away Communication in Parallel Query Optimization. In U. Dayal, P.M.D. Gray, and S. Nishio, editors, Proceedings of the 21st International Conference on Very Large Data Bases, pages 239--250, Zurich, Switzerland, 1995. Morgan Kaufmann Publishers.

....as the kind of information is either getting slowly out of date (e.g. books lists) or bringing few considerable incoherence (e.g. statistical studies data) Many studies have been carried out in the context of OLTP or QP. The main topics are data fragmentation [1,2] Parallel Execution Plans [3,4], and duplication strategies [5] Most of the work done was built on the assumption that parallel DBMS would run on Massively Parallel Machines (MPM) Such machines, while causing an incomparable rise of performance, are still quite few, and represent a big investment. This made hybrid ....

W. Hasan and R. Motwani. Coloring away Communication in Parallel Query Optimization. In Proceedings of 21 st VLDB Conference, Zurich, Switzerland, 1995.

....employed on Decision Support Systems. Speed of updates, and time in general, are less significant here, as insconsistencies appear slower, and are generally less dangerous. Many studies have been carried out in these contexts. The main topics are data fragmentation [1,2] Parallel Execution Plans [3,4], and duplication strategies [5] Most of the work done was designed for Massively Parallel Machines (MPM) which are powerful but quite expensive. This made hybrid architectures, such as workstation clusters, or networks of workstations come to the front page of research [6,7] More than ....

W. Hasan and R. Motwani, "Coloring Away Communication in Parallel Query Optimization," in Proceedings of the 21st VLDB Conference, (Zurich, Switzerland), pp. 36--47, 1995.

....this is acceptable for some algorithms implementing relational algebra operators, it is not true for object oriented methods with multiple inputs. Moreover, the delay between the start time of a node and the start time of its parent in a pipeline is almost uniformly ignored. Hasan and Motwani [HM94, HM95] study the problem of pipelined parallelism from the viewpoint of communication tradeoff. They propose heuristics to minimize communication overhead taking multitasking into account [HM94] and a method of specifying the operator to processors mapping [HM95] However, they do not study the impact ....

....ignored. Hasan and Motwani [HM94, HM95] study the problem of pipelined parallelism from the viewpoint of communication tradeoff. They propose heuristics to minimize communication overhead taking multitasking into account [HM94] and a method of specifying the operator to processors mapping [HM95]. However, they do not study the impact of the network configuration. This configuration affects the selection of physical channels to connect two communicating processors, as well as the load on the channels. A router may even dynamically choose different routes between the same processors during ....

Waqar Hasan and Rajeev Motwani. Coloring away communication in parallel query optimization. In Int. Conf. on Very Large Databases, pages 239--250, Zurich, Switzerland, 1995.

....producing a query plan, which is a tree of operation nodes. Each node represents a piece of work such as scanning a relation, sorting, or performing a join. The XPRS parallel database management system [HON92] used a single site optimizer during this phase and therefore ignored communication cost. [HM95] presents algorithms that incorporate communication costs. Dividing a plan into parts and scheduling the parts in an optimal way is itself an NP complete problem. CHM95] presents two approximation algorithms for dividing query plans into subplans for scheduling on a parallel machine. The original ....

W. Hasan, R. Motwani, "Coloring Away Communication in Parallel Query Optimization," 1995. Submitted for publication.

....an operator tree [GHK92, Hon92, Sch90] The second schedules the operator tree on a parallel machine. In this paper, we are only concerned with the second phase. Several approaches exist for the first phase; Hong and Stonebraker [HS91] used a conventional query optimizer while Hasan and Motwani [HM95] develop algorithms that incorporate communication costs. We will first discuss the forms of available parallelism and how they are captured by the operator tree representation. We then describe how we model communication. Finally, we describe a variety of scheduling problems. The reader is ....

....a job with zero cost. The practical importance of the problems presented in this paper rests on the premise that communication is a significant component of the cost of processing a query in parallel. The reader is referred to Pirahesh et al. PMC 90] Gray [Gra88] and Hasan et al. HM95] for such evidence. The parallelism communication trade off is not the only concern in parallel query optimization. We have assumed that a parallel machine consists of a set of processors that communicate over an inter connect. Enhancing the machine model to incorporate disks and memories ....

W. Hasan and R. Motwani. Coloring Away Communication in Parallel Query Optimization, 1995. Submitted for publication.

No context found.

W. Hasan and R. Motwani. Coloring away communication in parallel query optimization. In VLDB, 1995.

No context found.

W. Hasan and R. Motwani. Coloring away communication in parallel query optimization. In Proceedings of the 21th VLDB Conference, 1995.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC