M. Nakayama, M. Kitsuregawa, and M. Takagi. Hash-partitioned join method using dynamic destaging strategy. In Proc. Int. Conf. on Very Large Data Bases (VLDB), pages 468--478, 1988.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....function itself. When this disparity becomes large, the bucket no longer fits in main memory and hash based join degrades into nested loop join. Partition skew originates in the hash function chosen by the optimizer. Several papers have proposed ways to deal with partition skew in hash based join [3, 9, 10, 17, 20]. Intrinsic skew occurs when attributes are not distributed uniformly; it has also been called attribute value skew [20] Intrinsic skew impacts the performance of both hash and sortbased joins. Sort merge join works best when the join attributes are the primary key of both relations. This ....

Masaya Nakayama, Masaru Kitsuregawa and Mikio Takagi, "Hash-partitioned Join Method Using Dynamic Destaging Strategy," in Proceedings of the International Conference on Very Large Data Bases, Francois Bancilhon and David J. DeWitt (eds), pp. 469--478, Los Angeles, CA, August, 1988.

....similar to a sensor network [3, 49] Kabra and DeWitt proposed to reoptimize parts of queries after blocking operators [24] There is also a lot of work on adaptive query operators, an area we believe to be relevant to sensor networks. Examples include work on memory adaptive sorting and hashing [13, 28, 30, 34, 53, 54], and online aggregation algorithms [15, 18, 39, 48] Eddies push the idea of feedback on a tuple by tuple basis in online aggregation to adapting join orders at the same frequency [4] Other relevant work includes sequence query processing [42, 43] and temporal and time series databases [52] ....

M. Nakayama, M. Kitsuregawa, and M. Takagi. Hash-partitioned join method using dynamic desraging strategy. In F. Bancilhon and D. J. DeWitt, editors, Fourteenth International Conference on Very Large Data Bases, August 9 - September 1, 1988.

....randomize the input. In parallel systems, partition skew may result in an unbalanced workload, which can greatly degrade the performance of the whole system. Several papers have proposed ways to deal with partition skew in hash based join [DeWitt et al. 92, Hua and Lee 91, Kitsuregawa et al. 89, Nakayama et al. 88, Walton et al. 91] Intrinsic skew occurs when attributes are not distributed uniformly; it has also been called attribute value skew [Walton et al. 91] Intrinsic skew impacts the performance of both hash and sort based joins. Sort merge join works best when the join attributes are the primary ....

Masaya Nakayama, Masaru Kitsuregawa and Mikio Takagi, "Hash-partitioned Join Method Using Dynamic Destaging Strategy," in Proceedings of the International Conference on Very Large Data Bases, Francois Bancilhon and David J. DeWitt (eds), pp. 469--478, Los Angeles, CA, August, 1988.

.... the number of values in the building relation, or chooses a poor hash function, the first partition of the building relation is likely to be too large and have to be paged, resulting in as much as a random I O operation (seek and write) per tuple of the first partition of the probing relation [NKT88] By contrast, Hybrid Cache never over utilizes memory, since its first partition is dynamically grown to the appropriate size. Both algorithms can under utilize memory for the first partition if estimates are incorrect, but this is less dangerous than over utilizing memory, which results in ....

....however that duplicatefree inputs are common for primary key joins and semi joins, and hence the optimizations of Hybrid Cache may frequently be useful in hybrid hash join. More general (though somewhat complex) solutions to the problems of hybrid hash join have been proposed by Nakayama, et al. NKT88] 4.4 Sort vs. Hash Revisited In analyzing hashing and sorting, Graefe presents the interesting result that hash based algorithms typically have dual sorting algorithms that perform comparably [Gra93] However, we observe in this section that one of his dualities is based on an assumption ....

Masaya Nakayama, Masaru Kitsuregawa, and MikioTakagi. Hash-Partitioned Join Method Using Dynamic Destaging Strategy. In Proc. 14th International Conference on Very Large Data Bases, pages 468--477, Los Angeles, AugustSeptember 1988.

....are: 1. Before creating a new run. 8 2. Before beginning a new merge phase. 3. When the sort is done (to hand memory used back to the system) Our strategy for making hash joins adaptive would be to have them over partition their input, similar to GRACE by Nakayama, Kitsuregawa and Takagi [NKT88]. Some hueristic could be used to estimate how much memory a join would eventually be allowed to use. Based on this estimate, some number of partitions in excess of the amount that would be correct for a static hash join would be created initially. Twice as many partitions as the static version ....

Masaya Nakayama, Masaru Kitsuregawa, and Mikio Takagi. Hash-Partitioned Join Method Using Dynamic Destaging Strategy. In Proceedings of the 14th International Conference on VLDB, 1988.

....the anchor bucket is pre determined. This strategy could have problem since the anchor bucket may be too large to fit in memory or too small to be effective. Because usually the data distribution is unknown before join, the decision to choose the anchor bucket should be delayed as much as possible [32]. In DDS, the intended bucket size is chosen to be much smaller than the intended partitions to avoid bucket overflow. Actually in implementation not bucket sizes are set, rather, the number of buckets are set so that if data is distributed evenly, the bucket size will be small enough to fit in ....

..... The cost to read relation R into memory in the joining phase is [62] C(JRR ) Hs X t=1 C(read write group of t) H s ( j R j nwg Gamma 1) A seq (20) It is claimed in [62] that the performance of their technique is several times better than that of Dynamic Hashing GRACE Hash join [32, 35]. Orders of executions: It has long been accepted that if a query has projection, selection and join operations, it is usually more efficient to perform projection and selection before performing join. This practice may greatly reduce the size of data entering join 24 and since join is the most ....

M. Nakayama, M. Kitsuregawa, and M. Takagi. Hash-partitioned join method using dynamic destaging strategy. In Proc of the 14th Int'l Conf. on Very Large Data Bases, pp. 468-478, Los Angeles, California, USA, August 1988.

....loop join or merge join [BE76] Both these schemes are expensive since nested loop join performs a lot of disk I Os and merge join requires sorting of data prior to the join. Hence hash join was proposed as a better alternative and has since been enhanced to improve performance [Bra84, DKO 84, NKT88, KNT89, Sha86] Query processing schemes in commercial products currently use both merge join or hash join. Note that if the data is stored in a sorted order or if an index (like B tree) exists on the datasets, then data need not be sorted prior to merge join. Similarly hash indices can be ....

....smaller of the join inputs and probe this table for items in the large input. To overcome the memory size limitation while handling large inputs (table size greater than the size of the memory available) the hash join scheme was enhanced into the partition or hybrid hash join schemes [DKO 84, NKT88, KNT89, Sha86] where both inputs are first partitioned into disjoint subsets, and pairs of subsets, one from each relation, are matched using the basic hash join scheme. With the advent of parallel processing, there has been growing interest in exploiting parallelism for sort [Men86, LY89, STG ....

M. Nakayama, M. Kitsuregawa, and M. Takagi. Hash-partitioned join method using dynamic destaging strategy. In Proc. Int'l. Conf. on Very Large Data Bases, page 468, Los Angeles, CA, August 1988.

....uneven distribution of data which can result in large differences in the sizes of the partitions of a relation. The following chapters primarily use the GRACE or hybrid hash algorithms as examples of hash join algorithms. However, the following join algorithms could also be used. Nakayama et al. [56] proposed using a dynamic destaging strategy during the partitioning phase. It creates many buckets into which the input relations are partitioned. The number of buckets is typically much greater than P . All buckets initially start as internal, that is, they are held in a hash table in memory, as ....

....assume that the distribution of records to partitions is even, and thus they do not perform as well under non uniform distributions. A number of algorithms have been proposed which do not assume this. Some of these were discussed in Section 2.3. 3, and include the work by Kitsuregawa et al. [35, 56]. Any of these algorithms can be used in place of the GRACE hash join in our method without affecting the way in which we determine the optimal bit allocation for a file. Our method attempts to determine an optimal bit allocation for the hash join algorithm. It does this by attempting to ensure ....

[Article contains additional citation context not shown here]

M. Nakayama, M. Kitsuregawa, and M. Takagi. Hash-partitioned join method using dynamic destaging strategy. In Proceedings of the Fifteenth International Conference on Very Large Data Bases, pages 468--478, Los Angeles, California, USA, August 1988.

....servers and must be performed locally. To do this, two join operators have been added as additional primitives to the basic Kleisli system: the blocked nested loop join [22] and the indexed blocked nested loop join where indices are built on the fly (this is a variation of the hashed loop join of [30]) The join ruleset is dedicated to recognizing under what conditions to apply which join operator. For example, the indexed join can be used only if equality tests in the join condition can be turned into index keys. In addition, the optimizer also parallelizes joins involving remote sources to ....

M. Nakayama, M. Kitsuregawa, and M. Takagi. Hash-partitioned join method using dynamic destaging strategy. In Proceedings of Conference on Very Large Databases, pages 468--478, 1988.

....cost functions under estimate the cost of current algorithms and over estimate the cost of Seq , the actual performance gain of Seq is likely to be even greater. 1 Introduction Since the introduction of the GRACE hash join method [1] many other hash join algorithms have been proposed [2, 3, 4, 5, 6, 7]. Such work has focussed mainly on three goals: 1) reducing CPU costs, 2) reducing the amount of data transferred between memory and disk, and (3) improving the stability of hash joins in the presence of data skew. However, machine capabilities have changed dramatically since the introduction of ....

....hash joins and analyzes their costs. Section 3 describes our new method for hash join. Section 4 compares of the I O costs for our method and other current methods, and Section 5 concludes this paper. 2 Previous Work Traditional hash join algorithms have been discussed extensively in literature [2, 3, 4, 5, 6, 7]. Here we summarize only their relevant aspects. Given two relations to be joined, we assume without loss of generality that the smaller relation is the inner relation, denoted R, and the other relation is outer relation, denoted S. The result relation of the join is RES. The size of a relation ....

[Article contains additional citation context not shown here]

M. Nakayama, M. Kitusregawa, and M. Takagi, "Hash-partitioned join method using dynamic destaging strategy," in Proceedings of the 14th VLDB Conference, pp. 468--478, 1988.

.... product into a join; various data placement strategies, such as clustering, and the development of indexing, such as B trees and hash structures; optimization of selections to use indices where appropriate; various optimizations of joins, such as blocked nested loop join [16] and hashed loop joins [23]. While many of these optimizations fall outside of the semantics of the model itself, the existence of algebraic rewrite rule simplifies the process enormously since commutativity of various positive operators (cartesian product and union) is assured. Optimizations are even more important in a ....

Nakayama, M., Kitsuregawa, M., and Takagi, M. Hash-partitioned join method using dynamic destaging strategy. In Proceedings of Conference on Very Large Databases (1988), pp. 468--478.

....allocate to the join and the size of its R partitions. One possible cause of this discrepancy is due to incorrect estimation of the hash attribute distribution. This results in a situation where some R partitions are larger than the allocated memory, while other R partitions are under sized. In [Naka88], a modification of Hybrid Hash Join was proposed to deal with this memory misfit problem. Instead of deciding on the number of partitions at the beginning, the proposed modification splits the inner relation into smaller subsets, called buckets, which will later be grouped into partitions. The ....

....and the memory that is available to it is memory contention due to other transactions or queries (as discussed in the introduction) or by other processes that are running in the system concurrently with the DBMS. Zeller and Gray first addressed this situation in [Zell90] Like the algorithm in [Naka88], the algorithm that they proposed divides the inner relation into many buckets. Unlike the Nakayama et al. algorithm, the Zeller and Gray algorithm immediately groups these buckets into tentative partitions. The total number of buckets and the number of buckets per partition are both parameters of ....

[Article contains additional citation context not shown here]

M. Nakayama, M. Kitsuregawa, M. Takagi, "Hash-Partitioned Join Method Using Dynamic Destaging Strategy ", Proc. of the 14th Int. Conf. on Very Large Data Bases, August 1988.

....performance [1, 3] We have introduced two join operators as additional primitives to the basic Kleisli system. One of them is the blocked nested loop join [14] The other is the indexed blocked nested loop join where indices are built on the fly; this is a variation of the hashed loop join of [16]. Both operators have a good balance of memory consumption, response time, and total time behaviors. We use the former for general joins and the latter when equality tests in join conditions can be turned into index keys. These two operators are accompanied by 23 new optimization rules to help the ....

M. Nakayama, M. Kitsuregawa, and M. Takagi. Hash-partitioned join method using dynamic destaging strategy. In Proceedings of Conference on Very Large Databases, pages 468--478, 1988.

....the invention of relational database systems, tremendous effort has been undertaken in order to develop efficient join algorithms. Starting from a simple nested loop join algorithm, the first improvement was the introduction of the merge join [1] Later, the hash join [2, 6] and its improvements [21, 25, 30, 37] became alternatives to the merge join. For overviews see [29, 35] and for a comparison between the sort merge and hash joins see [12, 13] A lot of effort has also been spent on parallelizing join algorithms based on sorting [9, 27, 28, 33] and hashing [5, 10, 34] All of these algorithms ....

M. Nakayama, M. Kitsuregawa, and M. Takagi. Hash-partitioned join method using dynamic destaging strategy. In Proc. Int. Conf. on Very Large Data Bases (VLDB), pages 468--478, 1988.

....algebra instruction is carried out on a single RAP, but there is no fundamental reason why multiple RAPs cannot cooperate on a single task. We are working on schemes that would enable multiple RAPs to co operate in sorting a single relation in parallel, and on parallel hash based join algorithms [20]. At the opposite end of the spectrum, there are some tasks that are too small for RAPs. For example when Aditi evaluates predicates for list manipulation such as list reverse and append in a bottom up set at a time manner, each iteration of the RL code involves several relational algebra ....

M. Nakayama, M. Kitsuregawa, and M. Takagi. Hash-partitioned join method using dynamic destaging strategy. In Proceedings of the Fourteenth Conference on Very Large Data Bases, pages 468--477, Los Angeles, 1988.

....theoretically possible. 5 Non Uniform Data Distributions The problem of non uniform data distributions is a common one for all hash based techniques. To address this problem Nakayama, et al. proposed an extension to the hybrid hash method, called the Dynamic Hybrid GRACE Hash join method (DHGH) [11]. Their method dynamically determines which partitions will be stored in memory and which will be stored on disk. This depends upon the distribution of data. In [6] they provide an analysis of DHGH and show the effect of varying partition sizes and show that a large number of small partitions is ....

M. Nakayama, M. Kitsuregawa, and M. Takagi. Hash-partitioned join method using dynamic destaging strategy. In F. Bancilhon and D. J. DeWitt, editors, Proceedings of the Fifteenth International Conference on Very Large Data Bases, pages 468--478, Los Angeles, CA, USA, August 1988.

....the invention of relational database systems, tremendous effort has been undertaken in order to develop efficient join algorithms. Starting from a simple nested loop join algorithm, the first improvement was the introduction of the merge join [1] Later, the hash join [2, 7] and its improvements [20, 23, 28, 39] became alternatives to the merge join. For overviews see [27, 37] and for a comparison between the sort merge and hash joins see [13, 14] A lot of effort has also been spent on parallelizing join algorithms based on sorting [10, 25, 26, 34] and hashing [6, 12, 36] Another important research ....

M. Nakayama, M. Kitsuregawa, and M. Takagi. Hashpartitioned join method using dynamic destaging strategy. In Proc. Int. Conf. on Very Large Data Bases (VLDB), pages 468--478, 1988.

....the invention of relational database systems, tremendous effort has been undertaken in order to develop efficient join algorithms. Starting from a simple nested loop join algorithm, the first improvement was the introduction of the merge join [1] Later, the hash join [2, 7] and its improvements [19, 22, 28, 39] became alternatives to the merge join. For overviews see [27, 37] and for a comparison between the sort merge and hash joins see [13, 14] A lot of effort has also been spent on parallelizing join algorithms based on sorting [10, 25, 26, 34] and hashing [6, 12, 36] Another important research ....

M. Nakayama, M. Kitsuregawa, and M. Takagi. Hash-partitioned join method using dynamic destaging strategy. In Proc. Int. Conf. on Very Large Data Bases (VLDB), pages 468--478, 1988.

No context found.

M. Nakayama, M. Kitsuregawa, and M. Takagi. Hash-partitioned join method using dynamic destaging strategy. In Proc. Int. Conf. on Very Large Data Bases (VLDB), pages 468--478, 1988.

No context found.

M. Nakayama, M. Kitsuregawa, and M. Takagi. Hashpartitioned join method using dynamic destaging strategy. In VLDB 1988.

No context found.

M. Nakayama, M. Kitsuregawa, M. Takagi, Hash-Partitioned Join Method Using Dynamic Destaging Strategy, Proc. 14th VLDB (1988), pp. 468-478

No context found.

M. Nakayama, M. Kitsuregawa, and M. Takagi. Hash-Partitioned Join Method Using Dynamic Destaging Strategy. In Proceedings of the 14th VLDB, pages 468--478, Aug. 1988.

No context found.

Masaya Nakayama, Masaru Kitsuregawa and Mikio Takagi, "Hash-partitioned Join Method Using Dynamic Destaging Strategy," in Proceedings of the International Conference on Very Large Data Bases, Francois Bancilhon and David J. DeWitt (eds), pp. 469--478, Los Angeles, CA, August, 1988.

No context found.

M. Nakayama, M. Kitsuregawa, and M. Takagi. Hash-partitioned join method using dynamic destaging strategy. In 14th Conference on Very Large Data Bases, Los Angeles, California, 1988.

No context found.

M. Nakayama, M. Kitsuregawa, and M. Takagi. Hashpartitioned join method using dynamic destaging strategy.In Proceedings of ConferenceonVery Large Databases, pages 468#478, 1988.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC