Performance evaluation and optimization of join queries for association rule mining (1999) [11 citations — 1 self]
|
Abstract:
The rapid growth in data warehousing technology and the tremendous drop in storage prices has enabled collection of large volumes of data in an ever increasing number of business organizations. One of the greatest challenges today is how to turn these rapidly expanding data stores into nuggets of actionable knowledge. The state-of-the-art data mining tools integrate loosely with data stored in DBMSs, typically through a cursor interface. In this paper, we consider several formulations of association rule mining using SQL-92 queries primarily from a performance perspective. We study the performance of dierent join orders and join methods for executing the various queries. Based on the performance study we identify certain optimizations and develop the Set-oriented Apriori approach. We analyze the cost of the dierent execution plans which provides a basis to incorporate the semantics of association rule mining into future query optimizers. The cost expressions we develop are in terms of the relational operators and hence can be used as \addons " to the current optimizers. This work is an initial step towards developing \SQL-aware" mining algorithms and exploring the enhancements to current relational DBMSs to make them \mining-aware " thereby bridging the gap between the two. 1
