Mohammed J. Zaki and Ching-Jui Hsiao. CHARM: An Efficient Algorithm for Closed Association Rule Mining. Technical Report TR 99-10, RPI, 1999.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....and shrinks the dataset in each pass. Both FP tree and PDA greatly reduce the original dataset and also do not need to generate candidate sets. When the frequent patterns are long, mining FI is infeasible because of the exponential number of frequent itemsets. Thus, algorithms mining FCI [9,15,10] are proposed since FCI is enough to generate association rules. However, FCI could also be exponentially large as the FI. As a result, researchers now turn to find MFI. Given the set of MFI, it is easy to analyze many interesting properties of the dataset, such as the longest pattern, the overlap ....

M. J. Zaki and C. Hsiao. Charm: An efficient algorithm for closed association rule mining. In Technical Report 99-10, Computer Science, Rensselaer Polytechnic Institute, 1999.

....may come in different forms. In the case of association rules, a transaction is a set of items [2] In the case of episodes, a transaction corresponds to a sequence of events in a sliding time window [13] There are algorithms that do not rely on candidate generation (e.g. FP growth [8] CHARM [21], and GenMax [20] Given that the OSSM created with mmin segments will consume too much space, in Section 5 we consider the constrained segmentation problem. Given an input value m user (which is smaller than mmin ) the constrained segmentation problem seeks to find the m user segments that ....

M.J. Zaki and C.-J. Hsiao. CHARM: An efficient algorithm for closed association rule mining. Technical Report 99-10, Rensselaer Polytechnic Institute, USA, 1999.

....does not hold, without actually performing the subgraph isomorphism. 4.4.2 TID Lists Determining the geometric subgraph isomorphism described in Section 4.4. 1 is expensive and for this reason we also developed a frequency counting approach that uses Transaction ID (TID) lists, proposed by [6, 14, 21, 19, 20]. In this approach for each frequent subgraph we keep a list of transaction identifiers that support it. Now when we need to compute the frequency of gk , we first compute the intersection of the TID lists of its frequent k subgraphs. If the size of the intersection is below the support, gk is ....

M. J. Zaki and C.-J. Hsiao. CHARM: An efficient algorithm for closed association rule mining,. Technical Report 99-10, Department of Computer Science, Rensselaer Polytechnic Institute, October 1999. 17

....transaction to determine which of the itemsets in the hash tree it supports. Developing such an algorithm for frequent subgraphs, however, is challenging as there is no natural way to build the hash tree for graphs. For this reason, PSG instead uses Transaction identifier (TID) lists, proposed by [10, 23, 30, 28, 29]. In this approach for each frequent subgraph we keep a list of transaction identifiers that support it. Now when we need to compute the frequency of G k , we first compute the intersection of the TID lists of its frequent k subgraphs. If the size of the intersection is below the support, G is ....

M. J. Zaki and C.-J. Hsiao. CHARM: An efficient algorithm for closed association rule mining,. Technical Report 99-10, Department of Computer Science, Rensselaer Polytechnic Institute, October 1999. 27

....of surviving frequent itemsets. For each transaction, we have to do intersection with each surviving frequent itemsets. This makes the closure computation quite costly. CHARM CHARM is another efficient algorithm for finding closed frequent itemsets. It is proposed by M. J. Zaki and C. Hsiao in [49]. Different from A close, Charm explores a vertical Chapter 2. Related Work 29 data format, i.e. each item is associated with a set of transaction identifiers (tid for short) CHARM is not using the Apriori framework. Let us examine the following example. Example 2.4 Again, we use the same ....

M. Zaki and C. Hsiao. Charm: An efficient algorithm for closed association rule mining. In Technical Report 99-10, Computer Science, Rensselaer Polytechnic Institute, 1999.

....that need to be considered. So one must cleverly traverse the search space. 2.2 Different Traversal Strategies Let P denote the power set of the all itemset I. The ordered set (P(I) is a complete lattice, where the meet is given by set intersection, and the join is given by a set union [14]. For example, let I= a, b, c, d . In order to find all possible frequent itemsets, traverse in the following lattice: CHAPTER 2. RELATED WORK 8 Figure 2 1: Lattice for I= a, b, c, d The frequent itemsets are located in the upper part of Figure 2 1, whereas the infrequent ones are located in ....

....rules presented to the user, many of which are redundant. This is true even in a sparse database. For dense datasets it is simply d:4 Null:24 a:9 b:7 c:5 d:5 b:4 c:2 d:3 c:3 d:4 c:1 d:3 d:1 d:3 d:1 CHAPTER 3. MINING TOP k m 46 not feasible to mine all the possible frequent itemsets. In [11] [14], and other works, it is shown that it is only necessary to mine the closed frequent itemsets, not all frequent itemsets. Definition 3.5 (Closed Itemset) An itemset X is a closed itemset if there exists no itemset X , such that X is a proper superset of X and every transaction containing X also ....

[Article contains additional citation context not shown here]

M. J. Zaki and C. Hsiao. Charm: An Efficient Algorithm for Closed Association Rule Mining. In Technical Report 99-10, Computer Science, Rensselaer Polytechnic Institute, 1999. BIBLIOGRAPHY 64

....; a 2 ; a 50 ) a 51 ; a 52 ; a 100 ) since all the others can be derived from this one easily. In this paper, we study efficient mining of frequent closed itemsets in large databases. Pasquier et al. 9] propose an Apriori based mining algorithm, called A close. Zaki and Hsiao [10] propose another mining algorithm, CHARM, which improves mining efficiency by exploring an item based data structure. According to our analysis, A close and CHARM are still costly when mining long patterns or with low minimum support thresholds in large databases. As a continued study on frequent ....

....intersections of sets of transaction ids (tids) for itemsets. All the experiments are performed on a 233MHz Pentium PC with 128MB main memory, running on Microsoft Windows NT. All the programs are written in Microsoft Visual C 6.0. The A close and CHARM are implemented as described in [9] and [10]. We use runtime, i.e. the period between input and output, to report our result, instead of using CPU time measured in some literature. We test the three methods on various datasets, including synthetic ones generated by the standard procedure described in [2] and real datasets used in [4, ....

[Article contains additional citation context not shown here]

M. J. Zaki and C. Hsiao. Charm: An efficient algorithm for closed association rule mining. In Technical Report 99-10, Computer Science, Rensselaer Polytechnic Institute, 1999. 10

....structure that provides efficient support for such migration queries, as well as a simple algorithm for computing it. 1 Introduction Extensive work has been done on efficient mining of interesting patterns like associations, correlations, etc. and their variants, from large databases (e.g. see [AS94, MTV94, BMS97, LSW97, AS95, MTV97, DL99, HPY00, PHM00, Bay98, AAP00, ZaHs99]) At the core of extracting such patterns is the determination of frequent itemsets, to which many of these studies are devoted. All of these studies are concerned with answering the question which patterns hold in the database. We argue that many a time, it is at least as interesting and ....

M.J. Zaki and C. Hsiao. Charm: an efficient algorithm for closed association rule mining. Tech. Report., RPI, 1999.

....all possible 2 m 2 subsets of a m length pattern (m can easily be 30 or 40 or longer) is computationally unfeasible. Thus, there has been recent interest in mining maximal frequent patterns in these hard dense databases. Another recent promising direction is to mine only closed sets [9, 11]; a set is closed if it has no superset with the same frequency. Nevertheless, for some of the dense datasets we consider in this paper, even the set of all closed patterns would grow to be too large. The only recourse is to mine the maximal patterns in such domains. In this paper we introduce ....

....combine set in increasing order of support. This is likely to produce small combine sets in the next level, since the items with lower frequency are less likely to produce frequent itemsets at the next level. This heuristic was first used in MaxMiner, and has been used in other methods since then [1, 4, 11]. In addition to sorting the initial combine set at level 0 in increasing order of support, GenMax uses another novel reordering heuristic based on a simple lemma Lemma 1 Let IF (x) fy : y 2 F 1 ; xy is not frequent g, denote the set of infrequent 2 itemsets that contain an item x 2 F 1 , and ....

[Article contains additional citation context not shown here]

M. J. Zaki and C.-J. Hsiao. CHARM: An efficient algorithm for closed association rule mining. TR 99-10, CS Dept., RPI, Oct. 1999.

....for real (dense) datasets [20] At the same time, we don t loose any information; the closed itemsets uniquely determine the set of all frequent itemsets and their exact frequency. Thus instead of mining all the frequent itemsets we only mine the frequent closed itemsets using the CHARM algorithm [22] we recently developed. A detailed description of the algorithm is beyond the scope of this paper. Suffice it to say that CHARM can handle very large disk resident or external memory databases; it has been tested on databases with millions of examples, and it scales linearly in the database size. ....

....A detailed description of the algorithm is beyond the scope of this paper. Suffice it to say that CHARM can handle very large disk resident or external memory databases; it has been tested on databases with millions of examples, and it scales linearly in the database size. We refer the reader to [22] for the algorithm description and its efficiency. 3 HMMSTR: An HMM for local structure in proteins We describe here the hidden Markov model, HMMSTR [4] for general protein sequences based on the I sites library of sequence structure motifs [3] In the next section we will show how we apply ....

[Article contains additional citation context not shown here]

M. J. Zaki and C.-J. Hsiao. CHARM: An efficient algorithm for closed association rule mining. Technical Report 99-10, Computer Science Dept., Rensselaer Polytechnic Institute, October 1999.

....The mining task is to find all frequently accessed sets of pages. Figure 5 shows all the frequent k itemsets F k that are contained in at least three user transactions; i.e. min sup = 3. ABC, AF and CF , are the maximal frequent itemsets. We applied the Charm association mining algorithm [35] to a real LOGML document from the RPI web site (one day s logs) There were 200 user sessions with an average of 56 distinct nodes in each session. It took us 0.03s to do the mining with 10 minimum support. An example frequent set found is shown below: FREQUENCY = 22 , NODE IDS = 25854 5938 ....

M. J. Zaki and C.-J. Hsiao. CHARM: An efficient algorithm for closed association rule mining. Technical Report 99-10, Computer Science Dept., Rensselaer Polytechnic Institute, October 1999. 28

....property is preserved, i.e. all valid association rules can be found. Note that maximal sets do not have this property, since subset counts are not available. Methods for mining closed sets include the Apriori based A Close method [12] the Closet al..gorithm based on FP trees [13] and Charm [21]. Most of the previous work on association mining has utilized the traditional horizontal transactional database format. However, a number of vertical mining algorithms have been proposed recently for association mining [5, 17, 20, 21] as well as other mining tasks like classification [16] In a ....

....A Close method [12] the Closet al..gorithm based on FP trees [13] and Charm [21] Most of the previous work on association mining has utilized the traditional horizontal transactional database format. However, a number of vertical mining algorithms have been proposed recently for association mining [5, 17, 20, 21] (as well as other mining tasks like classification [16] In a vertical database each item is associated with its corresponding tidset, the set of all transactions (or tids) where it appears. Mining algorithms using the vertical format have shown to be very effective and usually outperform ....

[Article contains additional citation context not shown here]

M. J. Zaki and C.-J. Hsiao. CHARM: An efficient algorithm for closed association rule mining. Technical Report 99-10, Computer Science Dept., Rensselaer Polytechnic Institute, October 1999. 21

No context found.

Mohammed J. Zaki and Ching-Jui Hsiao. CHARM: An Efficient Algorithm for Closed Association Rule Mining. Technical Report TR 99-10, RPI, 1999.

No context found.

M. J. Zaki, and C. Hsiao, "CHARM: An Efficient Algorithm for Closed Association Rule Mining", in Technical Report 99-10, Computer Science, Rensselaer Polytechnic Institute, 1999.

No context found.

M.J. Zaki and C.-J. Hsiao. CHARM: An efficient algorithm for closed association rule mining. Technical Report 99-10, Department of Computer Science, Rensselaer Polytechnic Institute, Troy, NY, USA, 1999. 25

No context found.

M. J. Zaki and C. Hsiao. CHARM: An efficient algorithm for closed association rule mining. RPI Technical Report 99-10, 1999.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC