Gregory Piatetsky-Shapiro and Christopher J. Matheus. The interestingness of deviations. In Proceedings of KDD-94: AAAI-94 Knowledge Discovery in Databases Workshop, pages 25--36. AAAI Press, July 1994.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....volume and the fragmented nature of the information. Moreover, changes in equipment, software, and network load mean that the characteristics of the alarm data change. Our approach to the task of processing alarms is supplementary to alarm correlation,a central technique in fault identi cation [18]. In alarm correlation, a management center automatically analyzes the stream of alarms, noti cations, and clear messages it receives from a telecommunication network. Alarm correlation is typically based on looking at the active alarms within a time win dow, and interpreting them as a group. ....

....aid the experts in recalling and formulating correlation patterns. 3. Application of the correlation system in real time alarm management. We contribute to the rst step of the process, the semi autom atic discovery of recurrent patterns of alarms. Steps 2 and 3 have been discussed elsewhere [18]. The ideas expressed in this article have been implemented in a system called TASA, for Telecommunication Alarm Sequence Analyzer. The TASA system has been developed inco opera tion with four telecommunication compa nies. These companies have been using a prototype version of TASA with good ....

[Article contains additional citation context not shown here]

G. Piatetsky - Shap iro and C. J. Matheus, The interestingness of deviations. In U. M. Fayyad and R. Uthurusamy (eds.), Knowledge Discovery in Databases, Papers from AAAI Workshop (KDD ' 94), Seattle, Washington, July 1994, pp. 2536.

....mining rules and their discovery methods. One of the serious problems with many rule discovery methods is that they tend to generate large numbers of patterns, and often many of them, while being statistically acceptable, are trivial, spurious, or just not relevant to the application at hand [PSM94, ST96b, LH96, BMUT97, Ste97, PT98, PT99] Therefore, post analysis of discovered rules becomes an important issue, since there is a need to validate the discovered rules. For example, assume that a data mining method discovered the rule stating that, whenever customer ALW392 goes on a business ....

....metrics, besides confidence and support [AIS93] there are gain [FMMT96] variance and chisquared value [Mor98] gini [MFM 98] strength [DT93] conviction [BMUT97] sc and pcoptimality [BA99] etc. Subjective metrics include unexpectedness [ST96b, LH96, Suz97, PT98] and actionability [PSM94, ST96b, AT97] Any of these metrics can be used as a part of the interestingness based filtering operator, and the validation system can support di#erent interestingness criteria. Moreover, the domain expert can specify interestingness based filters using a syntax similar to the syntax of the ....

G. Piatetsky-Shapiro and C. J. Matheus. The interestingness of deviations. In Proceedings of the AAAI-94 Workshop on Knowledge Discovery in Databases, 1994.

....whether the deviation actually means a defection. We use a trend seasonal forecasting model to predict future performance for every customer, based on past performance in the various performance databases. With the predicted norms to serve as the references, we can then employ deviation analysis (Piatetsky Shapiro and Matheus, 1994) to identify trimming patterns among the customers. Without these predictions and analysis, human analysts can barely observe any phenomenon of gradual deviations at the initial stage of defection. 4.2. Deviation analysis Deviation analysis is the discovery of significant changes or deviations ....

....human analysts can barely observe any phenomenon of gradual deviations at the initial stage of defection. 4.2. Deviation analysis Deviation analysis is the discovery of significant changes or deviations of some pre defined measures from its normative value over a time period in a data set (Piatetsky Shapiro and Matheus, 1994). In most applications, the measured normative value is expressed as the expected value (expectation E t ) of some time series, or as a forecasted value calculated from applying some mathematical models like the seasonal model that describes the series. In our work, a deviation ffi t for time t ....

Piatetsky-Shapiro, G. and C. Matheus: 1994, `The Interestingness of Deviations'. In: Proceedings of the 11th Int'l Conference on Artificial Intelligence AAAI-94, Workshop on Knowledge Discovery in Databases. pp. 25--36.

....products. Technological trends have resulted in organizations accumulating enormous amounts of data on several facets of their operations. For example, many organizations such as credit card companies or retailing outlets record every single transaction a customer performs. It has been estimated [12] that businesses generate gigabytes of data every year and that the total quantity of data tracked doubles approximately every two years. Organizations may proactively build a technological infrastructure to capture and store such data. Firms build data warehouses to support various kinds of ....

....existing approaches in the literature on knowledge discovery and data mining use objective measures of interestingness, such as confidence and support [1] for the evaluation of the discovered patterns. These objective measures capture the statistical strength of a pattern. It has been argued in [8, 12, 16, 17] that besides objective measures of interestingness, subjective measures are equally important. These subjective measures, such as unexpectedness [8, 16, 17] and actionability [3, 12, 16, 17] assume that the interestingness of a pattern depends on the decision maker and does not solely depend on ....

[Article contains additional citation context not shown here]

Piatetsky-Shapiro, G. and Matheus, C.J., 1994. The Interestingness of Deviations. In Procs. of the AAAI-94 Workshop on Knowledge Discovery in Databases, pp. 25-36.

....both by researchers [FPM91, KMR 94,BMU 97,ST95,ST96a,LH96] and practitioners [S97,F97] that many existing tools generate a large number of valid but obvious or irrelevant patterns. To address this issue, some researchers have studied the discovery of novel [ST95,ST96a,LH96,LHC97,PT97a] and useful [PSM94, ST95, ST96a,AT97] patterns. In this paper, we continue the former stream of research and focus on the discovery of unexpected patterns. Unexpectedness of a rule relative to a belief system has been considered before in [ST95,ST96a,LH96,LHC97, PT97a] In [ST95,ST96a] unexpectedness of a rule is defined relative ....

Piatetsky-Shapiro, G. and Matheus, C.J., 1994. The Interestingness of Deviations. In Proc. of AAAI-94 Workshop on Know. Discovery in Databases, pp. 25-36.

....are user dependent, this means that each user may have different ideas about the interestingness of the discovered set of rules. Subjective interestingness measures include unexpectedness 2 [Silberschatz Tuzhilin 1995, Liu Hsu 1996, Padmanabhan Tuzhilin 1998, Freitas 1998] or actionability [Piatetsky Shapiro Matheus 1994, Adomavicius Tuzhilin 1997] The user can also define templates [Klemettinen, Mannila, Ronkainen, Toivonen Verkamo 1994] general impressions [Liu, Hsu Chen 1997] or define item constraints [Srikant, Vu Agrawal 1997] On the other hand, objective measures of rule interestingness are based ....

Piatetsky-Shapiro G., and Matheus C.J. (1994). The Interestingness of Deviations, in Proceedings of the AAAI-94 Workshop on Knowledge Discovery in Databases, 25-36.

....WEEV (Web Ecology and Evolution Visualization) which is a visualization tool to study the evolving relationship of Web usage, content and site topology with respect to time. 2. 2 Discovery of Interesting Patterns The notion of what makes discovered knowledge interesting has been addressed in [83, 101, 67, 79]. A survey of methods that have been used to characterize the interestingness of discovered patterns is given in [56] Four dimensions are used by [56] to classify interestingness measures pattern form, representation, scope, and class 1 . Pattern form de nes what type of patterns a measure is ....

G. Piatetsky-Shapiro and C. J. Matheus. The interestingness of deviations. In AAAI-94 Workshop on Knowledge Discovery in Databases, pages 25-36, 1994.

.... path analysis [9] Several research efforts [17, 13] have considered usage information for performing Web Content Mining [10] An overview of some of the challenges involved in Web Content Mining is given in [28] The notion of what makes discovered knowledge interesting has been addressed in [14, 18, 20, 26]. A common theme among the various criteria for interestingness is the concept of novelty or unexpectedness of a rule. Results that were previously known by the data analyst are not considered interesting. In Web Usage Mining, as with many data mining domains, thresholds for values such as support ....

G. Piatetsky-Shapiro and C. J. Matheus. The interestingness of deviations. In AAAI-94 Workshop on Knowledge Discovery in Databases, pages 25--36, 1994.

....are often used to limit the number of discovered rules to a manageable number. However, high thresholds rarely discover any new knowledge and low thresholds usually result in an unmanageable number of rules. The notion of what makes discovered knowledge interesting has been addressed in [PSM94, ST96, LHC97, PT98]. A common theme among the various criteria for interestingness is the concept of novelty or unexpectedness of a rule. Results that were previously known by the data analyst are not considered interesting. PT98] formally defines the unexpectedness of a rule in terms of its deviation from a set of ....

G. Piatetsky-Shapiro and C. J. Matheus. The interestingness of deviations. In AAAI-94 Workshop on Knowledge Discovery in Databases, pages 25--36, 1994.

....of analysis and prediction we can optimize decision making tasks. For example, the system Health KEFIR helps to save money by detecting situations in which a medical intervention will solve particular problems whose costs are higher than the cost of the indicated intervention (Matheus 1994) (Piatetsky Shapiro 1994). Much of the work in KDD area is based on machine learning methods that have been enhanced to better deal with issues particular to discovery in databases. Also, most of the real world databases are relational. For that reason we use machine learning and relational databases terminology. 2. ....

Piatetsky-Shapiro G. and Matheus C.J. (1994). The Interestingness of Deviations. In Proceedings of The AAAI-94 Workshop on Knowledge Discovery in Databases, KDD-94, 25-36. Seattle, Washington.

....all the mined rules manually. Previous proposals of interestingness measures include: rule templates [5, 4] for limiting attention to only those rules that match the templates, minimal rule covers [12] where rules implied by those presented to the user are eliminated, actionability of rules [8, 10] (some benefit can be obtained by doing something) and unexpectedness of rules [6, 11] Unexpectedness has been interpreted either in the statistical sense, as having higher chance than that under the independence assumption or as having higher chance than some threshold, or against user ....

....cover, focus on the importance of rules structures. The subjective ones, in contrast, depend not only on the structure of a rule and the data, but also on the user who examines the rules. Two useful subjective interestingnesses are actionability and unexpectedness. The notion of actionability [8, 10] of association rules is based on the usefulness of the rules to user whether the users can do something because of the rules to their advantage. Actionability is an important subjective measure of interestingness because users are mostly interested in the knowledge that permits them to do ....

G. Piatetsky-Shapiro and C. J. Matheus. The interestingness of deviations. In Proceedings of the AAAI-94 Workshop on Knowledge Discovery in Databases, pages 25-36, 1994.

....as a function of p(A) p(B) and p(A B) where p(ff) is the probability that condition ff is true. Typical examples of such objective measure of interestingness of a rule are its information content based on the J measure [10] a certainty factor [4] and a strength [2] It has been noted in [8] that objective measures of interestingness, although useful in many respects, usually do not capture all the complexities of the pattern discovery process, and that subjective measures of interestingness are needed to define interestingness of a pattern. These subjective measures do not depend ....

....to another user. For example, a pattern discovering some security trading irregularities, such as insider trading, may be of great interest to the officials from the Securities and Exchange Commission (SEC) However, it is of very little use to a homeless person living in New York City. In [8], subjective measures of interestingness were studied within the context of the discovery system KEFIR [6] that analyzes healthcare insurance claims for uncovering key findings. The key findings in KEFIR are the most important changes from the norms for various indicators assessing different ....

[Article contains additional citation context not shown here]

G. Piatetsky-Shapiro and C. J. Matheus. The interestingness of deviations. In Proceedings of the AAAI-94 Workshop on Knowledge Discovery in Databases, pages 25--36, 1994.

....this means that each user may have different ideas about the interestingness of the discovered set of rules. Subjective interestingness measures include unexpectedness 2 (Silberschatz and Tuzhilin, 1995; Liu and Hsu, 1996; Padmanabhan and Tuzhilin, 1998; Freitas 1998) or actionability (Piatetsky Shapiro and Matheus, 1994; Adomavicius and Tuzhilin, 1997) The user can also define templates (Klemettinen, Mannila, Ronkainen, Toivonen and Verkamo, 1994) general impressions (Liu, Hsu and Chen, 1997) or define item constraints (Srikant, Vu and Agrawal, 1997) On the other hand, objective measures of rule ....

Piatetsky-Shapiro G., and Matheus C.J. (1994). The Interestingness of Deviations, in Proceedings of the AAAI94 Workshop on Knowledge Discovery in Databases, 25-36.

....[3] Hence, an interest factor is defined to filter out such kind of misleading. However, the interestingness or the usefulness of a rule is often application dependent. There have been several studies on quantifying the interestingness or usefulness of the discovered rules in the literature [8] [13] Let N be the total number of tuples and j A j the number of tuples containing all items in the set A. Define support(X) P (X) j X j N (1) support(X Y ) P (X Y ) j X [ Y j N (2) confidence(X Y ) P (X Y ) P (X) j X [ Y j j X j (3) interest(X Y ) P (X Y ....

G. Piatetsky-Shapiro and C.J. Matheus, "The interestingness of deviations," presented at the AAAI Workshop on Knowledge Discovery in Databases, Seattle, 1994.

....are derived. 1 Introduction Data miners pursue the discovery of new knowledge. But knowledge based solely on statistical dominance is rarely new. The expert needs means for either instructing the miner to discover only interesting rules or for ranking the mining results by interestingness [7]. Tuzhilin et al. propose interestingness measures based on the notion of belief [1, 6] A belief reflects the expert s domain knowledge. Mining results that contradict a belief are more interesting than those simply confirming it. Hence, they propose methods to guide the miner in the discovery of ....

Gregory Piateski-Shapiro and Christopher J. Matheus. The interestingness of deviations. In AAAI'94 Workshop Knowledge Discocery in Databases, pages 25--36. AAAI Press, 1994.

....from large collections of data. What is interesting varies from one situation to another. The interestingness criteria such as what is on the right hand sides of the rules, or how significant the rules should be are in many KDD systems given as inputs to the pattern extraction process [14, 20]. If the user wants to change his or her point of view, the data has to be analyzed anew. This can require a lot of computational effort and cause delays in the KDD process. In the TASA system we aim at a KDD process where going back to the actual data for a new pattern extraction phase has to be ....

....guides for locating possible periodic relationships between the left and right hand sides, as is demonstrated by Figure 7. Note that the episode rule formalism does not capture the details of such relationships. User interface For the user interface system we have adopted the method used in [20]: we have based the implementation of the rule browsing system on the use of the HTML language and standard WWW browsers for HTML documents. This gives several immediate benefits, both in terms of added functionality and in terms of ease of implementation. This architecture also means that it is ....

Gregory Piatetsky-Shapiro and Christopher J. Matheus. The interestingness of deviations. In Usama M. Fayyad and Ramasamy Uthurusamy, editors, Knowledge Discovery in Databases, Papers from the 1994 AAAI Workshop (KDD'94), pages 25 -- 36, Seattle, Washington, July 1994.

....all the mined rules manually. Previous proposals of interestingness measures include: rule templates [5, 4] for limiting attention to only those rules that match the templates, minimal rule covers [12] where rules implied by those presented to the user are eliminated, actionability of rules [8, 10] (some benefit can be obtained by doing something) and unexpectedness of rules [6, 11] Unexpectedness has been interpreted either in the statistical sense, as having higher chance than that under the independence assumption or as having higher chance than some threshold, or against user beliefs. ....

....cover, focus on the importance of rules structures. The subjective ones, in contrast, depend not only on the structure of a rule and the data, but also on the user who examines the rules. Two useful subjective interestingnesses are actionability and unexpectedness. The notion of actionability [8, 10] of association rules is based on the usefulness of the rules to user whether the users can do something because of the rules to their advantage. Actionability is an important subjective measure of interestingness because users are mostly interested in the knowledge that permits them to do ....

G. Piatetsky-Shapiro and C. J. Matheus. The interestingness of deviations. In Proceedings of the AAAI-94 Workshop on Knowledge Discovery in Databases, pages 25-36, 1994.

....is the application of statistical and machine learning techniques for extracting interesting patterns from raw data. Formalizing what interesting means in this context has been an important problem in the data mining literature, which had not been addressed in a satisfactory way (see, e.g. [1, 17, 14, 16] for various attempts) Most research in data mining deals with the efficient discovery of patterns for subsequent human evaluation of the degree to which they are interesting, and not on techniques for automatically evaluating mined patterns, or for automatically focusing on interesting ....

G. Piatetsky-Schapiro, C. J. Matheus "The interestingness of deviations," KDD 1994, pp. 25-- 36, 1994.

....and repeat the pattern discovery. The patterns discovered by 49er [27] are contingency tables, equations, and logical equivalencies. The user can interactively change the focus, e.g. independent and dependent variables, and require for a new pattern discovery. The Key Finding Reporter (Kefir) [17, 21] discovers and explains deviations, and gives recommendations for corrective actions. Applications of Kefir are tailored with a lot of domain knowledge to be aware of the interestingness criteria, corrective actions, etc. of the domain. Given a database from the domain, a Kefir based application ....

....complex (e.g. Explora [13] IMACS [4] However, we are not aware of any system where the focus can be set after the pattern discovery. Domain knowledge is used by many systems to avoid redundancy or to infer different interestingness measures. For instance, the Key Finding Reporter (Kefir) [21] detects and explains deviations and recommends corrective actions. Kefir is an example of a system where a large body of domain knowledge is used to guide the search for patterns without interaction with the user. Once the domain knowledge has been acquired and coded, a Kefir based application ....

Gregory Piatetsky-Shapiro and Christopher J. Matheus. The interestingness of deviations. In Usama M. Fayyad and Ramasamy Uthurusamy, editors, Knowledge Discovery in Databases, Papers from the 1994 AAAI Workshop (KDD'94), pages 25 -- 36, Seattle, Washington, July 1994.

....association rules. 6] analyze the discovered classification rules against a set of general impressions that are specified using a representation language. Only those patterns conforming to these impressions are regarded as unexpected. The issue on interesting deviations is also discussed in [11]. As the interestingness of a pattern itself is subjective, a rule could be interesting to one user but not interesting to another. Thus, most of the previous work so far rely on users to distinguish those reliable exception patterns based on the existing concepts. One potential problem raised is ....

G. Piatetsky-Shapiro and C. Matheus. The interestingness of deviations. In AAAI Workshop on Knowledge Discovery in Database, pages 25--36, Seattle, Washington, July 1994.

....parts of the process. The aim of knowledge discovery in databases is to extract interesting knowledge from large collections of data. What is interesting varies from one situation to another. The interestingness criteria are in many KDD systems given as inputs to the pattern extraction process [9, 14]. If the users change their views, the data has to be analyzed anew. This can in the worst case require a lot of computational effort, and it can cause delays in the KDD process. In the TASA system we aim at a KDD process where going back to the actual data for a new pattern extraction process has ....

....of the left hand side to the nearest occurrence of the right hand side. Such histograms are valuable guides for locating possible periodic relationships between the left and right hand sides, as is demonstrated by Figure 6. For the user interface system we have adopted the method used in [14]: we have based the implementation of the rule browsing system on the use of the HTML language and the browsers for HTML documents. This gives several immediate benefits, both in terms of added functionality and in terms of easy implementation. This architecture also means that it is extremely ....

Gregory Piatetsky-Shapiro and Christopher J. Matheus. The interestingness of deviations. In Usama M. Fayyad and Ramasamy Uthurusamy, editors, Knowledge Discovery in Databases, Papers from the 1994 AAAI Workshop (KDD'94), pages 25 -- 36, Seattle, Washington, July 1994.

No context found.

Gregory Piatetsky-Shapiro and Christopher J. Matheus. The interestingness of deviations. In Proceedings of KDD-94: AAAI-94 Knowledge Discovery in Databases Workshop, pages 25--36. AAAI Press, July 1994.

No context found.

Gregory Piatetsky-Shapiro and Christopher J. Matheus. The interestingness of deviations. In Proceedings of KDD-94: AAAI-94 Knowledge Discovery in Databases Workshop, pages 25--36. AAAI Press, July 1994.

No context found.

Piatetsky-Shapiro, G. and Matheus, C.J., 1994. The Interestingness of Deviations. In Procs. of the AAAI-94 Workshop on Knowledge Discovery in Databases, pp. 25-36.

No context found.

G. Piatetsky-Shapiro and C.J. Matheus, "The Interestingness of Deviations," Proc. AAAI-94 Workshop Knowledge Discovery in Databases, AAAI Press, Menlo Park, Calif., 1994, pp. 25-36.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC