J.F. Elder IV and D. Pregibon, A Statistical Perspective on Knowledge Discovery in Databases. In Advances in Knowledge Discovery and Data Mining, U. Fayyad, et al., Eds. AAAI/MIT Press, Menlo Park, CA, pp. 83-113, 1996.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....comprehensible and interesting knowledge [18, 20] thus suggesting that several objectives have to be optimised while inducing a classifier. The comprehensibility of a classifier has practical importance. Some researchers argue that only comprehensible classifiers are actually adopted in practice [27, 28, 41]. One reason for this might be that domain experts are very wary and distrustful of the incomprehensible results generated by a computer [53] Comprehensibly can be achieved by symbolic classifiers, which correspond to an explicit knowledge representation form [1, 39] MERBIS produces Fuzzy ....

John F. Elder IV and Daryl Pregibon. A statistical perspective on knowledge discovery in databases. In Advances in Knowledge Discovery and Data Mining, pages 83--113. 1996.

....accesses. 1. INTRODUCTION The need for rigorous statistical analysis has not gone unnoticed in the data mining community. Statistical concepts such as latent variables, spurious correlation, and problems involving model search and selection have appeared in widely noted data mining literature [6, 12]. However, algorithms, model fitting methods that actually work on massive dataPermission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that ....

J. Elder and D. Pregibon. A statistical perspective on knowledge discovery in databases. In U. M. Fayyad, G. Piatetsky-Shapiro, P. Smyth, and R. Uthurusamy, editors, Advances in Knowledge Discovery and Data Mining, chapter 4. AAAI/MIT Press, 1996.

....e#ort at discovery, often is and should be preceded by theories of suitable specificity. But, many important discoveries have also been made in the absence of prior theories. The term exploratory data analysis (EDA) refers to the idea that statistical insights and modeling are driven by data [4], often defined over dozens of attributes and containing tens of thousands of records. While statistical methods have focused on problems with a manageable number of variables (a dozen, say) and cases (several hundred typically) 4] research in data mining and knowledge discovery aims to ....

....idea that statistical insights and modeling are driven by data [4] often defined over dozens of attributes and containing tens of thousands of records. While statistical methods have focused on problems with a manageable number of variables (a dozen, say) and cases (several hundred typically) [4], research in data mining and knowledge discovery aims to address much larger problems, and the methods draw from a variety of disciplines including statistics, artificial intelligence, and computing [5] Data mining problems typically consist of tens of thousands, if not millions, of records and ....

J. Elder IV and D. Pregibon. A Statistical Perspective on Knowledge Discovery in Databases, chapter 4 in [6]. AAAI Press/MIT Press, 1996.

....databases. One of the characteristics of a real world databases is high dimensionality. High Dimensionality increases the size of the search space in an exponential manner, and thus increases the chance that the algorithm will find spurious models that are not valid in general. Elder and Pregibon [7] define this phenomenon as the curse of dimensionality . Techniques that are efficient in low dimensions fail to provide meaningful results when the number of dimensions goes beyond a modest size of 10 attributes. Furthermore smaller data mining models, involving less attributes, are much more ....

J. Elder IV and D. Pregibon. A Statistical Perspective on Knowledge Discovery in Databases, in U. Fayyad, G. Piatetsky-Shapiro, P. Smyth, and R. Uthurusamy, editors, Advances in Knowledge Discovery and Data Mining, pp 83113. AAAI/MIT Press, 1996.

....a bank for accounting purposes) Thus, issues such as experimental design (the construction of an experiment to collect data to test a specific hypothesis) are not typically within the vocabulary or tool set of a data miner. For other general discussions on statistical aspects of data mining see [EP96, GMPS96, GMPS97, HPS97, Han98, Lam00, Smy00]. 3. A Reductionist View of Data Mining Let us consider a very high level view of data mining and try to reduce a generic data mining algorithm into its component parts. The particular reductionist viewpoint proposed here is not necessarily unique, but it nonetheless does provide some insight ....

Elder, J. F., and Pregibon, D. (1996) A statistical perspective on knowledge discovery in databases, in Advances in Knowledge Discovery and Data Mining, U. M. Fayyad, G. PiatetskyShapiro, P. Smyth, R. Uthurusamy, eds., Cambridge, MA: The MIT Press, pp. 83--115.

....dealt with in the proceedings of the International Conference on Knowledge Discovery and Data Mining series (the two most recent proceedings being [12] and [1] and the journal Data Mining and Knowledge Discovery. Papers discussing the relationship between statistics and data analysis include [8] [4], and [10] 5. ....

Elder J, IV, and Pregibon D. (1996) A statistical perspective on knowledge discovery in databases. In Fayyad U.M., Piatetsky-Shapiro G., Smyth P., and Uthurusamy R. (eds.) Advances in Knowledge Discovery and Data Mining. Menlo Park, California: AAAI Press. 83-113

....model which takes into account additional interactions. Besides the principal difficulty of discovering unexpected dependencies during this biased way of modelling, huge databases with many variables that interact in complex, unpredictable ways can render the manual design cycle unmanageable (Elder and Pregibon 1996). Vapnik emphasizes a complementary approach to data modelling, which is largely followed by the Neural Network community: Real life problems are such that there exist a large number of weak features whose smart linear combination approximates the unknown dependency well. Therefore, it is ....

Elder IV, J. F.; and Pregibon, D. 1996. A statistical perspective on knowledge discovery in databases. In: Advances in Knowledge Discovery and Data Mining. U. M. Fayyad, G. Piatetsky-Shapiro, P. Smyth, R. Uthurusamy, eds. The MIT Press, Menlo Park, CA.

....and the computation (model selection criteria) is only secondary, in DM the algorithm plays a much more central role due to proximity to computer science and machine learning. Similar opinions about DM and its relation to statistics can be found in numerous other publications on the subject (see [Elder Pregibon 1996], Glymour et al. 1997] and [Witten Frank 1999] amongst others) Our loose definition of DM which we will use throughout the article will therefore be: DM is the application of algorithms to extracting patterns from large data bases. DM emphasizes the central role of algorithms rather than ....

Elder J.F., Pregibon D.: A Statistical Perspective on Knowledge Discovery in Databases, in [Fayyad et al. 1996b], pp.83-116.

....naturally when appropriate. The mixture model does not require the specification of distance metrics, readily admitting categorical and continuous attributes (simultaneously) The EM algorithm [DLR77, CS96] has been shown to be superior to other alternatives for statistical modeling purposes [GMPS97, PE96, B95, CS96, NH99]. Utility of the statistical model computed via EM has been demonstrated in Scaling EM Clustering to Large Databases Bradley, Fayyad, and Reina 5 approximating OLAP aggregate queries on continuous data [SFB99] and approximating nearestneighbor queries [BFG99] These applications require the ....

....of clusters k (an open research problem, e.g. CS96,S96] The goal is to study scalability properties and performance for a given k and set of initial conditions. Comparing against alternatives is based on quality of obtained solutions. It is an established fact in the statistical literature [PE96,GMPS97,CS96] that EM modeling results in better quality models than other simpler alternatives like k Means (upon which algorithms like BIRCH [ZRL97] and CLARANS [NH94] are based) There is no prior work on scaling EM so we compare against de facto Scaling EM Clustering to Large Databases Bradley, Fayyad, and ....

D. Pregibon and J. Elder, "A statistical perspective on knowledge discovery in databases", in Advances in Knowledge Discovery and Data Mining, Fayyad, U., G. Piatetsky-Shapiro, P. Smyth, and R. Uthurusamy( Eds.), pp. 83-116. MIT Press, 1996.

.... extended and successfully applied powerful nonlinear techniques # from decision trees and neural networks, to adaptive splines and polynomial networks #toanumber of forecasting, classi#cation, and diagnosis challenges #Ripley 1993; Cheng Titterington 1994; Cherkassky,Friedman, Wechsler 1995; Elder Pregibon 1996; Ripley 1996#. Still, the number of distinct techniques one can employ to inductively create a classi# cation model is actually much smaller than it at #rst appears; many methods are actually reinventions of, or slight deviations from, others. For instance, case based reasoning is a form of ....

.... #Reducing the apparent diversityhowever, would require the hardship of communicating outside one s specialty discipline # Given a plethora of algorithms, a natural question to ask is Which works best A good study toward that end is described in #Michie, Spiegelhalter, Taylor 1994; reviewed in Elder 1996b#. Recently however, several researchers have found that an ensemble of models can be more accurate #on new data# than the best single model #e.g. Jacobs et al. 1991; Wolpert 1992; Perrone Cooper 1993; Hashem, Schmeiser, Yih 1994#. This process of bundling models together can be thought of ....

[Article contains additional citation context not shown here]

Elder, J. F. and Pregibon, D. #1996#. A Statistical Perspective on Knowledge Discovery in Databases, Ch. 4 of Advances in Knowledge Discovery and Data Mining, eds U.M.Fayyad, G.Piatetsky-Shapiro, P.Smith, and R.Uthurusamy. AAAI#MIT Press.

.... these data, checking data quality, summarising them into convenient and relevant forms for analysis, sampling them with minimum amount of bias, intelligent search for potentially useful structures, detecting anomalous and peculiar patterns and avoiding missing interesting ones (Hand, 1996; Elder IV and Pregibon, 1996). Second, data analysis is a complex process in which exploratory analysis and confirmatory analysis may be carried out iteratively. At any stage of the analysis process, there is often a large set of possible operations that could be performed and what to do next often depends on the results ....

Elder IV, J. and Pregibon, D. (1996) "A Statistical Perspective on Knowledge Discovery in Databases", in U M Fayyad, G Piatetsky-Shapiro, P Smyth and R Uthurusamy (eds): Advances in Knowledge Discovery and Data Mining, AAAI/MIT.

....confirming (or rejecting) things, we already know, and not about discovering something new in the data. Finally, the continuous lack of automation has lead the statisticians to focus on problems with a much more manageable number of variables and cases than may be encountered in modern databases [2]. Consequently, the machine learning methods (originally developed to deal, mainly, with the problems of pattern recognition) have been introduced into the data mining field. Today, the data mining has become one of the main application domains of machine learning [13] Artificial neural networks ....

J.F. Elder IV and D. Pregibon, A Statistical Perspective on Knowledge Discovery in Databases. In Advances in Knowledge Discovery and Data Mining, U. Fayyad, G. Piatetsky-Shapiro, P. Smyth, and R. Uthurusamy, Eds., AAAI/MIT Press, Menlo Park, CA, pp. 83-113, 1996.

....own class or in comparison with the contrasting classes to simplify the resulting rules or other forms of presentation. 2.2. 6 Variations and extensions at descriptive data mining Besides mining characteristic and discriminant rules, another interesting task is to mine descriptive statistics [6], especially data dispersion properties. One can demonstrate such properties using boxplot (which is a graph showing median, first and third quarters, whiskers, and outliers) quantile plot, scatter plot, histogram, outlier analysis tools, etc. Although most of data warehouse systems require users ....

J. Elder IV and D. Pregibon. A statistical perspective on knowledge discovery in databases. In U.M. Fayyad, G. Piatetsky-Shapiro, P. Smyth, and R. Uthurusamy, editors, Advances in Knowledge Discovery and Data Mining, pages 83--115. AAAI/MIT Press, 1996.

.... telecommunications and science analysis [FU96] Given attributes A , i=1, m, the problem is to estimate the probability of possible values C=c: 1 Pr( Am A cj C = There is a huge body of literature on this problem, especially in pattern recognition [DH73] and statistics [PE96]. We consider classification algorithms in general, and give specific examples for clients that build decision trees [B 84,Q93,FI92b,FI94] classification rules [GS89, Q93] and simple Bayesian networks [K96] Computational tools from statistics and machine learning communities have not taken into ....

.... stage [B 84, Q93, FI92a, FI93, DKS95, MAR96] Decision trees are good at dealing with data sets having many dimensions (common in data mining applications) unlike other approaches for classification such as the nearest neighbor [DH73] neural networks, regression based statistical techniques [PE96], and density estimation based methods. Decision trees can also be examined and understood by humans, particularly the leaves viewed as rules [Q93] Algorithms reported in the literature generate the tree topdown in a greedy fashion. A partition is selected which splits the data into two or more ....

D. Pregibon, J. Elder, 1996. "A statistical perspective on knowledge discovery in databases", in Advances in Knowledge Discovery and Data Mining, Fayyad et al ( Eds.), pp. 83-116. MIT Press.

.... readily admits categorical and continuous attributes (which is untrue of other clustering algorithms that either operate on continuous, e.g. k Means type algorithms, or categorical [GKR98] data exclusively) EM has been shown to be superior to other alternatives for statistical modeling purposes [GMPS97,PE96,B95,CS96,NH99]. The clustering problem has been formulated in various ways in the statistics [KR89,BR93,B95,S92,S86] pattern recognition [DH73,F90] optimization [BMS97,SI84] and machine learning literature [F87] The fundamental problem is that of grouping together (clustering) data items that are similar to ....

....of clusters k (an open research problem, e.g. CS96,S96] The goal is to study scalability properties and performance for a given k and set of initial conditions. Comparing against alternatives is based on quality of obtained solutions. It is an established fact in the statistical literature [PE96,GMPS97,CS96] that EM modeling results in better quality models than other simpler alternatives like k Means (upon which algorithms like BIRCH [ZRL97] and CLARANS [NH94] There is no prior work on scaling EM so we compare against de facto standard practices for dealing with large databases: sampling based and ....

D. Pregibon and J. Elder, "A statistical perspective on knowledge discovery in databases", in Advances in Knowledge Discovery and Data Mining, Fayyad, U., G. Piatetsky-Shapiro, P. Smyth, and R. Uthurusamy( Eds.), pp. 83-116. MIT Press, 1996.

.... mining association rules in transactional or relational databases [AIS93, AS94, HF95, PCY95, SON95, SA95] summarizing and generalizing data using datacube approach [GHQ95, HRU96, Wid95, YL95] and attribute oriented induction approach [HCC93, HF96] mining classification rules [Qui93, CS96, IP96, PS91, Zia94] data clustering analysis [CS96, EKX95, NH94, ZRL96] etc. An important point is that each technique typically suits some problems better than others. Thus, there is no universal data mining method and choosing an appropriate algorithm for a particular application is something of ....

J. Elder IV and D. Pregibon. A statistical perspective on knowledge discovery in databases. In U.M. Fayyad, G. Piatetsky-Shapiro, P. Smyth, and R. Uthurusamy, editors, Advances in Knowledge Discovery and Data Mining, pages 83--115. AAAI/MIT Press, 1996.

....People from both communities should could work and complement each other. In fact both fields share many common goals in mining data, in trying to extract information and recognizing patterns in data sets, and in fitting appropriate models to data for classification, interpretation, and prediction (Elder Pregibon 1995; Cherkassky, Friedman, Wechsler 1995; Michie, Spiegelhalter, Taylor 1994) In attempting to build bridges between the two paradigms, we will present an empirical study to integrate some neural and statistical classifiers. Instead of selecting the best model among a set of candidate ....

Elder, J. F. and Pregibon, D. 1995. "A Statistical Perspective on Knowledge Discovery in Databases," In Advances in Knowledge Discovery and Data Mining, eds U.M.Fayyad, G.Piatetsky-Shapiro, P.Smith, and R.Uthurusamy. AAAI/MIT Press.

....examination (e.g. 4] of these heuristic ideas an examination they are helping to provoke by simply showing that the ideas work, and hence are worthy of sustained attention. 4. AN INVITATION Over the decades, Statistics has been the major source of useful data based techniques (e.g. [3]) though the discipline has performed poorly at extending and exploiting those results. Jerry Friedman says If a statistician gets a good idea, he writes a paper. If a computer scientist does, he starts a company. As an engineer, I can get away with characterizing my Engineering and CS ....

Elder, J. F. IV & D. Pregibon (1996). A Statistical Perspective on Knowledge Discovery in Databases, Chapter 4 in Advances in Knowledge Discovery and Data Mining, eds. U. M. Fayyad, G. Piatetsky-Shapiro, P. Smyth, and R. Uthurusamy, AAAI/MIT Press.

No context found.

J.F. Elder IV and D. Pregibon, A Statistical Perspective on Knowledge Discovery in Databases. In Advances in Knowledge Discovery and Data Mining, U. Fayyad, et al., Eds. AAAI/MIT Press, Menlo Park, CA, pp. 83-113, 1996.

No context found.

John F. Elder and Daryl Pregibon. A statistical perspective on knowledge discovery in databases. In Usama M. Fayyad, Gregory Piatetsky-Shapiro, Padhraic Smyth, and Ramasamy Uthurusamy, editors, Advances in knowledge discovery and data mining, pages 83--113. AAAI/MIT Press, 1996. 200 Blanchard et al.

No context found.

John F. Elder IV and Daryl Pregibon. A Statistical Perspective on Knowledge Discovery in Databases. In Advances in Knowledge Discovery and Data Mining, pages 83--113. AAAI Press / The MIT Press, 1996.

No context found.

J. Elder, D. Pregibon, "A Statistical Perspective on Knowledge Discovery in Databases", in: U. Fayyad et al. (eds.), Advances in Knowledge Discovery and Data Mining, AAAI/MIT Press 1996.

No context found.

Elder IV J F, and Pregibon D A 1996 Statistical Perspective on Knowledge Discovery in Databases. In: Fayyad U, Piatetsky-Shapiro G, and Smyth P (eds) 1996 Advances in Knowledge Discovery and Data Mining, AAAI Press, Menlo Park, CA, pp 83-113

No context found.

J. Elder, D. Pregibon, "A Statistical Perspective on Knowledge Discovery in Databases", in: U. Fayyad et al. (eds.), Advances in Knowledge Discovery and Data Mining, AAAI/MIT Press 1996.

No context found.

D. Pregibon and J. Elder, "A statistical perspective on knowledge discovery in databases", in Advances in Knowledge Discovery and Data Mining, Fayyad, U., G. Piatetsky-Shapiro, P. Smyth, and R. Uthurusamy( Eds.), pp. 83-116. MIT Press, 1996.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC