Setiono, R., and Liu, H. Chi2: Feature selection and discretization of numeric attributes. In Proceedings of the Seventh IEEE International Conference on Tools with Artificial Intelligence (1995).  Home/Search   Document Details and Download   Summary   Related Articles   Check

....first sorts the values of a feature, and then attempts to find interval boundaries such that each interval has a strong majority of one particular class. The method is constrained to form intervals of some minimal size in order to avoid having intervals with very few instances. Setiono and Liu [SL95] present a statistically justified heuristic method for supervised discretization called Chi2. A numeric feature is initially sorted by placing each observed value into its own interval. The next step uses a chi square statistic 2 to determine whether the relative frequencies of the classes ....

....R is such an attribute, and C is the class attribute, then it is easy to show that 35 POSR (C) contains all the instances 4 and R (P ) 1. Similarly, for LVF, the feature R guarantees that there is no inconsistency in the data. 3.4. 2 Feature Selection Through Discretization Setiono and Liu [SL95] note that discretization has the potential to perform feature selection among numeric features. If a numeric feature can justifiably be discretized to a single value, then it can safely be removed from the data. The combined discretization and feature selection algorithm Chi2 (discussed in ....

R. Setiono and H. Liu. Chi2: Feature selection and discretization of numeric attributes. In Proceedings of the Seventh IEEE International Conference on Tools with Artificial Intelligence, 1995.

....intermediate rule, and one would want to determine a suitable extremum value in such a range. To make this tractable, each input range has to be discretized into a small number of values that can be subsequently examined. Thus, each input feature X i 2 (a i ; b i ) is discretized into k intervals [4]. When Full RE finds more than one discretization value of an input X i that can satisfy the intermediate rule (i.e. the rule has more than one feasible solution) then it chooses the minimum or the maximum of these values based on the sign of the corresponding effect parameter c i . If c i is ....

H. Liu and R. Setiono. Chi2: Feature selection and discretization of numeric attributes. In Proceedings of the Seventh International Conference on Tools with Artificial Intelligence, pages 388--391, November 1995.

.... Delta ; d i;k Gamma1 ; d i;k = b i g) where: d i;l Gamma1 and d i;l are the lower and upper boundary values of interval l of input X i respectively. Different discretization approaches can be exploited to compute discretization boundaries of input features X i s [39, 16] Full RE uses the Chi2 [19] algorithm 4 , a powerful discretization tool, to compute discretization boundaries of input features. When Full RE finds more than one discretization value of an input X i that satisfies the intermediate rule (i.e. the rule has more than one feasible solution) it chooses the minimum or the ....

H. Liu and R. Setiono. Chi2: Feature selection and discretization of numeric attributes. In Proceedings of the Seventh International Conference on Tools with Artificial Intelligence, pages 388--391, November 1995.

....; d i;k Gamma1 ; d i;k = b i g where: d i;l Gamma1 and d i;l are the lower and upper boundary values of interval l of input X i respectively. Different discretization approaches can be exploited to compute discretization boundaries of input features X i s [43, 8, 22, 24, 52] Full RE uses the Chi2 [23] algorithm 1 , a powerful discretization tool, to compute discretization boundaries of input features. When Full RE finds more than one discretization value of an input X i that can satisfy the intermediate rule (i.e. the rule has more than one feasible solution) then it chooses the minimum or ....

H. Liu and R. Setiono. Chi2: Feature selection and discretization of numeric attributes. In Proceedings of the Seventh International Conference on Tools with Artificial Intelligence, pages 388--391, November 1995.

....d i;k Gamma1 ; d i;k = b i g, where: d i;l Gamma1 and d i;l are the lower and upper boundary values of interval l of input X i respectively. Different discretization approaches can be exploited to compute discretization boundaries of input features X i s [46, 5, 23, 26, 56] Full RE uses the Chi2 [25] algorithm 2 , a powerful discretization tool, to compute discretization boundaries of input features. When Full RE finds more 2 We are thankful to Liu and Setiono for making their Chi2 source code available to us. 3 PROPOSED RULE EXTRACTION APPROACHES 14 than one discretization value of an ....

H. Liu and R. Setiono. Chi2: Feature selection and discretization of numeric attributes. In Proceedings of the Seventh International Conference on Tools with Artificial Intelligence, pages 388--391, November 1995.

....to discretize numeric attributes for subsequent classification. It is an agglomerative, hard clustering algorithm that uses the 2 statistic as the the similarity metric. We have also tried 2 in our experiments and found that the KL divergence average yields better performance. Chi2 [18] is an extension of ChiMerge for use as a feature selector of numeric attributes. Liu and Setiono observe that if all the values of any attribute are clustered together, then that value is irrelevant to the classification task and can be removed. Class based clustering [1] uses an agglomerative, ....

H. Liu and R. Setiono. Chi2: Feature selection and discretization of numeric attributes. In Proceedings of 7th IEEE Int'l Conference on Tools with Artificial Intelligence, 1995.

....to discretize numeric attributes for subsequent classification. It is an agglomerative, hard clustering algorithm that uses the 2 statistic as the the similarity metric. We have also tried 2 in our experiments and found that the KL divergence average yields better performance. Chi2 (Liu and Setiono 1995) is an extension of ChiMerge for use as a feature selector of numeric attributes. Liu and Setiono observe that if all the values of any attribute are clustered together, then that value is irrelevant to the classification task and can be removed. Class based clustering (Brown et al. 1992) uses an ....

H. Liu and R. Setiono. Chi2: Feature selection and discretization of numeric attributes. In Proceedings of 7th IEEE Int'l Conference on Tools with Artificial Intelligence, 1995.

....to select features. Since the last two algorithms do not try to explore all the combinations of features, it is certain that they fail on the parity problems (parity 5, parity 10, etc. where the combinations of a small number of attributes do not help in finding the relevant attributes. Chi2 [ Liu and Setiono, 1995 ] is another heuristic feature selector. It automatically discretizes the continuous attributes and removes irrelevant continuous attributes based on the chi square statistics and the inconsistency found in the data. If an attribute s values are discretized into one interval, the attribute can be ....

H. Liu and R. Setiono. Chi2:Feature selection and discretization of numeric attributes. In Proceedings of the 7th IEEE International Conference on Tools with Artificial Intelligence, 1995. http://www.iscs.nus.sg/ ~ liuh/tai95.ps

....and as a guide for future application of this method. 1 Introduction Classification rules are sought in many areas from automatic knowledge acquisition [ Russell and Norvig, 1995 ] to data mining [ Agrawal et al. 1993 ] neural network rule extraction [ Towell and Shavlik, 1993; Setiono and Liu, 1995 ] This is because classification rules possess some attractive features. They are explicit, understandable and verifiable by domain experts, and can be modified, extended and passed on as modular knowledge. The classification problem in the context of our discussion can be described as follows: ....

....it is then remotely likely that it is a corrupted pattern. The first line in RND is designed to find a good pattern (the most frequently occured one) The frequency of each pattern can be obtained by counting each distinct pattern s occurrence and removing duplicates. The output of Chi2 [ Liu and Setiono, 1995 ] which is a program that does discretization and feature selection for numeric data, also provides such information. When the frequency information is not available, RND goes ahead to generate rules, although it may induce redundant rules due to some ordering of the data and noise rules ....

[Article contains additional citation context not shown here]

H. Liu and R. Setiono. Chi2: Feature selection and discretization of numeric attributes. In Proceedings of the 7th IEEE International Conference on Tools with Artificial Intelligence, 1995. http://www.iscs.nus.sg/ ~ liuh/tai95.ps

....will just take one run of LVF to locate this kind of features 7 . Another run of LVF with the other features will identify the right set of features. LVF only works on discrete features since it relies on the inconsistency calculation. One way is to apply a discretization algorithm (e.g. Chi2 [13]) to discretize the continuous features first before one runs LVF. Other possibilities are (1) to simply treat a continuous feature as a discrete one in some cases; and (2) to apply LVF only to the discrete features when the number of features is large. More work is needed. The search for new ....

H. Liu and R. Setiono. Chi2: Feature selection and discretization of numeric attributes. In Proceedings of the Seventh IEEE International Conference on Tools with Artificial Intelligence, pages 388--391, 1995.

....S ij = P M k=1 jx ik = x jk j M (3) where jx ik = x jk j is 1 if x ik equals x jk and 0 otherwise, and M is the number of variables in the subset under consideration. For mixed data (i.e. both numeric and nominal variables) we can discretize numeric values first before applying our measure [8]. 3 Algorithm to Find Important Variables We use a Sequential Backward Selection algorithm [2] to determine the relative importance of variables for Unsupervised Data (SUD) In the algorithm M is the number of variables originally present, and D is the data set. SUD(D) T = Original Variable ....

....Thyroid 4,5,3,2,1 4,3,1,2,5 Wine 7,6,12,9,11,10,5,13,1,4,3,8,2 6,9,1,11,5,7,10,4,12,2,13,3,8 Table 3: Order of variables by SUD and Relief F. known. The important variables for Iris and Chemical Plant are based on the results of other researchers. For Iris data, Chiu [1] and Liu and Setiono [8] conclude that v 3 (petal length) and v 4 (petal width) are the most important variables. Both Chemical Plant and Non linear data sets are taken from Yasukawa and Sugeno s paper [9] For Chemical Plant data, they conclude that the first 3 variables are important. The Nonlinear data has an output ....

H. Liu and R. Setiono. Chi2: Feature selection and discretization of numeric attributes. In Proceedings of the 7th IEEE International Conference on Tools with Artificial Intelligence (TAT'95), pages 388--391, November 1995.

....of PCA is that it cannot take advantage of the class information, though it is available. By discarding those eigenvectors with eigenvalues less than the threshold, PCA cannot fully preserve the input information. Feature transformation may also be linked to techniques such as discretization [5, 11] and subset selection [10, 4] Discretization transforms continuous data into discrete one. Subset selection, as its name implies, chooses a subset of original features. Although both techniques can reduce the dimensionality of the data, it is clear that neither new nor compound features are ....

H. Liu and R. Setiono. Chi2: Feature selection and discretization of numeric attributes. In J.F. Vassilopoulos, editor, Proceedings of the Seventh IEEE International Conference on Tools with Artificial Intelligence, November 5-8, 1995, pages 388--391, Herndon, Virginia, 1995. IEEE Computer Society.

....hidden units is the interval [ Gamma1; 1] since they have been computed as the hyperbolic tangent of the weighted inputs (cf. Eqn. 4) In order to extract rules from the network, it is necessary that these values be grouped into a few clusters while preserving the accuracy of the network. Chi2 [11], an improved and automated version of ChiMerge [10] is the algorithm used for this purpose. Given a dataset where each pattern is described by the values of the continuous attributes A 1 ; A 2 ; and the class label of the pattern is known, Chi2 finds discrete representations of the ....

H. Liu and R. Setiono, Chi2: Feature selection and discretization of numeric attributes, in Proceedings of the 7th IEEE International Conference on Tools with Artificial Intelligence (1995) 388--391.

....In order to handle noise, the base algorithm is modified by adding the following at the very beginning of the algorithm. sort on freq(Data without Duplicates) The frequency of each pattern can be obtained by counting each distinct pattern s occurrence and removing duplicates. The output of Chi2 [9], which is a program that does discretization and feature selection for numeric data, also provides such information. More on handling noise in various cases can be found in [8] IV. Evaluation Measures The evaluation of rules is performed in two aspects. One is the estimation of error rates ....

....the Iris data is divided evenly into two sets (75 patterns each) for training and testing. Datasets CAR and Golf Playing are given in Tables 1 and 2. Iris data can be obtained from University of California, Irvine [12] In presence of numeric attributes (e.g. in Golf Playing and Iris data) Chi2 [9] is applied to discretizing these attributes. The resulting data contains discrete values only and is guaranteed that it can keep the original discriminating power of the data, shown in Tables 3 and 4. 10 displace fuelcap mass speed cyl cost 3 3 2 2 6 3 3 1 3 3 6 3 2 2 1 3 6 2 1 1 1 1 6 1 3 ....

[Article contains additional citation context not shown here]

H. Liu and R. Setiono. Chi2: Feature selection and discretization of numeric attributes. In Proceedings of the 7th IEEE International Conference on Tools with Artificial Intelligence, 1995. 23

No context found.

Setiono, R., and Liu, H. Chi2: Feature selection and discretization of numeric attributes. In Proceedings of the Seventh IEEE International Conference on Tools with Artificial Intelligence (1995).

No context found.

H. Liu and R. Setiono. Chi2: Feature selection and discretization of numeric attributes. In Proceedings of 7th IEEE International Conference on Tools with Artificial Intelligence, pages 388--391, 1995. 168

No context found.

H. Liu and R. Setiono. Chi2: Feature selection and discretization of numeric attributes. In Proceedings of 7th IEEE Int'l Conference on Tools with Artificial Intelligence, 1995.

No context found.

H. Liu and R. Setiono. Chi2: Feature selection and discretization of numeric attributes. Proceedings of the Seventh IEEE International Conference on Tools with Artificial Intelligence, 1995.

No context found.

H. Liu and R. Sentiono. Chi2: Feature selection and discretization of numeric attributes. In Proc. IEEE 7th Intl. Conf. on Tools with Artificial Intelligence, pp 338--391, 1995.

No context found.

H. Liu and R. Setiono. Chi2: Feature selection and discretization of numeric attributes. In Proc. of IEEE 7th International Conference on Tools with Artificial Intelligence, pages 338--391, 1995.

No context found.

Liu, H. and Setiono, R. (1995). Chi2 : Feature selection and discretization of numeric attributes. In Proc. of 7th IEEE Intl. Conf. on Tools with Artificial Intelligence.

No context found.

Liu, H. and Setiono, R. Chi2: feature selection and discretization of numeric attributes. In Proceedings of the 7 th IEEE International Conference on Tools with Artificial Intelligence, 1995. Pages 338-391.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC