Citation Context

...istic prediction [20], sparse representation [21]–[28], stability [29], eigenvalue sensitivity [30], and so on. The family of supervised feature selection algorithms can be divided into filters [14], =-=[15]-=-, [31], wrappers [32], [33], and embedded methods [21], [23], [34], according to whether a specific classifier is integrated into the learning framework [7]. Filters evaluate the correlation or releva...

Citation Context

...orithm is used as part of the evaluation function to determine the fitness of the selected feature subset. Wrappers can usually achieve better results than filter approaches, but the main drawbacks are their computational deficiency and loss of generality [3]. In a filter approach, feature selection is done as a preprocessing procedure and the search process is independent of a learning algorithm. Therefore, the performance of a filter approach relies mainly on the goodness of the evaluation criterion. Many different criteria have been used in filter approaches, including information measures [4], dependency measures [5], consistency measures [6], and distance measures [7]. Compared with wrappers, filter approaches are argued to be computationally less expensive and more general [1]. In feature selection, the size of the search space for n features is 2n. So in most situations, it is impractical to conduct an exhaustive search for feature selection [3]. Therefore, the search strategy can significantly influence the results of a feature selection approach. Many search techniques have been applied in feature selection such as greedy search, but most of them usually suffer from the probl...

Citation Context

...can be used to calculate the respective entropies, and finally the mutual information as presented in (5)–(8). The MI value can be normalized by dividing it by the minimum of entropies H(X) and H(Y ) =-=[6]-=- or by their sum [24]. H(X,Y ) = ∑ ∑ x∈p(x,y) y∈p(x,y) H(X) = ∑ x∈p(x) H(Y ) = ∑ y∈p(y) −p(x,y) ∗ log2(p(x,y)) , (5) −p(x) ∗ log2(p(x)) , (6) −p(y) ∗ log2(p(y)) , (7) MI(X,Y ) = H(X) + H(Y ) − H(X,Y )...

Citation Context

... paper, we propose a filter method that uses mutual information assa criterion for finding the optimal subset of features. Mutual information (MI)shas been widely applied in feature selection methods =-=[2, 3, 7, 8, 14, 19, 26]-=-.sThe more important properties of MI are: (i) its ability to quantify nonlinearsdependencies between features, and (ii) its invariance under transformationssof the space [15]. Battiti [2] proposed a ...

Citation Context

...step in the classification process. In general, it is desirable to keep the dimensionality of the input features as small as possible to reduce the computational cost of training a classifier as well as its complexity (Torkkola, 2003; Murillo & Rodriguez, 2007). Moreover, using large number of features, when the number of data is low, can cause degradation of the classification performance (Chow & Huang, 2005). Reduction of the number of input features can be done by selecting useful features and discarding others (i.e., feature selection) (Battiti, 1994; Kwak & Choi, 2002; Peng et al., 2005; Estèvez et al., 2009; Sindhwani et al., 2004) or extracting new features containing maximal information about the class label from the original ones (i.e., feature extraction) (Torkkola, 2003; Hild II et al., 2006; Kwak, 2007; Murillo & Rodriguez, 2007). In this paper, we focus on feature selection methods. A variety of linear feature extraction methods have been proposed. One well-known feature extraction methods may be principal component analysis (PCA) (Li et al., 2006). The purpose of PCA is to find an orthogonal set of projection vectors or principal components for feature extraction from given training data...

Citation Context

...step in the classification process. In general, it is desirable to keep the dimensionality of the input features as small as possible to reduce the computational cost of training a classifier as well as its complexity (Torkkola, 2003; Murillo & Rodriguez, 2007). Moreover, using large number of features, when the number of data is low, can cause degradation of the classification performance (Chow & Huang, 2005). Reduction of the number of input features can be done by selecting useful features and discarding others (i.e., feature selection) (Battiti, 1994; Kwak & Choi, 2002; Peng et al., 2005; Estèvez et al., 2009; Sindhwani et al., 2004) or extracting new features containing maximal information about the class label from the original ones (i.e., feature extraction) (Torkkola, 2003; Hild II et al., 2006; Kwak, 2007; Murillo & Rodriguez, 2007). In this paper, we focus on feature selection methods. A variety of linear feature extraction methods have been proposed. One well-known feature extraction methods may be principal component analysis (PCA) (Li et al., 2006). The purpose of PCA is to find an orthogonal set of projection vectors or principal components for feature extraction from given training data...