Feature selection based on mutual information: Criteria of max-dependency, max-relevance, and min-redundancy

DMCA

Feature selection based on mutual information: Criteria of max-dependency, max-relevance, and min-redundancy (2005)

Cached

Download Links

Other Repositories/Bibliography

DBLP

by Hanchuan Peng , Fuhui Long , Chris Ding

Venue:	IEEE TRANS. PATTERN ANALYSIS AND MACHINE INTELLIGENCE
Citations:	530 - 7 self

BibTeX

@ARTICLE{Peng05featureselection,
    author = {Hanchuan Peng and Fuhui Long and Chris Ding},
    title = {Feature selection based on mutual information: Criteria of max-dependency, max-relevance, and min-redundancy},
    journal = {IEEE TRANS. PATTERN ANALYSIS AND MACHINE INTELLIGENCE},
    year = {2005},
    pages = {2005}
}

OpenURL

Abstract

Feature selection is an important problem for pattern classification systems. We study how to select good features according to the maximal statistical dependency criterion based on mutual information. Because of the difficulty in directly implementing the maximal dependency condition, we first derive an equivalent form, called minimal-redundancy-maximal-relevance criterion (mRMR), for first-order incremental feature selection. Then, we present a two-stage feature selection algorithm by combining mRMR and other more sophisticated feature selectors (e.g., wrappers). This allows us to select a compact set of superior features at very low cost. We perform extensive experimental comparison of our algorithm and other methods using three different classifiers (naive Bayes, support vector machine, and linear discriminate analysis) and four different data sets (handwritten digits, arrhythmia, NCI cancer cell lines, and lymphoma tissues). The results confirm that mRMR leads to promising improvement on feature selection and classification accuracy.

Keyphrases

mutual information feature selection max-depe ndency naive bayes good feature minimal-redundancy-maximal-relevance criterion nci cancer cell line linear discriminate analysis handwritten digit equivalent form important problem extensive experimental comparison different data set first-order incremental feature selection abstract feature selection superior feature maximal dependency condition two-stage feature selection algorithm pattern classification system support vector machine maximal statistical dependency criterion low cost classification accuracy sophisticated feature selector different classifier compact set

DMCA