DMCA
Using mutual information for selecting features in supervised neural net learning (1994)
| Venue: | IEEE TRANSACTIONS ON NEURAL NETWORKS |
| Citations: | 358 - 1 self |
BibTeX
@ARTICLE{Battiti94usingmutual,
author = {Roberto Battiti},
title = {Using mutual information for selecting features in supervised neural net learning},
journal = {IEEE TRANSACTIONS ON NEURAL NETWORKS},
year = {1994},
volume = {5},
number = {4},
pages = {537--550}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
This paper investigates the application of the mutual infor“ criterion to evaluate a set of candidate features and to select an informative subset to be used as input data for a neural network classifier. Because the mutual information measures arbitrary dependencies between random variables, it is suitable for assessing the “information content ” of features in complex classification tasks, where methods bases on linear relations (like the correlation) are prone to mistakes. The fact that the mutual information is independent of the coordinates chosen permits a robust estimation. Nonetheless, the use of the mutual information for tasks characterized by high input dimensionality requires suitable approximations because of the prohibitive demands on computation and samples. An algorithm is proposed that is based on a “greedy” selection of the features and that takes both the mutual information with respect to the output class and with respect to the already-selected features into account. Finally the results of a series of experiments are discussed.
Keyphrases
mutual information supervised neural net learning input data already-selected feature complex classification task candidate feature abstract-this paper method base information content random variable neural network classifier greedy selection suitable approximation informative subset robust estimation linear relation output class neural network pruning prohibitive demand mutual information measure arbitrary dependency di-mensionality reduction mutual infor criterion index terms-feature extraction high input dimensionality
