DMCA
Incremental algorithms for hierarchical classification (2004)
Cached
Download Links
| Venue: | Journal of Machine Learning Research |
| Citations: | 109 - 9 self |
BibTeX
@INPROCEEDINGS{Cesa-bianchi04incrementalalgorithms,
author = {Nicolò Cesa-bianchi and Luca Zaniboni and Michael Collins},
title = {Incremental algorithms for hierarchical classification},
booktitle = {Journal of Machine Learning Research},
year = {2004},
pages = {31--54},
publisher = {MIT Press}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
We study the problem of classifying data in a given taxonomy when classifications associated with multiple and/or partial paths are allowed. We introduce a new algorithm that incrementally learns a linear-threshold classifier for each node of the taxonomy. A hierarchical classification is obtained by evaluating the trained node classifiers in a top-down fashion. To evaluate classifiers in our multipath framework, we define a new hierarchical loss function, the H-loss, capturing the intuition that whenever a classification mistake is made on a node of the taxonomy, then no loss should be charged for any additional mistake occurring in the subtree of that node. Making no assumptions on the mechanism generating the data instances, and assuming a linear noise model for the labels, we bound the H-loss of our on-line algorithm in terms of the H-loss of a reference classifier knowing the true parameters of the label-generating process. We show that, in expectation, the excess cumulative H-loss grows at most logarithmically in the length of the data sequence. Furthermore, our analysis reveals the precise dependence of the rate of convergence on the eigenstructure of the data each node observes. Our theoretical results are complemented by a number of experiments on texual corpora. In these experiments we show that, after only one epoch of training, our algorithm performs much better than Perceptron-based hierarchical classifiers, and reasonably close to a hierarchical support vector machine.
Keyphrases
hierarchical classification incremental algorithm precise dependence multipath framework data sequence linear-threshold classifier texual corpus algorithm performs true parameter partial path data instance classification mistake excess cumulative h-loss grows top-down fashion linear noise model new hierarchical loss function on-line algorithm trained node classifier hierarchical support vector machine label-generating process additional mistake perceptron-based hierarchical classifier theoretical result new algorithm reference classifier
