Information Theoretic Measures for Clusterings Comparison: Is a Correction for Chance Necessary?

by Nguyen Xuan Vinh , James Bailey

Citations:

14 - 0 self

Datum	Value	Source
TITLE	Information Theoretic Measures for Clusterings Comparison: Is a Correction for Chance Necessary?	SVM HeaderParse 0.2
AUTHOR NAME	Nguyen Xuan Vinh	SVM HeaderParse 0.2
AUTHOR NAME	James Bailey	SVM HeaderParse 0.2
AUTHOR AFFIL	The University of Melbourne, Vic. 3010, Australia	SVM HeaderParse 0.2
AUTHOR ADDR	Victoria Research Laboratory, National ICT Australia	SVM HeaderParse 0.2
ABSTRACT	Information theoretic based measures form a fundamental class of similarity measures for comparing clusterings, beside the class of pair-counting based and set-matching based measures. In this paper, we discuss the necessity of correction for chance for information theoretic based measures for clusterings comparison. We observe that the baseline for such measures, i.e. average value between random partitions of a data set, does not take on a constant value, and tends to have larger variation when the ratio between the number of data points and the number of clusters is small. This effect is similar in some other non-information theoretic based measures such as the well-known Rand Index. Assuming a hypergeometric model of randomness, we derive the analytical formula for the expected mutual information value between a pair of clusterings, and then propose the adjusted version for several popular information theoretic based measures. Some examples are given to demonstrate the need and usefulness of the adjusted measures. 1.	SVM HeaderParse 0.2
CITATIONS	12 found	ParsCit 1.0