Abstract

Kernel functions are typically viewed as providing an implicit mapping of points into a high-dimensional space, with the ability to gain much of the power of that space without incurring a high cost if the result is linearly-separable by a large margin #. However, the JohnsonLindenstrauss lemma suggests that in the presence of a large margin, a kernel function can also be viewed as a mapping to a low-dimensional space, one of dimension only O(1/# ). In this paper, we explore the question of whether one can e#ciently produce such low-dimensional mappings, using only black-box access to a kernel function. That is, given just a program that computes K(x, y) on inputs x, y of our choosing, can we e#ciently construct an explicit (small) set of features that e#ectively capture the power of the implicit high-dimensional space? We answer this question in the a#rmative if our method is also allowed black-box access to the underlying data distribution (i.e., unlabeled examples). We also give a lower bound, showing that if we do not have access to the distribution, then this is not possible for an arbitrary black-box kernel function; we leave as an open problem, however, whether this can be done for standard kernel functions such as the polynomial kernel.

Citations