Abstract

We derive a first-order approximation of the density of maximum entropy for a continuous 1-D random variable, given a number of simple constraints. This results in a density expansion which is somewhat similar to the classical polynomial density expansions by Gram-Charlier and Edgeworth. Using this approximation of density, an approximation of 1-D differential entropy is derived. The approximation of entropy is both more exact and more robust against outliers than the classical approximation based on the polynomial density expansions, without being computationally more expensive. The approximation has applications, for example, in independent component analysis and projection pursuit.

Citations

6515	Elements of Information Theory - Cover, Thomas - 1991
1034	Independent component analysis, a new concept - Comon - 1994
206	Exploratory projection pursuit - Friedman - 1987
189	Projection pursuit - HUBER - 1985
72	A new learning algorithm for blind source separation - Amari, Cichocki, et al. - 1996
71	What is Projection Pursuit - Jones, Sibson - 1987
16	Projection pursuit indexes based on orthonormal function expansions - Cook, Buja, et al. - 1993
8	Polynomial projection pursuit - Hall - 1989