Tools
Sorted by:
Try your query at:
 |
 |
 |
 |
 |
 |
Results 1 - 10
of
21,381
Rényi Divergence and Kullback-Leibler Divergence
"... Abstract—Rényi divergence is related to Rényi entropy much like Kullback-Leibler divergence is related to Shannon’s entropy, and comes up in many settings. It was introduced by Rényi as a measure of information that satisfies almost the same axioms as Kullback-Leibler divergence, and depends on a ..."
Abstract
-
Cited by 1 (0 self)
- Add to MetaCart
Abstract—Rényi divergence is related to Rényi entropy much like Kullback-Leibler divergence is related to Shannon’s entropy, and comes up in many settings. It was introduced by Rényi as a measure of information that satisfies almost the same axioms as Kullback-Leibler divergence, and depends
The Kullback-Leibler Divergence Rate between Markov Sources
- IEEE Trans. Information Theory
, 2004
"... Abstract—In this work, we provide a computable expression for the Kullback–Leibler divergence rate lim ( ) between two time-invariant finite-alphabet Markov sources of arbitrary order and arbitrary initial distributions described by the probability distributions and, respectively. We illustrate it n ..."
Abstract
-
Cited by 24 (1 self)
- Add to MetaCart
Abstract—In this work, we provide a computable expression for the Kullback–Leibler divergence rate lim ( ) between two time-invariant finite-alphabet Markov sources of arbitrary order and arbitrary initial distributions described by the probability distributions and, respectively. We illustrate
Kullback-Leibler Divergence and the Central Limit Theorem
"... Abstract—This paper investigates the asymptotics of Kullback-Leibler divergence between two probability distributions satisfying a Central Limit Theorem prop-erty. The basic problem is as follows. Let Xi, i ∈ N, be a sequence of independent random variables such that the sum Sn = ∑n i=1Xi has the sa ..."
Abstract
- Add to MetaCart
Abstract—This paper investigates the asymptotics of Kullback-Leibler divergence between two probability distributions satisfying a Central Limit Theorem prop-erty. The basic problem is as follows. Let Xi, i ∈ N, be a sequence of independent random variables such that the sum Sn = ∑n i=1Xi has
KULLBACK LEIBLER DIVERGENCE IN MIXTURE MODEL
"... Multiresolution data arise when an object or a phenomenon is described at several levels of detail Multiresolution data is prevalent in many application areas F Examples include biology, computer vision Faster growth of multiresolution data is expected in future Over the years, data accumulates ..."
Abstract
- Add to MetaCart
Multiresolution data arise when an object or a phenomenon is described at several levels of detail Multiresolution data is prevalent in many application areas F Examples include biology, computer vision Faster growth of multiresolution data is expected in future Over the years, data accumulates in multiple resolutions because
Maximal Kullback-Leibler Divergence Cluster Analysis
, 1987
"... In this paper we introduce a new procedure for performing a cluster analysis and prove a consistency result for the procedure. The method seems to perform well on data and a number of examples are presented. We will formulate the «clustering problem " in the following way. Suppose we observe X ..."
Abstract
- Add to MetaCart
-pereieicoe " of the space X. The partition which best describes the clustering structure of the da.ta. is defined to be the one which maximises a certain criterion function. This criterion function is a weighted sum of Kullback-Leibler divergences.
Kullback-Leibler Divergence Estimation of Continuous Distributions
- Proceedings of IEEE International Symposium on Information Theory
, 2008
"... Abstract—We present a method for estimating the KL divergence between continuous densities and we prove it converges almost surely. Divergence estimation is typically solved estimating the densities first. Our main result shows this intermediate step is unnecessary and that the divergence can be eit ..."
Abstract
-
Cited by 18 (0 self)
- Add to MetaCart
Abstract—We present a method for estimating the KL divergence between continuous densities and we prove it converges almost surely. Divergence estimation is typically solved estimating the densities first. Our main result shows this intermediate step is unnecessary and that the divergence can
Notes on kullback-leibler divergence and likelihood theory
- System Neurobiology Laboratory, Salk Institute for Biological Studies
, 2007
"... iv ..."
Optimism in Reinforcement Learning and Kullback-Leibler Divergence
"... Abstract. We consider model-based reinforcement learning in finite Markov Decision Processes (MDPs), focussing on so-called optimistic strategies. In MDPs, optimism can be implemented by carrying out ex-tended value iterations under a constraint of consistency with the esti-mated model transition pr ..."
Abstract
- Add to MetaCart
probabilities. The UCRL2 algorithm by Auer, Jaksch and Ortner (2009), which follows this strategy, has recently been shown to guarantee near-optimal regret bounds. In this paper, we strongly argue in favor of using the Kullback-Leibler (KL) divergence for this purpose. By studying the linear maximization
Fault tolerant learning using Kullback-Leibler Divergence
- in Proc. TENCON’2007
, 2007
"... Abstract — In this paper, an objective function for training a fault tolerant neural network is derived based on the idea of Kullback-Leibler (KL) divergence. The new objective function is then applied to a radial basis function (RBF) network that is with multiplicative weight noise. Simulation resu ..."
Abstract
-
Cited by 3 (3 self)
- Add to MetaCart
Abstract — In this paper, an objective function for training a fault tolerant neural network is derived based on the idea of Kullback-Leibler (KL) divergence. The new objective function is then applied to a radial basis function (RBF) network that is with multiplicative weight noise. Simulation
Results 1 - 10
of
21,381
