Results 1  10
of
22
Analyticity of entropy rate of a hidden Markov chain
 In Proc. of IEEE International Symposium on Information Theory, Adelaide, Australia, September 4September 9 2005
, 1995
"... We prove that under mild positivity assumptions the entropy rate of a hidden Markov chain varies analytically as a function of the underlying Markov chain parameters. A general principle to determine the domain of analyticity is stated. An example is given to estimate the radius of convergence for t ..."
Abstract

Cited by 31 (13 self)
 Add to MetaCart
(Show Context)
We prove that under mild positivity assumptions the entropy rate of a hidden Markov chain varies analytically as a function of the underlying Markov chain parameters. A general principle to determine the domain of analyticity is stated. An example is given to estimate the radius of convergence for the entropy rate. We then show that the positivity assumptions can be relaxed, and examples are given for the relaxed conditions. We study a special class of hidden Markov chains in more detail: binary hidden Markov chains with an unambiguous symbol, and we give necessary and sufficient conditions for analyticity of the entropy rate for this case. Finally, we show that under the positivity assumptions the hidden Markov chain itself varies analytically, in a strong sense, as a function of the underlying Markov chain parameters. 1
On the Optimality of Symbol by Symbol Filtering and Denoising
, 2003
"... We consider the problem of optimally recovering a finitealphabet discretetime stochastic process {X t } from its noisecorrupted observation process {Z t }. In general, the optimal estimate of X t will depend on all the components of {Z t } on which it can be based. We characterize nontrivial s ..."
Abstract

Cited by 19 (3 self)
 Add to MetaCart
We consider the problem of optimally recovering a finitealphabet discretetime stochastic process {X t } from its noisecorrupted observation process {Z t }. In general, the optimal estimate of X t will depend on all the components of {Z t } on which it can be based. We characterize nontrivial situations (i.e., beyond the case where (X t , Z t ) are independent) for which optimum performance is attained using "symbol by symbol" operations (a.k.a.
New bounds on the entropy rate of hidden Markov process.
 IEEE Information Theory Workshop,
, 2004
"... ..."
(Show Context)
From FiniteSystem Entropy to Entropy Rate for a
 Hidden Markov Process. Signal Processing Letters, IEEE, Volume 13, Issue 9, Sept. 2006 Page(s):517
, 2006
"... Abstract—A recent result presented the expansion for the entropy rate of a hidden Markov process (HMP) as a power series in the noise variable. The coefficients of the expansion around the noiseless @ aHAlimit were calculated up to 11th order, using a conjecture that relates the entropy rate of an H ..."
Abstract

Cited by 13 (0 self)
 Add to MetaCart
(Show Context)
Abstract—A recent result presented the expansion for the entropy rate of a hidden Markov process (HMP) as a power series in the noise variable. The coefficients of the expansion around the noiseless @ aHAlimit were calculated up to 11th order, using a conjecture that relates the entropy rate of an HMP to the entropy of a process of finite length (which is calculated analytically). In this letter, we generalize and prove the conjecture and discuss its theoretical and practical consequences.
Estimating the Entropy of Binary Time Series: Methodology, Some Theory and a Simulation Study
"... entropy ..."
(Show Context)
Derivatives of entropy rate in special families of hidden Markov chains
 IEEE TRANS. INFO. THEORY
, 2007
"... Consider a hidden Markov chain obtained as the observation process of an ordinary Markov chain corrupted by noise. Recently Zuk et al. showed how, in principle, one can explicitly compute the derivatives of the entropy rate of at extreme values of the noise. Namely, they showed that the derivatives ..."
Abstract

Cited by 9 (4 self)
 Add to MetaCart
(Show Context)
Consider a hidden Markov chain obtained as the observation process of an ordinary Markov chain corrupted by noise. Recently Zuk et al. showed how, in principle, one can explicitly compute the derivatives of the entropy rate of at extreme values of the noise. Namely, they showed that the derivatives of standard upper approximations to the entropy rate actually stabilize at an explicit finite time. We generalize this result to a naural class of hidden Markov chains called “Black Holes.” We also discuss in depth special cases of binary Markov chains observed in binarysymmetric noise, and give an abstract formula for the first derivative in terms of a measure on the simplex due to Blackwell.
Analyticity of Entropy Rate in Families of Hidden Markov Chains
, 2008
"... We prove that under a mild positivity assumption the entropy rate of a hidden Markov chain varies analytically as a function of the underlying Markov chain parameters. We give examples to show how this can fail in some cases. And we study two natural special classes of hidden Markov chains in more d ..."
Abstract

Cited by 6 (1 self)
 Add to MetaCart
(Show Context)
We prove that under a mild positivity assumption the entropy rate of a hidden Markov chain varies analytically as a function of the underlying Markov chain parameters. We give examples to show how this can fail in some cases. And we study two natural special classes of hidden Markov chains in more detail: binary hidden Markov chains with an unambiguous symbol and binary Markov chains corrupted by binary symmetric noise. Finally, we show that under the positivity assumption the hidden Markov chain itself varies analytically, in a strong sense, as a function of the underlying Markov chain parameters.
On analytic properties of entropy rate
, 2009
"... Entropy rate is a real valued functional on the space of discrete random sources for which it exists. However, it lacks existence proofs and/or closed formulas even for classes of random sources which have intuitive parameterizations. A good way to overcome this problem is to examine its analytic p ..."
Abstract

Cited by 6 (3 self)
 Add to MetaCart
(Show Context)
Entropy rate is a real valued functional on the space of discrete random sources for which it exists. However, it lacks existence proofs and/or closed formulas even for classes of random sources which have intuitive parameterizations. A good way to overcome this problem is to examine its analytic properties relative to some reasonable topology. A canonical choice of a topology is that of the norm of total variation as it immediately arises with the idea of a discrete random source as a probability measure on sequence space. It is shown that both upper and lower entropy rate, hence entropy rate itself if it exists, are Lipschitzian relative to this topology, which, by well known facts, is close to differentiability. An application of this theorem leads to a simple and elementary proof of the existence of entropy rate of random sources with finite evolution dimension. This class of sources encompasses arbitrary hidden Markov sources and quantum random walks.
THE THEORY OF TRACKABILITY AND ROBUSTNESS FOR PROCESS DETECTION
, 2006
"... Many applications of current interests involve detecting instances of processes from databases or streams of sensor reports. Detecting processes relies on identifying evidences for the existence of such processes from usually noisy and incomplete observable events through statistical inferences. The ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
Many applications of current interests involve detecting instances of processes from databases or streams of sensor reports. Detecting processes relies on identifying evidences for the existence of such processes from usually noisy and incomplete observable events through statistical inferences. The performance of inferences can vary dramatically, depending on the complexity of processes ’ behavioral patterns, sensor resolution and sampling rate, SNR, location and coverage, and so on. Stochastic models are mathematical representations of all these factors. In this dissertation, we intend to answer the following questions: Performance – How accurate are the inference results given the model? Trackability – What are the boundaries of the performance of inferences? Robustness – How sensitive is the performance of inferences to perturbations on input data or model parameters? Methodology – How can we improve the trackability and robustness of process detection? From the information theoretic point of view, we address the reason of errors in detection to the losses of source information during the sensing stage, measured as entropy in the Shannon sense. We propose a series of entropic measures of the trackability and robustness for a popular modeling technique – hidden Markov models (HMM). Our major contributions include: the theory of trackability; structural analysis of trackability for HMMs through its nonparametric counterpart – DFA/NFAs; an effective visualization method for analyzing the trackability for
Approximations for the Entropy Rate of a Hidden Markov Process
"... Abstract—Let {Xt} be a stationary finitealphabet Markov chain and {Zt} denote its noisy version when corrupted by a discrete memoryless channel. We present an approach to bounding the entropy rate of {Zt} by the construction and study of a related measurevalued Markov process. To illustrate its ef ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
(Show Context)
Abstract—Let {Xt} be a stationary finitealphabet Markov chain and {Zt} denote its noisy version when corrupted by a discrete memoryless channel. We present an approach to bounding the entropy rate of {Zt} by the construction and study of a related measurevalued Markov process. To illustrate its efficacy, we specialize it to the case of a BSCcorrupted binary Markov chain. The bounds obtained are sufficiently tight to characterize the behavior of the entropy rate in asymptotic regimes that exhibit a “concentration of the support”. Examples include the ‘high SNR’, ‘low SNR’, ‘rare spikes’, and ‘weak dependence’ regimes. Our analysis also gives rise to a deterministic algorithm for approximating the entropy rate, achieving the best known precisioncomplexity tradeoff, for a significant subset of the process parameter space. I.