J. Rissanen. Strong optimality of the normalized ML models as universal codes and information in data. IEEE Transactions on Information Theory, 47(5):1712--1717, July 2001.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....and if the code C that is used is allowed to depend on sample size n, 14) the Shtarkov NML and two part codes can be re ned to give [2] RC (x (n) k 2 log n 2 log Z k p det I( d o(1) 15) where I( is the (expected) Fisher information matrix of . Quite recently, Rissanen [25] showed that the regret (15) is the best that can be achieved under at least three di erent de nitions of optimality. LC (x (n) log p (x (n) x (n) RC (x (n) with RC (x (n) given by (15) is called the stochastic complexity of x (n) relative to M k . In the same ....

....the regret (15) is the best that can be achieved under at least three di erent de nitions of optimality. LC (x (n) log p (x (n) x (n) RC (x (n) with RC (x (n) given by (15) is called the stochastic complexity of x (n) relative to M k . In the same recent reference [25], Rissanen implicitly introduced a new type of universal code that achieves regret (15) We illustrate this kind of code for the simple case where M k is a k dimensional exponential family with nite sample space X . Let then M k = fP ( j 2 k g be a k parameter exponential family for X ....

[Article contains additional citation context not shown here]

J. Rissanen. Strong optimality of the normalized ML models as universal codes, 2001. To appear in IEEE Transactions on Information Theory.

....if the code C that is used is allowed to depend on sample size n, 15) the Shtarkov NML and two part codes can be re ned to give [2] RC (x (n) k 2 log n 2 log Z k p det I( d o(1) 16) where I( is the (expected) Fisher information matrix of . Quite recently, Rissanen [23] showed that the regret (16) is the best that can be achieved under at least three di erent de nitions of optimality. LC (x (n) log p (x (n) x (n) RC (x (n) with RC (x (n) given by (16) is called the stochastic complexity of x (n) relative to M k . In the same ....

....the regret (16) is the best that can be achieved under at least three di erent de nitions of optimality. LC (x (n) log p (x (n) x (n) RC (x (n) with RC (x (n) given by (16) is called the stochastic complexity of x (n) relative to M k . In the same recent reference [23], Rissanen implicitly introduced a new type of universal code that achieves regret (16) We illustrate this kind of code for the simple case where M k is a k dimensional exponential family with nite sample space X . Let then M k = fP ( j 2 k g be a k parameter exponential family for X with ....

J. Rissanen. Strong optimality of the normalized ML models as universal codes, 2001. To appear in IEEE Transactions on Information Theory.

....if the code C that is used is allowed to depend on sample size n, 14) the Shtarkov minimax and two part codes can be re ned to give [2] R C (x (n) k 2 log n 2 log Z k p det I( d o(1) 15) where I( is the (expected) Fisher information matrix of . Quite recently, Rissanen [25] showed that the regret (15) is the best that can be achieved under at least three di erent de nitions of optimality. LC (x (n) log p (x (n) x (n) RC (x (n) with RC (x (n) given by (15) is called the stochastic complexity of x (n) relative to M k . In the same ....

....the regret (15) is the best that can be achieved under at least three di erent de nitions of optimality. LC (x (n) log p (x (n) x (n) RC (x (n) with RC (x (n) given by (15) is called the stochastic complexity of x (n) relative to M k . In the same recent reference [25], Rissanen implicitly introduced a new type of universal code that achieves regret (15) We illustrate this kind of code for the simple case where M k is a k dimensional exponential family with nite sample space X . Let then M k = fP ( j 2 k g be a k parameter exponential family for X ....

[Article contains additional citation context not shown here]

J. Rissanen. Strong optimality of the normalized ml models as universal codes, 2001. To appear in IEEE Transactions on Information Theory.

....contribute. This means that we now can compare different classes of models, regardless of their structure, shape or size, simply by the amount of useful information they extract from the data, and we get a foundation for an authentic theory of model building if not for all statistics in general, [27]. The theory of universal models and stochastic complexity for families of models of random processes, which generate data of all lengths, has not yet reached the same level of maturity as the case is with models for data of fixed length. Such models are particularly important in prediction ....

....a wild guess. There exists also a stronger version of the theorem about the minmax bound log C (k) which in effect states that this worst case bound is not a rare event but that it cannot be beaten in essence even when we assume that the data were generated by the most benevolent opponents, [27]. 6.2.3 Universal Sufficient Statistics We begin by sketching the derivation of a quite accurate formula for the ideal code length of the fundamental NML model, which we write for density models. If we overlook some details, 25] the 49 derivation is quite simple. The main condition required ....

Rissanen, J. (2001), 'Strong Optimality of the Normalized ML Models as Universal Codes and Information in Data', IEEE Trans. Information Theory (to appear)

....attains a minmax cumulative prediction loss under specific restricted loss functions, where the maximum is taken over sequences. Inspired by these works we define an extension of stochastic complexity, which we call loss complexity, in a way analogous to that of the stochastic complexity, 9] and [10], namely, such that its mean provides an achievable lower bound for the mean accumulated loss. The mean is taken with respect to the worst case data generating distribution in a class that need not coincide with the class of models defined by the loss function. The loss complexity gives also a ....

.... jx n ; y n ; x n ) C n; 10) where C n; B n; Z n : 11) We mention in passing that the ideal code length is the most powerful loss function in the sense that the NML model captures all the regular features in the data that can be expressed in terms of the model class given, [10]. We derive next a few important properties of the models in the class M ;k for a simple loss function, which are shared by the exponential family of densities. First, 4 by differentiating the integral (4) with respect to we get for all y of type y = h(x; and all positive E ; ffi(Y; ....

[Article contains additional citation context not shown here]

Rissanen, J. (2001), `Strong Optimality of the Normalized ML Models as Universal Codes and Information in Data', IEEE Trans. Information Theory (to appear)

No context found.

J. Rissanen. Strong optimality of the normalized ML models as universal codes and information in data. IEEE Transactions on Information Theory, 47(5):1712--1717, July 2001.

No context found.

J. Rissanen. Strong optimality of the normalized ML models as universal codes and information in data. IEEE Transactions on Information Theory, 47(5):1712--1717, July 2001.

No context found.

J. Rissanen. Strong optimality of the normalized ML models as universal codes and information in data. IEEE Transactions on Information Theory, 47(5):1712--1717, July 2001.

No context found.

J. Rissanen. Strong optimality of the normalized MLmodels as universal codes and information in data. IEEE Transactions on Information Theory, 47(5):1712--1717, July 2001.

No context found.

Rissanen, J. (2001). Strong optimality of the normalized ML models as universal codes and information in data. IEEE Transactions on Information Theory 47, 1712-1717.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC