J. Rissanen, "A Universal Prior for Integers and Estimation by Minimum Description Length," The Annals of Statistics 2, pp. 416--431 (1983). Home/Search Document Not in Database Summary Related Articles Check |
This paper is cited in the following contexts:
First 50 documents Next 50
Universal Codes for Finite Sequences of Integers Drawn from .. - Dean Foster Robert (Correct)
....is the penultimate code, denoted by # in Elias [2] See [4] and [5] for extensions of Elias. This procedure encodes the positive integer x with an idealized length L # (x) c # log x log log x where the log (base 2) terms are accumulated while positive and c # 2. 865 (see [8]) The penultimate code is universal in the sense of (6) because its expected length is bounded by a linear function of the entropy: for X F [2] #F # M) H F . 7) The penultimate code is also asymptotically optimal as defined in [2] When coding one integer from a monotone distribution ....
J. Rissanen. A universal prior for integers and estimation by minimum description length. Annals of Statistics, 11:416--431, 1983.
Multiscale Bayesian Segmentation Using a Trainable Context Model - Cheng, Bouman (2001) (9 citations) (Correct)
..... The EM algorithm is a standard algorithm for estimating parameters of a mixture model [35] 36] We use the EM algorithm to estimate the means , the covariance matrices , and the weights for each Gaussian mixture density. The model order is chosen for each class using the Rissanen criteria [40]. Training data set are generated using the feature vectors and ground truth segmentation . The prediction coefficients defined in (9) are estimated from training data using the standard least squares estimation. V. SIMULATION RESULTS In this section, we apply our segmentation algorithm to the ....
J. Rissanen, "A universal prior for integers and estimation by minimum description length," Ann. Statist., vol. 11, pp. 417--431, Sept. 1983.
Kolmogorov's Structure Functions with an Application.. - Vereshchagin.. (2002) (2 citations) (Correct)
....in part by the EU fifth framework project QAIP, IST 1999 11234, the NoE QUIPROCONE IST 1999 29064, the ESF QiT Programmme, and the EU Fourth Framework BRA NeuroCOLT II Working Group EP 27150. Address: CWI, Kruislaan 413, 1098 SJ Amsterdam, The Netherlands. Email: Paul.Vitanyi cwi.nl. [17]) subject to a given model complexity constraint, as well as minimizing the one part code consisting of just the data to model code (essentially the maximum likelihood estimator) in every case (and not only with high probability) selects a model that is a best explanation (within O(log jxj) ....
....notion of sufficient statistic [6, 5] that is central in classical statistics. The paper [9] investigates the algorithmic notion in detail and formally establishes such a relation. The algorithmic (minimal) sufficient statistic is related in [20, 10] to the minimum description length principle [17, 1, 25] in statistics and inductive reasoning. Moreover, 9] observed that x( 6 hx( K(x) O(1) establishing a one sided relation between (4) and (5) and the question was raised whether the converse holds. This Work: The most fundamental result in this paper is the equality (8) x( hx ( ....
[Article contains additional citation context not shown here]
J.J. Rissanen, A Universal Prior for Integers and Estimation by Minimum Description Length, Annals of Statistics, 11:2(1983), 416--431.
The Competitive Complexity Ratio - Stine, Foster (2000) (7 citations) (Correct)
....problem. The bound is a solution of a numerical integration in general, but simplifies nicely in a simplified context. We then extend these results to multivariate problems and close with a short discussion. II. Stochastic Complexity Early versions of Rissanen s MDL model selection criterion [6] assess the ability of a model to represent data using the length of a two part code. Let Y denote n observations with probability distribution P #p (y) which is indexed by some p dimensional parameter vector #p . As shown by Rissanen, it is most e#cient in this type of coding to round the ....
....origin. The idealized length of the two part code obtained by the p dimensional model is then L(Y, p) #(p) #s (z p ) log # , 1) where #(p) is the length of a prefix code for the dimension p, #s (z p ) denotes the length of a spiraling prefix code for the rounded vector of z scores [6], and # denotes a small remainder due to rounding #p to standard error scale. This form of the MDL criterion selects the model class that obtains the shortest code for the data, choosing the dimension p which minimizes L(Y, p) All logs here and in what follows are to base 2 unless otherwise ....
[Article contains additional citation context not shown here]
Rissanen, J. (1983). A universal prior for integers and estimation by minimum description length. Annals of Statistics, 11, 416-431.
Gestalt theory and Computer Vision - Desolneux, Moisan, Morel (2002) (Correct)
....also show by numerical counterexamples that all partial gestalts are likely to lead to wrong scene interpretations. As we shall see, the wrong detections are always explainable by a con ict between gestalts. We eventually show some experiment suggesting that Minimal Description Length principles [26] may be adequate to resolve some of the con icts between gestalt laws. Our plan is as follows. We start in section 2 with an account of Gestalt theory, centered on the initial 1923 Wertheimer programme. In section 3, we focus on the problems raised by the synthesis of groups obtained by partial ....
....computational evidence in favor of a collaboration between partial gestalt laws, namely collinearity, parallelism, color, contrast and similar motion. 22 example of it, let us mention how the dilemma alignment versus parallelism can be solved by an easy minimal description length principle (MDL) [26], 8] Figure 24 shows the problem and its simple solution. On the middle, we see all detected alignments in the Brown image on the left. Clearly, those alignments make sense but many of them are slanted. The main reason is this : all straight edges are in fact blurry and therefore constitute a ....
Rissanen, J. A universal prior for integers and estimation by Minimum Description Length. Annals of Statistics 11 (2), 1983.
Subspace Based Signal Analysis using Singular Value.. - van der Veen.. (1993) (5 citations) (Correct)
....them being based on the asymptotic distribution of the covariance matrix related to H under the assumption of white Gaussian noise. These include the classical sequential hypothesis test [44, 45, 46] Akaike s Information Criterion (AIC) 47] Rissanen s Minimum Description Length (MDL) principle [48, 49], and the refinements of Zhao, et al. 50] Specific applications to DOA estimation have been studied in [51, 52, 53, 54, 55] It is beyond the scope of this paper to study the model order determination problem in any detail, so we will just assume in what follows that d has been correctly ....
J. Rissanen, "A universal prior for the integers and estimation by Min- imum Description Length," Annals of Statistics, vol. 11, pp. 417-431, 1983.
On The Effect Of A-Priori Information On Performance Of The.. - Fishler, Messer (2002) (Correct)
.... sensors has received a considerable amount of attention during the last two decades (see, among many others, 1, 2, 3] The most common approach for estimating this number is to apply information theoretic criteria, like the Minimum Description Length (MDL) or Akaike Information Criterion (AIC) [4]. Since 1985 [2] when first suggested for estimating the number of narrow band sources impinging on an array of sensors, the MDL estimator practically became the standard tool for accomplishing this task. Assume an array of p sensors and denote by x(t) the received, p dimensional, signal vector ....
....the number of sources, q, given N independent snapshots of the array output, x(t1 ) x(tN ) 2. THE MDL APPROACH 2.1. General MDL estimator The information theoretic criteria approach is a general approach for choosing a model, that fits the data mostly, from a family of possible models [4, 5]. That is, given a parameterized family of probability densities, fX #q for various q, select q such that: q = arg min L p(q) 2) where L = log fX is the log likelihood of the measurements, denoted by X = x1 , xN ] p(q) is a general penalty function ....
[Article contains additional citation context not shown here]
J. Rissanen, "A universal prior for integers and estimation by minimum description length," Ann. Stat., vol. 11, no. 2, pp. 431--466, 1983.
Detection of Signals by Information Theoretic Criteria.. - Fishler, Grosmann.. (2002) (Correct)
.... a considerable amount of attention during the last two decades (see, among many others, 1] 2] 3] 4] 5] 6] The most common approach for estimating this number is to apply information theoretic criteria, like the Minimum Description Length (MDL) or Akaike Information Criterion (AIC) [7]. Since 1985 [3] when first suggested for estimating the number of narrow band sources impinging on an array of sensors, the MDL estimator practically became the standard tool for accomplishing this task. A. Problem Formulation Assume an array of p sensors and denote by x(t) the received, ....
....problem is to estimate the number of sources, q, given N independent snapshots of the array output, x(t 1 ) x(t N ) B. The MDL approach The information theoretic criteria approach is a general approach for choosing a model, that fits the data mostly, from a family of possible models [7], 10] That is, given a parameterized family of probability densities, f X for various q, select q such that: q = arg min q L p(q) 2) where L # = log f X is the log likelihood of the measurements, denoted by X = x 1 , xN ] p(q) is a general penalty function ....
[Article contains additional citation context not shown here]
J. Rissanen, "A universal prior for integers and estimation by minimum description length," Ann. Stat., vol. 11, no. 2, pp. 431--466, 1983.
Learning from Cluster Examples - Kamishima (2001) (Correct)
....partitions themselves. Appendix A The Description Length for the Decision Lists and Example Sets The total description length of the decision lists and example sets is as follows. #(ex 1 , DL) log # (m) #(T i ) #(S(T i ) log # ( Rissanen s code length for natural numbers [23] m the number of terms in the decision list #(T i ) code length for the term T i #(S(T i ) code length for the example set covered by T i 91 Code length for the example set S [29] #(S) log(#S 1) log #S the number of examples in S the number of S s elements whose class is ....
J. Rissanen. A universal prior for integers and estimation by minimum description length. The Annals of Statistics, 11(2):416--431, 1983.
Minimum Message Length Grouping of Ordered Data - Fitzgibbon, Allison, Dowe (2000) (1 citation) (Correct)
....and retrieve the original data, we must encode the following: G, the number of segments; the cut point positions, c s; the parameter estimates, # i , for each segment s i ; and finally the data for each segment using the parameter estimates stated. We specify G using the universal log # code [13, 2], although we re normalise the probabilities because we know that G K. This simplifies the problem to the specification of: the cut point positions c s, the parameter estimates # i and data for each segment. From Wallace and Freeman [22] the formula for calculating the length of a ....
J. J. Rissanen. A universal prior for integers and estimation by minimum description length. Annals of Statistics, 11(2):416--431, 1983.
A Probabilistic Approach to Lexical Semantic Knowledge Acquisition.. - Li (1998) (Correct)
....weight update scheme. Littlestone (1988) has shown that winnow performs much better than perceptron when many attributes are irrelevant. 2. 7 Introduction to MDL The Minimum Description Length principle is a strategy (criterion) for data compression and statistical estimation, proposed by Rissanen (1978; 1983; 1984; 1986; 1989; 1996; 1997) Related strategies were also proposed and studied independently in (Solomono#, 1964; Wallace and Boulton, 1968; Schwarz, 1978) A number of important properties of MDL have been demonstrated by Barron and Cover (1991) and Yamanishi (1992a) MDL states that, for ....
....is also a learning criterion, one which stipulates that from among the class of models that satisfies certain constraints, the model which has the maximum entropy should be selected. Selecting a model with maximum entropy is, in fact, equivalent to selecting a model with minimum description length (Rissanen, 1983). Thus, MDL provides an information theoretic justification of MEP. MDL and stochastic complexity The sum of parameter description length and data description length given in (2.9) is still a loose approximation. Recently, Rissanen has derived this more precise formula: # # ....
Rissanen, Jorma. 1983. A universal prior for integers and estimation by minimum description length. The Annals of Statistics, 11(2):416--431.
An Intelligent Problem Selection Agent for SQL-Tutor Using.. - Wang (2002) (Correct)
....AIC Akaike s information criterion (AIC) 1] is used to measure the tradeoff between training performance and network size. The goal is to minimise this term to produce a network with the best generalisation. AIC(k) N ln(MSE) 2k (4.5) 4.1. 6 MDL Rissanen s minimum description length (MDL) [21] criterion is similar to the AIC in that it tries to combine the model s error with the number of degrees of freedom to determine the level of generalisation. MDL(k) N ln(MSE) 0.5k ln(N) 4.6) 4.2 Results 4.2.1 Error Predictor Performance The initial training measures after 600 epochs are ....
RISSANEN, J. A universal prior for integers and estimation by minimum description length. The Annals of Statistics 11, 2 (1983), 416--431.
Beyond Independence: Probgbilistic Models for Query.. - Pavlov, Mannila (2001) (Correct)
....depends directly on the query. One general strategy is to have a fast and accurate online answer at the expense of potentially much higher offline computational investments. Definition 11.2: Memory. Memory is the complexity of the learned model Pt (e.g. its minimum description length in bits [31]) We can expect more complex models to generate more accurate answers on the data used to build the model, but also to be more expensive in terms of online time taken to answer a query. Definition 11.3: Error. Let (Q) denote the query distribution , i.e. the probability that a particular query ....
J. Rissanen. A universal prior for integers and estimation by minimum description length. The Annals of Statistics, 11(2):416-431, 1983.
Hilbert-Schmidt Lower Bounds for Estimators on Matrix.. - Grenander, Miller.. (1998) (10 citations) (Correct)
....in determining the bound on recognition. This connection between parameter estimation and Fig. 6. b) Shows the bound on estimating the orientation of any algorithm using video images of the airplane shown in (a) identification in hypothesis testing is the basis for Schwartz [19] and Rissanen [18] complexity. We take a Bayesian hypothesis testing approach for target identification. To simplify to the binary case, let there be only two target types: 6 = a 0 = truck, a 1 = tank and let H 0 , H 1 be the associated hypotheses. Consider only the target pose as a nuisance parameter neglecting ....
# J. Rissanen, "A Universal Prior for Integers and Estimation by Minimum Description Length," Annals of Statistics, vol. 11, pp. 416-431, 1983.
Universal Codes for Finite Sequences of Integers Drawn From .. - Foster, Stine, Wyner (2001) (Correct)
....is the penultimate code, denoted by in Elias [2] See [4] and [5] for extensions of Elias. This procedure encodes the positive integer x 1 with an idealized length L (x) c log x log log x ; where the log (base 2) terms are accumulated while positive and c 2:865 (see [8]) The penultimate code is universal in the sense of (6) because its expected length is bounded by a linear function of the entropy: for X F [2] 8F 2 M) E L (X) 1 H F : 7) The penultimate code is also asymptotically optimal as de ned in [2] When coding one integer from a monotone ....
J. Rissanen. A universal prior for integers and estimation by minimum description length. Annals of Statistics, 11:416-431, 1983.
Building Optimal 2D Statistical Shape Models - Davies, Twining, Allen.. (Correct)
....choice of objective function to measure the quality of a model is based on the idea that the best description of the training set is the simplest , i.e. most compact and most likley to generalise to unseen examples. This can be formalised using the Minimum Description Length (MDL) Principle [20,21]. MDL is based on the idea of transmitting the training data set as a coded message, where the coding is based on some pre arranged set of parametric statistical models. The full transmission of the data set then has to include not only the encoded data values, but also the information required to ....
J. R. Rissanen, "A universal prior for integers and estimation by minimum description length", Annals of Statistics, vol. 11, no. 2, pp. 416-431, 1983.
An Optimization Framework for Feature Extraction - Fua And Hanson (Correct)
No context found.
J. Rissanen, "A Universal Prior for Integers and Estimation by Minimum Description Length," The Annals of Statistics 2, pp. 416--431 (1983).
Algorithms for Clustering High Dimensional and - Tao (Correct)
No context found.
J. Rissanen. A universal prior for integers and estimation by minimum description length. The Annals of Statistics, 11(2):416--431, 1983.
Model Selection and the Principle of Minimum Description Length - Hansen, Yu (1998) (203 citations) (Correct)
No context found.
Rissanen, J. (1983). A universal prior for integers and estimation by minimum description length. Ann. Statist., 11, 416--431.
Mesh Smoothing by Adaptive and Anisotropic Gaussian Filter - Applied To Mesh (Correct)
No context found.
J. Rissanen. A universal prior for integers and estimation by Minimal Description Length. The Annals of Statistics, 11:131--138, 1983.
3D Scattered Data Approximation with Adaptive Compactly.. - Ohtake, Belyaev, Seidel (Correct)
No context found.
J. Rissanen. A universal prior for integers and estimation by Minimal Description Length. The Annals of Statistics, 11:131--138, 1983.
Model Selection Techniques and Merging Rules for Range.. - Kishore Bubna Charles (2000) (1 citation) (Correct)
No context found.
J. Rissanen. A universal prior for integers and estimation by minimum description length. The Annals of Statistics, 11(2):416--431, 1983.
Bayesian Analysis of Polyphonic Western Tonal Music - Davy, Godsill, Idier (2006) (Correct)
No context found.
J. Rissanen, "A universal prior for integers and estimation by minimum description length," Annals of Statistics, vol. 11, no. 2, pp. 416--431, 1983.
Effective And Efficient Sports Highlights Extraction Using.. - Minimum Description Length (2004) (Correct)
No context found.
J. Rissanen, "A universal prior for integers and estimation by minimum description length," Annals of Statistics, vol. 11, no. 2, pp. 417 -- 431, 1983.
Probabilistic Query Models for Transaction Data - Dmitry Pavlov University (2001) (4 citations) (Correct)
No context found.
J. Rissanen. A universal prior for integers and estimation by minimum description length. The Annals of Statistics, 11(2):pages 416--431, 1983.
A Heuristic for Learning Decision Trees and Pruning.. - Ranilla, Luaces.. (Correct)
No context found.
J. Rissanen. A universal prior for integers and estimation by minimum description length. The Annals of Statistics, 11(2):416--431, 1983.
A Heuristic for Learning Decision Trees and Pruning.. - Ranilla, Luaces.. (Correct)
No context found.
J. Rissanen. A universal prior for integers and estimation by minimum description length. The Annals of Statistics, 11(2):416-431, 1983.
Automatic Construction of Optimal Statistical Shape Models - Davies, Twining, Cootes.. (Correct)
No context found.
Rissanen, J. R. "A universal prior for integers and estimation by minimum description length", Annals of Statistics, vol. 11, no. 2, pp. 416-431, 1983. AstraZeneca, Alderley Park, Macclesfield, Cheshire, UK
MML and Bayesianism: Similarities and Differences.. - Oliver, Baxter (1994) (18 citations) (Correct)
No context found.
J. Rissanen. A universal prior for integers and estimation by minimum description length. Annals of Statistics, 11:416--431, 1983.
Introduction to Minimum Encoding Inference - Oliver, Hand (1994) (13 citations) (Correct)
No context found.
J. Rissanen. A universal prior for integers and estimation by minimum description length. Annals of Statistics, 11:416--431, 1983.
Determining the Number of Communication - Sources Using Sensor (Correct)
No context found.
J. Rissanen. A Universal Prior for Integers and Estimation by Minimum Description Length. Ann. Stat., 11(2):431--466, 1983.
IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 50, NO. 5, .. - Criteria General.. (Correct)
No context found.
J. Rissanen, "A universal prior for integers and estimation by minimum description length," Ann. Stat., vol. 11, no. 2, pp. 431--466, 1983.
Minimum Message Length Grouping of Ordered - Fitzgibbon, Allison, Dowe (2000) (Correct)
No context found.
J. J. Rissanen. A universal prior for integers and estimation by minimum description length. Annals of Statistics, 11(2):416-431, 1983.
MML Inference of Single-Layer Neural Networks - Makalic, Allison, Dowe (2003) (Correct)
No context found.
J.J. Rissanen, A universal prior for integers and estimation by minimum description length, Annals of Statistics, 11(2), 1983, 416--431.
Statistical Analysis of Brain Optical Recording Data - Peshkin (1995) (Correct)
No context found.
Rissanen, J. (1983). A universal prior for integers and estimation by minimum description length. Ann. Statist., 11:416--431.
Kolmogorov's Structure Functions and Model Selection - Vereshchagin, Vitanyi (2003) (Correct)
No context found.
J.J. Rissanen, A Universal Prior for Integers and Estimation by Minimum Description Length, Annals of Statistics, 11:2(1983), 416-431.
An Optimization Framework for Feature Extraction - Fua And Hanson (1991) (9 citations) (Correct)
No context found.
J. Rissanen, "A Universal Prior for Integers and Estimation by Minimum Description Length," The Annals of Statistics 2, pp. 416--431 (1983).
The Types of Models - Allison (2003) (Correct)
No context found.
Rissanen, J.: A Universal Prior for Integers and Estimation by Minimum Description Length. Annals of Statistics 11(2) (1983) 416-431
Dynamic Enumeration Algorithm Using Array Antennas - Peter Green And (2001) (Correct)
No context found.
J. Rissanen, "A universal prior for integers and estimation by minimum description length," Ann. Stat., vol. 11, no. 2, pp. 416--431, 1983.
Unsupervised Image Restoration and Edge Location Using.. - Figueiredo, Leitao (1997) (7 citations) (Correct)
No context found.
J. Rissanen, "A universal prior for integers and estimation by minimum description length," Ann. Stat., vol. 11, pp. 416--431, 1983.
Functional Graphical Models - Marchadier (2003) (Correct)
No context found.
Rissanen J., "A Universal Prior For Integers and Estimation by Minimum Description Length", The Annals of Statistics, vol. 11(2), 1983, 416--431. 28
Model Selection and Surface Merging in Reconstruction Algorithms - Bubna, Stewart (1997) (1 citation) (Correct)
No context found.
J. Rissanen. A universal prior for integers and estimation by minimum description length. The Annals of Statistics, 11(2):416--431, 1983.
Learning When All Models Are Wrong: Additional Information - Grünwald (Correct)
No context found.
J. Rissanen. A universal prior for integers and estimation by minimum description length. The Annals of Statistics, 11:416-431, 1983.
Motion-Based Object segmentation and estimation using the MDL .. - Zheng, Blostein (1995) (12 citations) (Correct)
No context found.
J. Rissanen. "A universal prior for integers and estimation by minimum description length," Annals of Statistics, Vol. 11, No. 2, pp. 416-431, 1983.
From Synapses to Rules - Apolloni, Malchiodi, Orovas, Palmas (2001) (Correct)
No context found.
Rissanen, J. (1983). A universal prior for integers and estimation by minimum description length. Annals of Statistics, 11(2):416-431.
Unknown - (2002) (Correct)
No context found.
J. Rissanen, "A universal prior for integers and estimation by minimum description length," The Annals of Statistics, vol. 11, no. 2, pp. 416-431, 1983.
Clustered Components Analysis for Functional MRI - Chen, Bouman, Lowe (Correct)
No context found.
J. Rissanen, "A universal prior for integers and estimation by minimum description length," The Annals of Statistics, vol. 11, no. 2, pp. 417--431, September 1983.
What Is Better: GMM of Two . . . - Lapidot (2002) (Correct)
No context found.
J. Rissanen, "A universal prior for integers and estimation by minimum description length," The Annals of Statistics, vol. 11, no. 2, pp. 416-431, 1983.
A Probabilistic Criterion to Detect Rigid Point Matches.. - Moisan, Stival (2002) (Correct)
No context found.
Rissanen, J. \A universal prior for integers and estimation by Minimum Description Length", Annals of Statistics 11:2, pp. 416-431, 1983.
Multiscale Image Segmentation Using Joint Texture.. - Neelamani.. (2000) (1 citation) (Correct)
No context found.
J. Rissanen, \A universal prior for integers and estimation by minimum description length," Annals of Statistics 11, pp. 416-431, 1983.
First 50 documents Next 50
Online articles have much greater impact More about CiteSeer.IST Add search form to your site Submit documents Feedback
CiteSeer.IST - Copyright Penn State and NEC