R. J. Solomonoff. A formal theory of inductive inference. part II. Information and Control, 7(2):224--254, 1964.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....variant of m has astonishing performance in predicting sequences where the probability of the next element is computable from the initial segment. We now come to the punch line: For model selection, Bayes rule using the algorithmic universal prior distribution, suggested by Solomonoff already in [19], yields Occam s Razor principle in the MDL sense and is rigorously shown to work correctly in the companion paper [26] Namely, there it is shown that data compression is almost always the best strategy, both in hypothesis identification and prediction. 2.3. Minimum description length principle ....

R.J. Solomonoff, A formal theory of inductive inference, Part 1 and Part 2, Inform. Contr. 7 (1964) 1--22, 224-254.

....program P to be the number of bits in P (i.e. its length) i.e. the complexity #(p) of a program p is equal to [log 2 (p 1) the greatest integer not greater than the base 2 logarithm of p 1. We now introduce the simple program computers. Computers similar to them have been used in Solomono# [11], Kolmogorov [12] and in [13] Definition 5. A simple program computer # has the following property: For any computer #, there exists a natural number c ## such that ## (S) # # (S) c ## for all infinite computable sets S of natural numbers. To the extent that it is plausible to consider all ....

Solomonoff, R. J. A formal theory of inductive inference, Pt. I. Inform. Contr. 7 (1964), 1--22.

....measure P is plausible Occam s razor suggests that the simplest y should be more probable. But which exactly is the correct definition of simplicity The next section will offer an alternative to the celebrated but noncomputable algorithmic simplicity measure or Solomonoff Levin measure [24, 29, 25]. But let us first review Solomonoff s traditional approach. Roughly fourty years ago Solomonoff started the theory of universal optimal induction based on the apparently harmless simplicity assumption that P is computable [24] While Equation (1) makes predictions of the entire future, given the ....

....algorithmic simplicity measure or Solomonoff Levin measure [24, 29, 25] But let us first review Solomonoff s traditional approach. Roughly fourty years ago Solomonoff started the theory of universal optimal induction based on the apparently harmless simplicity assumption that P is computable [24]. While Equation (1) makes predictions of the entire future, given the past, Solomonoff [25] focuses just on the next bit in a sequence. Although this provokes surprisingly nontrivial problems associated with translating the bitwise approach to alphabets other than the binary one only recently ....

[Article contains additional citation context not shown here]

R.J. Solomonoff. A formal theory of inductive inference. Part I. Information and Control, 7:1-22, 1964.

....rules from experimental data. From these rules a theory is built up to predict future experiences. Some of the formal models gather some of the ideas about Some of the following sections originate on our thought that the following MDL principle ignores this. compression we have presented. Soon [Solomonoff 1964], it was recognised that the unsupervised learning of a grammar from raw data may be understood as information compression. The general field of machine learning has its more computational and formal part in the area of COmputational Learning Theory (COLT) With the ideas of compression and with ....

Solomonoff, R.J. "A formal theory of inductive inference" Inf. Control.. vol. 7, 1-22, Mar., 224-254, June 1964.

....generating functions, but rather to search for good models of the data where the goodness is defined in terms of code length , so to look for the model or class of models that best capture the regularity of the data. The results of the theory of algorithmic complexity, developed by Solomonoff [Sol64], Kolmogorov [Kol65] and Chaitin, can be applied to arrive at an ideal version of MDL. Roughly speaking the Kolmogorov Complexity of a string x is the length of the shortest computer program that prints x and then halts. Here the program should be written in some fixed universal language, such as ....

R. Solomonoff. A formal theory of inductive inference. Information and Control, 7, 1964.

....is, at best, extremely uncertain. On its own elaboration is, of course, an inadequate strategy. One can get into a position of diminishing returns where each elaboration brings decreasing improvements in the error rate, but at increasing cost. 3. Variously attributed to combinations of Solomonoff [15], Kolmogorov [8] and Chaitin [1] 4. In this I agree with Quine [12] and Judea Pearl [10] 5. For example, if we knew that nature had developed a certain class of systems starting simply and elaborating from this and our language of modelling reflected this structure we would expect there to be ....

Solomonoff, R.J. 1964. A Formal theory of Inductive Inference. Information and Control, 7, 1-22, 224-254.

....letter is the predecessor of. the corresponding letter in the previous 3 1etter subsequence. Early papers by Simon and Kotovsky [15, 32, 33] show that just a few relationships (such as successor, predecessor, and equality) are sufficient to represent most such patterns. Related work by Solomonoff [34] and Hedrick [13] has investigated grammatical approaches to describing letter sequences. The sequence prediction problem becomes more difficult when the sequence consists not of simple objects with only a single relevant attribute (like the problem just described) but instead of objects with ....

Solomonoff, R.S., A formal theory of inductive inference, Inform. and Control 7 (1964) 1-22;

....the Bayesian techniques is the fact they approximate quite well a suitably generalized maximum likelihood technique, especially for large amounts of data, as the subject matter of main interest in these lectures could also be called. The algorithmic theory of information, introduced by Solomonoff,[30], which we discuss briefly later, provides a different foundation for statistical inquiry, which in principle is free from the prob lems discussed above. What is most important is that data need not be regarded as a sample from any distribution, and the idea of a model is simply a computer ....

....and we have an infinite prefix code. The Kolmogorov complexity of a string x n, relative to a universal computer U, is defined as KU(X n) min Ipu(x )l . pj(x n) In words, it is the length of the shortest program in the language of U; ie, in the set of its programs, that generates the string, [30], 13] The set of all programs for U may be ordered, first by the length and then the programs of the same length alphabetically. Hence, each program p has an index i(p) in this list. Importantly, this set has a generating grammar, which can be programmed in another universal computer s language ....

Solomonoff, R.J. (1964), 'A Formal Theory of Inductive Inference', Part I, Information and Control 7, 1-22; Part II, Information and Control 7, 224-254 68

....them. When we have to stop, we know that we have exhausted all the means provided by the studied models, and no one can teach us more about the data without proposing new models. The MDL principle has its roots both in Shannon s information theory [21] and in the theories of inductive inference [9, 22] and algorithmic complexity [2, 10] As we shall see, the shortest code length, relative to a class of models, also called the stochastic complexity, extends Shannon s information in a natural manner. And just as his famous noiseless coding theorem sets the mean information as the lower bound ....

.... principle of searching for the model or model class that permits the shortest encoding of the data, together with the model and the class, is called the MDL (Minimum Description Length) principle, 14, 15] Finally, although Shannon s theorem (together with the algorithmic notion of information [22, 10, 2]) did provide the initial inspiration for this principle it is crucial to compare the models by the code length for the actually observed data sequence rather than by the mean code length. This is because we can always make the mean as small as we wish with the following worthless model: Assign a ....

R.J. Solomonoff, A formal theory of inductive inference, parts I and II, Information and Control 7 (1964), 1--22, 224--254. IBM Almaden Research Center, San Jose, CA 95120-6099 E-mail address: rissanen@almaden.ibm.com Department of Computer Science, Princeton University, Princeton, NJ 08544 E-mail address: ristad@princeton.edu

....first, we have a linearly ordered sequence of symbols that must be extrapolated. In the second we want to extrapolate an unordered set of finite strings. A very general formal solution to the first kind of problem is well known and much work has been done in obtaining good approximations to it [1, 3, 4, 5, 6, 9, 10]. Though the second kind of problem is of much practical importance, no general solution has been published. We present two general solutions for unordered data. We also show how machines can be constructed to summarize sequential and unordered data in optimum ways. 1. INTRODUCTION In the ....

Solomonoff, R. J. (1964) A formal theory of inductive inference. Information Control, 7, 224--254.

No context found.

R. J. Solomonoff. A formal theory of inductive inference. part II. Information and Control, 7(2):224--254, 1964.

No context found.

R. J. Solomonoff. A formal theory of inductive inference. part I. Information and Control, 7(1):1--22, 1964.

No context found.

R.J. Solomonoff, `A Formal Theory of Inductive Inference', Information and Control, Vol. 2, pp. 101--112, 1959

No context found.

R. J. Solomonoff. A formal theory of inductive inference. Information and Control, 7:1--22, 224--254, 1964.

No context found.

R.J. Solomonoff, A formal theory of inductive inference, Information and Control 7 39 (1964), pp. 1--22, 224--254.

No context found.

R. J. Solomonoff. A formal theory of inductive inference. Information and Control, 7:1--22, 224--254, 1964.

No context found.

R.J. Solomonoff. A formal theory of inductive inference. Information and Control, 7(1,2):1--22, 224--254, 1964. Parts I and II.

No context found.

R. J. Solomonoff. A formal theory of inductive inference. Part I. Information and Control, 7:1--22, 1964.

No context found.

R. J. Solomonoff. A formal theory of inductive inference. Part I. Information and Control, 7:1--22, 1964.

No context found.

R. J. Solomonoff, "A formal theory of inductive inference," Inform. Contr., pt. 1 and 2, vol. 7, pp. 224--254, 1964.

No context found.

R. J. Solomonoff, "A formal theory of inductive inference," Inform. Contr., pt. 1 and 2, vol. 7, pp. 1--22, 1964.

No context found.

R. Solomonoff, A formal theory of inductive inference, Information and Control 7 (1964) 1--22, 234--254.

No context found.

R.J. Solomonoff, A formal theory of inductive inference, Part 1 and Part 2, Inform. Contr., 7(1964), 1-22, 224-254.

No context found.

Solomonoff, R. J. A formal theory of inductive inference. Inform. and Contr. 7 (1964), 1--22, 224--254.

No context found.

R.J. Solomonoff. A formal theory of inductive inference. Part I. Information and Control, 7:1-22, 1964.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC