Jaynes, E. T. 1979. Where do we stand on maximum entropy? In Levine, R. D., and Tribus, M., eds., The Maximum Entropy Formalism. Cambridge, MA.: MIT Press. 15--118.  Home/Search   Document Not in Database   Summary   Related Articles   Check

.... Here, f(r) is (usually) a monotonic function with r : ky Gamma yk 0, and is a Gaussian noise term with zero mean and standard deviation oe = N (0; oe (r) 2) Assuming a normal noise distribution may be regarded as an approximation of reality based on the maximum entropy principle [19]. Furthermore, it will simplify the derviations considerably. The pdf (probability density function) of the noise reads p( 3) Note, the noise strength oe can be a function of r, allowing for the modeling of relative measuring errors. We will evaluate the performance of sGAs ....

E. T. Jaynes. Where Do We Stand on Maximum Entropy? In R.D. Levine and M. Tribus, editors, The Maximum Entropy Formalism, pages 15--118, 1979.

....of problems. It is also possible to introduce additional approximation algorithms with further variations of the mini bucket technique, as in Iterative Join Graph propagation [5] It would also be interesting to nd the constraint satisfying probability distribution that has maximum entropy [9], rather than choosing one arbitrarily. ....

E. T. Jaynes. Where do we stand on maximum entropy? In The Maximum Entropy Formalism. M. I. T. Press, 1979.

....satisfactorily: Je#reys non informative prior distribution works only for location parameters, such as the mean value of a parameter. For other properties, such as e.g. the standard deviation, other ignorance prior distributions are needed. Jaynes s approaches to find ignorance priors are [15, 16, 17, 18]: 1. Invariance under transformations. If the only thing one knows about the system is a model or an hypothesis, this ignorance should not change if the mathematical representation of the model is transformed into an equivalent representation. For example, a uniform distribution on x does not ....

E. T. Jaynes. Where do we stand on Maximum Entropy? In R. D. Levine and M. Tribus, editors, The maximum entropy formalism, pages 15--118. MIT Press, 1978. Reprinted in [29, p. 211--314].

....it successfully for synthesis. The maximum entropy density is optimal in the sense that it does not introduce any constraints on the RF beyond those of equation (3) The form of the maximum entropy density may be derived by solving the constrained optimization problem using Lagrange multipliers [34]: P ( x) k e k k ( x) 4) where x 2 IR corresponds to a (vectorized) image, and the k are the Lagrange multipliers. The values of the multipliers must be chosen such that the density satis es the constraints given in equation (3) But the multipliers are generally a ....

E T Jaynes. Where do we stand on maximum entropy? In R. D. Levine and M. Tribus, editors, The Maximal Entropy Formalism. MIT Press, Cambridge, MA, 1978.

....et al. s method defines, for example, a feature as follows f i = 1 (p, n 2 ) is attached to n 1 in ( ice cream, with, chocolate) 0 otherwise. It then incrementally selects features, and e#ciently estimates the conditional distribution by using the Maximum Entropy Estimation technique (see (Jaynes, 1978; Darroch and Ratcli#, 1972; Berger, Pietra, and Pietra, 1996) Another method of the quadruple approach is to employ transformation based error driven learning (Brill, 1995) as proposed in (Brill and Resnik, 1994) This method learns and uses IF THEN type rules, where the IF parts represent ....

Jaynes, E. T. 1978. Where do we stand on maximum entropy? In R. D. Levine and M. Tribus, editors, The Maximum Entropy Formalism. MIT Press.

.... by expansions of a Gaussian (also used in ES theory, see Beyer [62, 63] The peculiarity of this approach is, however, that the underlying microscopic description level is bypassed using inference methods gleaned from statistical mechanics, especially the maximum entropy principle (Jaynes [64]) For an introduction into this interesting method as well as further references, the reader is referred to PrugelBennett and Rogers [65] and Shapiro [66] Reviewing the history one may conclude that the theory on evolutionary algorithms has tried to obtain too general statements or too precise ....

E. T. Jaynes. Where do we stand on maximum entropy? In R.D. Levine and M. Tribus, editors, The Maximum Entropy Formalism, pages 15--118, Cambridge, MA, 1979. MIT Press.

....will answer a few questions of Tony and other participants of the workshop on the situations where we can use Maximum Entropy or Bayesian approaches or even the cases where we can actually use both of them. INTRODUCTION Dice problems have been analyzed many times (See mainly Ed. Jaynes papers [1, 2, 3, 4] and also [5, 6, 7, 8, 9] but it seems that still many questions are open. In this note, I will try to answer some of them. Before starting, we need to set up precise notations and describe precisely the context. Let consider an imaginary die with faces ( is the ordinary die) where on ....

E. T. Jaynes, "Where do we stand on maximum entropy ?," in The Maximum Entropy Formalism, R. D. Levine and M. Tribus, Eds. M.I.T. Press, Cambridge (MA), 1978.

....compute MINF(r i ) c) Select r j with minimal MINF(r i ) d) If MINF(r j ) INF let (r j ) 0 else let (r j ) s j MINV(r j ) Gamma MINF(r j ) 3] Assign ranks to models using equation (2) 4] Check constraints (1) to verify this is an me valid ranking. Fig. 3. The me algorithm [5]. If one has to select a PD from all possible ones, choosing one other than that which has maximum entropy means making additional assumptions or implicitly assuming extra constraints. It would be useful therefore to be able to compare systems of default reasoning with the answers obtained from ....

....represent rational consequence relations [3] The me rankings differ because the different strengths change the default information being encoded. However, the me ranking corresponding to any given set of strengths represents the least biased estimate of the underlying probability distribution [5]. In contrast, the lex ordering is unique and fixed for a given set of defaults [7] It follows that the lex ordering implies some additional assumptions are being made about what default information represents and it is reasonable to ask what these might be. By showing that the lex ordering can ....

[Article contains additional citation context not shown here]

E. Jaynes. Where do we stand on maximum entropy? In R. Levine and M. Tribus, editors, The Maximum Entropy Formalism, pages 15--118, Cambridge, MA, 1979. MIT Press.

....and C on 2048 occasions . Wen discusses how to reduce sets of relations into fourth normal form, which correspond to the cliques of the equivalent belief network, and from which the necessary conditional probabilities may be learnt. He also discusses methods, based on the maximum entropy principle [70] for completing sets of conditional probabilities. The other major disadvantage with network based formalisms is the fact that they are inherently propositional. Consider Pearl s [113] seminal example about Mr Holmes and his burglar alarm. Either an earthquake or a burglary would cause the alarm ....

Jaynes, E. T. (1979) Where do we stand on maximum entropy?, in The maximum entropy formalism, (R. D. Levine and M. Tribus, eds), MIT Press, Cambridge MA.

.... principle for inductive inference [6, 8, 26, 16, 27, 5, 11, 20] It has been applied to statistical and machine learning problems ranging from protein modeling so stock market prediction [18] One of its characterizations (some would say justi cations ) is the so called concentration phenomenon [14, 15]. Here is an informal version of this phenomenon, in Jaynes words: If the information incorporated into the maximum entropy analysis includes all the constraints actually operating in the random experiment, then the distribution predicted by maximum entropy is overwhelmingly the most likely to ....

....conditional limit theorem 3 , which says that lim n 1 n t2N Q 1 ( j T (n) t) P 1 ( 6) where Q 1 ( j T (n) t) and P 1 ( refer to the marginal distribution of X 1 under Q( j T (n) t) and P respectively. 2 We are referring here to the version in [14]. The theorem in [15] extends this in a direction di erent from the one we consider here. 3 This theorem too has later been extended in several directions di erent from the one considered here [7] see Section 4.3. STRONG ENTROPY CONCENTRATION, CODING, GAME THEORY AND RANDOMNESS 7 Our ....

[Article contains additional citation context not shown here]

E.T. Jaynes. Where do we stand on maximum entropy? In R.D. Levine and M. Tribus, editors, The Maximum Entropy Formalism, pages 15-118. MIT Press, Cambridge, MA, 1978.

.... principle for inductive inference [5, 7, 24, 15, 25, 4, 10, 19] It has been applied to statistical and machine learning problems ranging from protein modeling so stock market prediction [17] One of its characterizations (some would say justi cations ) is the so called concentration phenomenon [13, 14]. Here is an informal version of this phenomenon, in Jaynes words: Also: CWI, Kruislaan 413, 1098 SJ Amsterdam. The author would like to thank Richard Gill and Phil Dawid for very useful discussions regarding resp. conditioning on measure 0 events and the concentration phenomenon. Part of ....

....1 ( 6) where Q 1 ( j T (n) t) and P 1 ( refer to the marginal distribution of X 1 under Q( j T (n) t) and P respectively. Our Results Both theorems above say that for some sets A, Q n (A j T (n) t) P n (A) 7) 2 We are referring here to the version in [13]. The theorem in [14] extends this in a direction di erent from the one we consider here. 3 This theorem too has later been extended in several directions di erent from the one considered here [6] see the discussion at the end of Section 4. In the concentration phenomenon, the set A X n is ....

[Article contains additional citation context not shown here]

E.T. Jaynes. Where do we stand on maximum entropy? In R.D. Levine and M. Tribus, editors, The Maximum Entropy Formalism, pages 15-118. MIT Press, Cambridge, MA, 1978.

.... for inductive inference [7, 8, 26, 16, 27, 5, 11, 20] It has been applied to statistical and machine learning problems ranging from protein modeling [19] to stock market prediction [6] One of its characterizations (some would say justi cations ) is the so called concentration phenomenon [14, 15]. Here is an informal version of this phenomenon, in Jaynes words: If the information incorporated into the maximum entropy analysis includes all the constraints actually operating in the random experiment, then the distribution predicted by maximum entropy is overwhelmingly the most likely to ....

....that (7) holds asymptotically in a much wider sense, namely for just about any set whose probability one may be interested in. For examples of such sets see Example 1. In Theorems 1, 2 and 3 we show that (7) indeed holds for a very large class of sets; 1 We are referring here to the version in [14]. The theorem in [15] extends this in a direction di erent from the one we consider here. moreover, we give an explicit indication of the error one makes if one approximates Q(A j T (n) t) by P (A) In this way we unify and strengthen both the concentration phenomenon and the ....

[Article contains additional citation context not shown here]

E.T. Jaynes. Where do we stand on maximum entropy? In R.D. Levine and M. Tribus, editors, The Maximum Entropy Formalism, pages 15-118. MIT Press, Cambridge, MA, 1978.

....of interest to each other, there need not be any rational choice since different completions will prefer each of these to the other. We might, however, call a choice reasonable in such circumstances if it is best according to some compatible probability and utility measures. Some authors (e.g. (Jaynes, 1979; Cheeseman, 1983) suggest the further alternative of using a maximum entropy principle to fill in gaps in probability distributions. 4.1.2 Inertia Standard treatments of decision theory assume that the beliefs of the agent persist over time in the absence of new information and that beliefs ....

Jaynes, E. T. 1979. Where do we stand on maximum entropy? In Levine and Tribus, editors, The Maximum Entropy Formalism. M.I.T. Press.

....of the inverse problem, not a solution of it. The main diculty is, in general, before the application of the Bayes formula, i.e. how to formulate appropriately the problem and how to assign the direct probabilities. The use of ME principle to assign the direct probabilities also is not new [19, 20, 21, 22, 23, 24, 25, 26], but using ME principle to assign the direct probabilities and then using the Bayesian estimation theory is less usual [10, 27, 15, 28, 29, 30] In fact, ME has also been used directly in image restoration and reconstruction [31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, ....

.... 4 When our prior knowledge is or can be assumed to be in the form of some constraints on the probability distribution function (pdf) of the unknown parameters then we can then use the ME principle, to choose one prior law between all the possible probability laws satisfying these constraints [21, 22, 24]. For now, suppose now that we are able to assign the probability densities p(yjx) and p(x) We can then use the Bayes rule to nd p(xjy) p(xjy) p(yjx) p(x) 3) and use the maximum a posteriori (MAP) estimation rule to give a solution to the problem, i.e. b x = arg max x fp(xjy)g = arg ....

E. Jaynes, \Where do we stand on maximum entropy ?," in The Maximum Entropy Formalism (R. Levine and M. Tribus, eds.), Cambridge (MA): M.I.T. Press, 1978.

....completely by creating a Maximum Entropy Modeling Toolkit that can train and evaluate ME models in a generic encoding, suitable for any domain. The model described in Section 4 is then mapped into this encoding. 2 3 Maximum Entropy Modeling 3. 1 Introduction The maximum entropy framework [7, 8] is a powerful method for building statistical models. It is expressive, allowing modelers to easily represent their special insights into the data generating machinery. It is statistically efficient, because it models the intersection of complex events without increasing the number of parameters ....

....(x; y) in P that maximizes the entropy H(p) with respect to all distributions in P. H(p) Gamma X x;y p(x; y) log p(x; y) The maximum entropy distribution p (x; y) is the one that is most faithful to our constraints, because it makes no additional assumptions beyond what has been specified [7, 8, 3]. We also need a compact way to represent this distribution, that is, we require a model for this distribution. 3.2.2 The Exponential Model Class Now consider the class R of all exponential models m(x; y) defined over our features G R = fm : m(x; y) r(x; y) Z g whose numerators r(x; y) ....

E. T. Jaynes. Where do we stand on maximum entropy? In R. D. Levine and M. Tribus, editors, The Maximum Entropy Formalism. MIT Press, Cambridge, MA, 1978.

....i ) and p(AjB i ) for all i = 1; 2; n. This requirement of a full specification of probability distribution is often difficult to meet in real systems. Several methods have been developed to cope with a lack of statistical information. A celebrated one is the Principle of Maximum Entropy [11, 12, 35]. The notion of entropy, widely used in physics, was considered by Hartley [9] and Wiener [37] in connection with information transmission, and introduced into Information Theory by Claude Shannon [30] Let P be a probability distribution assigning probabilities p1 ; pn to n mutually ....

....can be traced back some 100 years, to the works of Boltzmann, Maxwell, Gibbs, and Planck, and has been introduced in its modern form by Jaynes [11] and Tribus [35] proved its efficiency in various fields of Physics, Statistics, Information Theory, Pattern Recognition, Signal Processing, etc. [1, 8, 11, 12, 13, 14, 18, 25, 32, 33, 34, 35, 38]. To be applied successfully, most of the known approaches to reasoning with uncertain and incomplete information require an extensive statistical knowledge (e.g. prior and posterior probabilities for Bayesian methods, basic probability assignment in Dempster Shafer theory [3, 4, 29] prior ....

Jaynes, E. T. Where do we stand on maximum entropy? In The Maximum Entropy Formalism. Ed. by R. D. Levine and M. Tribus (Cambridge, MA: MIT Press, 1979) 15-118.

....so that no further overhead has to be paid. It has to be noted that, when the system does not have a deeper knowledge of data distribution besides attribute cardinalities, the assumption of uniform distribution is the logical choice, since it is in accordance with the maximum entropy principle [19]. The PSE problem under the (AI) and (UD) assumptions is, nowadays, believed to be correctly solved by the model of Gardy and Puech [12] The Gardy and Puech s (GP ) model under (AI) and (UD) assumptions arises as a special case of a more general model where only the (AI) assump 4 tion is ....

E. Jaynes. Where do we stand on maximum entropy? In Levine and Tribus, editors, The Maximum entropy Formalism, pages 15--118. MIT Press, Cambridge, MA, 1979.

....2; n. This requirement of a full specification of probability distribution is often difficult to meet in real systems. 2.1 Evidence versus probability Several methods have been developed to cope with a lack of statistical information. A celebrated one is the Principle of Maximum Entropy [35, 36, 84]. The notion of entropy, widely used in physics, was considered by Hartley [30] and Wiener [86] in connection with information transmission, and introduced into Information Theory by Claude Shannon [78] Let P be a probability distribution assigning probabilities p 1 ; p n to n mutually ....

....can be traced back some 100 years, to the works of Boltzmann, Maxwell, Gibbs, and 4 Planck, and has been introduced in its modern form by Jaynes [35] and Tribus [84] proved its efficiency in various fields of Physics, Statistics, Information Theory, Pattern Recognition, Signal Processing, etc. [1, 28, 29, 35, 36, 37, 38, 81, 84, 87]. To be applied successfully, most of the known approaches to reasoning with uncertain and incomplete information require an extensive statistical knowledge (e.g. prior and posterior probabilities for Bayesian methods, basic probability assignment in Dempster Shafer theory [15, 16, 77] prior ....

Jaynes, E. T., 1979, Where do we stand on maximum entropy? In The Maximum Entropy Formalism, Ed. by R. D. Levine and M. Tribus (Cambridge, MA: MIT Press) 15--118.

....used it successfully for synthesis. The maximum entropy density is optimal in the sense that it does not introduce any constraints on the RF beyond those of Eq. 3) The form of the maximum entropy density may be derived by solving the constrained optimization problem using Lagrange multipliers (Jaynes, 1978). P(#x) # # k e # k # k (#x ) 4) where # x #R L corresponds to a (vectorized) image, and the # k are the Lagrange multipliers. The values of the multipliers must be chosen such that the density satisfies the constraints given in Eq. 3) But the multipliers are generally a ....

Jaynes, E.T. 1978. Where do we stand on maximum entropy? In The Maximal Entropy Formalism, R.D. Levine and M. Tribus (Eds.). MIT Press: Cambridge, MA.

....probabilities induces an equivalent set of conditional probabilities over the same lattice, though it may be an unreasonably large set. If there are any independence relationships to be exploited, equivalently a subset of marginal probabilities can be omitted and the maximumentropy principle (Jaynes 1979) can be applied to reconstruct the joint distribution. The advantages of this formulation are: 1) fewer parameters are required since it does not encode redundant distributional information in multiple dependent conditional probabilities, 2) consistency is easier to maintain because the ....

JAYNES, E. T. 1979. Where do we stand on maximum entropy. In The maximum entropy formalism, ed. by R. D. Levine & M. Tribus. Cambridge, MA: MIT Press.

....it successfully for synthesis. The maximum entropy density is optimal in the sense that it does not introduce any constraints on the RF beyond those of equation (3) The form of the maximum entropy density may be derived by solving the constrained optimization problem using Lagrange multipliers [33]: P ( x) Y k e Gamma k OE k ( x) 4) where x 2 IR jLj corresponds to a (vectorized) image, and the k are the Lagrange multipliers. The values of the multipliers must be chosen such that the density satisfies the constraints given in equation (3) But the multipliers are generally a ....

E T Jaynes. Where do we stand on maximum entropy? In R. D. Levine and M. Tribus, editors, The Maximal Entropy Formalism. MIT Press, Cambridge, MA, 1978.

....some of the more in uential earlier approaches. Previous justi cations by logical probabilists of particular solutions to the prior problem have tended to concentrate on purely logical or analytic arguments. Thus, whatever their di erences such authors as Laplace [9] Carnap [1] or Jaynes [7] or [8], have all emphasised the fundamental r ole of symmetry arguments, of which the prototype was Laplace s principle of indi erence (cf. Rosenkrantz [16] for a history of the problem) Despite the obviously unsatisfactory nature of the solutions given by the logical probabilists, a rather curious ....

E. T. Jaynes. Where do we stand on maximum entropy ?. The Maximum Entropy Formalisation, R. D. Levine and M. Tribus, eds., MIT Press, Cambridge, MA, 1978.

....this encoding. The encoding for a particular domain is not necessarily a trivial matter, and reflects the structural complexity of the domain. Still, the separation of tasks makes the overall implementation effort much less cumbersome. 3 Maximum Entropy Modeling The maximum entropy framework [8, 9] is a powerful method for building statistical models. It is expressive, allowing modelers to easily represent their special insights into the data generating machinery. It is statistically efficient, because it models the intersection of complex events without increasing the number of parameters ....

.... y) in P that maximizes the entropy H(p) with respect to all distributions in P: H(p) Gamma X x;y p(x; y) log p(x; y) 9) The maximum entropy distribution p (x; y) is the one that is most faithful to our constraints, because it makes no additional assumptions beyond what has been specified [8, 9, 3]. We also need a compact way to represent this distribution. 3.1.2 The Exponential Model Class Now consider the class R of all exponential models m(x; y) defined over our features G R = fm : m(x; y) r(x; y) Z g (10) 5 whose numerators r(x; y) are a product of exponentials, ff i = e i , ....

E. T. Jaynes. Where do we stand on maximum entropy? In R. D. Levine and M. Tribus, editors, The Maximum Entropy Formalism. MIT Press, Cambridge, MA, 1978.

....additional assumptions as minimal and clear as possible. We therefore enrich the probability calculus by two principles which have their common source in the concept of model quantification ( 8, 17] and find their dense representation in the well known principle of Maximum Entropy (MaxEnt [6]) As model quantification delivers a precise semantics to MaxEnt, the corresponding decisions make sense not only in our current project of medical diagnosis in Lexmed. 1 Introduction Probabilities deliver a well researched method of reasoning with uncertain knowledge. But if we express all our ....

....and independence ( 16, 18] ffl minimizes the amount of additional information, necessary to answer a query, as it chooses the model with minimal information content. 1 Accepted at SOR99 (Symposium on Operations Research) Magdeburg September Computing the MaxEnt Model is not a new idea ([6], for further insights into MaxEnt see also the axiomatic approaches [11, 19] but very expensive in the worst case. The main problem is that the number of interpretations (elementary events) grows exponentially with the number of variables. To avoid this effect in the average case, the ....

E.T. Jaynes. Where do we stand on Maximum Entropy? In R.D. Rosenkrantz, editor, Papers on Probability, Statistics and Statistical Physics, pages 210--314. Kluwer Academic Publishers, 1978.

....Because there is no unique best answer, this is a matter of major importance. There has been a great deal of work that can be viewed as attempts to generate degrees of belief given a database. Besides the work on reference classes mentioned above, much of Jaynes s work on maximum entropy [Jay78] can be viewed in this vein. Perhaps the work closest in spirit to ours is that of Carnap [Car52] Johnson [Joh32] and the more recent work of Paris and Vencovska [PV89, PV91] and Goodwin [Goo92] We compare our work with these others in some detail in the full paper; here we can only give brief ....

E. T. Jaynes. Where do we stand on maximum entropy? In R. D. Levine and M. Tribus, editors, The Maximum Entropy Formalism, pages 15--118. MIT Press, Cambridge, MA, 1978.

....and language modeling is used to find a good distribution that satisfies all constraints. According to the maximum entropy principle [12, 2, 1] the best distribution should have maximum entropy under these constraints since we should make no additional assumptions other than what we know [5, 1]. To get topic sensitive features, we first classify the training data according to topic and extract from the data for each topic words whose frequencies within the topic deviate significantly from the overall distributions. We 1 Casual conversations on assigned topics recorded over ....

E. Jaynes, Where do we stand on maximum entropy? In R. Levine and M. Tribus editors, The Maximum Entropy Formalism. MIT Press, Cambridge, MA, 1978

....concludes strictly stronger (see Figure 6) than the two principles combined. Both seem to be different from MaxEnt at first glance, and although they seem to be well known for a long time, they are far from clear when one looks at them in more detail: ffl The principle of Indifference, viewed by [Jay89] as a simple demand of consistency , is sometimes mixed with the problem of modelling probabilities; this leads to arguments against this principle. Therefore we have to specify how we use this principle, especially in the presence of linear constraints. ffl The principle of Independence is ....

....decisions with incomplete knowledge (e.g. the desired conclusions of the last example DB 2 ) This will be done in the next sections. 6.4.4 Conclusions on P models with Indifference What does Indifference mean The history of this famous principle goes back to Laplace and Keynes. Let us quote [Jay89] for a short and informal version of this principle: If the available evidence gives us no reason to consider proposition a 1 either more or less likely than a 2 , then the only honest way we can describe that state of knowledge is to assign them equal probabilities: P (a 1 ) P (a 2 ) Three ....

E.T. Jaynes. Where Do We Stand on Maximum Entropy ? In R.D. Rosenkrantz, editor, E.T. Jaynes: Papers on Probability, Statistics and Statistical Physics, pages 210--314. Kluwer Academic Publishers, 1989.

....is characterized by a family of probability functions bounded by such probability intervals. Inference using interval valued probability may be carried out in different ways. For example, one may convert probability intervals into a single probability function by using the maximumentropy principle [6]. Alternatively, one may consider probability intervals as constraints on the set of probability functions representing the uncertainty of the propositions. A probabilistic inference process is formulated as a constraint propagation [11] An advantage of the latter approach is that no ad hoc ....

Jaynes, E.T. Where do we stand on maximum entropy? In R.D. Levine and M. Tribus (Eds.), The Maximum Entropy Formalism, The MIT Press, Cambridge, Massachusetts, 1979.

....our belief in a specific proposition. Its value most often will be justified only by a subjective assessment of likelihood . This dual use of the term probability has caused a lot of controversy over what the true meaning of probability is: a measure of frequency, or of subjective belief (e.g. [Jay78]) A comprehensive study of both aspects of the term is [Car50] More recently, Bacchus has developed a probabilistic extension of firstorder logic that accomodates both notions of probability [Bac90] Now that we have stressed the differences in assigning a probability to subsets of a general ....

E.T. Jaynes. Where do we stand on maximum entropy? In R.D. Levine and M. Tribus, editors, The Maximum Entropy Formalism, pages 15--118. MIT Press, 1978.

....interpretation of the meaning of a probability, are intimately intertwined. The reader is referred to [Fin73, Hac65] for more in depth discussions of these issues. The Principle of Maximum Entropy: At the end of the 19th century, primarily as a result of the work of Maxwell, Boltzmann and Gibbs [Jay79], the area of Statistical Mechanics was born. As a consequence, the entropy of a physical system became associated with a probability distribution of the phase space of possible atomic configurations. In 1948, Claude Shannon published The Mathematical Theory of Communication and established the ....

E. T. Jaynes. Where do we stand on maximum entropy. In Raphael D. Levine and Myron Tribus, editors, The Maximum Entropy Formalism, pages 15--118, Cambridge, Massachusetts, May 1979. MIT Press.

....N ( jKB) assuming the limit exists. Pr rw 1 ( jKB) is the degree of belief in given KB according to the random worlds method. 2. 3 Maximum entropy and cross entropy The entropy of a probability distribution over a finite space Omega is Gamma P 2 Omega ( ln( It has been argued [Jay78] that entropy measures the amount of information in a probability distribution, in the sense of information theory. The uniform distribution has the maximum possible entropy. In general, given some constraints on the probability distributions, the distribution with maximum entropy that satisfies ....

E. T. Jaynes. Where do we stand on maximum entropy? In R. D. Levine and M. Tribus, editors, The Maximum Entropy Formalism, pages 15--118. MIT Press, Cambridge, Mass., 1978.

....the identity of an object given the value of each of its attributes. In this section we describe Shastri s solution to this problem [Shastri 1985] The solution is based on the concept of maximum entropy, a notion that is fundamentally related to information theory and statistical mechanics [Jaynes 1979] and also bears resemblance to statistical methods developed for error estimation and hypothesis testing. In general, an agent s knowledge about a concept A may be represented as an n dimensional matrix where n = j(A)j. Each dimension of A corresponds to an applicable property and the marginals ....

E. T. Jaynes. Where do we stand on maximum entropy. In R. D. Levine and M. Tribus, editors, The Maximum Entropy Formalism. The MIT Press, Cambridge, Massachusetts, 1979.

No context found.

Jaynes, E. T. 1979. Where do we stand on maximum entropy? In Levine, R. D., and Tribus, M., eds., The Maximum Entropy Formalism. Cambridge, MA.: MIT Press. 15--118.

No context found.

E. T. Jaynes. Where do we stand on maximum entropy? In R. D. Levine and M. Tribus, editors, The Maximum Entropy Formalism. MIT Press, Cambridge, 1978.

No context found.

E. T. Jaynes. Where do we stand on maximum entropy? In The Maximum Entropy Formalism, pages 15---118, Cambridge MA, 1979. MIT Press.

No context found.

Jaynes, E. T. 1979 Where do we stand on maximum entropy? In The maximum entropy formalism (ed. R. D. Levine & M. Tribus), pp. 15-118. Cambridge, MA: MIT Press.

No context found.

Jaynes, E. T., "Where do we stand on maximum entropy," in The Maximum Entropy Formalism, edited by R. D. Levine and M. Tribus, The MIT Press, Cambridge, 1979, pp. 15--118.

No context found.

Jaynes E.T. Where do we stand on maximum entropy? In The Maximum Entropy Formalism (eds. R. D. Levine & M. Tribus), pp. 15--118, Cambridge:MIT Press, 1979.

No context found.

Jaynes, E. T., "Where do we stand on maximum entropy," in The Maximum Entropy Formalism, edited by R. D. Levine and M. Tribus, The MIT Press, Cambridge, 1979, pp. 15--118.

No context found.

E. T. Jaynes, "Where Do We Stand on Maximum Entropy?," in The Maximum Entropy Formalism,R. D. Levine and M. Tribus (eds.), M. I. T. Press, Cambridge, MA,, p. 15, 1978.

No context found.

E. T. Jaynes. Where do we stand on maximum entropy? In The Maximum Entropy Formalism. M. I. T. Press, 1979.

No context found.

E. T. Jaynes, "Where do we stand on maximum entropy?," in The Maximum Entropy Formalism,R. D. Levine and M. Tribus (eds.), M. I. T. Press, Cambridge, MA,, p. 15, 1978.

No context found.

E. T. Jaynes, "Where Do We Stand on Maximum Entropy?," in The Maximum Entropy Formalism,R. D. Levine and M. Tribus (eds.), M. I. T. Press, Cambridge, MA,, p. 15, 1978.

No context found.

E. T. Jaynes, "Where Do We Stand on Maximum Entropy?," in The Maximum Entropy Formalism,R. D. Levine and M. Tribus (eds.), M. I. T. Press, Cambridge, MA,, p. 15, 1978.

No context found.

E. T. Jaynes. Where do we stand on maximum entropy? In The Maximum Entropy Formalism. M. I. T. Press, 1979.

No context found.

E. T. Jaynes, Where do we stand on maximum entropy?, in The MaximumEntropyFormalism, R. D. Levine and M. Tribus, eds., MIT Press, Cambridge, Mass., 1978, pp. 15--118.

No context found.

E. T. Jaynes. Where do we stand on maximum entropy? In R. D. Levine and M. Tribus, editors, The Maximum Entropy Formalism, pages 15--118. MIT Press, Cambridge, Mass., 1978.

No context found.

E. T. Jaynes, " Where do we Stand Maximum Entropy?," in The Maximum Entropy Formalism, Ed. R. Levine and M. Tribus ( MIT, Boston, 1979).

No context found.

E. T. Jaynes. Where do we stand on maximum entropy? In R. D. Levine and M. Tribus, editors, The Maximum Entropy Formalism, pages 15--118. MIT Press, Cambridge, MA, 1978.

No context found.

Jaynes, E.T. (1979) Where do we stand on maximum entropy? In The Maximum Entropy Formalism Ed. R.D. Levine and M. Tribus, MIT Press, Cambridge, pp. 15-118.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC