D.V. Lindley. On the measure of information provided by an experiment. The Annals of Mathematical Statistics, 27(4):986--1005, 1956.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....experiments. 1 Introduction There are various measures of the information content of a statistical experiment E, among others the decision theoretic deficiency distance to the most informative experiment and the Shannon capacity (which was introduced in the statistical context by D.V. Lindley [9] and J.M. Bernardo [1] These numbers are not easily computed. Therefore it is desirable to describe at least their asymptotic behaviour when the experiments get more and more informative. To our knowledge the asymptotics of the Shannon capacity has not been studied for finite parameter ....

....f (E; is the value m E (f ) of the conical measure m E associated with the experiment E at the positively homogeneous function f : z 7 f( 1 z 1 ; n z n ) cf. 7] ch.3) So in this sense our paper deals with large deviations of conical measures . Extending the approach of D.V. Lindley [9] from entropy to general similarities one introduces the expected amount of information (measured in terms of the functional f) which one gains by passing from the prior to the posterior distribution. This corresponds to the transmission rate in Shannon theory. The maximal information gain ....

D.V. Lindley. On a measure of the information provided by an experiment. Ann. Math. Stat., 27:986--1005, 1956.

....rather than making the Kullback Leibler distance zero, as in the case of sucient statistics, MI statistics are found at local minima of the Kullback Liebler distance viewed as a functional of the statistic. This demonstrates how the approach of this paper generalizes that performed by Lindley [4]. 6 MI Statistics for the Gaussian distribution This section details the inference of the one dimensional MI statistic for the one dimensional Gaussian distribution. We take the position parameter of the Gaussian to be q, and the the goal is to nd R (x) so that (19) holds. From there note ....

D. V. Lindley. On a measure of the information provided by an experiment. Annals of Mathematical Statistics, 27:986-1005, 1961.

....problem with squared error loss. designing observation times for interval censored data 449 3. Information Theoretic Loss Information theory offers general measures of the amount of information gained about a parameter from experimentation. Information theoretic criteria have been introduced by Lindley (1956) and are widely used in experimental design. Lindley s approach is the method of choice for measuring the information contained in an experiment, when the purpose of the investigation is not tied to a specific decision Verdinelli (1992) Parmigiani and Berry (1994) Traditional information ....

Lindley, D. V. (1956), On a measure of the information provided by an experiment. Ann. Math. Statist., 27, 986--1005.

....from which the sample is drawn remains unknown. Hence it must be the case that H(p) H(p 0 ) and H(p) H(p 1 ) The excess of H(p) over 0 H(p 0 ) 1 H(p 1 ) is the average gain of information one can expect about the distribution itself. This quantity, called the mutual information [60, 61], J(p 0 ; p 1 ; 0 ; 1 ) H( 0 p 0 1 p 1 ) 0 H(p 0 ) 1 H(p 1 ) 2.101) is the natural candidate for distinguishability that we seek in this section. If the two distributions p 0 (b) and p 1 (b) are completely distinguishable, then all the information gained in a sampling ....

D. V. Lindley, \On a measure of the information provided by an experiment," The Annals of Mathematical Statistics, vol. 27, pp. 986-1005, 1956.

.... 1991) Therefore, we have established that Rn (w) I w ( Theta; X n ) 56) 39 The quantity I w measures the average amount of information contained in the data X n about the parameter Theta and has been used to measure information in a statistical context by Lindley as early as 1956 (cf. Lindley, 1956). Let R Gamma n denote the worst case minimal Bayes redundancy among all priors w: R Gamma n = sup w Rn (w) 57) This quantity also carries with it an information theoretic interpretation. Here, R Gamma n is referred to as the channel capacity, C ( Theta; X n ) Following Cover and ....

Lindley, D. V. (1956). On a measure of the information provided by an experiment. Ann. Math. Statist., 27, 986--1005.

....and Thomas, 1991#. Therefore, wehave established that Rn #w#=Iw ##; X n #: #56# 39 The quantity I w measures the average amount of information contained in the data X n about the parameter # and has been used to measure information in a statistical context by Lindley as early as 1956 #cf. Lindley, 1956#. Let R , n denote the worst case minimal Bayes redundancy among all priors w: R , n = sup w Rn #w#: #57# This quantity also carries with it an information theoretic interpretation. Here,R , n is referred to as the channel capacity,C##; X n #. Following Cover and Thomas #1991#, weenvision ....

Lindley, D. V. #1956#.On a measure of the information provided by an experiment. Ann. Math. Statist., 27, 986#1005.

....have been developed by di erent authors, rst Oehlert, then Nychka and Saltzman, and nally the approach developed in a series of papers by Sun, Le and Zidek. 6.1. A Bayesian formulation of optimal design An early discussion of the design of experiments from a Bayesian point of view was given by Lindley (1956). Lindley proposed a criterion which amounts to maximization of the expected Shannon information to be gained from an experiment, when the objective 501 is not to reach decisions but rather to gain knowledge about the world . Later Bernardo (1979) showed that this criterion can also be derived ....

....density of X will be denoted p X (x) R p(x j ) d . Since will not enter the following discussion except through , henceforth we write ( instead of ( The posterior density of given data X = x, evaluated at = will be ( j x) Bernardo (1979) following earlier work by Lindley (1956), proposed the following measure of the information contained in E when the prior density is ( IfE; g = Z p X (x) Z ( j x) log ( j x) d dx: 6:1) Given that the inner integral is essentially an information divergence between the prior and posterior densities, the expression ....

Lindley, D.V. (1956), On a measure of the information provided by an experiment.

.... 1991) Therefore, we have established that Rn (w) I w ( Theta; X n ) 59) The quantity I w measures the average amount of information contained in the data X n about the parameter Theta and has been used to measure information in a statistical context by Lindley as early as 1956 (cf. Lindley, 1956). Let R Gamma n denote the worst case minimal Bayes redundancy among all priors w: R Gamma n = sup w Rn (w) 60) This quantity also carries with it an information theoretic interpretation. Here, R Gamma n is referred to as the channel capacity, C ( Theta; X n ) Following Cover and ....

Lindley, D. V. (1956). On a measure of the information provided by an experiment. Ann. Math. Statist., 27, 986--1005.

....unifies the objectives of exploration and exploitation into a single objective, thereby providing a quantitative principle for managing this trade off. We are not aware of any previous attempts of this nature, although there has been extensive research into objective functions for exploration [15], and many techniques for managing the exploration exploitation tradeoff have been invented [20, 16] One of our current research directions is to place some of these methods into the general context. Here we have set out only the first steps of this approach, and demonstrated that it does indeed ....

D. V. Lindley. On a measure of the information provided by an experiment. Ann. Math. Statist., 27:986--1005, 1956.

....design is to select experiments in a way that their outcomes, which correspond to labels, give sufficient information for constructing a hypothesis that maximizes some criterion of accuracy. One natural criterion is the accuracy with which the parameters that define the hypothesis can be estimated [Lindley, 1956]. In the context of Bayesian estimation a very general measure of the quality of a query is the reduction in the probability of the set of possible hypotheses that is induced by the answer to the query. Similar suggestions have been made in the perceptron learning literature[Kinzel and Ruj an, ....

D. V. Lindley. On a measure of the information provided by an experiment. Ann. Math. Statist., 27:986--1005, 1956.

....X1 0.0 0.6 Figure 8: a) Maximum entropy designs for p = 2, n = 1 16, and the Gaussian correlation function with = 0:5; 0:5) the ultimate goal of the computer experiment, the first design block might be utilized to refine the design and reduce the design space. 5. 1 Entropy designs Lindley [31] introduced a measure, based upon Shannon s entropy [70] of the amount of information provided by an experiment. This Bayesian measure uses the expected reduction in entropy as a design criterion. This criterion has been used in Box and Hill [9] and Borth [6] for model discrimination. Shewry and ....

Lindley, D.V. On a measure of the information provided by an experiment. Annuals of Mathematical Statistics, 27:986--1005, 1956.

....design is to select experiments in a way that their outcomes, which correspond to labels, give sufficient information for constructing a hypothesis that maximizes some criterion of accuracy. One natural criterion is the accuracy with which the parameters that define the hypothesis can be estimated [Lin56]. In the context of Bayesian estimation a very general measure of the quality of a query is the reduction in the probability of the set of possible hypotheses that is induced by the answer to the query. Similar suggestions have been made in the perceptron learning literature[KR90] A different ....

D. V. Lindley. On a measure of the information provided by an experiment. Ann. Math. Statist., 27:986--1005, 1956.

....instance by making a so called query to a teacher and add the example to the training set. The goal here is to minimize the number of queries made to the teacher. This framework has also been studied by Cohn [ Cohn et al. 1990 ] and is related to the problem of sequential experimental design [ Lindley, 1956; Fedorov, 1972; Atkinson and Donev, 1992 ] Seung et al. Seung et al. 1992 ] suggest a general instance selection method, which they call Query by Committee (QBC) Their method can be applied when there exists a distribution measure over the concept class. This learning framework is usually ....

Lindley, D. V. 1956. On a measure of the information provided by an experiment. Ann. Math. Statist.

....has several interpretations in information theory and statistics, here it will be used as a risk functional in a parameter estimation context. In 1956 D.V. Lindley adapted the concept of the information theoretic transmission or information rate to the theory of statistical experiments, see [Lin56]. An experiment, i.e. a family of probability distributions fP : 2 Thetag over an observation space X , is considered as a Shannon information transmitting channel, the true parameter being the unknown character sent, and the data being the characters observed after some trials. A prior s ....

D.V. Lindley. On a Measure of the Information provided by an Experiment. Ann. Math. Stat., 27:986--1005, 1956.

....experiments. 1 Introduction There are various measures of the information content of a statistical experiment E, among others the decision theoretic deficiency distance to the most informative experiment and the Shannon capacity (which was introduced in the statistical context by D.V. Lindley [9] and J.M. Bernardo [1] These numbers are not easily computed. Therefore it is desirable to describe at least their asymptotic behaviour when the experiments get more and more informative. To our knowledge the asymptotics of the Shannon capacity has not been studied for finite parameter ....

....f (E; is the value m E (f ) of the conical measure m E associated with the experiment E at the positively homogeneous function f : z 7 f( 1 z 1 ; n z n ) cf. 7] ch.3) So in this sense our paper deals with large deviations of conical measures . Extending the approach of D.V. Lindley [9] from entropy to general similarities one introduces the expected amount of information (measured in terms of the functional f) which one gains by passing from the prior to the posterior distribution. This corresponds to the transmission rate in Shannon theory. The maximal information gain ....

D.V. Lindley. On a measure of the information provided by an experiment. Ann. Math. Stat., 27:986--1005, 1956.

....divergence between the marginal prior and the marginal posterior on the treatment effects. The design criterion is not crucial to the example, except for being notoriously expensive computationally (Carlin and Polson 1992, M ller and Parmigiani 1994) The design criterion was introduced by Lindley (1956), based on information theoretic foundations. DeGroot (1984) discusses properties of I, and shows how it can be interpreted as a gain in utility for the terminal decision problem of reporting a posterior distribution. Formally, the preposterior information measure is given by U(d) Z Y ....

Lindley, D.V. (1956), "On the measure of information provided by an experiment, " Annals of Statistics 27, 986-1005.

....design is to select experiments in a way that their outcomes, which correspond to labels, give sufficient information for constructing a hypothesis that maximizes some criterion of accuracy. One natural criterion is the accuracy with which the parameters that define the hypothesis can be estimated (Lindley,1956). In the context of Bayesian estimation a very general measure of the quality of a query is the reduction in the entropy of the posterior distribution that is induced by the answer to the query. Similar suggestions have been made in the perceptron learning literature(Kinzel Ruj an,1990) A ....

D. V. Lindley. On a measure of the information provided by an experiment. Ann. Math. Statist., 27:986--1005, 1956.

....rate coincide, where the former is measured by the minimal Bayes risk or equivalently by the deficiency distance to the most informative experiment and the latter is the rate of the entropy risk under the optimal ( reference ) prior. This concept has been studied in Bayesian analysis starting with [Lin56] and [Ber79] Our main results in this paper are stated in sections 3, 4 and 6. In analogy to Chernoff we show (Theorem 1) for the simple alternative of two irreducible transition matrices 0 ; 1 with the same zeroes that the rate of the risk R n after n observations is given by lim n 1 n p ....

D.V. Lindley. On a Measure of the Information provided by an Experiment. Ann. Math. Stat., 27:986--1005, 1956.

....put forward to give a rational foundation to the use of information theoretic measures such as Shannon s information. One especially insightful exposition on this matter is DeGroot (1984) The concept has also been used to construct a Bayesian theory of comparison of experiment and optimal design (Lindley, 1956, 1957) to construct diagnostics measures, to derive prior distributions that put as much weight as possible on the data (Zellner 1977, Bernardo 1979) and in many other ways. Yet, in applications, the direct use of information theoretic measures is not as widespread as their theoretical ....

....C(D; y)dP; 1) where KL is the Kullback Leibler divergence between prior and posterior as defined in the previous section, mD (y) denotes the marginal distribution of y (under design choice D) and C(D; y) is the sampling cost associated with design D, expressed in units of information. See Lindley (1956), DeGroot (1984) for a discussion of the properties of U and of the terminal decision problem that determines this choice. See Parmigiani and Berry (1993) for applications of such criteria to the design of clinical experiments. Finding the optimal design in this context involves for each D, ....

Lindley, D.V. (1956), On the measure of information provided by an experiment, Annals of Statistics 27, pp. 986-1005.

.... and Thomas, 1991) Therefore, we have established that R n (w) I w (Q;X n ) 50) The quantity I w measures the average amount of information contained in the data X n about the parameter Q and has been used to measure information in a statistical context by Lindley as early as 1956 (cf. Lindley, 1956). Let R Gamma n denote the worst case minimal Bayes redundancy among all priors w: R Gamma n = sup w R n (w) 51) This quantity also carries with it an information theoretic interpretation. Here, R Gamma n is referred to as the channel capacity, C(Q;X n ) Following Cover and Thomas ....

Lindley, D. V. (1956). On a measure of the information provided by an experiment. Annals of Mathematical Statistics, 27, 986-1005.

....design is to select experiments in a way that their outcomes, which correspond to labels, give sufficient information for constructing a hypothesis that maximizes some criterion of accuracy. One natural criterion is the accuracy with which the parameters that define the hypothesis can be estimated [Lin56]. In the context of Bayesian estimation a very general measure of the quality of a query is the reduction in the entropy of the posterior distribution that is induced by the answer to the query. Similar suggestions have been made in the perceptron learning literature[KR90] A different ....

D. V. Lindley. On a measure of the information provided by an experiment. Ann. Math. Statist., 27:986--1005, 1956.

....is natural to continue to call it fidelity . 3.4.4 Shannon distinguishability Now we come to the last, and maybe most important notion of distinguishability. Mutual information, as defined by Shannon [SH48] can also be used as a distin 66 guishability measure between probability distributions [LI56, BE82]. Recall that the (Shannon) entropy function H is defined as H(X) def = Gamma X x2X Prob[X=x] lg Prob[X=x] 3.18) where lg denotes the logarithm function with base 2. The argument of H can be either a stochastic variable or a probability distribution. A special case is the function H 2 ....

LINDLEY, D. V., "On a measure of the information provided by an experiment", Ann. Math. Statist. 27 (1956), pp. 986--1005.

....of Lindley Information Measure to the Design of Clinical Experiments Giovanni Parmigiani Donald A. Berry Institute of Statistics and Decision Sciences, Duke University Summary In a celebrated work, Lindley (1956) introduced a measure of the information provided by an experiment. In this paper we consider applications of Lindley information measure to the design of clinical experiments. We review the decision theoretic foundations underlying the use of Lindley information, and discuss its role in ....

....in utility can actually be used as a quantitative measure of the worth of an experiment in any given situation. This idea is about as old as Bayesian statistics (see Ramsey, 1990) and is discussed by Raiffa and Schlaifer (1961) and DeGroot (1984) The well known measure of information proposed by Lindley (1956) is the object of investigation in this paper. It can be seen as a very important special case of this general approach. Consider the decision problem of reporting a distribution regarding an unknown quantity Theta with values in Omega Gamma This is a Bayesian way of modelling a situation in ....

[Article contains additional citation context not shown here]

Lindley, D.V. (1956) On the measure of information provided by an experiment, Annals of Statistics 27, pp. 986-1005.

No context found.

D.V. Lindley. On the measure of information provided by an experiment. The Annals of Mathematical Statistics, 27(4):986--1005, 1956.

No context found.

Lindley, D. V. (1956). On a measure of the information provided by an experiment. Ann. Math. Statist., 27, 986--1005.

No context found.

Lindley D.V. (1956). On the measure of information provided by an experiment. Ann. Math. Statist. Vol. 27, pp. 986--1005.

No context found.

Lindley, D. V. 1956 On the measure of information provided by an experiment. Ann. Math. Statist. 27, 986-1005.

No context found.

Lindley D.V. On the measure of information provided by an experiment. Ann. Math. Statist. Vol. 27, pp. 986--1005, 1956.

No context found.

Lindley, D. V. (1956), `On a measure of information provided by an experiment', Ann. Math. Stat., 29: 986-1005.

No context found.

D.V. Lindley. On a Measure of the Information provided by an Experiment. Ann. Math. Stat., 27:986--1005, 1956.

No context found.

D. V. Lindley. On the measure of information provided by an experiment. Ann. Stat., 27:986--1005, 1956.

No context found.

Lindley, D. V. (1956), \On a measure of information provided by an experiment", Ann. Math. Stat., 29, pp. 986-1005.

No context found.

Lindley, D. V. (1956), "On a measure of information provided by an experiment", Ann. Math. Stat., 29, pp. 986-1005.

No context found.

Lindley, D. V. (1956), `On a measure of information provided by an experiment', Ann. Math. Stat., 29: 986-1005.

No context found.

Lindley, D. V. (1956), "On a measure of information provided by an experiment", Ann. Math. Stat., 29, pp. 986-1005.

No context found.

Lindley, D. V. (1956), \On a measure of information provided by an experiment", Ann. Math. Stat., 29, pp. 986-1005.

No context found.

Lindley, D. (1956). On a Measure of Information provided by an experiment. Ann. Math. Stat. 27, 986-996.

No context found.

Lindley, D.V. (1956), On a measure of the information provided by an experiment.

No context found.

Dennis Lindley. On a measure of the information provided by an experiment. Annals of Mathematical Statistics, 27:986--1005, 1956.

No context found.

D. V. Lindley. On a measure of the information provided by an experiment. Ann. Math. Statist., 27:986--1005, 1956.

No context found.

D. V. Lindley. On a measure of the information provided by an experiment. Annals of Mathematical Statistics, 27(4):986--1005, December 1956.

No context found.

Lindley, D. V. (1956). On the measure of information provided by an experiment. Ann. Math. Statist. 27, 986--1005.

No context found.

D.V. Lindley (1956). `On a measure of the information provided by an experiment', Ann. Math.

No context found.

Lindley, D.V. (1956), "On a measure of information provided by an experiment, " Ann. Math. Statist., 27, pp. 986-1005.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC