Madigan, D., Raftery, A., and Hoeting, J. (1997). Bayesian model averaging for linear regression models. J. Amer. Statist. Assoc., 92, 179--191.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....where they are combined. Ensemble learning [3] is often used as a means of combining models built at geographically distributed sites. Methods for combining models in an ensemble include voting schemas [3] meta learning [11] knowledge probing [7] Bayesian model averaging and model selection [10], stacking [12] mixture of experts [13] etc. Several systems for analysis of distributed data havebeen developed in recentyears. These include the JAM system developed by Stolfo et al. [11] the Kensington system developed by Guo et al. [7] and BODHI developed by Kargupta et al. [8] 9] These ....

A. E. Raftery, D. Madigan, and J. A. Hoeting. Bayesian model averaging for linear regression models. Journal of the American Statistical Association, 92:179-191, 1996.

....as the desired output [9] Multiple models, or what are often called ensembles or committees of models, have been used for quite a while in (centralized) data mining. A variety of methods have been studied for combining models in an ensemble, including Bayesian model averaging and model selection [15], stacking [20] partition learning [4] and other statistical methods, such as mixture of experts [21] JAM employs meta learning, while Kensington employs knowledge probing. Papyrus is designed to support different data, task and model strategies. For example, in contrast to JAM and Kensington, ....

A. E. Raftery, D. Madigan, and J. A. Hoeting, 1996. Bayesian Model Averaging for Linear Regression Models, Journal of the American Statistical Association, Volume92, pages 179--191, 1996.

....allow for exible nonlinear functions and interactions in these variables. Because of the large number of possible models, we use Stochastic Search Variable Selection (SSVS) George and McCulloch 1993, 1997, Smith and Kohn 1996) to identify important models, and use Bayesian Model Averaging (BMA) (Raftery, Madigan, and Hoeting 1997, Clyde 1999) to incorporate uncertainty about which predictor variables should be incorporated into the model. Rather than selecting a single model to make predictions, as is common practice, predictions under BMA are based on a weighted average of several models, where weights are based on the ....

....1999) However, in the conjugate framework, the posterior model probabilities are known up to the normalizing constant. Using this additional information, we can alternatively estimate the posterior model probabilities by renormalizing over the models in the sample (Clyde et al. 1996, Clyde 1999, Raftery et al. 1997). A simulation consistent estimate of the posterior model probabilities obtained by renormalizing the posterior model probabilities (RPP) within the set of sampled models is d p( jY ) RPP = 1 c) q =2 S( n=2 p( P 0 2S (1 c) q 0 =2 S( 0 ) n=2 p( 0 ) 10) for ....

[Article contains additional citation context not shown here]

Raftery, A. E., Madigan, D. and Hoeting, J. A., (1997), \Bayesian Model Averaging for Linear Regression Models," Journal of the American Statistical Association 92, 179-191.

....one for scienti c reporting. On the other hand, in many applications there are quantities of interest, like predictive inference for future patients, that do not depend on a particular model and might be averaged across models. General issues of model averaging and model selection are discussed in Raftery et al. 1997), Wasserman (1997) and Clyde (1999) Hoeting et al. 1999) provide a recent tutorial on Bayesian model averaging. Let M = f1; 2; Mg denote the set of indeces representing all models under consideration and assume that is an outcome of interest, such as the future pro le of a new patient ....

Raftery, A.E., Madigan, D. and Hoeting, J.A. (1997) Bayesian model averaging for linear regression models. Journal of the American Statistical Association, 92, 179-191.

....and thus greedy algorithms frequently get stuck in local maxima, not nding the optimal model(s) Stochastic algorithms have a clear advantage here. This section presents an algorithm, Bayesian Random Searching (BARS) which is motivated by Markov chain Monte Carlo Model Composition (MC 3 ) (Raftery, Madigan, and Hoeting 1997). The basic idea of MC 3 is to perform MCMC on the model space (instead of the parameter space, as usual) thus estimating the posterior probabilities of the models by the fraction of time the chain spends visiting each model. This can be accomplished using the Metropolis algorithm to move ....

Raftery, A. E., Madigan, D., and Hoeting, J. A. (1997). \Bayesian Model Averaging for Linear Regression Models." Journal of the American Statistical Association, 437, 179-191.

....of the predictive distributions from each model, the weights being determined from the posterior probabilities of each model. This is known as Bayesian model averaging, and is widely used today as the basic methodology of accounting for model uncertainty. See Geisser (1993) Draper (1995) Raftery, Madigan and Hoeting (1997), and Clyde (1999) for discussion and references. Although keeping all models in the analysis is an ideal, this can be cumbersome for communication and descriptive purposes. If only one or two models receive substantial posterior probability, it would not be an egregious sin to eliminate the ....

....which computations need to be done, can be enormous, especially in model selection problems such as variable selection. We do not address computational issues here; some recent papers on the subject are Carlin and Chib (1995) Green (1995) Kass and Raftery (1995) Verdinelli and Wasserman (1995) Raftery, Madigan and Hoeting (1997) and Clyde (1999) 7 Diculty 2. When the models have parameter spaces of di ering dimensions, use of improper noninformative priors yields indeterminate answers. To see this, suppose that improper noninformative priors N i and N j are entertained for models M i and M j , respectively. ....

Raftery, A., Madigan, D. and Hoeting, J. (1997), \Bayesian Model Averaging for Linear Regression Models," J. Amer. Statist. Assoc. 92, 179-191.

....i (y) P k j=1 m j (y) Indeed, the m i are often presented as the default posterior model probabilities. Whether one wants to simply select the model with the largest posterior probability, nd the best single predictive model (Barbieri Berger, 2000) or engage in Bayesian model averaging (Raftery et al. 1997; Clyde, 1999) it is necessary to compute the Bayes factors or the m i (y) or, at least, to be able to sample from the latter) 1 2 Challenges in Bayesian model selection The attractive features of the Bayesian approach to model selection are discussed, for instance, in Kass Raftery ....

Raftery, A., Madigan, D., & Hoeting, J. (1997). Bayesian model averaging for linear regression models. Journal of the American Statistical Association 92, 179-191.

....especially in the preliminary stages of modelling when careful specification of subjective priors for all models under consideration is typically not feasible. For discussion of these issues, see Jeffreys (1961) Berger and Sellke (1987) Berger and Delampady (1987) Kass and Raftery (1995) Raftery, Madigan and Hoeting (1997), Berger and Pericchi (1995, 1996, 1997a) O Hagan (1995, 1997) and Berger (1997) There are two main difficulties with the development of default Bayes factors. The first is the well known difficulty that, when the models or hypotheses have parameter spaces of differing dimension, one cannot ....

Raftery, A., Madigan, D. and Hoeting, J. (1997), "Bayesian Model Averaging for Linear Regression Models, J. Amer. Statist. Assoc. 92, 179--191.

....typically used. Second, I use the full posterior to address the question of model selection for both the size of the network and the subset of explanatory variables. As part of the implementation of model selection, I use Bayesian Random Searching, which builds upon model space searching work by Raftery, Madigan, and Hoeting (1997) and is similar to the approach of Chipman, George, and McCulloch (1998) in their implementation of Bayesian CART . Finally I present results on the asymptotic properties of the posterior for neural network models. An example which illustrates the ideas of this paper is the ozone data of Breiman ....

.... averaging is Hoeting et al. 1999) In some cases, one will nd more than one model with high posterior probability, and these models will give di erent predictions (see, for example, the heart attack data in Raftery 1996) Model averaging can also result in dramatic decreases in prediction errors (Raftery, Madigan, and Hoeting 1997). Picking only a single model will grossly underestimate the variability of the estimate, since it ignores the fact that another model with signi cant posterior probability made a di erent prediction. Instead, one should calculate predictions (or statistics thereof) by using a weighted average ....

[Article contains additional citation context not shown here]

Raftery, A. E., Madigan, D., and Hoeting, J. A. (1997). \Bayesian Model Averaging for Linear Regression Models." Journal of the American Statistical Association, 437, 179-191.

....over the collection of possible models. Candidate chains for exploring this space include the Gibbs sampler of George and McCulloch (1993) the importance sampler of Clyde, DeSimone and Parmigiani (1996) applicable when the predictor variables are orthogonal; and the Occam s window scheme of Madigan, Raftery and Hoeting (1997). In the simulation study described below, however, the number of covariates is small, so that we could simply evaluate SIC and iMDL on all possible models to identify the best. To understand the characteristics of each MDL criterion, we consider three simulated examples. These have been adapted ....

Madigan, D., Raftery, A., and Hoeting, J. (1997). Bayesian model averaging for linear regression models. J. Amer. Statist. Assoc., 92, 179--191.

....over the collection of possible models. Candidate chains for exploring this space include the Gibbs sampler of George and McCulloch #1993#; the importance sampler of Clyde,DeSimone and Parmigiani #1996#,applicable when the predictor variables are orthogonal; and the Occam s window scheme of Madigan,Raftery and Hoeting #1997#. In the simulation study described below,however,the number of covariates is small, so that we could simply evaluate SIC and iMDL on all possible models to identify the best. To understand the characteristics of each MDL criterion, we consider three simulated examples. These have been adapted ....

Madigan, D., Raftery, A., and Hoeting, J. #1997#.Bayesian model averaging for linear regression models.

....the results are very sensitive to this choice then the method will not be of practical use. As will be seen below, our approach to hyperparameter selection makes use of a rough combination of training data information and prior beliefs, and is similar in spirit to the prior selection approach of Raftery, Madigan, and Hoeting (1997) (although the authors of that article may not agree ) We begin by standardizing the training data by a linear transformation so that each x and y has an average value of 0 and a range of 1. The purpose of this standardization is to make it easier to intuitively gauge parameter values. For ....

Raftery, A. E., Madigan, D. and Hoeting, J. A. (1997) \Bayesian Model Averaging for Linear Regression Models". Journal of the American Statistical Association, 92, 179-191.

....Siow (1980) Draper (1995) and Phillips (1995) and the discussions of these papers. Secondly, there is the recent statistics literature on computational aspects. Markov chain Monte Carlo methods are proposed in George and McCulloch (1993) Green (1995) Madigan and York (1995) Geweke (1996) and Raftery, Madigan and Hoeting (1997), while Laplace approximations are found in Gelfand and Dey (1995) and Raftery (1996) Finally, there exists a large literature on information criteria, mostly in the context of time series, see e.g. Hannan and Quinn (1979) Akaike (1981) Atkinson (1981) Chow (1981) This paper provides a ....

.... to as Bayesian model averaging (BMA) is in fact the standard Bayesian procedure under model uncertainty, since it follows directly from the rules of probability calculus upon which the Bayesian paradigm is based [see e.g. Leamer (1978) Min and Zellner (1993) Osiewalski and Steel (1993) and Raftery et al. 1997)] Posterior model probabilities are given by P (M j y) l y (M j )P (M j ) # 2 k h=1 l y (M h )P (M h ) # # 2 k # h=1 P (M h ) P (M j ) l y (M h ) l y (M j ) # # 1 , 1.7) where l y (M j ) the marginal likelihood of model M j , is obtained as l y (M j ) # p(y ....

[Article contains additional citation context not shown here]

Raftery, A.E., Madigan, D. and Hoeting, J.A. (1997), "Bayesian Model Averaging for Linear Regression Models," Journal of the American Statistical Association, 92, 179-191.

....to use for each of these subsets. Since it is not feasible to try all of these models, one needs a strategy for searching the model space. Traditional search methods include algorithms such as stepwise regression. A modern Bayesian method is Markov Chain Monte Carlo Model Composition (MC 3 ) (Raftery, Madigan, and Hoeting 1997). I will present a technique, Bayesian Random Searching (BARS) motivated by MC 3 . The idea behind MC 3 is to create a Markov chain with state space equal to the set of models under consideration and equilibrium distribution equal to the posterior probabilities of the models. If one simulates ....

Raftery, A. E., Madigan, D., and Hoeting, J. A. (1997). \Bayesian Model Averaging for Linear Regression Models." Journal of the American Statistical Association, 437, 179-191.

....over the collection of possible models. Candidate chains for exploring this space include the Gibbs sampler of George and McCulloch (1993) the importance sampler of Clyde, DeSimone and Parmigiani (1996) applicable when the predictor variables are orthogonal; and the Occam s window scheme of Madigan, Raftery and Hoeting (1997). In the simulation study described below, however, the number of covariates is small, so that we could simply evaluate SIC and iMDL on all possible models to identify the best. To understand the characteristics of each MDL criterion, we consider three simulated examples. These have been adapted ....

Madigan, D., Raftery, A., and Hoeting, J. (1997). Bayesian model averaging for linear regression models. J. Amer. Statist. Assoc., 92, 179--191.

....to be chosen. The covariate coefficients, are assumed to be independent a priori. When the predictors include a set of indicator (dummy) variables for a categorical variable with c categories, the posterior distribution should be invariant to the choice of c Gamma 1 indicator variables. Raftery, Madigan, and Hoeting (1997) suggest an appropriate prior distribution set up for dummy variables. Finally, the hyperprior for the spatial parameter is defined (fi) Gamma ( ff) where and ff are hyperparameters to be chosen. Thus the spatial parameter is constrained to be greater than 0. In the examples we have ....

Raftery, A., Madigan, D., and Hoeting, J. (1997). Bayesian model averaging for linear regression models. Journal of the American Statistical Association, 92:179--191.

....analysis of age, or of risk scores, can take into account information from every model in the model class. In several reported experiments with real data, BMA has yielded improved predictive performance and parameter estimation (Madigan and Raftery 1994; Madigan et al. 1995; Volinsky et al. 1997; Raftery et al. 1997). An S PLUS function to do Bayesian model averaging for Cox regression models, bic.surv, is available from the BMA homepage (www.research.att.com volinsky bma.html) or by contacting the first author. First, we specify a uniform prior over the space of the 2 23 models associated with the ....

Raftery, A. E., D. Madigan, and J. A. Hoeting (1997). Bayesian model averaging for linear regression models. Journal of the American Statistical Association 92, 179--191.

No context found.

Madigan, D., Raftery, A., and Hoeting, J. (1997). Bayesian model averaging for linear regression models. J. Amer. Statist. Assoc., 92, 179--191.

No context found.

Raftery, A. E., Madigan, D., and Hoeting, J. (1997). Bayesian model averaging for linear regression models. Journal of the American Statistical Association, 92(437):179--191.

No context found.

A. E. Raftery, D. Madigan, and Jennifer A. Hoeting, Bayesian Model Averaging for Linear Regression Models, submitted for publication.

No context found.

A. E. Raftery, D. Madigan, and J. A. Hoeting, 1996. Bayesian Model Averaging for Linear Regression Models. Journal of the American Statistical Association 92:179- 191.

No context found.

Raftery, A.E., Madigan, D., and Hoeting, J.A. (1997). Bayesian model averaging for linear regression models. Journal of the American Statistical Association 92:179-191.

No context found.

Raftery, A.E., Madigan, D., and Hoeting, J.A. (1997), "Bayesian Model Averaging for Linear Regression Models," Journal of the American Statistical Association,

No context found.

Raftery, A.E., Madigan, D., and Hoeting, J.A. (1997), \Bayesian Model Averaging for Linear Regression Models," Journal of the American Statistical Association,

No context found.

Raftery, A. E., Madigan, D. and Hoeting, J. A. (1997) Bayesian Model Averaging for Linear Regression Models. J. Amer. Statist. Asso., 92, 179-191.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC