Results 1  10
of
15
Forecast aggregation via recalibration
 Machine Learning:129
, 2013
"... Your article is protected by copyright and all rights are held exclusively by The Author(s). This eoffprint is for personal use only and shall not be selfarchived in electronic repositories. If you wish to selfarchive your article, please use the accepted manuscript version for posting on your ow ..."
Abstract

Cited by 4 (1 self)
 Add to MetaCart
(Show Context)
Your article is protected by copyright and all rights are held exclusively by The Author(s). This eoffprint is for personal use only and shall not be selfarchived in electronic repositories. If you wish to selfarchive your article, please use the accepted manuscript version for posting on your own website. You may further deposit the accepted manuscript version in any repository, provided it is only made publicly available 12 months after official publication or later and provided acknowledgement is given to the original source of publication and a link is inserted to the published article on Springer's website. The link must be accompanied by the following text: "The final publication is available at link.springer.com”.
An analytic method for evaluating the performance of aggregation rules for probability densities
 Operations Research
, 2010
"... It is shown how infinite sequences of densities with defined properties can be used to evaluate the expected performance of mathematical aggregation rules for elicited densities. The performance of these rules is measured through the expected variance, calibration, and expected Brier score of the ag ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
It is shown how infinite sequences of densities with defined properties can be used to evaluate the expected performance of mathematical aggregation rules for elicited densities. The performance of these rules is measured through the expected variance, calibration, and expected Brier score of the aggregate. A general result for the calibration of the arithmetic average of densities from wellcalibrated independent experts is given. Arithmetic and geometric aggregation rules are compared in several demonstrations using sequences of uniform, normal, and exponential densities. Sequences are developed that exhibit dependence among experts and lack of calibration. The impact of correlation, number of experts, and degree of calibration on the performance of the aggregation is demonstrated with normal densities.
Bayesian models · Individual differences · Wisdom of the crowd
, 2012
"... Abstract It is known that the average of many forecasts about a future event tends to outperform the individual assessments. With the goal of further improving forecast performance, this paper develops and compares a number of models for calibrating and aggregating forecasts that exploit the wellkn ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract It is known that the average of many forecasts about a future event tends to outperform the individual assessments. With the goal of further improving forecast performance, this paper develops and compares a number of models for calibrating and aggregating forecasts that exploit the wellknown fact that individuals exhibit systematic biases during judgment and elicitation. All of the models recalibrate judgments or mean judgments via a twoparameter calibration function, and differ in terms of whether (1) the calibration function is applied before or after the averaging, (2) averaging is done in probability or logodds space, and (3) individual differences are captured via hierarchical modeling. Of the nonhierarchical models, the one that first recalibrates the individual judgments and then averages them in logodds is the best relative to simple averaging, with 26.7 % improvement in Brier score and better performance on 86 % of the individual problems. The hierarchical version of this model does slightly better in terms of mean Brier score (28.2 %) and slightly worse in terms of individual problems (85 %).
Advances: Aggregate Probabilities Page 1 of 39 Ch 09 060430 V07 9 Aggregation of Expert Probability Judgments
"... probability distributions from experts in risk analysis, ” Risk Analysis, 19, 187203. This chapter is concerned with the aggregation of probability distributions in decision and risk analysis. Experts often provide valuable information regarding important uncertainties in decision and risk analyses ..."
Abstract
 Add to MetaCart
probability distributions from experts in risk analysis, ” Risk Analysis, 19, 187203. This chapter is concerned with the aggregation of probability distributions in decision and risk analysis. Experts often provide valuable information regarding important uncertainties in decision and risk analyses because of the limited availability of “hard data ” to use in those analyses. Multiple experts are often consulted in order to obtain as much information as possible, leading to the problem of how to combine or aggregate their information. Information may also be obtained from other sources such as forecasting techniques or scientific models. Because uncertainties are typically represented in terms of probability distributions, we consider expert and other information in terms of probability distributions. We discuss a variety of models that lead to specific combination methods. The output of these methods is a “combined probability distribution, ” which can be viewed as representing a summary of the current state of information regarding the uncertainty of interest. After presenting the models and methods, we discuss empirical evidence on the performance of the methods. In the conclusion we highlight important
Appendix S1. Definitions of Acronyms 3rdG: 3rd Generation Technologies CCS: Carbon Capture and Storage CDF: Cumulative Distribution Function ChemL: Chemical Looping DICE: Dynamic Integrated Model of Climate and the Economy FR: Fast Burner Reactors GCAM: G
"... c(µ): generic representation for cost of emissions abatement cR(µ): cost of emissions abatement in the reduced form R&D model DR(µ): cost of climate damages in the reduced form R&D model cD(µt): cost of emissions abatement in the DICE model DD(τt): cost of climate damages in the DICE model e ..."
Abstract
 Add to MetaCart
c(µ): generic representation for cost of emissions abatement cR(µ): cost of emissions abatement in the reduced form R&D model DR(µ): cost of climate damages in the reduced form R&D model cD(µt): cost of emissions abatement in the DICE model DD(τt): cost of climate damages in the DICE model et: total carbon emissions in period t Gs(·) : representative function modeling the constraint set for stage s, s=N,L H(τt−1, et) : representative function linking greenhouse gas emissions to atmospheric temperature to the carbon cycle h(α): shift effect in the marginal abatement curve due to technological success Jωψ (·): representative function modeling the constraint ψ for scenario ω kt: capital stock in period t lt: investment in traditional capital in period t ot: consumption of goods/services in period t ut: social utility in period t Us(·) : social utility in stage s, s=N,L xijk: 1 if project j of technology i is funded at level k in the reduced form R&D model, 0 otherwise xs: generic vector representing all decision variables other than abatement decisions in stage s, s=N,L x: generic vector representing all decision variables other than abatement decisions yt: net output of goods/services in period t ygt: unadjusted output in period t ys: vector representing net output of goods/services in stage s, s=N,L αi: pivot effect in the marginal abatement curve due to success in technology i µt: level of emissions abatement in period t µs: vector representing emissions abatement decisions in stage s, s=N,L Φ(c (µ),α): functional representing cost of emissions abatement after technical change τt: atmospheric temperature in period t τs: vector representing atmospheric temperature in stage s, s=N,L
©2004 INFORMS Anniversary Article Decision Analysis in Management Science
"... As part of the 50th anniversary of Management Science, the journal is publishing articles that reflect on thepast, present, and future of the various subfields the journal represents. In this article, we consider decision analysis research as it has appeared in Management Science. After reviewing th ..."
Abstract
 Add to MetaCart
As part of the 50th anniversary of Management Science, the journal is publishing articles that reflect on thepast, present, and future of the various subfields the journal represents. In this article, we consider decision analysis research as it has appeared in Management Science. After reviewing the foundations of decision analysis and the history of the journal’s decision analysis department, we review a number of key developments in decision analysis research that have appeared in Management Science and offer some comments on the current state of the field. Key words: decision analysis; probability assessment; utility theory; game theory 1.
UNIVERSITY
, 2013
"... This material has been approved for public release and unlimited distribution except as restricted below. Internal use: * Permission to reproduce this material and to prepare derivative works from this material for internal use is granted, provided the copyright and “No Warranty ” statements are in ..."
Abstract
 Add to MetaCart
(Show Context)
This material has been approved for public release and unlimited distribution except as restricted below. Internal use: * Permission to reproduce this material and to prepare derivative works from this material for internal use is granted, provided the copyright and “No Warranty ” statements are included with all reproductions and derivative works. External use: * This material may be reproduced in its entirety, without modification, and freely distributed in written or electronic form without requesting formal permission. Permission is required for any other external and/or commercial use. Requests for permission should be directed
This report was prepared for the SEI Administrative Agent
, 2013
"... This material has been approved for public release and unlimited distribution except as restricted below. Internal use: * Permission to reproduce this material and to prepare derivative works from this material for internal use is granted, provided the copyright and “No Warranty ” statements are in ..."
Abstract
 Add to MetaCart
(Show Context)
This material has been approved for public release and unlimited distribution except as restricted below. Internal use: * Permission to reproduce this material and to prepare derivative works from this material for internal use is granted, provided the copyright and “No Warranty ” statements are included with all reproductions and derivative works. External use: * This material may be reproduced in its entirety, without modification, and freely distributed in written or electronic form without requesting formal permission. Permission is re
Calibration of Expert Judgments in Counterterrorism Risk Assessment
, 2014
"... This project will develop an elicitation process to assess important counterterrorism values such as adversary intent and capabilities, target vulnerabilities, and countermeasure costs, by aggregating judgments of a diverse panel of intelligence experts. In this process, experts would be asked to gi ..."
Abstract
 Add to MetaCart
This project will develop an elicitation process to assess important counterterrorism values such as adversary intent and capabilities, target vulnerabilities, and countermeasure costs, by aggregating judgments of a diverse panel of intelligence experts. In this process, experts would be asked to give medians and prediction intervals that reflect their best knowledge and levels of confidence (or uncertainty) both for the uncertain quantities of interest, and for a set of “seed variables ” whose true values are verifiable using historical or anticipated nearfuture data. A consensus probability distribution for the quantity of interest can then be achieved by weighting the various experts based on performance on the seed variables (and possibly other factors). Project Technical Description 1. Keywords: Risk assessment, expert elicitation, uncertainty, calibration, aggregation of judgments