Results 11  20
of
55
Forecast aggregation via recalibration
 Machine Learning:129
, 2013
"... Your article is protected by copyright and all rights are held exclusively by The Author(s). This eoffprint is for personal use only and shall not be selfarchived in electronic repositories. If you wish to selfarchive your article, please use the accepted manuscript version for posting on your ow ..."
Abstract

Cited by 4 (1 self)
 Add to MetaCart
(Show Context)
Your article is protected by copyright and all rights are held exclusively by The Author(s). This eoffprint is for personal use only and shall not be selfarchived in electronic repositories. If you wish to selfarchive your article, please use the accepted manuscript version for posting on your own website. You may further deposit the accepted manuscript version in any repository, provided it is only made publicly available 12 months after official publication or later and provided acknowledgement is given to the original source of publication and a link is inserted to the published article on Springer's website. The link must be accompanied by the following text: "The final publication is available at link.springer.com”.
A Bayesian latent group analysis for detecting poor effort in the assessment of malingering
, 2012
"... Abstract Despite their theoretical appeal, Bayesian methods for the assessment of poor effort and malingering are still rarely used in neuropsychological research and clinical diagnosis. In this article, we outline a novel and easytouse Bayesian latent group analysis of malingering whose goal is ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
(Show Context)
Abstract Despite their theoretical appeal, Bayesian methods for the assessment of poor effort and malingering are still rarely used in neuropsychological research and clinical diagnosis. In this article, we outline a novel and easytouse Bayesian latent group analysis of malingering whose goal is to identify participants displaying poor effort when tested. Our Bayesian approach also quantifies the confidence with which each participant is classified and estimates the base rates of malingering from the observed data. We implement our Bayesian approach and compare its utility in effort assessment to that of the classic belowchance criterion of symptom validity testing (SVT). In two experiments, we evaluate the accuracy of both a Bayesian latent group analysis and the belowchance criterion of SVT in recovering the membership of participants assigned to the malingering group. Experiment 1 uses a simulation research design, whereas Experiment 2 involves the differentiation of patients with a history of stroke from coached malingerers. In both experiments, sensitivity levels are high for the Bayesian method, but low for the belowchance criterion of SVT. Additionally, the Bayesian approach proves to be resistant to possible effects of coaching. We conclude that Bayesian latent group methods complement existing methods in making more informed choices about malingering.
Cognitive models and the wisdom of crowds: A case study using the bandit problem
 In R
, 2010
"... The “wisdom of the crowds ” refers to the idea that the aggregated performance of a group of people on a challenging task may be superior to the performance of any of the individuals. For some tasks, like estimating a single quantity, it is straightforward to aggregate individual behavior. For more ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
The “wisdom of the crowds ” refers to the idea that the aggregated performance of a group of people on a challenging task may be superior to the performance of any of the individuals. For some tasks, like estimating a single quantity, it is straightforward to aggregate individual behavior. For more complicated multidimensional or sequential tasks, however, it is not so straightforward. Cognitive models of behavior are needed, to infer what people know from how they behave, and allow aggregation to be done on the inferred knowledge. We provide a case study of this role for cognitive modeling in the wisdom of crowds, using a multidimensional sequential optimization problem, known as the bandit problem, for which there are large differences in individual ability. We show that, using some established cognitive models of people’s decisionmaking on these problems, aggregate performance approaches optimality, and exceeds the performance of the vast majority of individuals.
A modelbased fMRI analysis with hierarchical at
 UNIV OF PENNSYLVANIA on November 24, 2014cpx.sagepub.comDownloaded from Decision Making and Mental Illness 781 Bayesian
, 2011
"... A recent trend in decision neuroscience is the use of modelbased functional magnetic resonance imaging (fMRI) using mathematical models of cognitive processes. However, most previous modelbased fMRI studies have ignored individual differences because of the challenge of obtaining reliable paramet ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
(Show Context)
A recent trend in decision neuroscience is the use of modelbased functional magnetic resonance imaging (fMRI) using mathematical models of cognitive processes. However, most previous modelbased fMRI studies have ignored individual differences because of the challenge of obtaining reliable parameter estimates for individual participants. Meanwhile, previous cognitive science studies have demonstrated that hierarchical Bayesian analysis is useful for obtaining reliable parameter estimates in cognitive models while allowing for individual differences. Here we demonstrate the application of hierarchical Bayesian parameter estimation to modelbased fMRI using the example of decision making in the Iowa Gambling Task. First, we used a simulation study to demonstrate that hierarchical Bayesian analysis outperforms conventional (individualor grouplevel) maximum likelihood estimation in recovering true parameters. Then we performed modelbased fMRI analyses on experimental data to examine how the fMRI results depend on the estimation method.
Correcting the SIMPLE Model of Free Recall
"... Abstract The SIMPLE (ScaleInvariant Memory, Perception, and LEarning) model developed by The SIMPLE Model and a Correction The SIMPLE model assumes that items in memory are represented along the single dimension corresponding to a logarithmically compressed representation of ..."
Abstract
 Add to MetaCart
Abstract The SIMPLE (ScaleInvariant Memory, Perception, and LEarning) model developed by The SIMPLE Model and a Correction The SIMPLE model assumes that items in memory are represented along the single dimension corresponding to a logarithmically compressed representation of
THEORETICAL REVIEW Using Bayesian hierarchical parameter estimation to assess the generalizability of cognitive models of choice
, 2014
"... Abstract To be useful, cognitive models with fitted parameters should show generalizability across time and allow accurate predictions of future observations. It has been proposed that hierarchical procedures yield better estimates of model parameters than do nonhierarchical, independent approache ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract To be useful, cognitive models with fitted parameters should show generalizability across time and allow accurate predictions of future observations. It has been proposed that hierarchical procedures yield better estimates of model parameters than do nonhierarchical, independent approaches, because the formers ’ estimates for individuals within a group can mutually inform each other. Here, we examine Bayesian hierarchical approaches to evaluating model generalizability in the context of two prominent models of risky choice— cumulative prospect theory (Tversky&Kahneman, 1992) and the transferofattentionexchange model (Birnbaum & Chavez, 1997). Using empirical data of risky choices collected for each individual at two time points, we compared the use of hierarchical versus independent, nonhierarchical Bayesian estimation techniques to assess two aspects of model generalizability: parameter stability (across time) and predictive accuracy. The relative performance of hierarchical versus independent estimation varied across the different measures of generalizability. The hierarchical approach improved parameter stability (in terms of a lower absolute discrepancy of parameter values across time) and predictive accuracy (in terms of deviance; i.e., likelihood). With respect to test–retest correlations and posterior predictive accuracy, however, the hierarchical approach did not outperform the independent approach. Further analyses suggested that this was due to strong correlations between some parameters within both models. Such intercorrelations make it difficult to identify and interpret single parameters and can induce high degrees of shrinkage in hierarchical models. Similar findings may also occur in the context of other cognitive models of choice.
Using cognitive models to combine probability estimates
"... Abstract We demonstrate the usefulness of cognitive models for combining human estimates of probabilities in two experiments. The first experiment involves people's estimates of probabilities for general knowledge questions such as "What percentage of the world's population speaks En ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract We demonstrate the usefulness of cognitive models for combining human estimates of probabilities in two experiments. The first experiment involves people's estimates of probabilities for general knowledge questions such as "What percentage of the world's population speaks English as a first language?" The second experiment involves people's estimates of probabilities in football (soccer) games, such as "What is the probability a team leading 10 at half time will win the game?", with ground truths based on analysis of large corpus of games played in the past decade. In both experiments, we collect people's probability estimates, and develop a cognitive model of the estimation process, including assumptions about the calibration of probabilities and individual differences. We show that the cognitive model approach outperforms standard statistical aggregation methods like the mean and the median for both experiments and, unlike most previous related work, is able to make good predictions in a fully unsupervised setting. We also show that the parameters inferred as part of the cognitive modeling, involving calibration and expertise, provide useful measures of the cognitive characteristics of individuals. We argue that the cognitive approach has the advantage of aggregating over latent human knowledge rather than observed estimates, and emphasize that it can be applied in predictive settings where answers are not yet available.
unknown title
"... Scientists who study cognition infer underlying processes either by observing behavior (e.g., response times, percentage correct) or by observing neural activity (e.g., the BOLD response). These two types of observations have traditionally supported two separate lines of study. The first is led by c ..."
Abstract
 Add to MetaCart
(Show Context)
Scientists who study cognition infer underlying processes either by observing behavior (e.g., response times, percentage correct) or by observing neural activity (e.g., the BOLD response). These two types of observations have traditionally supported two separate lines of study. The first is led by cognitive modelers, who rely on behavior alone to support their computational theories. The second is led by cognitive neuroimagers, who rely on statistical models to link patterns of neural activity to experimental manipulations, often without any attempt to make a direct connection to an explicit computational theory. Here we present a flexible Bayesian framework for combining neural and cognitive models. Joining neuroimaging and computational modeling in a single hierarchical framework allows the neural data to influence the parameters of the cognitive model and allows behavioral data, even in the absence of neural data, to constrain the neural model. Critically, our Bayesian approach can reveal interactions between behavioral and neural parameters, and hence between neural activity and cognitive mechanisms. We demonstrate the utility of our approach with applications to simulated fMRI data with a recognition model and to diffusionweighted imaging data
Bayesian models · Individual differences · Wisdom of the crowd
, 2012
"... Abstract It is known that the average of many forecasts about a future event tends to outperform the individual assessments. With the goal of further improving forecast performance, this paper develops and compares a number of models for calibrating and aggregating forecasts that exploit the wellkn ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract It is known that the average of many forecasts about a future event tends to outperform the individual assessments. With the goal of further improving forecast performance, this paper develops and compares a number of models for calibrating and aggregating forecasts that exploit the wellknown fact that individuals exhibit systematic biases during judgment and elicitation. All of the models recalibrate judgments or mean judgments via a twoparameter calibration function, and differ in terms of whether (1) the calibration function is applied before or after the averaging, (2) averaging is done in probability or logodds space, and (3) individual differences are captured via hierarchical modeling. Of the nonhierarchical models, the one that first recalibrates the individual judgments and then averages them in logodds is the best relative to simple averaging, with 26.7 % improvement in Brier score and better performance on 86 % of the individual problems. The hierarchical version of this model does slightly better in terms of mean Brier score (28.2 %) and slightly worse in terms of individual problems (85 %).
NeuroImage 72 (2013) 193–206 Contents lists available at SciVerse ScienceDirect
"... journal homepage: www.elsevier.com/locate/ynimg ..."
(Show Context)