Results 1  10
of
27
Default Bayes factors for ANOVA designs.
 Journal of Mathematical Psychology,
, 2012
"... Abstract Bayes factors have been advocated as superior to pvalues for assessing statistical evidence in data. Despite the advantages of Bayes factors and the drawbacks of pvalues, inference by pvalues is still nearly ubiquitous. One impediment to adoption of Bayes factors is a lack of practical ..."
Abstract

Cited by 25 (4 self)
 Add to MetaCart
(Show Context)
Abstract Bayes factors have been advocated as superior to pvalues for assessing statistical evidence in data. Despite the advantages of Bayes factors and the drawbacks of pvalues, inference by pvalues is still nearly ubiquitous. One impediment to adoption of Bayes factors is a lack of practical development, particularly a lack of readytouse formulas and algorithms. In this paper, we discuss and expand a set of default Bayes factor tests for ANOVA designs. These tests are based on multivariate generalizations of Cauchy priors on standardized effects, and have the desirable properties of being invariant with respect to linear transformations of measurement units. Moreover, these Bayes factors are computationally convenient, and straightforward sampling algorithms are provided. We cover models with fixed, random, and mixed effects, including random interactions, and do so for withinsubject, betweensubject, and mixed designs. We extend the discussion to regression models with continuous covariates. We also discuss how these Bayes factors may be applied in nonlinear settings, and show how they are useful in differentiating between the power law and the exponential law of skill acquisition. In sum, the current development makes the computation of Bayes factors straightforward for the vast majority of designs in experimental psychology. * Correspondence: 210 McAlester Hall, Columbia, MO 65203, rouderj@missouri.edu. We thank Brandon Turner and EricJan Wagenmakers for detailed and constructive comments. This research is supported by NSF SES 1024080.
Bayesian assessment of null values via parameter estimation and model comparison
 Perspectives on Psychological Science
, 2011
"... Psychologists have been trained to do data analysis by asking whether null values can be rejected. Is the difference between groups nonzero? Is choice accuracy not at chance level? These questions have been traditionally addressed by null hypothesis significance testing (NHST). NHST has deep problem ..."
Abstract

Cited by 23 (6 self)
 Add to MetaCart
(Show Context)
Psychologists have been trained to do data analysis by asking whether null values can be rejected. Is the difference between groups nonzero? Is choice accuracy not at chance level? These questions have been traditionally addressed by null hypothesis significance testing (NHST). NHST has deep problems that are solved by Bayesian data analysis. As psychologists transition to Bayesian data analysis, it is natural to ask how Bayesian analysis assesses null values. The article explains and evaluates two different Bayesian approaches. One method involves Bayesian model comparison (and uses Bayes factors). The second method involves Bayesian parameter estimation and assesses whether the null value falls among the most credible values. Which method to use depends on the specific question that the analyst wants to answer, but typically the estimation approach (not using Bayes factors) provides richer information than the model comparison approach. Keywords Bayes, model comparison, parameter estimation Psychologists are routinely trained to frame their research design and analysis in terms of rejecting null values. For example, when studying the influence of distraction on response time, we might ask whether the change in response time is dif
Bayesian Estimation Supersedes the t Test
"... This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. Bayesian estimation for 2 groups provides complete distributions of credible valu ..."
Abstract

Cited by 12 (2 self)
 Add to MetaCart
(Show Context)
This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. Bayesian estimation for 2 groups provides complete distributions of credible values for the effect size, group means and their difference, standard deviations and their difference, and the normality of the data. The method handles outliers. The decision rule can accept the null value (unlike traditional t tests) when certainty in the estimate is high (unlike Bayesian model comparison using Bayes factors). The method also yields precise estimates of statistical power for various research goals. The software and programs are free and run on Macintosh, Windows, and Linux platforms.
Introduction to special section on Bayesian data analysis
 Perspectives on Psychological Science
"... ..."
Rethinking Statistical Analysis Methods for CHI
"... CHI researchers typically use a significance testing approach to statistical analysis when testing hypotheses during usability evaluations. However, the appropriateness of this approach is under increasing criticism, with statisticians, economists, and psychologists arguing against the use of routin ..."
Abstract

Cited by 6 (0 self)
 Add to MetaCart
(Show Context)
CHI researchers typically use a significance testing approach to statistical analysis when testing hypotheses during usability evaluations. However, the appropriateness of this approach is under increasing criticism, with statisticians, economists, and psychologists arguing against the use of routine interpretation of results using “canned ” p values. Three problems with current practice the fallacy of the transposed conditional, a neglect of power, and the reluctance to interpret the size of effects can lead us to build weak theories based on vaguely specified hypothesis, resulting in empirical studies which produce results that are of limited practical or scientific use. Using publicly available data presented at CHI 2010 [19] as an example we address each of the three concerns and promote consideration of the magnitude and actual importance of effects, as opposed to statistical significance, as the new criteria for evaluating CHI research.
Beyond theory and data in preference modeling: Bringing humans into the loop
 In Proceedings of the 4th International Conference on Algorithmic Decision Theory (ADT
, 2015
"... Abstract. Many mathematical frameworks aim at modeling human preferences, employing a number of methods including utility functions, qualitative preference statements, constraint optimization, and logic formalisms. The choice of one model over another is usually based on the assumption that it can ..."
Abstract

Cited by 3 (3 self)
 Add to MetaCart
(Show Context)
Abstract. Many mathematical frameworks aim at modeling human preferences, employing a number of methods including utility functions, qualitative preference statements, constraint optimization, and logic formalisms. The choice of one model over another is usually based on the assumption that it can accurately describe the preferences of humans or other subjects/processes in the considered setting and is computationally tractable. Verification of these preference models often leverages some form of real life or domain specific data; demonstrating the models can predict the series of choices observed in the past. We argue that this is not enough: to evaluate a preference model, humans must be brought into the loop. Human experiments in controlled environments are needed to avoid common pitfalls associated with exclusively using prior data including introducing bias in the attempt to clean the data, mistaking correlation for causality, or testing data in a context that is different from the one where the data were produced. Human experiments need to be done carefully and we advocate a multidisciplinary research environment that includes experimental psychologists and AI researchers. We argue that experiments should be used to validate models. We detail the design of an experiment in order to highlight some of the significant computational, conceptual, ethical, mathematical, psychological, and statistical hurdles to testing whether decision makers ’ preferences are consistent with a particular mathematical model of preferences. 1
A reinvestigation of the reference frame of the tiltadaptation aftereffect
"... A central issue in research on visual perception and eye movements is the extent to which a detailed representation of our visual surroundings is preserved across eye ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
A central issue in research on visual perception and eye movements is the extent to which a detailed representation of our visual surroundings is preserved across eye
The effect of horizontal eye movements on free recall: A preregistered adversarial collaboration
 Journal of Experimental Psychology: General
, 2015
"... A growing body of research has suggested that horizontal saccadic eye movements facilitate the retrieval of episodic memories in free recall and recognition memory tasks. Nevertheless, a minority of studies have failed to replicate this effect. This article attempts to resolve the inconsistent resul ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
(Show Context)
A growing body of research has suggested that horizontal saccadic eye movements facilitate the retrieval of episodic memories in free recall and recognition memory tasks. Nevertheless, a minority of studies have failed to replicate this effect. This article attempts to resolve the inconsistent results by introducing a novel variant of proponentskeptic collaboration. The proposed approach combines the features of adversarial collaboration and purely confirmatory preregistered research. Prior to data collection, the adversaries reached consensus on an optimal research design, formulated their expectations, and agreed to submit the findings to an academic journal regardless of the outcome. To increase transparency and secure the purely confirmatory nature of the investigation, the 2 parties set up a publicly available adversarial collaboration agreement that detailed the proposed design and all foreseeable aspects of the data analysis. As anticipated by the skeptics, a series of Bayesian hypothesis tests indicated that horizontal eye movements did not improve free recall performance. The skeptics suggested that the nonreplication may partly reflect the use of suboptimal and questionable research practices in earlier eye movement studies. The proponents countered this suggestion and used a p curve analysis to argue that the effect of horizontal eye movements on explicit memory did not merely reflect selective reporting.
Frequent Words Do Not Break Continuous Flash Suppression Differently from Infrequent or Nonexistent Words: Implications for Semantic Processing of Words in the Absence of Awareness
, 2014
"... Continuous flash suppression (CFS) has been used as a paradigm to probe the extent to which word stimuli are processed in the absence of awareness. In the two experiments reported here, no evidence is obtained that word stimuli are processed up to the semantic level when suppressed through CFS. In E ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
(Show Context)
Continuous flash suppression (CFS) has been used as a paradigm to probe the extent to which word stimuli are processed in the absence of awareness. In the two experiments reported here, no evidence is obtained that word stimuli are processed up to the semantic level when suppressed through CFS. In Experiment 1, word stimuli did not break suppression faster than their pseudoword variants nor was suppression time modulated by word frequency. Experiment 2 replicated these findings, but more critically showed that differential effects can be obtained with this paradigm using a simpler stimulus. In addition, pixel density of the stimuli did prove to be related to suppression time in both experiments, indicating that the paradigm is sensitive to differences in detectability. A third and final experiment replicated the wellknown face inversion effect using the same setup as Experiments 1 and 2, thereby demonstrating that the employed methodology can capture more highlevel effects as well. These results are discussed in the context of previous evidence on unconscious semantic processing and two potential explanations are advanced. Specifically, it is argued that CFS might act at a level too low in the visual system for highlevel effects to be observed or that the widely used breaking CFS paradigm is merely illsuited to capture effects in the context of words.
Intuitive Logic Revisited: New Data and a Bayesian Mixed Model MetaAnalysis
, 2014
"... Recent research on syllogistic reasoning suggests that the logical status (valid vs. invalid) of even difficult syllogisms can be intuitively detected via differences in conceptual fluency between logically valid and invalid syllogisms when participants are asked to rate how much they like a conclus ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
(Show Context)
Recent research on syllogistic reasoning suggests that the logical status (valid vs. invalid) of even difficult syllogisms can be intuitively detected via differences in conceptual fluency between logically valid and invalid syllogisms when participants are asked to rate how much they like a conclusion following from a syllogism (Morsanyi & Handley, 2012). These claims of an intuitive logic are at odds with most theories on syllogistic reasoning which posit that detecting the logical status of difficult syllogisms requires effortful and deliberate cognitive processes. We present new data replicating the effects reported by Morsanyi and Handley, but show that this effect is eliminated when controlling for a possible confound in terms of conclusion content. Additionally, we reanalyze three studies (n~287) without this confound with a Bayesian mixed model metaanalysis (i.e., controlling for participant and item effects) which provides evidence for the nullhypothesis and against Morsanyi and Handley’s claim.