The nature of empirical simplicity and its relationship to scientific truth are long-standing puzzles. In this paper, empirical simplicity is explicated in terms of empirical effects, which are defined in terms of the structure of the inference problem addressed. Problem instances are classified according to the number of empirical effects they present. Simple answers are satisfied by simple worlds. An efficient solution achieves optimum worst-case cost over each complexity class with respect to such costs such as the number of retractions or errors prior to convergence and elapsed time to convergence. It is shown that always choosing the simplest theory compatible with experience and hanging onto it while it remains simplest is both necessary and sufficient for efficiency. 1 The Simplicity Puzzle Machine learning, statistics, and the philosophy of science all recommend the selection of simple theories or models on the basis of empirical data, where simplicity has something to do with minimizing independent entities, principles, causes, or equational coefficients. This intuitive preference for simplicity is called Ockham’s razor, after the fourteenth century theologian and logician William of Ockham, whose work exemplified a similar tendency. But in spite of its intuitive appeal, how could Ockham’s razor possibly help us find the true theory? For if we already know that the simplest theory is true or probably true, we don’t need Ockham’s razor to infer that it is. And if we don’t know that the simplest theory is true or probably true, how do we know that simplicity steers us in the right direction? It doesn’t help to say that simplicity is associated with other virtues such as testability (Popper 1968), unity (Friedman 1983), better explanations (Harman 1965), higher “confirmation ” (Carnap 1950, Glymour 1980), minimization of predictive risk (Akaike 1973), or minimum description length (Vitanyi and Li 2000), since if the truth weren’t simple, it wouldn’t have these nice properties either. To assume otherwise is to engage in wishful thinking (vanFraassen 1981). 1 Over-fitting arguments based upon minimization of predictive risk might seem to

In science, one faces the problem of selecting the true theory from a range of alternative theories. The typical response is to select the simplest theory compatible with available evidence, on the authority of “Ockham’s Razor”. But how can a fixed bias toward simplicity help one find possibly complex truths? A short survey of standard answers to this question reveals them to be either wishful, circular, or irrelevant. A new explanation is presented, based on minimizing the reversals of opinion prior to convergence to the truth. According to this alternative approach, Ockham’s razor does not inform one which theory is true but is, nonetheless, the uniquely most efficient strategy for arriving at the true theory, where efficiency is a matter of minimizing reversals of opinion prior to finding the true theory. 1

This paper presents a new explanation of how preferring the simplest theory compatible with experience assists one in finding the true answer to a scientific question when the answers are theories or models. Science is portrayed as an infinite game between science and nature. Simplicity is a structural invariant reflecting sequences of theory choices nature could force the scientist to produce. It is demonstrated that among the methods that converge to the truth in an empirical problem, the ones that do so with a minimum number of reversals of opinion prior to convergence are exactly the ones that prefer simple theories. The idea explains not only simplicity tastes in model selection, but aspects of theory testing and the unwillingness of natural science to break symmetries without a reason. In natural science, one typically faces a situation in which several (or even infinitely many) available theories are compatible with experience. Standard practice is to choose the simplest theory among them and to cite “Ockham’s razor ” as the excuse (figure

Abstract not found

Abstract. We explore the notion of duality for defeasible entailment relations induced by preference orderings on states. We then show that such preferential entailment relations may be used to characterise Peircean inductive and abductive reasoning. Interpreting the preference relations as accessibility relations establishes modular Gödel-Löb logic as the modal logic of inductive and abductive reasoning. §1. Introduction. The heart of logic is entailment – a relation between information-bearers X and Y according to which Y in some sense follows from X. We take X and Y to be discrete strings (sentences) in some object language. The entailment relation that may hold between premiss X and consequence Y is induced by the choice of a relation E between possibly independent representations

Reliable Reasoning is a simple, accessible, beautifully explained introduction to Vapnik and Chervonenkis’s statistical learning theory. It includes a modest discussion of the application of the theory to the philosophy of induction; the purpose of these remarks is to say something more. 1. A Patient Pessimist’s Guide to Induction Philosophical Learning Theory Vapnik and Chervonenkis’s statistical learning theory may be compared to formal learning theory, familiar to philosophers from the work of Putnam (1963) and Kelly (1996). There are significant technical differences between the two theories, but considered as philosophical frameworks for thinking about inductive reasoning, they have much in common. I will say that they are both—in their epistemological incarnations—species of philosophical learning theory. The programmatic goal of formal learning theory is to investigate methods for learning from experience that are guaranteed to converge on the truth. (or at least guaranteed to come as close as possible) under some given set of circumstances. If you

Both in learning and in natural science, one faces the problem of selecting among a range of theories, all of which are compatible with the available evidence. The traditional response to this problem has been to select the simplest such theory on the basis of “Ockham’s Razor”. But how can a fixed bias toward simplicity help us find possibly complex truths? I survey the current, textbook answers to this question and find them all to be wishful, circular, or irrelevant. Then I present a new approach based on minimizing the number of reversals of opinion prior to convergence to the truth. According to this alternative approach, Ockham’s razor is a good idea when it seems to be (e.g., in selecting among parametrized models) and is not a good idea when it feels dubious (e.g., in the inference of arbitrary computable functions). Hence, the proposed vindication of Ockham’s razor can be used to separate vindicated applications In science and learning, one must eventually face up to the problem of choosing among several or even infinitely many theories compatible with all available information. How ought one to choose? The traditional answer is to choose the “simplest ” and to invoke

Ockham’s razor is the principle that, all other things being equal, scientists ought to prefer simpler theories. In recent years, philosophers have argued that simpler theories make better predictions, possess theoretical virtues like explanatory power, and have other pragmatic virtues like computational tractability. However, such arguments fail to explain how and why a preference for simplicity can help one find true theories in scientific inquiry, unless one already assumes that the truth is simple. One new solution to that problem is the Ockham efficiency theorem (Kelly 2002, 2004, 2007a-d, Kelly and Glymour 2004), which states that scientists who heed Ockham’s razor retract their opinions less often and sooner than do their non-Ockham competitors. The theorem neglects, however, to consider competitors following random (“mixed”) strategies and in many applications random strategies are known to achieve better worst-case loss than deterministic strategies. In this paper, we describe two ways to extend the result to a very general class of random, empirical strategies. The first extension concerns expected retractions, retraction times, and errors and the second extension concerns retractions in chance, times of retractions in chance, and chances of errors.