Sequential decision making is a fundamental task faced by any intelligent agent in an extended interaction with its environment; it is the act of answering the question "What should I do now?" In this thesis, I show how to answer this question when "now" is one of a finite set

In this report we show how the class of adaptive prediction methods that Sutton called "temporal difference," or TD, methods are related to the theory of squential decision making. TD methods have been used as "adaptive critics" in connectionist learning systems, and have been

Abstract We connect a broad class of generative models through their shared reliance on sequential decision making. Motivated by this view, we develop extensions to an existing model, and then explore the idea further in the context of data imputation -perhaps the simplest setting in which

The Theory of Adaptive Utility for sequential decision making under uncertainty provides a generalisation of the standard Bayesian approach, permitting initial utility uncertainty and the learning of preferences. In this paper we discuss the motivation for the use of Adaptive Utility and comment

Automatically constructing novel representations of tasks from analysis of state spaces is a longstanding fundamental challenge in AI. I review recent progress on this problem for sequential decision making tasks modeled as Markov decision processes. Specifically, I discuss three classes

We investigate the problem of using function approximation in reinforcement learning (RL) where the agent’s control policy is represented as a classifier mapping states to actions. The innovation of this paper lies with introducing a measure of state’s decision-making importance. We then use

We study a multi-round optimization setting in which in each round a player may select one of several actions, and each action produces an outcome vector, not observable to the player until the round ends. The final payoff for the player is computed by applying some known function f to the sum of all outcome vectors (e.g., the minimum of all coordinates of the sum). We show that standard notions of performance measure (such as comparison to the best single action) used in related expert and bandit settings (in which the payoff in each round is scalar) are not useful in our vector setting. Instead, we propose a different performance measure, and design algorithms that have vanishing regret with respect to our new measure.

Abstract not found

Abstract not found

: This paper formalizes the notion of flexibility in sequential decision making and investigates conditions under which the use of flexibility as an additional criterion may be justified. The correlations between flexibility and value, and flexibility and risk, are studied under various assumptions