Sequential decision making is a fundamental task faced by any intelligent agent in an extended interaction with its environment; it is the act of answering the question "What should I do now?" In this thesis, I show how to answer this question when "now" is one of a finite set of states, "do" is one of a finite set of actions, "should" is maximize a long-run measure of reward, and "I" is an automated planning or learning system (agent). In particular,

Markov decision problems (MDPs) provide the foundations for a number of problems of interest to AI researchers studying automated planning and reinforcement learning. In this paper, we summarize results regarding the complexity of solving MDPs and the running time of MDP solution algorithms. We argue that, although MDPs can be solved efficiently in theory, more study is needed to reveal practical algorithms for solving large problems quickly. To encourage future research, we sketch some alternative methods of analysis that rely on the structure of MDPs.

We introduce the smoothed analysis of algorithms, which continuously interpolates between the worst-case and average-case analyses of algorithms. In smoothed analysis, we measure the maximum over inputs of the expected performance of an algorithm under small random perturbations of that input. We measure this performance in terms of both the input size and the magnitude of the perturbations. We show that the simplex algorithm has smoothed complexity polynomial in the input size and the standard deviation of

Abstract. This paper gives an algcmthm for solving linear programming problems. For a problem with tz constraints and d variables, the algorithm requires an expected O(d’n) + (log n)o(d)d’’+(’(’) + o(dJA log n) arithmetic operations, as rz ~ ~. The constant factors do not depend on d. Also, an algorlthm N gwen for integer hnear programmmg. Let p bound the number of bits required to specify the ratmnal numbers defmmg an input constraint or the ob~ective function vector. Let n and d be as before. Then, the algorithm requires expected 0(2d dn + S~dm In n) + dc)’d) ~ in H operations on numbers with O(1~p bits d ~ ~ ~z + ~, where the constant factors do not depend on d or p. The expectations are with respect to the random choices made by the algorithms, and the bounds hold for any gwen input. The techmque can be extended to other convex programming problems. For example, m algorlthm for finding the smallest sphere enclosing a set of /z points m Ed has the same t]me bound

We show that with recently developed derandomization techniques, one can convert Clarkson's randomized algorithm for linear programming in fixed dimension into a lineartime deterministic one. The constant of proportionality is d O(d) , which is better than for previously known such algorithms. We show that the algorithm works in a fairly general abstract setting, which allows us to solve various other problems (such as finding the maximum volume ellipsoid inscribed into the intersection of n halfspaces) in linear time.

We review the recent progress in the design of efficient algorithms for various problems in geometric optimization. We present several techniques used to attack these problems, such as parametric searching, geometric alternatives to parametric searching, prune-and-search techniques for linear programming and related problems, and LPtype problems and their efficient solution. We then describe a variety of applications of these and other techniques to numerous problems in geometric optimization, including facility location, proximity problems, statistical estimators and metrology, placement and intersection of polygons and polyhedra, and ray shooting and other query-type problems.

This thesis establishes a connection between the Helly theorems, a collection of results from combinatorial geometry, and the class of problems whichwe call Generalized Linear Programming, or GLP, which can be solved by combinatorial linear programming algorithms like the simplex method. We use these results to explore the class GLP and show new applications to geometric optimization, and also to prove Helly theorems. In general, a GLP is a set...

An Abstract Optimization Problem (AOP) is a triple (H, <, Φ) where H is a finite set, < a total order on 2 H and Φ an oracle that, for given F ⊆ G ⊆ H, either reports that F = min<{F ′ | F ′ ⊆ G} or returns a set F ′ ⊆ G with F ′ < F. To solve the problem means to find the minimum set in H. We present a randomized algorithm that solves any AOP with an expected number of at most e 2 √ n+O ( 4 √ n ln n) oracle calls, n = |H|. In contrast, any deterministic algorithm needs to make 2 n − 1 oracle calls in the worst case. The algorithm is applied to the problem of finding the distance between two n-vertex (or nfacet) convex polyhedra in d-space, and to the computation of the smallest ball containing n points in d-space; for both problems we give the first subexponential bounds in the arithmetic model of computation.

We suggest a new randomized algorithm for solving parity games with worst case time complexity roughly

We suggest the first strongly subexponential and purely combinatorial algorithm for solving the mean payoff games problem. It is based on iteratively improving the longest shortest distances to a sink in a possibly cyclic directed graph. We identify a new “controlled” version of the shortest paths problem. By selecting exactly one outgoing edge in each of the controlled vertices we want to make the shortest distances from all vertices to the unique sink as long as possible. Under reasonable assumptions the problem belongs to the complexity class NP∩coNP. Mean payoff games are easily reducible to this problem. We suggest an algorithm for computing longest shortest paths. Player Max selects a strategy (one edge in each controlled vertex) and player Min responds by evaluating shortest paths to the sink in the remaining graph. Then Max locally changes choices in controlled vertices looking at attractive switches that seem to increase shortest paths lengths (under the current evaluation). We show that this is a monotonic strategy improvement, and every locally optimal strategy is globally optimal. This allows us to construct a randomized algorithm of complexity min(poly · W, 2 O( √ n log n)), which is simultaneously pseudopolynomial (W is the maximal absolute edge weight) and subexponential in the number of vertices n. All previous algorithms for mean payoff games were either exponential or pseudopolynomial (which is purely exponential for exponentially large edge weights).