Can cooperation be learnt through reinforcement learning? This is the central question we pose in this paper. To answer it first requires an examination of what constitutes reinforcement learning. We also examine some of the issues associated with the design of a reinforcement learning system; these include: the choice of an update rule, whether or not to implement an eligibility trace. In this paper we set ourselves four tasks that need solving, each task shows us certain aspects of reinforcement learning. Each task is of increasing complexity, the first two allow us to explore reinforcement learning on its own, while the last two allow us to examine reinforcement learning in a multi-agent setting. We begin with a system that learns to play blackjack; it allows us to examine how robust reinforcement learning algorithms are. The second system learns to run through a maze; here we learn how to correctly implement an eligibility trace, and explore different updating rules. The two multi-agent systems involve a traffic simulation, as well as a cellular simulation. The traffic simulation shows the weaknesses in reinforcement learning that show up when applying it to a multi-agent setting. In our cellular simulation, we show that it is possible to implement a reinforcement learning algorithm in continuous statespace.

We present a survey of policy search algorithms in reinforcement learning. The foundations of reinforcement learning and the historical development of policy search are discussed. Policy search algorithms are divided and examined along three axes. First, we examine the search methodology utilized by the algorithm. Second, we examine the representational structure of the policy. Finally, we examine the types problems that the algorithms are designed to solve. We conclude by examining practical applications, future trends and other issues that pertain to current day policy search techniques. 1