A new reinforcement learning (RL) methodology is proposed to design multi-agent systems. In the realistic setting of situated agents with local perception, the task of automatically building a coordinated system is of crucial importance. We use simple reactive agents which learn their own behavior in a decentralized way. To cope with the difficulties inherent to RL used in that framework, we have developed an incremental learning algorithm where agents face more and more complex tasks. We illustrate this general framework on a computer experiment where agents have to coordinate to reach a global goal. 1

Designing individual agents so that, when put together, they reach a given global goal is not an easy task. One solution to automatically build such large Multi-Agent Systems is to use decentralized learning : each agent learns by itself its own behavior. To that purpose, Reinforcement Learning methods are very attractive as they do not require a solution of the problem to be known before hand. Nevertheless, many hard points need to be solved for such a learning process to be viable. Among others, the credit assignement problem, combinatorial explosion and local perception of the world seem the most crucial and prevent optimal behavior. In this paper, we propose a framework based on a gradual learning of harder and harder tasks until the desired global behavior is reached. The applicability of our paradigm is tested on computer experiments where many agents have to coordinate to reach a global goal. Our results show that incremental learning leads to better performances than more classical techniques. We then discuss several improvements which could lead to even better performances.