Abstract. In this paper we present a novel method, called Evolutionary

A modular approach to neural behavior control of autonomous robots is presented. It is based on the assumption that complex internal dynamics of recurrent neural networks can eciently solve complex behavior tasks. For the development of appropriate neural control structures an evolutionary algorithm is introduced, which is able to generate neuromodules with specic functional properties, as well as the connectivity structure for a modular synthesis of such modules. This so called ENS 3 -algorithm does not use genetic coding. It is primarily designed to develop size and connectivity structure of neuro-controllers. But at the same time it optimizes also parameters of individual networks like synaptic weights and bias terms. For demonstration, evolved networks for the control of miniature Khepera robots are presented. The aim is to develop robust controllers in the sense that neuro-controllers evolved in a simulator show comparably good behavior when loaded to a real robot acting in a physical environment. Discussed examples of such controllers generate obstacle avoidance and phototropic behaviors in non-trivial environments. 1 1

To study the relevance of recurrent neural network structures for the behavior of autonomous agents a series of experiments with miniature robots is performed. A special evolutionary algorithm is used to generate networks of different sizes and architectures. Solutions for obstacle avoidance and phototropic behavior are presented. Networks are evolved with the help of simulated robots, and the results are validated with the use of physical robots. 1 1

Evolving neuro-modules and their interfaces to control autonomous robots by

Abstract ii The graceful animal motion we see in nature has proven extremely difficult to reproduce algorith-mically. There is a need for further research into motion control techniques to address this problem adequately for computer animation and robotics applications. In this thesis, we describe a novel method for the synthesis of compact control policies for a variety of motion control problems. Direct policy search is applied to a nearest-neighbour control policy, which uses a Voronoi cell discretization of the observable state space, as induced by a set of control nodes located in this space. Such a semi-parametric representation allows for policy refinement through the adaptive addition of nodes. Thus, a coarse-to-fine policy search can be performed in such a way that the problem is shaped for easy learning. We apply our method to developing policies for steering various vehicles around a winding track. In particular, a policy is learned to steer a double-trailer truck backwards, a problem of considerable difficulty given the instability of the trailers. We also generate policies for the problems of balancing an inverted pendulum on a moving cart and driving the state of a point mass around a user-specified

One of the main challenges in automatic controller synthesis is to develop methods that can successfully be applied for complex tasks. The difficulty is increased even more in case of settings with multiple interacting agents. We apply the Artificial Homeostatic Hormone Systems (AHHS) approach, which is inspired by the signaling network of unicellular organisms, to control a system of several independently acting agents decentrally. The approach is designed for evaluation-minimal, artificial evolution in order to be applicable to complex modular robotics scenarios. The performance of AHHScontrollers is compared to NeuroEvolution of Augmenting Topologies (NEAT) in the coupled inverted pendulums benchmark. AHHS controllers are found to be better for multi-modular settings. We analyze the evolved controllers concerning the usage of sensory inputs, the emerging oscillations, and we give a nonlinear dynamics interpretation. The generalization of evolved controllers to initial conditions far from the original conditions is investigated and found to be good. Similarly the performance of controllers scales well even with module numbers different from the original domain the controller was evolved for. Two reference implementations of a similar controller approach are reported and shown to have shortcomings. We discuss the related work and conclude by summarizing the main contributions of our work.

Mobile agents can be considered to be a new design paradigm in the area of distributed programming. This paper deals with problems of mobile agents which should be able to achieve a user-given task autonomously. In search of a "good enough" solution, the agent should be able to find new places and should move "fast enough" through the network. Can "migration intelligence" help to solve these problems?

The presented evolutionary algorithm is especially designed to generate recurrent neural networks with non-trivial internal dynamics. It is not based on genetic algorithms, and sets no constraints on the number of neurons and the architecture of a network. Network topology and parameters like synaptic weights and bias terms are developed simultaneously. It is well suited for generating neuromodules acting in sensorimotor loops, and therefore it can be used for evolution of neurocontrollers solving also nonlinear control problems. We demonstrate this capability by applying the algorithm successfully to the following task: A rotating pendulum is mounted on a cart; stabilize the rotator in an upright position, and center the cart in a given finite interval.

We show that using recurrent networks with finite time constants is not contradictory to arbitrary fast signal transmission in a closed loop with appropriate feedbacks. This surprising result is due to the occurrence of infinitely amplifying subloops, which we formally describe by differential inclusions. The theory then shows that the transmission speed depends crucially on the gain of the transfer function. Generalising the theoretical framework we demonstrate how to build efficient, fast and robust neuro-controllers with pre-specified performance by application to... the benchmark problem of balancing the inverted pendulum.