this paper, a network of the former type will be analyzed in the following. Figure 24 shows a characteristic trajectory of a successful robot controller of this type. As above, the robot starts off facing the upper left obstacle. It turns away from it to the left, enters the zone, and collects three objects on its first pass through the zone, turning slightly to the left towards each of them. As soon as it has left the zone it starts moving in a semi-circle to the left, which takes it back into the zone. In the zone it starts moving straight ahead again, takes a slight turn to the right to collect the upper object, and continues straight ahead out of the zone. The same pattern is repeated: as soon as it leaves the zone, it moves in a semi-circle to the left, which takes it back into the zone, where it starts moving straight forward again. Once more it performs a slight turn to the right to collect an object it would otherwise have missed. It continues to move straight ahead, leaves the zone, returns in another semi-circle, enters once more and moves straight ahead until the evaluation period ends.

Abstract not found

We show how event extraction can be used for handling delayed response tasks with arbitrary delay periods between the stimulus and the cue for response. Our approach is based on a number of information processing levels, where the lowest level works on raw time-stepped based sensory data. This data is classified using an unsupervised clustering mechanism. The second level works on this classified data, but still on the individual time-step basis. An event extraction mechanism detects and signals transitions between classes; this forms the basis for the third level. As this level only is updated when events occur, it is independent of the time-scale of the lower level interaction. We also sketch how an event filtering mechanism could be constructed which discards irrelevant data from the event stream. Such a mechanism would output a fourth level representation which could be used for delayed response tasks where irrelevant, or distracting, events could occur during the delay.

Research in embodied cognitive science emphasizes that a close interaction of brain, body and environment is central to the emergence of cognitive processes. Much work on embodied artificial intelligence has therefore shifted focus from purely computational modeling to autonomous mobile robotics. Many researchers emphasize the importance of working with real robots rather than simulations which usually cannot fully capture the complexities of the physical world. However, from a cognitive science point of view, robot simulations nevertheless have an important, complementary role to play, due to the fact that in many cases they allow for more extensive, systematic experimentation as well as for experiments, e.g. with evolving robot morphologies, that can only be carried out in very limited form on real robots. Furthermore, it will be argued in this paper, robot simulations are very useful tools in experimentation with active adaptation of non-trivial environments, an aspect that is still largely ignored in much embodied artificial intelligence research. 1

Abstract not found

In autonomous agent systems, internal memory can be an important element to overcome the limitations of purely reactive agent behaviour. This paper presents an analysis of memory requirements for T-maze tasks well known as the road sign problem. In these tasks a robot agent should make a decision of turning left or right at the T-junction in the approach corridor, depending on a history of perceptions. The robot agent in simulation can sense the light intensity influenced by light lamps placed on the bank of the wall. We apply the evolutionary multiobjective optimization approach to finite state controllers with two objectives, behaviour performance and memory size. Then the internal memory is quantified by counting internal states needed for the T-maze tasks in noisy environments. In particular, we focused on the influence of noise on internal memory and behaviour performance, and it is shown that state machines with variable thresholds can improve the performance with a hysteresis effect to filter out noise. This paper also provides an analysis of noise effect on perceptions and its relevance on performance degradation in state machines. keywords: T-maze, delayed response task, evolutionary robotics, finite state machines, evolutionary multiobjective optimization, internal memory 1 1

We show how a simple layered system can selforganize into a set of distinct states and qualitatively different behaviors as a result of learning a robotic delayed response task. Our approach is based on an architecture where higher levels are able to dynamically modulate the lower reactive mapping when needed.

HE AMOUNT OF DATA AVAILABLE to a mobile robot controller is staggering. This thesis investigates how extensive continuous-valued data streams of noisy sensor and actuator activations can be stored, recalled, and processed by robots equipped with only limited memory buffers. We address three robot memorisation problems, namely Route Learning (store a route), Novelty Detection (detect changes along a route) and the Lost Robot Problem (find best match along a route or routes). A robot learning problem called the Road-Sign Problem is also addressed. It involves a long-term delayed response task where temporal credit assignment is needed. The limited memory buffer entails that there is a trade-off between memorisation and learning. A traditional overall data compression could be used for memorisation, but the compressed representations are not always suitable for subsequent learning. We present a novel unsupervised on-line data reduction technique which focuses on change detection rather than overall data compression. It produces reduced sensory flows which are suitable for storage in the memory buffer while preserving underrepresented inputs. Such inputs can be essential when using temporal credit assignment for learning a task. The usefulness of the technique is evaluated through a number of experiments on the identified robot problems. Results show that a learning ability can be introduced while at the same time maintaining memorisation capabilities. The essentially symbolic representation, resulting from the unsupervised online reduction could in the extension also help bridge the gap between the raw sensory flows and the symbolic structures useful in prediction and communication.

Abstract — The road sign problem is tackled in this work with Reservoir Computing (RC) networks. These networks are made of a fixed recurrent neural network where only a readout layer is trained. In the road sign problem, an agent has to decide at some point in time which action to take given relevant information gathered in the past. We show that RC can handle simple and complex T-maze tasks (which are a subdomain of the road sign problem). Keywords — Reservoir Computing, road sign problem, T-maze, long-term memory. 1

We report on an under-explored animat learning problem, that of learning sequences of goals, and present an algorithm for solving it using observation and imitation. The presentation introduces the problem as well as a neural agent control architecture we use as a framework in which to use the algorithm. We demonstrate that, by using the algorithm, a learning agent can learn to satisfy a sequence of goals by observing the actions of a teaching agent, and later imitate the teaching agent.