We propose that the ability to extract regularities from time series through prediction learning can be enhanced if we use a hierarchical architecture in which higher layers are trained to predict the internal state of lower layers when such states change significantly. This hierarchical organization has two functions: (a) it forces the system to progressively re-code sensory information so as to enhance useful regularities and filter out useless information; (b) it progressively reduces the length of the sequences which should be predicted going from lower to higher layers. This, in turn, allows higher levels to extract higher level regularities which are hidden at the sensory level. By training an architecture of this type to predict the next sensory state of a robot navigating in a environment divided into two rooms we show how the first level prediction layer extracts low level regularities such as `walls', `corners', and `corridors' while the second level prediction laye...

Recently there has been a good deal of interest in using techniques developed for learning from reinforcement to guide learning in robots. Motivated by the desire to find better robot learning methods, this thesis presents a number of novel extensions to existing techniques for controlling exploration and inference in reinforcement learning. First I distinguish between the well known exploration-exploitation trade-off and what I term exploration for future exploitation. It is argued that there are many tasks where it is more appropriate to maximise this latter measure. In particular it is appropriate when we want to employ learning algorithms as part of the process of designing a controller. Informed by this insight I develop a number of novel measures of the agent's task knowledge. The first of these is a measure of the probability of a particular course of action being the optimal course of action. Estimators are developed for this measure for boolean and non-boolean processes. These...

This paper presents an investigation of the the suitability of the robot controller presented in [Nehmzow et al. 89] and [Nehmzow et al. 90] for a computationally cheap expansion of the behavioural repertoire of a mobile robot. Experiments with mobile robots are presented that show that this is possible by simply adding further so-called instinct-rules without altering the controller itself: through the robot's interaction with its environment effective associations between sensors and actuators arise in an artificial neural network which serves as an associative memory. 1 Introduction Designing intelligent controllers for autonomous mobile robots is a task often underestimated by the designer. Sensor signals turn out to differ from what is expected in theory, and actuators produce different effects than anticipated. Whilst many of these differences between expected and observed behaviour can be overcome by building actual robots, there remain uncertainties and variations that are due...

Experiments with the Edinburgh R2 mobile robot are presented that show how robots can be taught to accomplish various different tasks, without the need for re-programming the controller, and without using self-tuition. In an externally supervised teaching process --- not unlike the process of "shaping" known in animal learning --- the robot acquires competences such as obstacle avoidance, contour following, box pushing, phototaxis or route learning (mazes). The learning is fast. 1 Introduction It has been shown that competences fundamental to robot control, such as obstacle avoidance or contour following can, in behaviour-based robotics, be achieved through the interaction of independent, so-called behavioural modules ([Malcolm et al. 89, Brooks 85]). The question of how to re-combine and orchestrate several of those fundamental competences in order to synthesize new, more complex global behaviours is as yet an unresolved problem. The experiments presented here show one way to approa...

Introduction Since the time that "God made man in His own image", humans have been fascinated by stories about artifacts coming to life. Ancient myth, fairytales, literature, and science fiction abound with stories of artificial beings. In the older stories, although it is people who make the beings, it is the power of supernatural forces that bestows life. Ovid's story of Pygmalion is perhaps one the most famous from mythology; a sculptor falls in love with his sculpture of a woman which the goddess Venus then brings to life. Then there is the ancient Guianan Indian fairytale about a witch doctor who carved himself a daughter out of a plum tree because he needed a son-in-law to look after him. Similarly, there is the story of the wooden puppet Pinocchio who desires and eventually obtains boyhood (a desire that parallels that of the android Commander Data in "Star Trek: the Next Generation"). In days of old, the breath of life into the inanimate was a mystery that extolled th

Fixed robot controllers are suitable for tasks whose main characteristics are known a priori. However, for tasks such as exploration of unknown territory, fixed controllers tend to be too brittle as they rely on predefined knowledge, supplied by the designer. For such tasks (i.e. tasks in which a-priori knowledge is limited) we propose to use self-organising controllers which allow the robot to acquire the necessary competences autonomously ([Nehmzow et al. 89]). The paper describes experiments with mobile robots, in which the robots autonomously determine effective connections between their sensory input and their motor response to it. In these experiments, artificial neural networks are used to store associations between sensory input and motor responses, the networks are trained in a selfsupervised way, making use of so-called instinct-rules which govern the robots' behaviour. The fast learning achieved enables the robots to adapt to changing circumstances such as changes in their e...

It is argued that the following three properties are foundations of robust robot navigation: ffl The use of landmarks (and, in particular, the use of a compass sense), ffl the use of canonical paths, and ffl the use of topological rather than geometrical maps. Some examples of successful animal navigation are presented that support this view. We have performed initial experiments with mobile robots to investigate mechanisms suitable to implement such navigational architectures. Experiments concerning navigation by dead reckoning are presented, and a differential light compass is introduced to aid robot navigation. 1 Introduction Most of the work to date concerning navigation of mobile robots uses internal geometrical representations of the robot's environment to perform navigational tasks. MOBOT III, for example, constructs such a geometrical representation autonomously from sensor data ([Knieriemen & v.Puttkamer 91]), other robots use maps supplied by the designer ([Kampmann & Sch...

Mobile robot navigation under controlled laboratory conditions is, by now, state of the art and reliably achievable. To transfer navigation mechanisms used in such small-scale environments to applications in untreated, large environments, however, is not trivial, and typically requires modifications to the original navigation mechanism: scaling up is hard. In this paper, we discuss the difficulties of mobile robot navigation in general, the various options to achieve navigation in large environments, and experiments with Manchester's FortyTwo, which investigate how scaling up of navigational competencies can be achieved. We were particularly interested in autonomous mobile robot navigation in unmodified, large and varied environments, without the aid of pre-installed maps or supplied CAD models of the environment. This paper presents a general approach to achieve this. FortyTwo regularly travels the corridors of the Department of Computer Science at Manchester University, using topol...

A self-organising controller for mobile robots is presented that allows robots to acquire sensor-motor competences autonomously, and thus adapt to changes in the environment, the task, or the robot itself, without external supervision and feedback. Using artificial neural networks, the robot acquires these fundamental sensor-motor competences rapidly, in a couple of learning steps, taking a few tens of seconds. Results of experiments with a number of mobile robots are presented, both by us and by other research groups, using the same controller architecture. 1 c flU. Nehmzow, 1994 1 1 Introduction As mobile robot technology advances and sophisticated mobile robots become widely available, at comparably low cost --- the Nomad 200 robot described in section 4.1 of this paper being one example --- the question of how to control such robots intelligently has become more interesting, both from a research and an industrial point of view. Intelligent control here means that the r...

This paper gives an overview of the bottom-up approach to artificial intelligence (AI), commonly referred to as behavior-oriented AI. The behavior-oriented approach, with its focus on the interaction between autonomous agents and their environments, is introduced by contrasting it with the traditional approach of knowledge-based AI. Different notions of autonomy are discussed, and key problems of generating adaptive and complex behavior are identified. A number of techniques for the generation of behavior are introduced and evaluated regarding their potential for realizing different aspects of autonomy as well as adaptivity and complexity of behavior. It is concluded that in order to realize truly autonomous and intelligent agents, the behavior-oriented approach will have to focus even more on life-like qualities in both agents and environments.