Ulrich Nehmzow. Experiments in Competence Acquisition for Autonomous Mobile Robots. PhD thesis, Department of Arti cial Intelligence, University of Edinburgh, 1992.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....offer possibilities for training the embodied control system within its own environment. In particular, real time neural control systems that make no distinction between a learning and a performance phase are promising candidates for the control of adaptive behaviour (Verschure et al. 1992; Nehmzow, 1992; Fagg, 1993; Touzet, 1994) As an example to show the need for adaptive behaviour we return to our Beast 1 . Now Beast 1 has come equipped with a built in control structure that uses a heat sensor to protect it from having its sensors melted off. Such an event would leave it blind and paralysed ....

....to associate their behaviours with the rewards and punishments. Neural network reinforcement learning is a methodology that attempts to capture the essence of conditioning. There are numerous examples in the literature where robots are trained using this kind of learning (Meeden et al. 1993; Nehmzow, 1992; Prescott, 1994; Verschure et al. 1992; Ziemke, 1996) Consider a simple example in which a robot has to learn to avoid obstacles. At the start it does not have any knowledge about the relation between obstacles and sensor information and what actions to take. The robot starts with a (random) ....

Nehmzow, U. (1992). Experiments in Competence Acquisition for Autonomous Mobile Robots. PhD thesis, University of Edinburgh.

....if the environment is stochastic. 5 This is not necessarily the case e.g. the two armed bandit task used here. 3 Problems putting RL onto robots Recently a lot of work has been done trying to put RL algorithms onto real robots and there have been a number of successful implementations to date [7, 9, 14, 15, 16, 19]. There are, however, a number of difficulties associated with RL methods per se and these are especially pertinent to the problem of using them with real robots. In short RL makes assumptions which do not apply to real world tasks [16] First RL assumes that the environment as perceived by the ....

Ulrich Nehmzow. Experiments in Competence Acquisition for Autonomous Mobile Robots. Ph.d. thesis, Department of Artificial Intelligence, Edinburgh University, 1992.

....using descriptive categories which are not necessary to describe the behaviour of the constituent components. An emergent behaviour leads to emergent functionality if the behaviour contributes to the system s self preservation and if the system can build further upon it. An example is given by Nehmzow (1994) who describes the emergence of a corridor following behavior in a simple robot from three underlying particular behaviors: turning right when perceiving a wall on the left, turning left when perceiving a wall on the right, and moving straight forward otherwise. Despite its central role in ....

....autonomy, i.e. they depend on sufficient input to be available from the environment at any point in time. The complexity of behavior that can be achieved with such an agent is of course limited. Nevertheless it has been shown that such a system can learn to acquire far more than trivial behavior. Nehmzow (1994), for example, used a simple robotic vehicle controlled by a feed forward network mapping the input from two whisker sensors at the front of the vehicle to four possible actions (left, right, forward, backward) It was shown that, using reinforcement learning, this vehicle could successfully be ....

Nehmzow, U. (1994). Experiments in Competence Acquisition for Autonomous Mobile Robots (Doctoral dissertation). University of Edinburgh, UK.

....of finite state automaton. The most commonly known approach of this type is the subsumption architecture [ 4 ] Learning approaches consist in endowing each agent with some learning capabilities. Thus, the agent can adapt its behavior (self learning through artificial neural networks, 28 ] [ 24 ] , 13 ] or inherit it from earlier generations of agents (evolutionary learning) 2.2 Embodiment Autonomous agents are embodied systems by definition, however there are different forms of embodiment. Some work, for instance [ 5 ] 19 ] 20 ] considers physically embodied agents ....

U. Nehmzow. Experiments in Competence Acquisition for Autonomous Mobile Robots. PhD thesis, University of Edimburg, 1994.

....some form of finite state automaton. The most commonly known approach of this type is the subsumption architecture [7] Learning approaches consist in endowing each agent with some learning capabilities. Thus, the agent can adapt its behavior (selflearning through artificial neural networks, 66] [51], 35] or inherit it from earlier generations of agents (evolutionary learning) Learning paradigms (supervised, reinforcement, self organized learning) and algorithms (error correction, hebbian competitive, 31] are strongly inspired by neuro physiology and behavioral psychology. 3.3 ....

U. Nehmzow. Experiments in Competence Acquisition for Autonomous Mobile Robots. PhD thesis, University of Edimburg, 1994.

....optimal. 17 An infinite number of times if the environment is stochastic. CHAPTER 1. INTRODUCTION 11 1. 5 Problems putting RL onto robots Recently a lot of work has been done trying to put RL algorithms onto real robots and there have been a number of successful implementations to date [26, 37, 49, 19, 50, 57]. There are, however, a number of difficulties associated with RL methods per se and these are especially pertinent to the problem of using them with real robots. In short RL makes assumptions which do not apply to real world tasks [50] First RL assumes that the environment as perceived by the ....

Ulrich Nehmzow. Experiments in Competence Acquisition for Autonomous Mobile Robots. Ph.d. thesis, Department of Artificial Intelligence, Edinburgh University, 1992.

....As we said above the behavior of the robot might strongly affect its ability to learn to predict the next sensory states. We choose this behavior because it is easy to implement and because it allows the robot to explore environments with different topologies. A similar approach has been used by Nehmzow (1992). 9 after each cycle (a cycle may correspond to one or more steps depending on the level of the layer involved in the prediction, see below) In doing so we expect that our robot will extract a dynamical representation of the environment that will incorporate the topological structure of the ....

Nehmzow, U. (1992). Experiments in Competence Acquisition for Autonomous Mobile Robots, Ph.D. Thesis, University of Edinburgh, U.K.

.... behaviour is simply reversed for traversal in the opposite direction (i.e. wall following to the left between places A and B becomes wall following to the right between B and A) The wall following behaviour used in our experiments was a learned behaviour based on work carried out by Nehmzow [11]. This model was augmented by Duckett and Nehmzow [3] in work on localisation to enable the robot s turret to operate independently of the base. In using wall following, the map building process itself can be made more autonomous. In areas where wall following is appropriate, the robot can be ....

U. Nehmzow, Experiments in competence acquisition for autonomous mobile robots, Ph.D. Thesis, Department of Artificial Intelligence, University of Edinburgh, 1992.

....successfully and in real time ( Nehmzow 94, Nehmzow 95] Published at Intern. Conf. Intelligent Autonomous Vehicles 95, Helsinki. Names appear in alphabetical order, with both being principal authors. The experiments presented in this paper have taken our previous approach to robot control ( Nehmzow 92] autonomous competence acquisition by means of connectionist computing further: feedback for the reinforcement learning process is provided by the user, allowing the robot to acquire task achieving competences through observation of a human operator. This approach is advantageous for ....

....the desired task, the same robot can be used for a variety of different tasks without the need for (expensive) reprogramming, thus reducing setup costs. Contrary to reinforcement learning architectures enabling autonomous competence acquisition (for example [Mahadevan Connell 91, Kaelbling 91, Nehmzow 92] learning here is supervised by a human operator, an aspect that can be relevant regarding safety aspects of industrial applications. 1 Figure 1: The Manchester Mobile Robot, FortyTwo 2 THE ROBOT FortyTwo (see figure 1) a Nomad 200 mobile robot, is fully autonomous and operates ....

Ulrich Nehmzow, Experiments in Competence Acquisition for Autonomous Mobile Robots, PhD thesis, University of Edinburgh 1992.

....and symbolic representation, minimalist use of computational resources, and tight coupling between sensory input and effector output. Examples for behaviour based robot controllers are the MIT robots ( Brooks 85, Brooks 86, Brooks et al. 88] or the Edinburgh robots ( Malcolm Smithers 88, Nehmzow 92] Nehmzow et al. 93] see as well [Arkin 87, Payton 86] So, on the one hand there are symbol based planning programs that can determine series of actions leading towards specified goal situations, provided the internal model of environment, agent, and agent world interaction are adequate and ....

....input to the network may vary from experiment to experiment, the output nodes denote motor actions. The result of this process is that effective associations between input stimuli and output signals (motor actions) arise. The instinct rules and sensors we used are described in more detail in [Nehmzow 92] the following sections of this paper briefly describe the experiments and contain references to the relevant literature. 4 Experiments The controller described in general in section 3 has been implemented on a number of different mobile robots by different research groups. The consistent ....

[Article contains additional citation context not shown here]

Ulrich Nehmzow, Experiments in Competence Acquisition for Autonomous Mobile Robots, PhD thesis, University of Edinburgh 1992.

....perceptions, to a larger degree than, for instance, stationary robots. Any sensor signal processing mechanism that can cope with such data is therefore particularly suitable for mobile robot control. Artificial neural networks have been shown to be a suitable mechanism for this purpose (e.g. Nehmzow 92, Zalzala Morris 96] In this paper, we discuss two experimental scenarios, in which a Nomad 200 mobile robot (see figure 1) acquires fundamental sensory motor competences through neural network learning, using input from a CCD camera. In the first example, supervised teaching is used to train ....

.... if not necessary alternative ( Barto 89, Sutton 91] Earlier experiments have demonstrated that such techniques enable robots to learn from reinforcement, successfully and in real time ( Nehmzow 94, Nehmzow 95a] The following experiments have taken our previous approach to robot control ( Nehmzow 92] autonomous competence acquisition by means of connectionist computing further: feedback for the reinforcement learning process is provided by the user, allowing the robot to acquire task achieving competences through observation of a human operator. This approach is advantageous for ....

[Article contains additional citation context not shown here]

Ulrich Nehmzow, Experiments in Competence Acquisition for Autonomous Mobile Robots, PhD thesis, University of Edinburgh 1992.

....Mobile Robotics Laboratory is, firstly, to provide access to mobile robot resources so that robotics experiments can be conducted even if no local robotics facilities are available. Secondly, we aim to improve remote robot control through machine learning techniques. In work reported elsewhere ( Nehmzow 92, Nehmzow McGonigle 94, Martin Nehmzow 95] we have used artificial neural networks for acquisition of fundamental sensor motor competences; first experiments show that such learning controllers can be used for remote robot control as well. Thirdly, a collaborative project between the Robotics ....

U. Nehmzow, Experiments in Competence Acquisition for Autonomous Mobile Robots, PhD thesis, University of Edinburgh 1992.

No context found.

Ulrich Nehmzow, Experiments in Competence Acquisition for Autonomous Mobile Robots, PhD thesis, Edinburgh University, 1991.

....identify a curriculum of robot learning At the bottom level, there is probably the learning of relexive sensor motor associations, expressed in competences such as obstacle avoidance, wall following, and following attractors. Competences at this level are state of the art in mobile robotics (see [Nehmzow 92] for a review) How to combine several fundamental competences to achieve more complicated tasks, how to identify components of plans and how to assemble them to form complete plans, must be elements of the robot learning curriculum. Here is an attempt to define a curriculum of robot learning: ....

....light seeking behaviour. Whilst Walter was dependent on electronic circuitry to implement his learning controller, and robots were therefore built for specific tasks, we now have controllers that allow a robot to acquire different tasks by merely redefining the reward function (see, for instance, Nehmzow 92] for an overview) Learning from immediate reward is the most effective way of learning. Competences such as phototaxis can be acquired within minutes by robots ( Nehmzow 94] In some cases, however, immediate reward is not available. If the task has been described in such a way that reward is ....

U. Nehmzow, Experiments in Competence Acquisition for Autonomous Mobile Robots, PhD thesis, University of Edinburgh 1992.

....local landmarks such as passive or active beacons, walls and corners, and reference landmarks (a compass sense) canonical paths and a topological representation of the environment. We have already shown that such a system can be used for successful location identification and limited navigation ( Nehmzow 92] The robots used in those early experiments were equipped with tactile sensors only, whereas the robots used here have far more sensors (see section 5.1) For short travelling distances (up to four infrared sensor ranges, approximately 4 m) we propose a navigation system that is based on both ....

Ulrich Nehmzow, Experiments in Competence Acquisition for Autonomous Mobile Robots, Ph.D. Thesis, Edinburgh University, 1992.

....of a particular colour. In this way, other distractor objects can be avoided, even within the crucial search space. It is also clear that there are many other ways of implementing solutions to the navigation problem we pose. Location recognition whilst following canonical paths is one way ( Nehmzow 92] Route learning is another powerful yet low level solution which many animals use 4 . There is no reason in principle why a 4 Particularly those which occupy subterranean niches, such as rodents in burrows. succession of landmarks could not be learnt by a robot in an invariant chain in ....

Ulrich Nehmzow, 1992. Experiments in Competence Acquisition for Autonomous Mobile Robots. Ph.D. Thesis, Edinburgh University, 1992.

....a priori defined strategies is (in practice) impossible. One possible solution to the problem of coping with such unforeseen situations is the use of self organising controllers that determine the effective wiring between sensors and actuators autonomously. Such an approach is described in [Nehmzow 92] for example. An additional property of the proposed architecture is discussed in this paper: that it allows an easy expansion of the robot s behavioural repertoire by adding so called instinct rules, without necessitating changes to the controller architecture itself. This increases the ....

....robot with its environment. 2 Strictly speaking, the robot only monitors the signals coming from the whiskers, the whiskers being off when they are straight and on when they are bent. There is no notion of a bent whisker being identical with an obstacle, for instance. 3 For more details see [Nehmzow 92] 4 If more than one instinct rule is used instinct rules are tested for violation beginning at the latest (the newest) and ending at the first instinct rule. 5 This value is mostly dependent on the velocity of the robot. 6 At the beginning of the learning phase, when no associations are ....

[Article contains additional citation context not shown here]

Ulrich Nehmzow, Experiments in Competence Acquisition for Autonomous Mobile Robots, PhD Thesis, University of Edinburgh 1992

....investigates the use of learning controllers in telerobotic applications. Experiments have already been conducted in which FortyTwo acquires sensor motor competences through remote control, and uses these skills to move autonomously, with only occasional interference by the operator (see [Nehmzow 92, Nehmzow 95] regarding the controller used for these experiments) A second area of investigation is to add further sensor modalities to the telerobotics control system. In particular, we intend to use the robot s visual sensor for this purpose. Using image processing and data compression ....

U. Nehmzow, Experiments in Competence Acquisition for Autonomous Mobile Robots, PhD thesis, University of Edinburgh, 1992.

....to acquire fundamental sensor motor couplings rapidly, requiring a couple of learning steps only. The following sections describe the robots, the controller architecture and the experiments we conducted regarding learning in mobile robots. More details about these experiments can be found in [Nehmzow 92] 2 Competence Acquisition, using Artificial Neural Networks 2.1 The Robots The experiments described in this paper were conducted with two small mobile robots, shown in figure 1. Alder is controlled by an ARC52 controller, based on the 8052 microprocessor, having 16k of RAM. Cairngorm is ....

.... layer (see figure 3) The fact that it learns very quickly is bought at the price that it can only learn functions that are linearly separable ( Minsky Papert 88] The functions Alder and Cairngorm had to learn in the particular experiments discussed in this paper are linearly separable (see [Nehmzow 92] for a discussion of this point) it was therefore possible to use the Pattern Associator in these experiments as an associative memory, associating stimuli with robot motor actions. The function of input and output units is as follows: input units simply pass the received input signals i on ....

Ulrich Nehmzow, Experiments in Competence Acquisition for Autonomous Mobile Robots, Ph.D. thesis, Edinburgh University, 1992.

....(an approach we consider unsuitable for all but the most basic behaviours) or self tuition. The latter involves autonomous acquisition of taskachieving competences, and has successfully been used to accomplish tasks such as obstacle avoidance and contour following ( Nehmzow et al. 89, Nehmzow 91] box pushing ( Mahadevan Connell 91] leg coordination ( Maes Brooks 90] or phototaxis ( Kaelbling 90, Colombetti Dorigo 93] The advantage of providing external feedback (shaping) is, firstly, the considerable acceleration of the learning process. The fundamental robot behaviours ....

.... w j : o j (t) i(t) Delta w j (t) 1) 2.4.2 Teaching the Network Training a Pattern Associator is done in a supervised way, i.e. a teaching signal, signifying the desired output vector , given a particular input vector i is provided from outside. In work reported elsewhere ( Nehmzow 91, Nehmzow et al. 93] such teaching signals were generated internally by the controller, using so called instinct rules; in the experiments described here, however, they are provided externally by the experimenter: by covering or uncovering the upward facing light sensor of the robot positive or ....

Ulrich Nehmzow, Experiments in Competence Acquisition for Autonomous Mobile Robots, PhD thesis, Edinburgh University, 1991.

....of learning steps, within seconds. In order to achieve as flexible a control as possible, we believe it is essential to reduce the amount of predefined knowledge used by the robot. In this paper we present an extension to earlier work on autonomous competence acquisition ( Nehmzow et al. 89, Nehmzow 92] that reduces the amount of necessary predefined knowledge even further by using artificial motor neurons that drive motors directly. 1.1 Background: Reinforcement Learning Reinforcement Learning the learning mechanism used here covers techniques of learning by trial and error through ....

....a couple of learning steps only. It is our experience that, in order to increase the robot s flexibility in unforeseen situations, it is beneficial to reduce the amount of predefined knowledge used in the controller as much possible. This paper describes extensions to work presented earlier ( Nehmzow 92] that do exactly this: by introducing the concept of an artificial motor neuron the output space of the artificial neural network is reduced, while the repertoire of possible robot actions is increased, now covering the complete range of physically possible actions. 2 The Manchester Mobile ....

[Article contains additional citation context not shown here]

Ulrich Nehmzow, Experiments in Competence Acquisition for Autonomous Mobile Robots, Ph.D. thesis, Edinburgh University, 1992.

No context found.

Ulrich Nehmzow. Experiments in Competence Acquisition for Autonomous Mobile Robots. PhD thesis, Department of Arti cial Intelligence, University of Edinburgh, 1992.

No context found.

U. Nehmzow. Experiments in Competence Acquisition for Autonomous Mobile Robots. PhD thesis, Department of Arti cial Intelligence, University of Edinburgh, 1992.

No context found.

U. Nehmzow, "Experiments in Competence Acquisition for Autonomous Mobile Robots", PhD thesis, Department of Artificial Intelligence, University of Edinburgh, 1992.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC