In this article we compare methods for temporal sequence learning (TSL) across the disciplines machine-control, classical conditioning, neuronal models for TSL as well as spiketiming dependent plasticity. This review will briefly introduce the most influential models and focus on two questions: 1) To what degree are reward-based (e.g. TD-learning) and correlation based (hebbian) learning related? and 2) How do the different models correspond to possibly underlying biological mechanisms of synaptic plasticity? We will first compare the different models in an open-loop condition, where behavioral feedback does not alter the learning. Here we observe, that reward-based and correlation based learning are indeed very similar. Machine-control is then used to introduce the problem of closed-loop control (e.g. “actor-critic architectures”). Here the problem of evaluative (“rewards”) versus nonevaluative (“correlations”) feedback from the environment will be discussed showing that both learning approaches are fundamentally different in the closed-loop condition. In trying to answer the second question we will compare neuronal versions of the different learning architectures to the anatomy of the involved brain structures (basal-ganglia, thalamus and

this article we will analytically demonstrate that this process can be understood in terms of control theory showing that the system learns the inverse controller of its own reflex. Thereby this system is able to learn a simple form feed-forward motor control

Biped walking remains a difficult problem and robot models can greatly facilitate our understanding of the underlying biomechanical principles as well as their neuronal control. The goal of this study is to specifically demonstrate that stable biped walking can be achieved by combining the physical properties of the walking robot with a small, reflex based neuronal network, which is governed mainly by local sen-sor signals. Building on earlier work (Taga, 1995; Cruse et al., 1998), this study shows that human-like gaits emerge without specific posi-tion or trajectory control and that the walker is able to compensate small disturbances through its own dynamical properties. The re-flexive controller used here has the following characteristics, which are different from earlier approaches: (1) Control is mainly local. Hence, it uses only two signals (AEA=Anterior Extreme Angle and GC=Ground Contact) which operate at the inter-joint level. All other

Abstract not found

Biped walking remains a difficult problem and robot models can greatly facilitate our understanding of the underlying biomechanical principles as well as their neuronal control. The goal of this study is to specifically demonstrate that stable biped walking can be achieved by combining the physical properties of the walking robot with a small, reflex based neuronal network, which is governed mainly by local sensor signals. Building on earlier work (Taga, 1995; Cruse et al., 1998), this study shows that human-like gaits emerge without specific position or trajectory control and that the walker is able to compensate small disturbances through its own dynamical properties. The reflexive controller used here has the following characteristics, which are different from earlier approaches: (1) Control is mainly local. Hence, it uses only two signals (AEA=Anterior Extreme Angle and GC=Ground Contact) which operate at the inter-joint level. All other

Abstract. Spiking Neuron Networks (SNNs) are often referred to as the 3 rd generation of neural networks. They derive their strength and interest from an accurate modelling of synaptic interactions between neurons, taking into account the time of spike emission. SNNs overcome the computational power of neural networks made of threshold or sigmoidal units. Based on dynamic event-driven processing, they open up new horizons for developping models with an exponential capacity of memorizing and a strong ability to fast adaptation. Today, the main challenge is to discover efficient learning rules that might take advantage of the specific features of SNNs while keeping the nice properties (general-purpose, easy-to-use, available simulators, etc.) of current connectionist models (such as MLP, RBF or SVM). The present survey relates the history of the “spiking neuron ” and summarizes the most currenlty in use models of neurons and networks, in Section 1. The computational power of SNNs is addressed in Section 2 and the problem of learning in networks of spiking neurons is tackled in Section 3, with insights into the tracks currently explored for solving it. Section 4 reviews the tricks of implementation and discuss several simulation frameworks. Examples of application domains are proposed in Section 5, mainly in speech processing and computer vision, emphasizing the temporal aspect of pattern recognition by SNNs.

In this article we will compare traditional reinforcement learning techniques with a novel correlation based algorithm. We will discuss several problems which occur in reward-based reinforcement learning and outline alternative solutions. An example of a robot control task shown at the end will support our claims.

Abstract not found

Vision can serve many purposes when employed on a mobile robot. In order to explore the capabilities of a visually guided mobile robot, an existing robot kit used in the Intelligent Autonomous Robotics course at the University of Edinburgh was enhanced through the addition of a webcam. An API was then created for controlling the webcam within the development platform used by the robot kit. The API was used to implement two initial vision-based behaviours on a mobile robot. The first was a colour target tracking behaviour that was found to perform significantly better than a blind robot in locating coloured balls. The second was a deformable line following behaviour that extracted the line information using a classic method. The initial scope of the project proposal was then extended by implementing two more behaviours. The third behaviour relied on a biologically inspired learning technique that enabled a mobile robot to learn to follow a line using only a small amount of visual data. Finally, the same learning algorithm was combined with classic line detection so as to enable the robot to avoid obstacles. Computational efficiency and working in the face of general hardware limitations were of primary importance throughout, often requiring novel