Results 1 - 10
of
394
The neural basis of human error processing: Reinforcement learning, dopamine, and the error-related negativity
- PSYCHOLOGICAL REVIEW 109:679–709
, 2002
"... The authors present a unified account of 2 neural systems concerned with the development and expression of adaptive behaviors: a mesencephalic dopamine system for reinforcement learning and a “generic ” error-processing system associated with the anterior cingulate cortex. The existence of the error ..."
Abstract
-
Cited by 430 (20 self)
- Add to MetaCart
(Show Context)
The authors present a unified account of 2 neural systems concerned with the development and expression of adaptive behaviors: a mesencephalic dopamine system for reinforcement learning and a “generic ” error-processing system associated with the anterior cingulate cortex. The existence of the error-processing system has been inferred from the error-related negativity (ERN), a component of the event-related brain potential elicited when human participants commit errors in reaction-time tasks. The authors propose that the ERN is generated when a negative reinforcement learning signal is conveyed to the anterior cingulate cortex via the mesencephalic dopamine system and that this signal is used by the anterior cingulate cortex to modify performance on the task at hand. They provide support for this proposal using both computational modeling and psychophysiological experimentation. Human beings learn from the consequences of their actions. Thorndike (1911/1970) originally described this phenomenon with his law of effect, which made explicit the commonsense notion that actions that are followed by feelings of satisfaction are more likely to be generated again in the future, whereas actions that are followed by negative outcomes are less likely to reoccur. This
Learning to Perceive the World as Articulated: An Approach for Hierarchical Learning in Sensory-Motor Systems
- NEURAL NETWORKS
, 1999
"... This paper describes how agents can learn an internal model of the world structurally by focusing on the problem of behavior-based articulation. We develop an on-line learning scheme -- the so-called mixture of recurrent neural net (RNN) experts -- in which a set of RNN modules becomes self-organ ..."
Abstract
-
Cited by 141 (31 self)
- Add to MetaCart
This paper describes how agents can learn an internal model of the world structurally by focusing on the problem of behavior-based articulation. We develop an on-line learning scheme -- the so-called mixture of recurrent neural net (RNN) experts -- in which a set of RNN modules becomes self-organized as experts on multiple levels in order to account for the different categories of sensory-motor flow which the robot experiences. Autonomous switching of activated modules in the lower level actually represents the articulation of the sensory-motor flow. In the meanwhile, a set of RNNs in the higher level competes to learn the sequences of module switching in the lower level, by which articulation at a further more abstract level can be achieved. The proposed scheme was examined through simulation experiments involving the navigation learning problem. Our dynamical systems analysis clarified the mechanism of the articulation; the possible correspondence between the articulation...
The case for motor involvement in perceiving conspecifics
- Psychological Bulletin
, 2005
"... Perceiving other people’s behaviors activates imitative motor plans in the perceiver, but there is disagreement as to the function of this activation. In contrast to other recent proposals (e.g., that it subserves overt imitation, identification and understanding of actions, or working memory), here ..."
Abstract
-
Cited by 137 (2 self)
- Add to MetaCart
(Show Context)
Perceiving other people’s behaviors activates imitative motor plans in the perceiver, but there is disagreement as to the function of this activation. In contrast to other recent proposals (e.g., that it subserves overt imitation, identification and understanding of actions, or working memory), here it is argued that imitative motor activation feeds back into the perceptual processing of conspecifics’ behaviors, generating top-down expectations and predictions of the unfolding action. Furthermore, this account incorporates recent ideas about emulators in the brain—mental simulations that run in parallel to the external events they simulate—to provide a mechanism by which motoric involvement could contribute to perception. Evidence from a variety of literatures is brought to bear to support this account of perceiving human body movement.
Imitation as a dual-route process featuring predictive and learning components: a biologically plausible computational model
, 2002
"... ..."
Abnormalities in the Awareness and Control of Action
- Philos Trans R Soc Lond B Biol Sci
, 2000
"... this paper is to consider the extent to which we are aware of the functioning of some ..."
Abstract
-
Cited by 111 (2 self)
- Add to MetaCart
(Show Context)
this paper is to consider the extent to which we are aware of the functioning of some
Emergence of functional hierarchy in a multiple timescale neural network model: A humanoid robot experiment”, PLoS
- Computational Biology
"... It is generally thought that skilled behavior in human beings results from a functional hierarchy of the motor control system, within which reusable motor primitives are flexibly integrated into various sensori-motor sequence patterns. The underlying neural mechanisms governing the way in which cont ..."
Abstract
-
Cited by 93 (15 self)
- Add to MetaCart
(Show Context)
It is generally thought that skilled behavior in human beings results from a functional hierarchy of the motor control system, within which reusable motor primitives are flexibly integrated into various sensori-motor sequence patterns. The underlying neural mechanisms governing the way in which continuous sensori-motor flows are segmented into primitives and the way in which series of primitives are integrated into various behavior sequences have, however, not yet been clarified. In earlier studies, this functional hierarchy has been realized through the use of explicit hierarchical structure, with local modules representing motor primitives in the lower level and a higher module representing sequences of primitives switched via additional mechanisms such as gate-selecting. When sequences contain similarities and overlap, however, a conflict arises in such earlier models between generalization and segmentation, induced by this separated modular structure. To address this issue, we propose a different type of neural network model. The current model neither makes use of separate local modules to represent primitives nor introduces explicit hierarchical structure. Rather than forcing architectural hierarchy onto the system, functional hierarchy emerges through a form of self-organization that is based on two distinct types of neurons, each with different time properties (‘‘multiple timescales’’). Through the introduction of multiple timescales, continuous sequences of behavior are segmented into reusable primitives, and the primitives, in turn, are flexibly integrated into novel sequences. In experiments, the proposed network model, coordinating the physical body of a humanoid robot through
Multiple model-based reinforcement learning
- Neural Computation
, 2002
"... We propose a modular reinforcement learning architecture for non-linear, non-stationary control tasks, which we call multiple model-based reinforcement learn-ing (MMRL). The basic idea is to decompose a complex task into multiple domains in space and time based on the predictability of the environme ..."
Abstract
-
Cited by 85 (5 self)
- Add to MetaCart
(Show Context)
We propose a modular reinforcement learning architecture for non-linear, non-stationary control tasks, which we call multiple model-based reinforcement learn-ing (MMRL). The basic idea is to decompose a complex task into multiple domains in space and time based on the predictability of the environmental dynamics. The 1 system is composed of multiple modules, each of which consists of a state predic-tion model and a reinforcement learning controller. The “responsibility signal,” which is given by the softmax function of the prediction errors, is used to weight the outputs of multiple modules as well as to gate the learning of the predic-tion models and the reinforcement learning controllers. We formulate MMRL for both discrete-time, finite state case and continuous-time, continuous state case. The performance of MMRL was demonstrated for discrete case in a non-stationary hunting task in a grid world and for continuous case in a non-linear, non-stationary control task of swinging up a pendulum with variable physical parameters. 1
Hierarchical attentive multiple models for execution and recognition of actions
- ROBOTICS AND AUTONOMOUS SYSTEMS
, 2005
"... According to the motor theories of perception, the motor systems of an observer are actively involved in the perception of actions when these are performed by a demonstrator. In this paper we review our computational architecture, HAMMER (Hierarchical Attentive Multiple Models for Execution and Reco ..."
Abstract
-
Cited by 84 (20 self)
- Add to MetaCart
According to the motor theories of perception, the motor systems of an observer are actively involved in the perception of actions when these are performed by a demonstrator. In this paper we review our computational architecture, HAMMER (Hierarchical Attentive Multiple Models for Execution and Recognition), where the motor control systems of a robot are organised in a hierarchical, distributed manner, and can be used in the dual role of (a) competitively selecting and executing an action, and (b) perceiving it when perfomed by a demonstrator. We subsequently demonstrate that such arrangement can provide a principled method for the top-down control of attention during action perception, resulting in significant performance gains. We assess these performance gains under a variety of resource allocation strategies.
Learning from Observation Using Primitives
, 2004
"... This paper describes the use of task primitives in robot learning from observation. A framework has been developed that uses observed data to initially learn a task and then the agent goes on to increase its performance through repeated task performance (learning from practice). Data that is collect ..."
Abstract
-
Cited by 70 (5 self)
- Add to MetaCart
(Show Context)
This paper describes the use of task primitives in robot learning from observation. A framework has been developed that uses observed data to initially learn a task and then the agent goes on to increase its performance through repeated task performance (learning from practice). Data that is collected while a human performs a task is parsed into small parts of the task called primitives. Modules are created for each primitive that encode the movements required during the performance of the primitive, and when and where the primitives are performed. The feasibility of this method is currently being tested with agents that learn to play a virtual and an actual air hockey game. 1
Challenge Point: A Framework for Conceptualizing the Effects of Various Practice Conditions in Motor Learning
"... ABSTRACT. The authors describe the effects of practice conditions in motor learning (e.g., contextual interference, knowledge of results) within the constraints of 2 experimental variables: skill level and task difficulty. They use a research framework to conceptualize the interaction of those varia ..."
Abstract
-
Cited by 70 (0 self)
- Add to MetaCart
(Show Context)
ABSTRACT. The authors describe the effects of practice conditions in motor learning (e.g., contextual interference, knowledge of results) within the constraints of 2 experimental variables: skill level and task difficulty. They use a research framework to conceptualize the interaction of those variables on the basis of concepts from information theory and information processing. The fundamental idea is that motor tasks represent different challenges for performers of different abilities. The authors propose that learning is related to the information arising from performance, which should be optimized along functions relating the difficulty of the task to the skill level of the performer. Specific testable hypotheses arising from the framework are also described. Key words: augmented feedback, contextual interference, motor learning P ractice is generally considered to be the single most important factor responsible for the permanent improvement in the ability to perform a motor skill (i.e.,
