Citation Context

...ct the evolution of the reward vector R(t).s3. Evaluation : For each evolution {SMRmi} an expected reward rmi is computed as the sum of all the future expected rewards. t+T X ˛ rmi(t) = ˛ ˛ ˛R(j) ˛ ˛ =-=(5)-=- 4. Selection : The motor command {mi} corresponding to the highest expected reward is chosen. The action selection module monitors how well the system is behaving by computing the average reward < ar...

Citation Context

...FORCEMENT LEARNING MODELS AND EXTINCTION 787 where s(t) was the agent’s current state; and � was a parameter that balanced exploration and exploitation (Daw, O’Doherty, Dayan, Seymour, & Dolan, 2006; =-=Doya, 2002-=-). High � drove the agent to be more likely to select the high-value action, whereas low � made the agent more likely to select actions more randomly. On every time step, � was calculated as the diffe...

Citation Context

...lection and switching behavior can emerge from the unique anatomy and dynamics of these structures. Kenji Doya and colleagues have proposed a theory that encompasses multiple neuromodulatory systems (=-=Doya, 2002-=-, 2008). In this theory, Doya suggests that each neuromodulator has a specific parametric role in the temporal difference learning; that is, dopamine represents the global learning signal for the pred...

Citation Context

...cal areas temporarily increases the responsivity (or gain) of these areas to their afferent input, selectively potentiating the processing of these task-relevant stimuli (Berridge & Waterhouse, 2003; =-=Doya, 2002-=-; ServanSchreiber, Printz, & Cohen, 1990). Conversely, the tonic LC mode is characterized by an elevated level of LC baseline activity and tonic NE release, and the absence of phasic responses.1 Accor...

Citation Context

...n error signal of the temporal difference (TD) learning algorithm (Sutton, 1988). In this scheme, a phasic dopamine signal interpreted as TD-error plays a double role (Baldassarre, 2002; Barto, 1995; =-=Doya, 2002-=-; Houk et al., 1995; Khamassi et al., 2005; Montague et al., 1996; Schultz et al., 1997; Suri and Schultz, 2001). First, this error is used as a classical training signal to improve future prediction....

Citation Context

... are probably to be made between habitslearning and goal-directed learning [12]. Finally, the role of neuromodulators in the setting of learningsmetaparameters should be investigated, as suggested in =-=[15]-=-.sFrom an engineering point of view, the system described above raises several important issues. Considering thatsliving systems are the product of roughly 3.5 billions years of evolutionary tinkering...

Citation Context

...l., 1997) which respond to environmental cues signalling reward prediction, uncertainty, and novelty. Theoretical models based of these systems and their effect on brain function have been developed (=-=Doya, 2002-=-; Friston et al., 1994; Montague et al., 1996; Yu and Dayan, 2005) and embedded in real world behaving devices (Arleo et al., 2004; Krichmar and Edelman, 2002; Sporns and Alexander, 2002). 2.6 Compari...

Citation Context

...L agent. Further, we envision to experiment with controlling metaparameters (such as exploration/exploitation and learning rate) based on the agent’s internal emotional state or social rewards [3][11]=-=[18]-=-. Currently we use simulated annealing-like mechanisms to control these parameters. Further, the agent could try to learn what an emotional expression predicts. In this case, the agent would use the e...