T. Ziemke, "Adaptive Behavior in Autonomous Agents", PRESENCE, 7(6) (1998) 564-587.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....many robotic systems have included learning techniques. Adaptation is also needed in order to be able to perform in different and changing environments. There isn t yet an established methodology to develop adaptive behavior based systems. A commonly used approach is Reinforcement Learning [24]. Reinforcement Learning (RL) 9,19] is a class of learning algorithm in which an agent tries to maximize a scalar evaluation (reward or punishment) of its interaction with the environment. The evaluation is generated by the critic using an utility function. A RL system tries to map the states of ....

Ziemke, T. Adaptive Behavior in Autonomous Agents. Presence, vol. 7, is. 6, pp. 564-587, 1998.

....of science and technology, in the relatively pragmatic and demanding field of entertainment, an artificial system is best instilled from the beginning with as much knowledge as its designers can impart. This has been referred to as the engineering approach to artificial intelligence development (Ziemke, 1998), and follows from our work on Edmund and Ymir. Similarly, AI developers should not necessarily be expected to be sufficiently skilled artists that they can create the plots and characters needed for a fully engaging interactive play experience. AI attracts (and perhaps requires) developers with ....

Ziemke, T. (1998). Adaptive behavior in autonomous agents. Presence, 7(6).

....and experience or that they have genuine autonomy, subjectivity, qualia, experience and perception, or that the type of learning and evolution we discuss is the same as in living organisms. That is an incorrect impression, as will be discussed in further detail in Section 5. 4 (cf. also Sharkey and Ziemke, 1998; Ziemke and Sharkey, in press) However, instead of marking each term with quotes or qualifications such as it has been 33 argued that , we have put in this disclaimer so that we can simplify and improve the flow of the discussion. 4.2 Artificial Life Models One of the earliest autonomous ....

....autonomous agents structurally coupled with their environment. Similarly there is disagreement about what exactly is meant by autonomy . A number of researchers have criticized Brooks original approach for the lack of adaptivity, something they consider an essential aspect of autonomy (cf. Ziemke, 1998). Sharkey and Heemskerk (1997) for example, used the metaphor of the environmental puppeteer guiding the robot by the strings of predetermined reactive mechanisms and they pointed out: An important goal of modern scientific robotics is to cut the strings and give the robot its autonomy. We ....

[Article contains additional citation context not shown here]

Ziemke, Tom (1998). Adaptive Behavior in Autonomous Agents. Presence, 7(6):564-587.

....they artificially learn, develop and evolve in interaction with their environments, typically using computational learning techniques, such as artificial neural networks or evolutionary algorithms. Due to the biological inspiration and motivation underlying much of this research (cf. Sharkey and Ziemke 1998), autonomous agents are often referred to as artificial organisms , artificial life , animats (short for artificial animals ) Wilson 1985) creatures (Brooks 1990) or biorobots (Ziemke and Sharkey 1998) These terms do not necessarily all mean exactly the same; some of them refer to ....

....autonomous systems , and does not at all distinguish between living and non living in his discussion of semiosis in such systems. We have previously discussed this distinction in an examination of the biological and psychological foundations of modern autonomous robotics research (Sharkey and Ziemke 1998). In that paper we investigated differences between the embodiment of living and non living systems, and their implications for the possibility of cognitive processes in artifacts. In this paper the issues are further analyzed with reference to Jakob von Uexkll s theory of meaning. As a result ....

[Article contains additional citation context not shown here]

Ziemke, Tom (1998). Adaptive Behavior in Autonomous Agents. Presence, 7(6):564-587.

....by the agents in order to process a transaction, but rather to concentrate on the basic coordination mechanisms that come into play in the interactions between agents. These aspects (negotiation and control algorithms) will be tackled in a next stage. In this implementation, agents are autonomous [10] in the sense that they can make decisions on their own, without user intervention. We briefly describe the organization of the agents in our trading system and their interactions. Company or private customers (end user agents) create trade queries to buy or sell goods. These queries are written ....

T. Ziemke. Adaptive Behavior in Autonomous Agents. Autonomous Agents, Adaptive Behaviors and Distributed Simulations' journal, 1997.

....components [ 9 ] Steels in [ 27 ] adds that an emergent behavior leads to emergent functionality if the behavior contributes to the system s self preservation and if the system can build further upon it. 2. 1 Autonomy In Autonomous Agents, two types of autonomy are commonly pointed out [ 30 ] : operational autonomy and behavioral autonomy 1 . Both types agree in the idea that automaticity is necessary to autonomy. Ac 1 This distinction is only meaningful at the level of the designer. cording, to Steels in [ 26 ] an agent, to be autonomous, must first be automatic: it must be ....

T. Ziemke. Adaptive Behavior in Autonomous Agents. Autonomous Agents, Adaptive Behaviors and Distributed Simulations' journal, 1997.

....for instance) The general architecture of an agent is displayed on figure 2. An agent possesses some sensors to perceive the world within which it moves, and some effectors to act in this world, so that it complies with the prescriptions of physically embodied agents and simulated embodied agents [28]. The implementation of the different modules presented on figure 2, namely perception, state, actions and control algorithm depends on the application and is under the user s responsibility. The control algorithm module is particularly important because it defines the type of autonomy of the ....

....on the application and is under the user s responsibility. The control algorithm module is particularly important because it defines the type of autonomy of the agent: it is precisely inside this module that the designer decides whether to implement an operational autonomy or a behavioral autonomy [28]. Operational autonomy is defined as the capacity to operate without human intervention, without being remotely controlled. Behavioral autonomy supposes that the basis of self steering originates in the agent s own capacity to form and adapt its principles of behavior: an agent, to be behaviorally ....

T. Ziemke, `Adaptive Behavior in autonomous agents', Autonomous Agents, Adaptive Behaviors and Distributed Simulations' journal, (1997). A Pragmatic Approach to Conflict 7 F. Chantemargue & al.

....formalization of emergence [50] 26] some others are concerned with bringing to the fore some experimental on emergence [22] 16] some others even still discuss the relevance of the concept of emergence [58] 3. 2 Autonomy In Autonomous Agents, two types of autonomy are commonly pointed out [70]: operational autonomy and behavioral autonomy. Both types agree in the idea that automaticity is necessary to autonomy. According, to Steels in [63] an agent, to be autonomous, must first be automatic: it must be able to operate in an environment, to sense this environment and to impact in ways ....

T. Ziemke. Adaptive Behavior in Autonomous Agents. Autonomous Agents, Adaptive Behaviors and Distributed Simulations' journal, 1997.

....constitute a whole discipline of Artificial Intelligence, whose description would be prohibitive to do here; as it is not the main concern of this paper, only the concepts of Autonomous Agents necessary to understand our implementation will be presented. More information can be found in [17] and [22]. We will focus exclusively on autonomous agents that are considered to be embodied systems, which are designed to fulfill internal or external goals by their own actions in continuous long term interaction with the environment (possibly unpredictable and dynamical) in which they are situated. ....

....(embodiment) The implementation of the different modules presented on Figure 2, namely Perception, State, Actions and Control Algorithm depends on the application and is the user s responsibility. The Control Algorithm module is particularly important because it defines the type of autonomy [22] of the agent: for instance, a very basic autonomy would consist of randomly choosing the type of action to take, a more sophisticated one would consist of implementing some learning capabilities, e.g. by using an adaptive neural network. 3.2 A Typical Application We illustrate with a simulation ....

T. Ziemke. Adaptive Behavior in autonomous agents. To appear in Autonomous Agents, Adaptive Behaviors and Distributed Simulations' journal, 1997.

....systems) that allows them to reason within this domain. These systems are very efficient in their domain of expertise and provide a good framework for high level reasoning. But they suffer from lacking ability to relate internal representations to the external world [4] and to extend them [22] [50]. Moreover (and consequently) they tend to utterly fail when facing problems even slightly outside their domain of expertise [27] or when slight changes occur in the problem structure. Most common drawbacks inherent to classical AI are known as the frame problem (problem of maintaining a model of ....

T. Ziemke. Adaptive Behavior in Autonomous Agents. Autonomous Agents, Adaptive Behaviors and Distributed Simulations' journal, 1997.

....of a large number of interacting entities exhibit global properties that do not exist at the level of the individual entities, that we do characterize as emergent properties. Finally, another important concept in Autonomous Agents is that of autonomy. Two types of autonomy are commonly pointed out [30]: operational autonomy and behavioral autonomy. Operational autonomy is defined as the capacity to operate without human intervention, without being remotely controlled. Behavioral autonomy supposes that the basis of self steering originates in the agent s own capacity to form and adapt its ....

T. Ziemke. Adaptive Behavior in Autonomous Agents. Autonomous Agents, Adaptive Behaviors and Distributed Simulations' journal, 1997.

....BEGIN CONTAINER You SIZE 15 WHEIGHT ALLOWANCE 15 SKILLS look, move, get, drop, open, close CONTENTS Book3, Glasses CONTROL ByTelnet END CONTAINER You Fig. 1. a typical EMud initialization file for an agent, different levels of autonomy can be achieved. Three fundamental levels can be distinguished [26]. From automatic agents, automaticity implying that the agent has access to its environment and can influence its own future existence through actions in that environment (a prerequisite for an agent to be autonomous [22] on to operationally and behaviorally autonomous agents. Operational ....

T. Ziemke. Adaptive Behavior in autonomous agents. To appear in Autonomous Agents, Adaptive Behaviors and Distributed Simulations' journal, 1997.

....features of one cell, or a small subset of cells, at a given time. The control algorithm module is particularly important because it defines the type of autonomy of the agent: it is precisely inside this module that the designer decides whether to implement an operational or a behavioral autonomy [33]. Operational autonomy is defined as the capacity to operate without human intervention, without being remotely controlled. Behavioral autonomy supposes that the basis of self steering originates in the agent s own capacity to form and adapt its principles of behavior: an agent, to be behaviorally ....

T. Ziemke. Adaptive Behavior in autonomous agents. Autonomous Agents, Adaptive Behaviors and Distributed Simulations' journal, 1997.

....provide powerful coordination mechanisms that do not alter the model s conceptual prescriptions. 1 Introduction Artificial Intelligence (AI) aims at synthesizing intelligence in artefacts. However two families of approaches exist disagreeing in their notion of what intelligence actually means [ Ziemke, 1997 ] Franklin, 1995 ] On the one hand, Top Down AI considers intelligence as the capacity to form and manipulate internal representational models of the world. On the other hand, Bottom Up AI (or Autonomous Agents) considers intelligence as a biological feature [ Maturana and Varela, 1980 ] ....

....it at a given time. The architecture of an agent is displayed on figure 1. An agent possesses some sensors to perceive the world within which it moves, and some effectors to act in this world, so that it complies with the prescriptions of physically embodied agents and simulated embodied agents [ Ziemke, 1997 ] The implementation of the different modules presented on Figure 1, namely Perception, State, Actions and Control Algorithm depends on the application and is the user s responsibility. In the Perception module, the designer specifies the type of perception of the agent, e.g. if the agent ....

[Article contains additional citation context not shown here]

T. Ziemke. Adaptive Behavior in autonomous agents. To appear in Autonomous Agents, Adaptive Behaviors and Distributed Simulations ' journal, 1997.

....to reach out into its environment and directly interact with it, i.e. they offer a way to escape the internalist trap. Physical grounding does, however, only offer a pathway for hooking an agent to its environment. It does, by itself, not ground behaviour or internal mechanisms (cf. Sharkey Ziemke 1998; Rylatt Czarnecki 1998; cf. also Searle s (1980) discussion of the robot reply to the CRA) as will be discussed in detail in the following. Grounding Behaviour: Instead of the central modelling and control typical for the cognitivist paradigm, enactive systems typically consist of a number ....

.... Brooksian notion of situatedness) that determines an agent s behaviour, i.e. if the agent is merely reacting to its current environment, then the agent is best described as controlled by the environmental puppeteer (Sharkey Heemskerk 1997) rather than as an autonomous agent (cf. Ziemke 1997; Ziemke 1998). This is also reflected in Pfeifer s (1995) more encompassing definition of a situated agent: a situated agent is one which can bring to bear its own experience onto a particular situation, and the interaction of its experience with the current situation will determine the agent s ....

[Article contains additional citation context not shown here]

Ziemke, T. (1998) Adaptive Behavior in Autonomous Agents. Presence, 5(6).

.... divide andconquer approach is that the divide step, i.e. the decomposition of a control task, is usually carried out be a designer, based on his her high level understanding from a distal perspective, which is not at all guaranteed to be compatible with the robot s proximal perspective (cf. also Ziemke, 1996, 1998). Nolfi s therefore proposed his emergent modular architecture, in which a maximum number of artificial neural network (ANN) modules is pregiven, but it is left to the robot itself to determine in a process of self organization (using an evolutionary algorithm) a) how many of these modules to ....

....modular architectures (with or without feedback) This alternative is therefore examined in detail throughout the rest of this paper. 3. Recurrent Neural Networks and Diachronic Structure RNNs are commonly used as control and learning mechanisms for autonomous agents (e.g. Beer, 1996; Bir and Ziemke, 1998; Nolfi and Tani, 1999; Tani, 1996; Tani and Nolfi, 1998; Rylatt et al., 1998; Ziemke, 1996; for an overview see also Ziemke, 1999) What makes RNNs particularly interesting for the study of adaptive behavior is their use of feedback, which allows the input output mapping to vary with the network s ....

[Article contains additional citation context not shown here]

Ziemke, Tom (1998). Adaptive Behavior in Autonomous Agents. Presence, 7(6):564-587.

....C evolved in experiment 2 shows that successful networks remember having picked up an object and thus significantly less often than the others try to release an object while in fact they are not carrying one. Thus, these networks realise a form of virtual modularity through feedback (cf. [7, 11, 12]) That means, via their internal feedback and memory units they dynamically adapt their behavioural dispositions, such that they act as if they were using different modules for finding objects and for carrying them out of the arena respectively. This allows the robot to attribute different ....

T. Ziemke, Adaptive behavior in autonomous agents, Presence 7(6), pp. 564 - 586, December 1998.

....private subjective experiences . Searle s view is biological. He holds that the phenomenal mind is caused by a real living brain. 3 Embodied Cognition Behaviour based robotics represents an alternative approach to AI that has been gaining ground over the last decade (for overviews see [11, 1, 60, 41]) Although the basis for the approach has been around in biology for nearly a century and in robotics for over fifty years, it entered mainstream AI in the 1980s through the work and ideas of Rodney Brooks [6 11] His approach to the study of intelligence was through the construction of ....

Tom Ziemke. Adaptive Behavior in Autonomous Agents. Presence, 7(6):564--587, 1998.

.... and van der Smagt, 1997, Sharkey, 1997a] ANNs offer flexibility, robustness to noise and the capability of learning by example or by trial and error (reinforcement) They are therefore commonly considered to be effective mechanisms for controlling autonomous robots, e.g. Meeden et al. 1993, Ziemke, 1998] Connectionist ideas also mesh well with reactive or behaviour based robotics although this is not always appreciated. A key feature of the use of ANNs for robot control is their capacity for learning and selforganisation. Unlike traditional AI methods, ANNs are not based on explicit, symbolic ....

....to the representationalist computationalist framework of cognitivism. In particular, Brooks put forward his behaviour based AI approach [Brooks, 1986b, Brooks, 1990, Brooks, 1991a] and Wilson formulated the animat approach to AI [Wilson, 1985, Wilson, 1991] For a more detailed review see [Ziemke, 1998]. In this approach, robotic agents are typically considered physically grounded as Brooks explains. Nouvelle AI is based on the physical grounding hypothesis. This hypothesis states that to build a system that is intelligent it is necessary to have its representations grounded in the physical ....

Ziemke, T. (1998). Adaptive Behavior in Autonomous Agents. Presence, 7(6):564--587.

No context found.

T. Ziemke, "Adaptive Behavior in Autonomous Agents", PRESENCE, 7(6) (1998) 564-587.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC