This paper describes the use of task primitives in robot learning from observation. A framework has been developed that uses observed data to initially learn a task and then the agent goes on to increase its performance through repeated task performance (learning from practice). Data that is collected while a human performs a task is parsed into small parts of the task called primitives. Modules are created for each primitive that encode the movements required during the performance of the primitive, and when and where the primitives are performed. The feasibility of this method is currently being tested with agents that learn to play a virtual and an actual air hockey game. 1

In multiagent domains with adversarial and cooperative team agents, team agents should be adaptive to the current environment and opponent. We introduce an online method to provide the agents with team plans that a "coach" agent generates in response to the specific opponents. The coach agent can observe the agents' behaviors but it has only periodic communication with the rest of the team. The coach uses a Simple Temporal Network to represent team plans as coordinated movements among the multiple agents and the coach searches for an opponent-dependent plan for its teammates. This plan is then communicated to the agents, who execute the plan in a distributed fashion, using information from the plan to maintain consistency among the team members. In order for these plans to be effective and adaptive, models of opponent movement are used in the planning. The coach is then able to quickly select between different models online by using a Bayesian style update on a probability distribution over the models. Planning then uses the model which is found to be the most likely. The system is fully implemented in a simulated robotic soccer environment. In several recent games with completely unknown adversarial teams, the approach demonstrated a visible adaptation to the different teams.

The area of agent modeling deals with the task of observing other agents and modeling their behavior, in order to predict their future behavior, coordinate with them, assist them, or counter their actions.

is action" or very general, such as "Your goal should now be this." Also, for more general advice, the coach may want the agents to be independent and not follow advice in all situations. Given the explanation of the coach role, the coaching problem can be stated quite simply: "How can an agent in Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. AAMAS'02, July 15-19, 2002, Bologna, Italy. Copyright 2002 ACM 1-58113-480-0/02/0007 ...$5.00. a coach role improve the performance of the team?" The coach agent may be a separate agent whose only role is coach, or a team member may fulfill the coach role in addition to others. For exa

In a team-based multiagent system, the ability to construct a model of an opponent team’s joint behavior can be useful for determining an agent’s expected distribution over future world states, and thus can inform its planning of future actions. This paper presents an approach to team opponent modeling in the context of the RoboCup simulation coach competition. Specifically, it introduces an autonomous coach agent capable of analyzing past games of the current opponent, advising its own team how to play against this opponent, and identifying patterns or weaknesses on the part of the opponent. Our approach is fully implemented and tested within the RoboCup soccer server, and was the champion of the RoboCup 2005 simulation coach competition.

Abstract. The UT Austin Villa 2003 simulated online soccer coach was a first time entry in the RoboCup Coach Competition. In developing the coach, the main research focus was placed on treating advice-giving as a machine learning problem. Competing against a field of mostly handcoded coaches, the UT Austin Villa coach earned first place in the competition. In this paper, we present the multi-faceted learning strategy that our coach used and examine which aspects contributed most to the coach’s success. 1

In this paper, we investigate the hypothesis that plan recognition can significantly improve the performance of a casebased reinforcement learner in an adversarial action selection task. Our environment is a simplification of an American football game. The performance task is to control the behavior of a quarterback in a pass play, where the goal is to maximize yardage gained. Plan recognition focuses on predicting the play of the defensive team. We modeled plan recognition as an unsupervised learning task, and conducted a lesion study. We found that plan recognition was accurate, and that it significantly improved performance. More generally, our studies show that plan recognition reduced the dimensionality of the state space, which allowed learning to be conducted more effectively. We describe the algorithms, explain the reasons for performance improvement, and also describe a further empirical comparison that highlights the utility of plan recognition for this task. 1. Motivation

This paper describes a methodology for recording, representing, and recognizing team behaviors performed by human players in an Unreal Tournament MOUT (Military Operations in Urban Terrain) scenario.

In order to be truly robust, deployed robot systems must be capable of adaptation. No matter how much we know about a particular task or environment, some amount of on-site tweaking is inevitable if we want the system to perform as well as it can. This tweaking causes the behavior of the robot to change, often by only a small amount, to better fit its environment. Machine learning can be used to “program ” mobile robots, given some initial knowledge. We present a method of gathering this knowledge without the need for technical expertise in robotics or programming. The method relies on semanitic information from restricted natural language descriptions of the task to be performed. The behavior can then be modified interactively without knowledge of the underlying control mechanisms. This method allows us to tap into the experience of local experts who have done the task before, without having to train them first in computers or robotics.

The introduction of a coach competition in the RoboCup2001 simulation league raised many questions concerning the development of a "coachable" team. This paper addresses the issues of dealing with conflicting advice and knowing when to listen to advice. An action-selection architecture is proposed to support the integration of advice into an agent's set of beliefs. The results from the coach competition are discussed and provide a basis for experiments. Results are provided to support the claim that the architecture is well-suited for such a task.