Results 21  30
of
111
Integrating Learning in Interactive Gaming Simulators
 in Challenges of Game AI: Proceedings of the AAAI’04 Workshop, AAAI
, 2004
"... Many developers of simulations for computergenerated forces and realtime strategy games seek to incorporate learning or learned behaviors in their systems. Likewise, many researchers seek to evaluate their learning systems in these simulators. However, these integrations require great effort. We d ..."
Abstract

Cited by 17 (5 self)
 Add to MetaCart
(Show Context)
Many developers of simulations for computergenerated forces and realtime strategy games seek to incorporate learning or learned behaviors in their systems. Likewise, many researchers seek to evaluate their learning systems in these simulators. However, these integrations require great effort. We describe our initial work on a testbed, named TIELT that we are designing to facilitate these integrations.
An Integrated Agent for Playing RealTime Strategy Games
 In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI
, 2008
"... We present a realtime strategy (RTS) game AI agent that integrates multiple specialist components to play a complete game. Based on an analysis of how skilled human players conceptualize RTS gameplay, we partition the problem space into domains of competence seen in expert human play. This partitio ..."
Abstract

Cited by 16 (7 self)
 Add to MetaCart
(Show Context)
We present a realtime strategy (RTS) game AI agent that integrates multiple specialist components to play a complete game. Based on an analysis of how skilled human players conceptualize RTS gameplay, we partition the problem space into domains of competence seen in expert human play. This partitioning helps us to manage and take advantage of the large amount of sophisticated domain knowledge developed by human players. We present results showing that incorporating expert highlevel strategic knowledge allows our agent to consistently defeat established scripted AI players. In addition, this work lays the foundation to incorporate tactics and unit micromanagement techniques developed by both man and machine.
Learning continuous action models in a realtime strategy environment
 Proceedings of the TwentyFirst Annual Conference of the Florida Artificial Intelligence Research Society (pp. 257262). Coconut
, 2008
"... Although several researchers have integrated methods for reinforcement learning (RL) with casebased reasoning (CBR) to model continuous action spaces, existing integrations typically employ discrete approximations of these models. This limits the set of actions that can be modeled, and may lead to ..."
Abstract

Cited by 15 (3 self)
 Add to MetaCart
(Show Context)
Although several researchers have integrated methods for reinforcement learning (RL) with casebased reasoning (CBR) to model continuous action spaces, existing integrations typically employ discrete approximations of these models. This limits the set of actions that can be modeled, and may lead to nonoptimal solutions. We introduce the Continuous Action and State Space Learner (CASSL), an integrated RL/CBR algorithm that uses continuous models directly. Our empirical study shows that CASSL significantly outperforms two baseline approaches for selecting actions on a task from a realtime strategy gaming environment. 1.
Transferring state abstractions between mdps
 In ICML Workshop on Structural Knowledge Transfer for Machine Learning
, 2006
"... Decision makers that employ state abstraction (or state aggregation) usually find solutions faster by treating groups of states as indistinguishable by ignoring irrelevant state information. Identifying irrelevant information is essential for the field of knowledge transfer where learning takes plac ..."
Abstract

Cited by 15 (1 self)
 Add to MetaCart
(Show Context)
Decision makers that employ state abstraction (or state aggregation) usually find solutions faster by treating groups of states as indistinguishable by ignoring irrelevant state information. Identifying irrelevant information is essential for the field of knowledge transfer where learning takes place in a general setting for multiple environments. We provide a general treatment and algorithm for transferring state abstractions in MDPs. 1.
Factored value iteration converges
 Acta Cyb
"... Abstract. In this paper we propose a novel algorithm, factored value iteration (FVI), for the approximate solution of factored Markov decision processes (fMDPs). The traditional approximate value iteration algorithm is modified in two ways. For one, the leastsquares projection operator is modified ..."
Abstract

Cited by 12 (2 self)
 Add to MetaCart
(Show Context)
Abstract. In this paper we propose a novel algorithm, factored value iteration (FVI), for the approximate solution of factored Markov decision processes (fMDPs). The traditional approximate value iteration algorithm is modified in two ways. For one, the leastsquares projection operator is modified so that it does not increase maxnorm, and thus preserves convergence. The other modification is that we uniformly sample polynomially many samples from the (exponentially large) state space. This way, the complexity of our algorithm becomes polynomial in the size of the fMDP description length. We prove that the algorithm is convergent. We also derive an upper bound on the difference between our approximate solution and the optimal one, and also on the error introduced by sampling. We analyze various projection operators with respect to their computation complexity and their convergence when combined with approximate value iteration. factored Markov decision process, value iteration, reinforcement learning 1.
Exploration in Relational Domains for Modelbased Reinforcement Learning
"... A fundamental problem in reinforcement learning is balancing exploration and exploitation. We address this problem in the context of modelbased reinforcement learning in large stochastic relational domains by developing relational extensions of the concepts of the E 3 and RMAX algorithms. Efficien ..."
Abstract

Cited by 10 (1 self)
 Add to MetaCart
A fundamental problem in reinforcement learning is balancing exploration and exploitation. We address this problem in the context of modelbased reinforcement learning in large stochastic relational domains by developing relational extensions of the concepts of the E 3 and RMAX algorithms. Efficient exploration in exponentially large state spaces needs to exploit the generalization of the learned model: what in a propositional setting would be considered a novel situation and worth exploration may in the relational setting be a wellknown context in which exploitation is promising. To address this we introduce relational count functions which generalize the classical notion of state and action visitation counts. We provide guarantees on the exploration efficiency of our framework using count functions under the assumption that we had a relational KWIK learner and a nearoptimal planner. We propose a concrete exploration algorithm which integrates a practically efficient probabilistic rule learner and a relational planner (for which there are no guarantees, however) and employs the contexts of learned relational rules as features to model the novelty of states and actions. Our results in noisy 3D simulated robot manipulation problems and in domains of the international planning competition demonstrate that our approach is more effective than existing propositional and factored exploration techniques.
Firstorder decisiontheoretic planning in structured relational environments
, 2008
"... We consider the general framework of firstorder decisiontheoretic planning in structured relational environments. Most traditional solution approaches to these planning problems ground the relational specification w.r.t. a specific domain instantiation and apply a solution approach directly to the ..."
Abstract

Cited by 10 (2 self)
 Add to MetaCart
We consider the general framework of firstorder decisiontheoretic planning in structured relational environments. Most traditional solution approaches to these planning problems ground the relational specification w.r.t. a specific domain instantiation and apply a solution approach directly to the resulting ground Markov decision process (MDP). Unfortunately, the space and time complexity of these solution algorithms scale linearly with the domain size in the best case and exponentially in the worst case. An alternate approach to grounding a relational planning problem is to lift it to a firstorder MDP (FOMDP) specification. This FOMDP can then be solved directly, resulting in a domainindependent solution whose space and time complexity either do not scale with domain size or can scale sublinearly in the domain size. However, such generality does not come without its own set of challenges and the first purpose of this thesis is to explore exact and approximate solution techniques for practically solving FOMDPs. The second purpose of this thesis is to extend the FOMDP specification to succinctly capture factored actions and additive rewards while extending the exact and approximate solution techniques to directly exploit this structure. In addition, we provide a proof of correctness of the firstorder symbolic dynamic programming approach w.r.t. its wellstudied ground MDP
Logical Markov Decision Programs and the Convergence of Logical TD(λ)
 Proc. of ILP’04
, 2004
"... Recent developments in the area of relational reinforcement learning (RRL) have resulted in a number of new algorithms. A theory, however, that explains why RRL works, seems to be lacking. In this paper, we provide some initial results on a theory of RRL. To realize this, we introduce a novel re ..."
Abstract

Cited by 10 (0 self)
 Add to MetaCart
(Show Context)
Recent developments in the area of relational reinforcement learning (RRL) have resulted in a number of new algorithms. A theory, however, that explains why RRL works, seems to be lacking. In this paper, we provide some initial results on a theory of RRL. To realize this, we introduce a novel representation formalism, called logical Markov decision programs (LOMDPs), that integrates Markov Decision Processes (MDPs) with Logic Programs. Using LOMDPs one can compactly and declaratively represent complex MDPs. Within this framework we then devise a relational upgrade of TD(#) called logical TD(#) and prove convergence.
Guiding inference through relational reinforcement learning
 Proceedings of the Fifteenth International Conference on Inductive Logic Programming
, 2005
"... Abstract. Reasoning plays a central role in intelligent systems that operate in complex situations that involve time constraints. In this paper, we present the Adaptive Logic Interpreter, a reasoning system that acquires a controlled inference strategy adapted to the scenario at hand, using a variat ..."
Abstract

Cited by 9 (5 self)
 Add to MetaCart
(Show Context)
Abstract. Reasoning plays a central role in intelligent systems that operate in complex situations that involve time constraints. In this paper, we present the Adaptive Logic Interpreter, a reasoning system that acquires a controlled inference strategy adapted to the scenario at hand, using a variation on relational reinforcement learning. Employing this inference mechanism in a reactive agent architecture lets the agent focus its reasoning on the most rewarding parts of its knowledge base and hence perform better under time and computational resource constraints. We present experiments that demonstrate the benefits of this approach to reasoning in reactive agents, then discuss related work and directions for future research. 1
Evolutionary multiagent systems
 In Proceedings of the 8th International Conference on Parallel Problem Solving from Nature PPSN04
, 2004
"... Abstract. In MultiAgent learning, agents must learn to select actions that maximize their utility given the action choices of the other agents. Cooperative Coevolution offers a way to evolve multiple elements that together form a whole, by using a separate population for each element. We apply this ..."
Abstract

Cited by 9 (0 self)
 Add to MetaCart
(Show Context)
Abstract. In MultiAgent learning, agents must learn to select actions that maximize their utility given the action choices of the other agents. Cooperative Coevolution offers a way to evolve multiple elements that together form a whole, by using a separate population for each element. We apply this setup to the problem of multiagent learning, arriving at an evolutionary multiagent system (EAMAS). We study a problem that requires agents to select their actions in parallel, and investigate the problem solving capacity of the EAMAS for a wide range of settings. Secondly, we investigate the transfer of the COllective INtelligence (COIN) framework to the EAMAS. COIN is a proved engineering approach for learning of cooperative tasks in MASs, and consists of reengineering the utilities of the agents so as to contribute to the global utility. It is found that, as in the Reinforcement Learning case, the use of the Wonderful Life Utility specified by COIN also leads to improved results for the EAMAS. 1