Results 1  10
of
10
Active learning of inverse models with intrinsically motivated goal exploration in robots
 ROBOTICS AND AUTONOMOUS SYSTEMS
, 2013
"... ..."
(Show Context)
The Strategic Student Approach for LifeLong Exploration and Learning
 in IEEE Conference on Development and Learning / EpiRob
, 2012
"... Abstract—This article introduces the strategic student metaphor: a student has to learn a number of topics (or tasks) to maximize its mean score, and has to choose strategically how to allocate its time among the topics and/or which learning method to use for a given topic. We show that under which ..."
Abstract

Cited by 23 (17 self)
 Add to MetaCart
(Show Context)
Abstract—This article introduces the strategic student metaphor: a student has to learn a number of topics (or tasks) to maximize its mean score, and has to choose strategically how to allocate its time among the topics and/or which learning method to use for a given topic. We show that under which conditions a strategy where time allocation or learning method is chosen from the easier to the more complex topic is optimal. Then, we show an algorithm, based on multiarmed bandit techniques, that allows empirical online evaluation of learning progress and approximates the optimal solution under more general conditions. Finally, we show that the strategic student problem formulation allows to view in a common framework many previous approaches to active and developmental learning. I.
Continually Adding SelfInvented Problems to the Repertoire: First Experiments with POWERPLAY
"... Abstract—Pure scientists do not only invent new methods to solve given problems. They also invent new problems. The recent POWERPLAY framework formalizes this type of curiosity and creativity in a new, general, yet practical way. To acquire problem solving prowess through playing, POWERPLAYbased ar ..."
Abstract

Cited by 2 (2 self)
 Add to MetaCart
(Show Context)
Abstract—Pure scientists do not only invent new methods to solve given problems. They also invent new problems. The recent POWERPLAY framework formalizes this type of curiosity and creativity in a new, general, yet practical way. To acquire problem solving prowess through playing, POWERPLAYbased artificial explorers by design continually come up with the fastest to find, initially novel, but eventually solvable problems. They also continually simplify or speed up solutions to previous problems. We report on results of first experiments with POWERPLAY. A selfdelimiting recurrent neural network (SLIM RNN) is used as a general computational architecture to implement the system’s solver. Its weights can encode arbitrary, selfdelimiting, halting or nonhalting programs affecting both environment (through effectors) and internal states encoding abstractions of event sequences. In openended fashion, our POWERPLAYdriven RNNs learn to become increasingly general problem solvers, continually adding new problem solving procedures to the growing repertoire, exhibiting interesting developmental stages. I.
First Experiments with POWERPLAY
, 2012
"... Like a scientist or a playing child, POWERPLAY [24] not only learns new skills to solve given problems, but also invents new interesting problems by itself. By design, it continually comes up with the fastest to find, initially novel, but eventually solvable tasks. It also continually simplifies or ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
(Show Context)
Like a scientist or a playing child, POWERPLAY [24] not only learns new skills to solve given problems, but also invents new interesting problems by itself. By design, it continually comes up with the fastest to find, initially novel, but eventually solvable tasks. It also continually simplifies or compresses or speeds up solutions to previous tasks. Here we describe first experiments with POWERPLAY. A selfdelimiting recurrent neural network SLIM RNN [25] is used as a general computational problem solving architecture. Its connection weights can encode arbitrary, selfdelimiting, halting or nonhalting programs affecting both environment (through effectors) and internal states encoding abstractions of event sequences. Our POWERPLAYdriven SLIM RNN learns to become an increasingly general solver of selfinvented problems, continually adding new problem solving procedures to its growing skill repertoire. Extending a recent conference paper [28], we identify interesting, emerging, developmental stages of our openended system. We also show how it automatically selfmodularizes, frequently reusing code for previously invented skills, always trying to invent novel tasks that can be quickly validated because they do not require too many weight changes affecting too many previous tasks. 1
ResourceBounded Machines are Motivated to be Effective, Efficient, and Curious
"... Resourceboundedness must be carefully considered when designing and implementing artificial general intelligence (AGI) algorithms and architectures that have to deal with the real world. But not only must resources be represented explicitly throughout its design, an agent must also take into accou ..."
Abstract
 Add to MetaCart
(Show Context)
Resourceboundedness must be carefully considered when designing and implementing artificial general intelligence (AGI) algorithms and architectures that have to deal with the real world. But not only must resources be represented explicitly throughout its design, an agent must also take into account their usage and the associated costs during reasoning and acting. Moreover, the agent must be intrinsically motivated to become progressively better at utilizing resources. This drive then naturally leads to effectiveness, efficiency, and curiosity. We propose a practical operational framework that explicitly takes into account resource constraints: activities are organized to maximally utilize an agent’s bounded resources as well as the availability of a teacher, and to drive the agent to become progressively better at utilizing its resources. We show how an existing AGI architecture called AERA can function inside this framework. In short, the capability of AERA to perform selfcompilation can be used to motivate the system to not only accumulate knowledge and skills faster, but also to achieve goals using less resources, becoming progressively more effective and efficient.
New Millennium AI and the Convergence of History: Update of 2012
, 2012
"... ...millennium brought the first mathematically sound, asymptotically optimal, universal problem solvers, providing a new, rigorous foundation for the previously largely heuristic field of General AI and embedded agents. There also has been rapid progress in not quite universal but still rather gener ..."
Abstract
 Add to MetaCart
(Show Context)
...millennium brought the first mathematically sound, asymptotically optimal, universal problem solvers, providing a new, rigorous foundation for the previously largely heuristic field of General AI and embedded agents. There also has been rapid progress in not quite universal but still rather general and practical artificial recurrent neural networks for learning sequenceprocessing programs, now yielding stateoftheart results in real world applications. And the computing power per Euro is still growing by a factor of 1001000 per decade, greatly increasing the feasibility of neural networks in general, which have started to yield humancompetitive results in challenging pattern recognition competitions. Finally, a recent formal theory of fun and creativity identifies basic principles of curious and creative machines, laying foundations for artificial scientists and artists. Here I will briefly review some of the new results of my lab at IDSIA, and speculate about future developments, pointing out that the time intervals between the most notable events in over 40,000 years or 2 9 lifetimes of human history have sped up exponentially, apparently converging to zero within the next few decades. Or is this impression just a byproduct of the way humans allocate memory space to past events?
Explore to See, Learn to Perceive, Get the Actions for Free: SKILLABILITY
, 2014
"... How can a humanoid robot autonomously learn and refine multiple sensorimotor skills as a byproduct of curiosity driven exploration, upon its highdimensional unprocessed visual input? We present SKILLABILITY, which makes this possible. It combines the recently introduced Curiosity Driven Modular In ..."
Abstract
 Add to MetaCart
(Show Context)
How can a humanoid robot autonomously learn and refine multiple sensorimotor skills as a byproduct of curiosity driven exploration, upon its highdimensional unprocessed visual input? We present SKILLABILITY, which makes this possible. It combines the recently introduced Curiosity Driven Modular Incremental Slow Feature Analysis (Curious Dr. MISFA) with the wellknown options framework. Curious Dr. MISFA’s objective is to acquire abstractions as quickly as possible. These abstractions map highdimensional pixellevel vision to a lowdimensional manifold. We find that each learnable abstraction augments the robot’s state space (a set of poses) with new information about the environment, for example, when the robot is grasping a cup. The abstraction is a function on an image, called a slow feature, which can effectively discretize a highdimensional visual sequence. For example, it maps the sequence of the robot watching its arm as it moves around, grasping randomly, then grasping a cup, and moving around some more while holding the cup, into a step function having two outputs: when the cup is or is not currently grasped. The new state space includes this grasped/not grasped information. Each abstraction is coupled with an option. The reward function for the option’s policy (learned through Least Squares Policy Iteration) is high for transitions that produce a large change in the stepfunctionlike slow features. This corresponds to finding bottleneck states, which are known good subgoals for hierarchical reinforcement learning in the example, the subgoal corresponds to grasping the cup. The final skill includes both the learned policy and the learned abstraction. SKILLABILITY makes our iCub the first humanoid robot to learn complex skills such as to topple or grasp an object, from raw highdimensional video input, driven purely by its intrinsic motivations.
Active Learning for Autonomous Intelligent Agents: Exploration, Curiosity, and Interaction
, 2014
"... In this survey we present different approaches that allow an intelligent agent to explore autonomous its environment to gather information and learn multiple tasks. Different communities proposed different solutions, that are in many cases, similar and/or complementary. These solutions include act ..."
Abstract
 Add to MetaCart
(Show Context)
In this survey we present different approaches that allow an intelligent agent to explore autonomous its environment to gather information and learn multiple tasks. Different communities proposed different solutions, that are in many cases, similar and/or complementary. These solutions include active learning, exploration/exploitation, onlinelearning and social learning. The common aspect of all these approaches is that it is the agent to selects and decides what information to gather next. Applications for these approaches already include tutoring systems, autonomous grasping learning, navigation and mapping and humanrobot interaction. We discuss how these approaches are related, explaining their similarities and their differences in terms of problem assumptions and metrics of success. We consider that such an integrated discussion will improve interdisciplinary research and applications.1 1