Results 1  10
of
54
Natural Evolution Strategies
"... Abstract — This paper presents Natural Evolution Strategies (NES), a novel algorithm for performing realvalued ‘black box ’ function optimization: optimizing an unknown objective function where algorithmselected function measurements constitute the only information accessible to the method. Natura ..."
Abstract

Cited by 41 (22 self)
 Add to MetaCart
(Show Context)
Abstract — This paper presents Natural Evolution Strategies (NES), a novel algorithm for performing realvalued ‘black box ’ function optimization: optimizing an unknown objective function where algorithmselected function measurements constitute the only information accessible to the method. Natural Evolution Strategies search the fitness landscape using a multivariate normal distribution with a selfadapting mutation matrix to generate correlated mutations in promising regions. NES shares this property with Covariance Matrix Adaption (CMA), an Evolution Strategy (ES) which has been shown to perform well on a variety of highprecision optimization tasks. The Natural Evolution Strategies algorithm, however, is simpler, less adhoc and more principled. Selfadaptation of the mutation matrix is derived using a Monte Carlo estimate of the natural gradient towards better expected fitness. By following the natural gradient instead of the ‘vanilla ’ gradient, we can ensure efficient update steps while preventing early convergence due to overly greedy updates, resulting in reduced sensitivity to local suboptima. We show NES has competitive performance with CMA on unimodal tasks, while outperforming it on several multimodal tasks that are rich in deceptive local optima. I.
Hoeffding and bernstein races for selecting policies in evolutionary direct policy search
 In Proceedings of the 26 th International Conference on Machine Learning (ICML
, 2009
"... Uncertainty arises in reinforcement learning from various sources, and therefore it is necessary to consider statistics based on several rollouts for evaluating behavioral policies. We add an adaptive uncertainty handling based on Hoeffding and empirical Bernstein races to the CMAES, a variable me ..."
Abstract

Cited by 39 (4 self)
 Add to MetaCart
(Show Context)
Uncertainty arises in reinforcement learning from various sources, and therefore it is necessary to consider statistics based on several rollouts for evaluating behavioral policies. We add an adaptive uncertainty handling based on Hoeffding and empirical Bernstein races to the CMAES, a variable metric evolution strategy proposed for direct policy search. The uncertainty handling adjusts individually the number of episodes considered for the evaluation of a policy. The performance estimation is kept just accurate enough for a sufficiently good ranking of candidate policies, which is in turn sufficient for the CMAES to find better solutions. This increases the learning speed as well as the robustness of the algorithm. 1.
Autonomous evolution of topographic regularities in artificial neural networks
 Neural Computation
"... Looking to nature as inspiration, for at least the last 25 years researchers in the field of neuroevolution (NE) have developed evolutionary algorithms designed specifically to evolve artificial neural networks (ANNs). Yet the ANNs evolved through NE algorithms lack the distinctive characteristics o ..."
Abstract

Cited by 31 (18 self)
 Add to MetaCart
(Show Context)
Looking to nature as inspiration, for at least the last 25 years researchers in the field of neuroevolution (NE) have developed evolutionary algorithms designed specifically to evolve artificial neural networks (ANNs). Yet the ANNs evolved through NE algorithms lack the distinctive characteristics of biological brains, perhaps explaining why NE is not yet a mainstream subject of neural computation. Motivated by this gap, this paper shows that when geometry is introduced to evolved ANNs through the Hypercubebased NeuroEvolution of Augmenting Topologies (HyperNEAT) algorithm, they begin to acquire characteristics that indeed are reminiscent of biological brains. That is, if the neurons in evolved ANNs are situated at locations in space (i.e. if they are given coordinates), then, as experiments in evolving checkersplaying ANNs in this paper show, topographic maps with symmetries and regularities can evolve spontaneously. The ability to evolve such maps is shown in this paper to provide an important advantage in generalization. In fact, the evolved maps are sufficiently informative that their analysis yields the novel insight that the geometry of the connectivity patterns of more general players is significantly more smooth and contiguous than less general ones. Thus, the results in this paper reveal a correlation between generality and smoothness in connectivity patterns. This result hints at the intriguing possibility that, as NE matures as a field, its algorithms can evolve
Coevolution of rolebased cooperation in multiagent systems
, 2007
"... In certain tasks such as pursuit and evasion, multiple agents need to coordinate their behavior to achieve a common goal. An interesting question is, how can such behavior be best evolved? A powerful approach is to control the agents with neural networks, coevolve them in separate subpopulations, an ..."
Abstract

Cited by 27 (3 self)
 Add to MetaCart
(Show Context)
In certain tasks such as pursuit and evasion, multiple agents need to coordinate their behavior to achieve a common goal. An interesting question is, how can such behavior be best evolved? A powerful approach is to control the agents with neural networks, coevolve them in separate subpopulations, and test them together in the common task. In this paper, such a method, called MultiAgent ESP (Enforced SubPopulations), is proposed and demonstrated in a preycapture task. First, the approach is shown more efficient than evolving a single central controller for all agents. Second, cooperation is found to be most efficient through stigmergy, i.e. through rolebased responses to the environment, rather than direct communication between the agents. Together these results suggest that rolebased cooperation is an effective strategy in certain multiagent domains. [ This paper is a revision of AI01287.]
Ontogenetic and Phylogenetic Reinforcement Learning
"... Reinforcement learning (RL) problems come in many flavours, as do the algorithms for solving them. It is currently not clear which of the commonly used RL benchmarks best measure an algorithm’s capacity for solving realworld problems. Similarly, it is not clear which types of RL algorithms are best ..."
Abstract

Cited by 14 (5 self)
 Add to MetaCart
(Show Context)
Reinforcement learning (RL) problems come in many flavours, as do the algorithms for solving them. It is currently not clear which of the commonly used RL benchmarks best measure an algorithm’s capacity for solving realworld problems. Similarly, it is not clear which types of RL algorithms are best suited to solve which kinds of RL problems. Here we present some dimensions along the axes o which RL problems and algorithms can be varied to help distinguish them from each other. Based on results and arguments in the literature, we present some conjectures as to what algorithms should work best for particular types of problems, and argue that tunable RL benchmarks are needed in order to further understand the capabilities of RL algorithms. 1
High Dimensions and Heavy Tails for Natural Evolution Strategies
 In Genetic and Evolutionary Computation Conference (GECCO
, 2011
"... The family of natural evolution strategies (NES) offers a principled approach to realvalued evolutionary optimization by following the natural gradient of the expected fitness on the parameters of its search distribution. While general in its formulation, existing research has focused only on multi ..."
Abstract

Cited by 14 (10 self)
 Add to MetaCart
(Show Context)
The family of natural evolution strategies (NES) offers a principled approach to realvalued evolutionary optimization by following the natural gradient of the expected fitness on the parameters of its search distribution. While general in its formulation, existing research has focused only on multivariate Gaussian search distributions. We address this shortcoming by exhibiting problem classes for which other search distributions are more appropriate, and then derive the corresponding NESvariants. First, we show how simplifying NES to separable distributions reduces its complexity from O(d 3) to O(d), and apply it to problems of previously unattainable dimensionality, recovering lowestenergy structures on the LennardJones atom clusters and stateoftheart results on neuroevolution benchmarks. Second, we develop a new, equivalent formulation based on invariances, which allows us to generalize NES to heavytailed distributions, even if their variance is undefined. We theninvestigate howthisvariant aids inovercoming deceptive local optima.
POWERPLAY: Training an Increasingly General Problem Solver by Continually Searching for the Simplest Still Unsolvable Problem
, 2011
"... Most of computer science focuses on automatically solving given computational problems. I focus on automatically inventing or discovering problems in a way inspired by the playful behavior of animals and humans, to train a more and more general problem solver from scratch in an unsupervised fashion. ..."
Abstract

Cited by 10 (4 self)
 Add to MetaCart
Most of computer science focuses on automatically solving given computational problems. I focus on automatically inventing or discovering problems in a way inspired by the playful behavior of animals and humans, to train a more and more general problem solver from scratch in an unsupervised fashion. At any given time, the novel algorithmic framework POWERPLAY searches the space of possible pairs of new tasks and modifications of the current problem solver, until it finds a more powerful problem solver that provably solves all previously learned tasks plus the new one, while the unmodified predecessor does not. The new task and its corresponding tasksolving skill are those first found and validated. Newly invented tasks may require making previously learned skills more efficient. The greedy search of typical POWERPLAY variants orders candidate pairs of tasks and solver modifications by their conditional computational complexity, given the stored experience so far. This biases the search towards pairs that can be described compactly and validated quickly. Standard problem solver architectures of personal computers or neural networks tend to generalize by solving numerous tasks outside the selfinvented training set; POWERPLAY’s ongoing search for novelty keeps fighting to extend beyond the generalization abilities of its present solver. The continually increasing repertoire of problem solving procedures can be exploited
Efficient representation of recurrent neural networks for markovian/nonmarkovian nonlinear control problems
 in Proceedings of the 10th International Conference on Intelligent Systems Design and Applications (ISDA2010) (2010) 615–620
"... Abstract—A novel representation of Recurrent Artificial neural network is proposed for nonlinear markovian and nonmarkovian control problems. The network architecture is inspired by Cartesian Genetic Programming. The neural network attributes namely weights, topology and functions are encoded usi ..."
Abstract

Cited by 7 (5 self)
 Add to MetaCart
(Show Context)
Abstract—A novel representation of Recurrent Artificial neural network is proposed for nonlinear markovian and nonmarkovian control problems. The network architecture is inspired by Cartesian Genetic Programming. The neural network attributes namely weights, topology and functions are encoded using Cartesian Genetic Programming. The proposed algorithm is applied on the standard benchmark control problem: double pole balancing for both markovian and nonmarkovian cases. Results demonstrate that the network has the ability to generate neural architecture and parameters that can solve these problems in substantially fewer number of evaluations in comparison to earlier neuroevolutionary techniques. The power of Recurrent Cartesian Genetic Programming Artificial Neural Network (RCGPANN) is its representation which leads to a thorough evolutionary search producing generalized networks.
Evolution of neural networks using cartesian genetic programming, in
 IEEE Congress on Evolutionary Computation
"... are encoded and evolved using a representation adapted from the CGP. We have tested the new approach on the single pole balancing problem. Results show that CGPANN evolves solutions faster and of higher quality than the most powerful algorithms of Neuroevolution in the literature. I. ..."
Abstract

Cited by 5 (1 self)
 Add to MetaCart
(Show Context)
are encoded and evolved using a representation adapted from the CGP. We have tested the new approach on the single pole balancing problem. Results show that CGPANN evolves solutions faster and of higher quality than the most powerful algorithms of Neuroevolution in the literature. I.