Results 1  10
of
359
Metaheuristics in combinatorial optimization: Overview and conceptual comparison
 ACM COMPUTING SURVEYS
, 2003
"... The field of metaheuristics for the application to combinatorial optimization problems is a rapidly growing field of research. This is due to the importance of combinatorial optimization problems for the scientific as well as the industrial world. We give a survey of the nowadays most important meta ..."
Abstract

Cited by 294 (16 self)
 Add to MetaCart
The field of metaheuristics for the application to combinatorial optimization problems is a rapidly growing field of research. This is due to the importance of combinatorial optimization problems for the scientific as well as the industrial world. We give a survey of the nowadays most important metaheuristics from a conceptual point of view. We outline the different components and concepts that are used in the different metaheuristics in order to analyze their similarities and differences. Two very important concepts in metaheuristics are intensification and diversification. These are the two forces that largely determine the behaviour of a metaheuristic. They are in some way contrary but also complementary to each other. We introduce a framework, that we call the I&D frame, in order to put different intensification and diversification components into relation with each other. Outlining the advantages and disadvantages of different metaheuristic approaches we conclude by pointing out the importance of hybridization of metaheuristics as well as the integration of metaheuristics and other methods for optimization.
Populationbased incremental learning with memory scheme for changing environments
 in Proc. 2005 Genetic Evol. Comput. Conf., 2005
"... Abstract—In recent years, interest in studying evolutionary algorithms (EAs) for dynamic optimization problems (DOPs) has grown due to its importance in realworld applications. Several approaches, such as the memory and multiple population schemes, have been developed for EAs to address dynamic pro ..."
Abstract

Cited by 64 (30 self)
 Add to MetaCart
(Show Context)
Abstract—In recent years, interest in studying evolutionary algorithms (EAs) for dynamic optimization problems (DOPs) has grown due to its importance in realworld applications. Several approaches, such as the memory and multiple population schemes, have been developed for EAs to address dynamic problems. This paper investigates the application of the memory scheme for populationbased incremental learning (PBIL) algorithms, a class of EAs, for DOPs. A PBILspecific associative memory scheme, which stores best solutions as well as corresponding environmental information in the memory, is investigated to improve its adaptability in dynamic environments. In this paper, the interactions between the memory scheme and random immigrants, multipopulation, and restart schemes for PBILs in dynamic environments are investigated. In order to better test the performance of memory schemes for PBILs and other EAs in dynamic environments, this paper also proposes a dynamic environment generator that can systematically generate dynamic environments of different difficulty with respect to memory schemes. Using this generator, a series of dynamic environments are generated and experiments are carried out to compare the performance of investigated algorithms. The experimental results show that the proposed memory scheme is efficient for PBILs in dynamic environments and also indicate that different interactions exist between the memory scheme and random immigrants, multipopulation schemes for PBILs in different dynamic environments. Index Terms—Associative memory scheme, dynamic optimization problems (DOPs), immune systembased genetic algorithm (ISGA), memoryenhanced genetic algorithm, multipopulation scheme, populationbased incremental learning (PBIL), random immigrants.
Modelbased search for combinatorial optimization
, 2001
"... Abstract In this paper we introduce modelbased search as a unifying framework accommodating some recently proposed heuristics for combinatorial optimization such as ant colony optimization, stochastic gradient ascent, crossentropy and estimation of distribution methods. We discuss similarities as ..."
Abstract

Cited by 60 (13 self)
 Add to MetaCart
(Show Context)
Abstract In this paper we introduce modelbased search as a unifying framework accommodating some recently proposed heuristics for combinatorial optimization such as ant colony optimization, stochastic gradient ascent, crossentropy and estimation of distribution methods. We discuss similarities as well as distinctive features of each method, propose some extensions and present a comparative experimental study of these algorithms. 1
Feature Subset Selection by Bayesian networks: a comparison with genetic and sequential algorithms
"... In this paper we perform a comparison among FSSEBNA, a randomized, populationbased and evolutionary algorithm, and two genetic and other two sequential search approaches in the well known Feature Subset Selection (FSS) problem. In FSSEBNA, the FSS problem, stated as a search problem, uses the E ..."
Abstract

Cited by 55 (15 self)
 Add to MetaCart
In this paper we perform a comparison among FSSEBNA, a randomized, populationbased and evolutionary algorithm, and two genetic and other two sequential search approaches in the well known Feature Subset Selection (FSS) problem. In FSSEBNA, the FSS problem, stated as a search problem, uses the EBNA (Estimation of Bayesian Network Algorithm) search engine, an algorithm within the EDA (Estimation of Distribution Algorithm) approach. The EDA paradigm is born from the roots of the GA community in order to explicitly discover the relationships among the features of the problem and not disrupt them by genetic recombination operators. The EDA paradigm avoids the use of recombination operators and it guarantees the evolution of the population of solutions and the discovery of these relationships by the factorization of the probability distribution of best individuals in each generation of the search. In EBNA, this factorization is carried out by a Bayesian network induced by a chea...
Algorithms for hyperparameter optimization
 In NIPS
, 2011
"... Several recent advances to the state of the art in image classification benchmarks have come from better configurations of existing techniques rather than novel approaches to feature learning. Traditionally, hyperparameter optimization has been the job of humans because they can be very efficient ..."
Abstract

Cited by 46 (10 self)
 Add to MetaCart
(Show Context)
Several recent advances to the state of the art in image classification benchmarks have come from better configurations of existing techniques rather than novel approaches to feature learning. Traditionally, hyperparameter optimization has been the job of humans because they can be very efficient in regimes where only a few trials are possible. Presently, computer clusters and GPU processors make it possible to run more trials and we show that algorithmic approaches can find better results. We present hyperparameter optimization results on tasks of training neural networks and deep belief networks (DBNs). We optimize hyperparameters using random search and two new greedy sequential methods based on the expected improvement criterion. Random search has been shown to be sufficiently efficient for learning neural networks for several datasets, but we show it is unreliable for training DBNs. The sequential algorithms are applied to the most difficult DBN learning problems from [1] and find significantly better results than the best previously reported. This work contributes novel techniques for making response surface models P (yx) in which many elements of hyperparameter assignment (x) are known to be irrelevant given particular values of other elements. 1
Natural Evolution Strategies
"... Abstract — This paper presents Natural Evolution Strategies (NES), a novel algorithm for performing realvalued ‘black box ’ function optimization: optimizing an unknown objective function where algorithmselected function measurements constitute the only information accessible to the method. Natura ..."
Abstract

Cited by 42 (23 self)
 Add to MetaCart
Abstract — This paper presents Natural Evolution Strategies (NES), a novel algorithm for performing realvalued ‘black box ’ function optimization: optimizing an unknown objective function where algorithmselected function measurements constitute the only information accessible to the method. Natural Evolution Strategies search the fitness landscape using a multivariate normal distribution with a selfadapting mutation matrix to generate correlated mutations in promising regions. NES shares this property with Covariance Matrix Adaption (CMA), an Evolution Strategy (ES) which has been shown to perform well on a variety of highprecision optimization tasks. The Natural Evolution Strategies algorithm, however, is simpler, less adhoc and more principled. Selfadaptation of the mutation matrix is derived using a Monte Carlo estimate of the natural gradient towards better expected fitness. By following the natural gradient instead of the ‘vanilla ’ gradient, we can ensure efficient update steps while preventing early convergence due to overly greedy updates, resulting in reduced sensitivity to local suboptima. We show NES has competitive performance with CMA on unimodal tasks, while outperforming it on several multimodal tasks that are rich in deceptive local optima. I.
Evolutionary algorithm with the guided mutation for the maximum clique problem
 IEEE Transactions on Evolutionary Computation
, 2005
"... Abstract—Estimation of distribution algorithms sample new solutions (offspring) from a probability model which characterizes the distribution of promising solutions in the search space at each generation. The location information of solutions found so far (i.e., the actual positions of these solutio ..."
Abstract

Cited by 42 (15 self)
 Add to MetaCart
(Show Context)
Abstract—Estimation of distribution algorithms sample new solutions (offspring) from a probability model which characterizes the distribution of promising solutions in the search space at each generation. The location information of solutions found so far (i.e., the actual positions of these solutions in the search space) is not directly used for generating offspring in most existing estimation of distribution algorithms. This paper introduces a new operator, called guided mutation. Guided mutation generates offspring through combination of global statistical information and the location information of solutions found so far. An evolutionary algorithm with guided mutation (EA/G) for the maximum clique problem is proposed in this paper. Besides guided mutation, EA/G adopts a strategy for searching different search areas in different search phases. Marchiori’s heuristic is applied to each new solution to produce a maximal clique in EA/G. Experimental results show that EA/G outperforms the heuristic genetic algorithm of Marchiori (the best evolutionary algorithm reported so far) and a MIMIC algorithm on DIMACS benchmark graphs. Index Terms—Estimation of distribution algorithms, evolutionary algorithm, guided mutation, heuristics, hybrid genetic algorithm, maximum clique problem (MCP). I.
MILEPOST GCC: machine learning based research compiler
 In GCC
, 2008
"... Tuning hardwired compiler optimizations for rapidly evolving hardware makes porting an optimizing compiler for each new platform extremely challenging. Our radical approach is to develop a modular, extensible, selfoptimizing compiler that automatically learns the best optimization heuristics based ..."
Abstract

Cited by 37 (13 self)
 Add to MetaCart
Tuning hardwired compiler optimizations for rapidly evolving hardware makes porting an optimizing compiler for each new platform extremely challenging. Our radical approach is to develop a modular, extensible, selfoptimizing compiler that automatically learns the best optimization heuristics based on the behavior of the platform. In this paper we describe MILEPOST 1 GCC, a machinelearningbased compiler that automatically adjusts its optimization heuristics to improve the execution time, code size, or compilation time of specific programs on different architectures. Our preliminary experimental results show that it is possible to considerably reduce execution time of the MiBench benchmark suite on a range of platforms entirely automatically. 1
On the convergence of a class of estimation of distribution algorithms, conditionally
 IEEE Trans. Evol. Comput
"... Abstract—We investigate the global convergence of estimation of distribution algorithms (EDAs). In EDAs, the distribution is estimated from a set of selected elements, i.e., the parent set, and then the estimated distribution model is used to generate new elements. In this paper, we prove that: 1) i ..."
Abstract

Cited by 37 (8 self)
 Add to MetaCart
Abstract—We investigate the global convergence of estimation of distribution algorithms (EDAs). In EDAs, the distribution is estimated from a set of selected elements, i.e., the parent set, and then the estimated distribution model is used to generate new elements. In this paper, we prove that: 1) if the distribution of the new elements matches that of the parent set exactly, the algorithms will converge to the global optimum under three widely used selection schemes and 2) a factorized distribution algorithm converges globally under proportional selection. Index Terms—Convergence, estimation of distribution algorithms (EDAs), factorized distribution algorithms (FDA). I.
The estimation of distributions and the minimum relative entropy principle
 Evolutionary Computation
, 2005
"... Estimation of Distribution Algorithms EDA have been proposed as an extension of genetic algorithms. In this paper the relation of EDA to algorithms developed in statistics, artificial intelligence, and statistical physics is explained. The major design issues are discussed within a general interdisc ..."
Abstract

Cited by 32 (3 self)
 Add to MetaCart
(Show Context)
Estimation of Distribution Algorithms EDA have been proposed as an extension of genetic algorithms. In this paper the relation of EDA to algorithms developed in statistics, artificial intelligence, and statistical physics is explained. The major design issues are discussed within a general interdisciplinary framework. It is shown that maximum entropy approximations play a crucial role. All proposed algorithms try to minimize the KullbackLeibler divergence ÃÄ � between the unknown distribution Ô Ü and a class Õ Ü of approximations. The KullbackLeibler divergence is not symmetric. Approximations which suppose that the function to be optimized is additively decomposed (ADF) minimize ÃÄ � Õ�Ô, the methods which learn the approximate model from data minimize ÃÄ � Ô�Õ. This minimization is identical to maximizing the loglikelihood. In the paper three classes of algorithms are discussed. FDA uses the ADF to compute an approximate factorization of the unknown distribution. The factors are marginal distributions, whose values are computed from samples. The BetheKikuchi approach developed in statistical physics uses bivariate or higher order marginals. The values of the marginals are computed from a difficult minimization problem. The third class learns the factorization from the data. We analyze our learning algorithm LFDA in detail. It is shown that learning is faced with two problems: first, to detect the important dependencies between the variables, and second, to create an acyclic Bayesian network of bounded clique size.