The hierarchical Bayesian optimization algorithm (hBOA) can solve nearly decomposable and hierarchical problems of bounded difficulty in a robust and scalable manner by building and sampling probabilistic models of promising solutions. This paper analyzes probabilistic models in hBOA on two common test problems: concatenated traps and 2D Ising spin glasses with periodic boundary conditions. We argue that although Bayesian networks with local structures can encode complex probability distributions, analyzing these models in hBOA is relatively straightforward and the results of such analyses may provide practitioners with useful information about their problems. The results show that the probabilistic models in hBOA closely correspond to the structure of the underlying problem, the models do not change significantly in subsequent iterations of BOA, and creating adequate probabilistic models by hand is not straightforward even with complete knowledge of the optimization problem. Categories and Subject Descriptors

Estimation of distribution algorithms (EDAs) are stochastic optimization techniques that explore the space of potential solutions by building and sampling explicit probabilistic models of promising candidate solutions. While the primary goal of applying EDAs is to discover the global optimum or at least its accurate approximation, besides this, any EDA provides us with a sequence of probabilistic models, which in most cases hold a great deal of information about the problem. Although using problem-specific knowledge has been shown to significantly improve performance of EDAs and other evolutionary algorithms, this readily available source of problem-specific information has been practically ignored by the EDA community. This paper takes the first step towards the use of probabilistic models obtained by EDAs to speed up the solution of similar problems in future. More specifically, we propose two approaches to biasing model building in the hierarchical Bayesian optimization algorithm (hBOA) based on knowledge automatically learned from previous hBOA runs on similar problems. We show that the proposed methods lead to substantial speedups and argue that the methods should work well in other applications that require solving a large number of problems with similar structure.

Evolutionary Algorithms are largely used search and optimization procedures. They have been successfully applied for several problems and with proper care on the design process they can solve hard problems accurately, efficiently and reliably. The proper design of the algorithm turns some problems from intractable to tractable. We can go even further, using efficiency enhancements to turn them from tractable to practical. In this paper we show preliminary results of two efficiency enhancements proposed for Extended Compact Genetic Algorithm. First, a model building enhancement was used to reduce the complexity of the process from O(n 3) to O(n 2), speeding up the algorithm by 1000 times on a 4096 bits problem. Then, a local-search hybridization was used to reduce the population size by at least 32 times, reducing the memory and running time required by the algorithm. These results draw the first steps toward a competent and efficient Genetic Algorithm.

Abstract not found

Genetic Algorithms are a class of metaheuristics with applications on several fields including biology, engineering and even arts. However, simple Genetic Algorithms may suffer from exponential scalability on hard problems. Estimation of Distribution Algorithms, a special class of Genetic Algorithms, can build complex models of the iterations among variables in the problem, solving several intractable problems in tractable polynomial time. However, the model building process can be computationally expensive and efficiency enhancements are oftentimes necessary to make tractable problems practical. This paper presents a new model building approach, called ClusterMI, inspired both on the Extended Compact Genetic Algorithm and the Dependency Structure Matrix Genetic Algorithm. The new approach has a more efficient model building process, resulting in speed ups of 10 times for moderate size problems and potentially thousands of times for large problems. Moreover, the new approach may be easily extended to perform incremental evolution, eliminating the burden of representing the population explicitly.

This paper analyzes the effects of restricting probabilistic models in the hierarchical Bayesian optimization algorithm (hBOA) by defining a distance metric over variables and disallowing dependencies between variables at distances greater than a given threshold. We argue that by using prior problem-specific knowledge, it is often possible to develop a distance metric that closely corresponds to the strength of interactions between variables. This distance metric can then be used to speed up model building in hBOA. Three test problems are considered: 3D Ising spin glasses, random additively decomposable problems, and the minimum vertex cover.

This paper discusses automated selection of estimation of distribution algorithms for problem solving. A specific method inspired in the parameter-less GA is proposed. Other alternatives are also briefly mentioned as promising research directions to address the problem.