R. Battiti and G. Tecchiolli. Parallel Biased Search for Combinatorial Optimization: Genetic Algorithms and TABU. Microprocessors and Microsystems, 16(7):351-367, 1992.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....between the iterative steps of these methods (see Greening [18] and Yannakakis [41] for this topic) Cooperative search algorithms are not based on an explicit decomposition of the problem domain. Rather, parallelism is achieved by launching concurrently several independent search programs [1,11,22,34,40]. Di erent initial solutions and di erent values for the search parameters may be used for each independent search, which yields an implicit decomposition of the solution space. In this way, the solution space can be explored in parallel without being limited by the sequential dependencies of the ....

R. Battiti and G. Tecchiolli. Parallel Biased Search for Combinatorial Optimization: Genetic Algorithms and TABU. Microprocessors and Microsystems, 16(7):351-367, 1992.

....start with identical random number generator seeds. These speedups are linear for a number of metaheuristics, including simulated annealing [31, 71] iterated local search algorithms for the traveling salesman problem [33] tabu search, provided that the search starts from a local optimum [17, 94]; and WalkSAT [93] on hard random 3 SAT problems [56] A typical example is described in [74] for a PVM implementation of a parallel GRASP for the MAX SAT problem. This observation can be explained if the random variable time to find a solution within some target value is exponentially ....

R. Battiti and G. Tecchiolli. Parallel biased search for combinatorial optimization: Genetic algorithms and tabu. Microprocessors and Microsystems, 16:351--367, 1992.

....found a new optimum for the Steinberg problem (QAP of size 36) The numbers of processors used to run this problem were 16, 32, and 64. The 64 processor implementation (on a system with distributed memory) gave by far the best results in terms of computational time. Battiti and Tecchiolli [9] presented parallelization schemes of genetic algorithms and tabu search for combinatorial optimization problems and in particular for quadratic assignment and N k problems giving indicative experimental results. Tanese [115] presents a parallel genetic algorithm implemented on a 64 processor ....

....n = 42 to 100, using n 2 processors. The new tabu strategy gave good results in terms of quality of solutions. For problems up to size 90, it obtained the best known solutions or close to those. For size n = 100, every previously best solution found, was improved upon. Battiti and Tecchiolli [9] describe a parallelization scheme for tabu search called reactive tabu scheme, and apply it to the quadratic assignment problem and the N k problem with the objective of achieving a speedup in the order of the number of processors. In the reactive tabu search, each processor executes an ....

[Article contains additional citation context not shown here]

R. Battiti and G. Tecchiolli, Parallel Biased Search for Combinatorial Optimization: Genetic Algorithms and TABU, Microprocessors and Microsystems 16 (1992), pp. 351--367.

....that they are very easy to implement. Our comparison to other state of the art algorithms for the QAP has shown that the proposed ILS extensions are currently among the top performing algorithms for structured QAP instances. Run time distributions have been observed occasionally in the literature [52, 2, 53, 57], but they have been mainly used for simple descriptive purposes or to get hints on the obtainable speed up for parallel processing based in multiple independent runs of a sequential algorithm. In [53] general conditions are given when parallel independent runs on p processors of an algorithm for ....

....outline of ILS given in Figure 1. Such extensions could strongly benefit from the use of long term memory techniques used in tabu search [17] An interesting research direction arises from the observation of the stagnation behavior observed in the run time behavior of the ILS algorithm. In [2, 52] it has been observed that for two specific tabu search algorithms for the QAP, one of these is the robust tabu search algorithm applied in Section 8, the required run time to find the pseudo optimal solution for unstructured instances of type i (defined in Section 4) follows an exponential ....

R. Battiti and G. Tecchiolli. Parallel Biased Search for Combinatorial Optimization: Genetic Algorithms and TABU. Microprocessor and Microsystems, 16(7):351--367, 1992.

.... of Instrument and Control Engineers, 19] Lettre du Transputer et des Calculateurs Distribu es, 507] Mechatronics, 121, 541] Memoirs of the Faculty of Engineering, Fukui University, 515] Microprocessing and Microprogramming EURO Micro Journal, 546] Microprocessors and Microsystems (UK) [360] Neural Computation, 467] Neural Networks, 338] Neural Parallel Sci. Comput, 63] Neural Parallel Sci. Comput. USA) 66] New Generation Computing Journal, 18] Nuclear Instruments Methods in Physics Research A, 38] Nuclear Technology, 317] Parallel Computing, 65, 210, 67, 74, 75, ....

....Ronald C. 189] Axmann, Joachim K. 100, 171, 205, 249, 285, 307, 317] Baba, T. 253] Baden, Scott, 300] Balio, R. Del, 384, 385] Baluja, Shumeet, 201, 356, 357, 358, 432, 433] Banks, S. P. 42] Banzhaf, Wolfgang, 532, 533] Barak, Amnon, 211] Baskaran, Subbiah, 359] Battiti, R. [360] Beaumont, P. M. 156] Becker, Douglas E. 554, 555] Beckers, Mischa L. M. 231] Belding, Theodore C. 157] Belew, Richard K. 300] Bennett III, Forrest H. 26, 33] Bessiere, P. 269] Bessi ere, Pierre, 195, 361, 502, 503, 505, 506] Beyer, Hans Georg, 362] Bhattacharjya, Anoop K. ....

[Article contains additional citation context not shown here]

R. Battiti and G. Tecchiolli. Parallel biased search for combinatorial optimization: genetic algorithms and TABU. Microprocessors and Microsystems (UK), 16(7):351-367, September 1992. * CCA 2413/93 ga:Battiti92.

.... du Transputer et des Calculateurs Distribu es, 657] Machine Learning, 88] Mathematical and Computer Modelling, 241, 339] Memoirs of the Faculty of Engineering, Fukui University, 670] Microprocessing and Microprogramming EURO Micro Journal, 167] Microprocessors and Microsystems (UK) [80] Network: Computation in Neural Systems, 489] Neural Networks, 108] New Generation Computing, 395] Optical Engineering, 605] OR Spektrum, 396] ORSA Journal on Computing, 391] Parallel Computing, 62] Parallel Processing Letters, 148] Physical Review Letters, 363] Physics of the ....

....30] Baba, N. 65] Back, Thomas, 66, 67, 68, 69, 70, 71, 72, 73, 74, 6] Bala, Jerzy W. 75, 149, 150] Balio, R. Del, 148] Baluja, Shumeet, 76, 438] Bank van der, Dirk Johannes, 722] Barclay, A. R. 497] Barth, N. H. 77] Baskaran, Subbiah, 78] Bassus, R. C. 79] Battiti, R. [80] Battle, S. A. 198] Bean, James C. 81, 82, 83] Becker, B. D. 18] Becker, Douglas E. 590, 591] Becks, K. H. 84] Bedau, M. A. 714] Beer, Randall D. 85] Belew, Richard K. 86, 87, 88] Ben Kiki, Oren, 142] Benson, R. 218] Bergman, Aviv, 89] Bersini, Hugues, 90, 91, 92, ....

[Article contains additional citation context not shown here]

R. Battiti and G. Tecchiolli. Parallel biased search for combinatorial optimization: genetic algorithms and TABU. Microprocessors and Microsystems (UK), 16(7):351--367, September 1992. y(CCA 2413/93) ga:Battiti92.

.... algorithms for the traveling salesman problem [7] where it is shown that the probability of finding a sub optimal solution is given by a shifted exponential distribution, allowing for the time to find the first local optimum; and tabu search, provided that the search starts from a local optimum [1, 33]. The objective of this paper is to determine if the solution times for GRASP also have this property. i.e. they fit a two parameter exponential distribution. To do this, we consider five GRASPs that have been reported in the literature and for which we have source code: 1. maximum independent set ....

....component of the heuristic. Let # # = max # i , # i x i yet unassigned and # # = min # i , # i x i yet unassigned , and let # (0 # # # 1) be the restricted candidate parameter. A new value for # is selected, at random, at each iteration, from the uniform distribution U[0, 1]. A candidate x i = true is inserted into the RCL if # i # # # # (# # # # ) Likewise, a candidate x i = false is inserted if # i # # # # (# # # # ) 2.4.3. Local search phase. To define the local search procedure, some preliminary definitions have to be made. ....

R. Battiti and G. Tecchiolli. Parallel biased search for combinatorial optimization: Genetic algorithms and TABU. Microprocessors and Microsystems, 16:351--367, 1992.

....allowing for the time to find the first local GRASP SOLUTION TIME DISTRIBUTION 3 0 0.2 0.4 0.6 0.8 1 0 2 4 6 8 10 12 probability time to sub optimal Figure 1. Probability distribution plot of measured data. optimum; and tabu search, provided that the search starts from a local optimum [1, 33]. The objective of this paper is to determine if the solution times for GRASP also have this property. i.e. they fit a two parameter exponential distribution. To do this, we consider five GRASPs that have been reported in the literature and for which we have source code: 1. maximum independent set ....

....i , # i x i yet unassigned 12 R.M. AIEX, M.G.C. RESENDE, AND C.C. RIBEIRO and # # = min # i , # i x i yet unassigned , and let # (0 # # # 1) be the restricted candidate parameter. A new value for # is selected, at random, at each iteration, from the uniform distribution U[0, 1]. A candidate x i = true is inserted into the RCL if # i # # # # (# # # # ) Likewise, a candidate x i = false is inserted if # i # # # # (# # # # ) 2.4.3. Local search phase. To define the local search procedure, some preliminary definitions have to be made. ....

R. Battiti and G. Tecchiolli. Parallel biased search for combinatorial optimization: Genetic algorithms and TABU. Microprocessors and Microsystems, 16:351--367, 1992.

....1 Introduction One of the simplest and most often used techniques to parallelize iterative heuristic search methods such as tabu search (TS) performs many concurrent independent search threads on the same problem. This approach have been used to parallelize the TS method applied to the QAP [1, 15], the Job Shop Scheduling problem [16] the location allocation problem with balancing requirements [6] Parallelization based on independent search threads is computationally equivalent to a sequential approach where repeated executions of the same algorithm are performed with different initial ....

....executions of the same algorithm are performed with different initial solutions and eventually different search parameters. The behavior of these algorithms approximates an exponential distribution with respect to the probability of failure p(t) to find the optimal solution before iteration t [1, 15]. As reported in the taxonomy of Crainic, Toulouse and Gendreau [5] the parallelization of a method such as TS can also be realized based on knowledge sharing and cooperation between the different search threads. Huberman [10] conjectures 1 a performance improvement when concurrent search ....

R. Battiti and G. Tecchiolli. Parallel Biased Search for Combinatorial Optimization: Genetic Algorithms and TABU. 16(7):351--367, 1992.

....is possible by this strategy. See also Chakrapani Skorin Kapov [5] for another example of this type. High level parallelization strategies. The simplest strategy for high level parallelization executing p independent runs is also an attractive option here. Experimental work (see e.g. [3, 28]) suggests that as long as each run is sufficiently long, the solution quality is not deteriorated as compared to the corresponding sequential algorithm. By sufficiently long it is meant that each search should at least sample some minimum number of local optima. Determining how many moves it ....

....least sample some minimum number of local optima. Determining how many moves it takes to travel from one local optimum to another is in general not a trivial question (see Johnson and Papadimitriou [18] for a theoretical discussion) but for the QAP at least, which is the test problem used in [3] it appears to be not larger than the problem size n. Also, the minimal number of local optima which should be visited by each searcher is quite low, so the minimal number of iterations needed is probably proportional to the problem size, with a fairly small constant. It is also appealing to try ....

[Article contains additional citation context not shown here]

Battiti, R., Tecchioli, G., Parallel biased search for Combinatorial Optimization: Genetic Algorithms and TABU, Microprocessors and Microsystems 16 (1992) 351-- 367.

....operator protects against such irrecoverable losses. Genetic Algorithms are known to be robust and have attracted researchers for their wide range of applicability. However, the simple GAs described above have only limited power, and recent research has pursued the development of extended GAs [4, 8, 38, 9, 67]. Furthermore, GAs are suitable for parallel processing, such as the neural computation [25, 69, 72, 73, 30] In particular, with the emergence of massively parallel computers, researchers have begun to pay more and more attention to the needs of high performance computing that is required by ....

....100.00 120.00 140.00 160.00 0.00 10.00 20.00 30.00 40.00 50.00 60.00 CPU TIME(1 60 msec) TOUR COST Worst Mean Best Mutation rate = 0 Crossover rate = 0.7 Figure 5.2: Tour Length for 100 city TSP by GA 1 3 4 5 6 7 0 2 Parent1 [ 4 7 . Parent2 [ 4 5 . Child(1,2) 4 7 . Parent1 [ 2 3 6 1 0 4 7 5 ] Parent2 [ 7 0 1 4 5 2 6 3 ] Child(1,2) 4 7 5 2 6 3 1 0 ] Random Choose 5,7 Heuristic Crossover (By Grefenstette [1985] Figure 5.3: Heuristic Greedy Crossover the greedy crossover introduced by [24] Figure 5.3) This operator constructs offspring from two parent tours as follows: 1. Pick a ....

[Article contains additional citation context not shown here]

R. Battiti and G. Tecchiolli. Parallel biased search for combinatorial optimization: Genetic algorithms and tabu. Computation: The Micro and the Macro View, Vol. 10, pp. 351--367, September 1992.

....systems, such as parallel processing systems. Good resource scheduling strategies make good use of the complex parallel systems. Many parallel search algorithms have been developed and studied theoretically and empirically. Significant speedups have been achieved on existing parallel computers [19, 37 39, 44,79,174,175]. Among the seven issues of search algorithms, we identify the two most critical ones for nonlinear optimization: handling nonlinear constraints and escaping from local minima. The representation of a search space is derived from the feasible domain of an optimization problem, and is usually ....

....If there are no better solutions in the neighborhood, the tabu search moves to the best neighboring solution. The tabu list prevents returning to the local optimum from which the search has recently escaped. Tabu search has obtained good results in solving some large discrete optimization problems [18, 19,44,112,223]. 2.2 Proposed Methods for Handling Nonlinear Constraints In this section, we present a new Lagrangian formulation to handle inequality constraints and propose new methods to search for saddle points of Lagrangian functions more efficiently. Our method employs adaptive control of relative ....

R. Battiti and G. Tecchiolli. Parallel biased search for combinatorial optimization: Genetic algorithms and TABU. Microprocessors and Microsystems, 16:351--367, 1992.

....is preferable to avoid reaching the same local minimum repeatedly. The problem is how to save a solution so it can be identified in a later iteration using a fast and compact memory structure. A solution to this problem is to use a hashing table. A very simple example of this technique is used in [Battiti et al., 1992] where the hashing function consists of the simplest (non unique) identification of a solution its objective value. They use these tools for detecting cycles in a TS like method tested on the quadratic assignment problem (QAP) and on a similar problem. Their argument for using the simple hashing ....

....the TS memory, it contributes with its own search experience to the memory, and receives the experience from all other processors from the memory. In [Crainic et al., 1993] the conclusion is that TS benefits most from the asynchronous, global TS memory parallelization. In [Malek et al., 1989] and [Battiti et al., 1992] some simple parallelizations of TS are presented, and both papers conclude that TS can benefit very much from parallelization. Final remarks After the completion of a smaller project on tabu search [Dam and Zachariassen, 1993] we made some additional experiments, which encouraged us to ....

Battiti; Tecchiolli, G. "Parallel biased search for combinatorial optimization: Genetic algorithms and TABU ", Microprocessors and Microsystems 16, 7, 1992.

....problem with sparse flow matrix, of mapping tasks to processors in a multi processor system in order to minimize the time spent in inter processor communication. It is noteworthy that, due to the sparsity of the task graph, implementing a move (swap a single pair of tasks) does not signifi11 [1] Battiti Tecchiolli p RS MPSS p KS MPSS p RS MPSS 1 RS SPSS 1 KS SPDS p RS MPSS 1 RS SPSS Knowledge Synchronous Collegial Knowledge Collegial Rigid Synchronous SPSS SPDS MPSS MPDS 1 Search Differentiation 21 Control and Communication Type Control Cardinality [23,25] Taillard [23] Taillard [21] ....

.... SPSS Knowledge Synchronous Collegial Knowledge Collegial Rigid Synchronous SPSS SPDS MPSS MPDS 1 Search Differentiation 21 Control and Communication Type Control Cardinality [23,25] Taillard [23] Taillard [21] Malek all [11] Fiechter [3,4,5] Chakrapani Skorin Kapov [24] Taillard [3,4,5,23] [1,23,25] [11,24] p Figure 1: Taxonomy Dimensions cantly affect the values of most other possible moves; hence, most improving moves are still improving. Two operations are therefore performed in parallel: the candidate moves are identified and evaluated and, second, multiple moves are implemented. To ....

[Article contains additional citation context not shown here]

R. Battiti and G. Tecchiolli. Parallel Biased Search for Combinatorial Optimization: Genetic Algorithms and TABU. Microprocessors and Microsystems, 16:351--367, 1992.

....the search space in an efficient way. It can be demonstrated that the probability of visiting points at large Hamming distances with respect to a starting configuration is much higher than in the case of a random walk in the search space. The RTS algorithm is studied in detail in [6] while [5] is dedicated to a study of the parallel properties. Benchmarks and comparisons with respect to Simulated Annealing [9] Repeated Local Minima Search, Genetic Algorithms and Mean Field Theory neural nets [10] have been executed with fully satisfactory results. III. TIE APPLICATION FOR TRAINING ....

R. Battiti and G. Tecchiolli, "Parallel biased search for combi- natorial optimization: genetic algorithms and TABU," Microprocesso $ and Microsystems, vol. 16, no. 7, pp. 351 367, 1992.

....classes. For example if f is differentiable and well conditioned, steepest descent, conjugate gradient, or task specific algorithms can be profitably used. 3 THE COMBINATORIAL COMPONENT: RTS The Reactive Tabu Search (RTS) for combinatorial optimization has been proposed by the authors in [2] and [3] This section describes the basic principles of RTS while the algorithm details and the modifications regarding the interface between combinatorial and continuous optimization are described in APPENDIX I. RTS is a local search algorithm that generates a trajectory X of points in the ....

.... can be reduced to some bytes per iteration, while the time is reduced to that needed to calculate a memory address from the current configuration and to execute a small number of comparisons and updates of variables [3] RTS has been used for problems ranging from combinatorial optimization [2, 3, 7, 8], minimization of continuous functions [3] and sub symbolic machine learning tasks [5, 6] 4 THE LOCAL MINIMIZER: AFFINE SHAKER This section summarizes the main features of the Affine Shaker algorithm of [4] an adaptive random search algorithm based on the theoretical framework of [28] The ....

R. Battiti and G. Tecchiolli, Parallel biased search for combinatorial optimization: genetic algorithms and TABU, Microprocessors and Microsystems 16 (1992) 351--367.

....on the trajectory, but a smaller spread means a smaller entropy (HZ = H f Gamma log(1= p (1 Gamma ff ) The uncertainty, and therefore the information, for points on the trajectory is larger than that for points in the neighborhood. Autoregressive models for the QAP task are considered in [1]. discrete dynamical system that generates the search trajectory is regulated by the entire past history. The asymptotic space time requirements can be reduced to one bit and a small number of CPU cycles per RTS iteration by using hashing strategies. The criterion for prohibiting moves is the same ....

R. Battiti and G. Tecchiolli, Parallel biased search for combinatorial optimization: genetic algorithms and TABU, Microprocessors and Microsystems 16, 351--367 (1992).

....(Sec. 6.3) Special purpose VLSI systems have been designed and realized for real time pattern recognition applications with machine learning techniques (Sec. 6.4) 6.1 Combinatorial tasks Reactive Search has been applied to the following list of problems. ffl Quadratic Assignment Problem (QAP) [6, 7, 8] ffl N k model, a model of biological inspiration [6, 10] ffl 0 1 Knapsack [7] ffl Multi Knapsack (with multiple constraints) 10] ffl Max Clique [18] ffl Biquadratic Assignment Problem (Bi QAP) 5] In many cases the results obtained with alternative competitive heuristics have been ....

....and realized for real time pattern recognition applications with machine learning techniques (Sec. 6.4) 6.1 Combinatorial tasks Reactive Search has been applied to the following list of problems. ffl Quadratic Assignment Problem (QAP) 6, 7, 8] ffl N k model, a model of biological inspiration [6, 10] ffl 0 1 Knapsack [7] ffl Multi Knapsack (with multiple constraints) 10] ffl Max Clique [18] ffl Biquadratic Assignment Problem (Bi QAP) 5] In many cases the results obtained with alternative competitive heuristics have been duplicated with low computational complexity, and without intensive ....

[Article contains additional citation context not shown here]

R. Battiti and G. Tecchiolli (1992) Parallel biased search for combinatorial optimization: Genetic algorithms and tabu. Microprocessor and Microsystems 16:351--367.

....analysis of this behavior is given in Appendix II, where we show that the probability of visiting points at large Hamming distances with respect to a starting configuration is much higher than in the case of a random walk in the search space. The RTS algorithm is studied in detail in [7] while [6] is dedicated to a study of the parallel properties. 3 THE APPLICATION FOR TRAINING NEURAL NETWORKS We consider two paradigmatic systems in the area of neural networks (sub symbolic machine learning) the Multi Layer Perceptron (MLP) of [41] see the applications considered in Sections 4.1 4.3, ....

R. Battiti and G. Tecchiolli, "Parallel biased search for combinatorial optimization: genetic algorithms and TABU," Microprocessors and Microsystems, vol. 16, no. 7, pp. 351--367, 1992.

....in all cases and the number of iterations is not a#ected in a critical way, justifying the choice of fixed values 1.1 and 0.9 for all tests. In the last column of Table 5 we modified the memory mechanism so that the function value is recorded instead of the configuration, the same method used in [2]. Because the same function value can be associated with di#erent configurations, there is a small probability of false alarms , i.e. reactions of the algorithm when there is no actual repetition of configurations. The advantage of the method is that the memory requirement is reduced: only a ....

....to the strict cycle avoidance confirms that avoiding cycles is not the ultimate goal of the search process [8] the broader objective being that of stimulating a bold exploration of the search space. A straightforward parallel implementation of a primitive version of R TABU was presented in [2], where independent searches are executed in the di#erent nodes. We are now experimenting the use the above mentioned memory structures in the fully parallel case, where the information contained in a set of suboptimal configurations is used to create a new set of candidate points (see also [9] ....

R. BATTITI and G. TECCHIOLLI, 1992. Parallel Biased Search for Combinatorial Optimization, Microprocessors and Microsystems 16(7), 351--367.

....in all cases and the number of iterations is not affected in a critical way, justifying the choice of fixed values 1.1 and 0.9 for all tests. In the last column of Table 5 we modified the memory mechanism so that the function value is recorded instead of the configuration, the same method used in [2]. Because the same function value can be associated with different configurations, there is a small probability of false alarms , i.e. reactions of the algorithm when there is no actual repetition of configurations. The advantage of the method is that the memory requirement is reduced: only a ....

....to the strict cycle avoidance confirms that avoiding cycles is not the ultimate goal of the search process [8] the broader objective being that of stimulating a bold exploration of the search space. A straightforward parallel implementation of a primitive version of R TABU was presented in [2], where independent searches are executed in the different nodes. We are now experimenting the use the above mentioned memory structures in the fully parallel case, where the information contained in a set of suboptimal configurations is used to create a new set of candidate points (see also ....

R. BATTITI and G. TECCHIOLLI, 1992. Parallel Biased Search for Combinatorial Optimization, Microprocessors and Microsystems 16(7), 351--367.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC