#### DMCA

## Filter Pattern Search Algorithms for Mixed Variable Constrained Optimization Problems (2004)

### Cached

### Download Links

Venue: | SIAM Journal on Optimization |

Citations: | 55 - 6 self |

### Citations

10047 |
Genetic Algorithms
- Goldberg
- 1989
(Show Context)
Citation Context ...ally done randomly, with increased probability of selection for individuals with better fitness values. The most common approaches select parents based on proportional values of the fitness functions =-=[63]-=-, ordinal rankings of individuals by fitness value [11], or a combination of the two [63, 110]. In genetic algorithms, since the individuals are usually represented by binary strings, the crossover (r... |

5156 | Optimization by Simulated Annealing,
- Kirkpatrick, Gelatt, et al.
- 1983
(Show Context)
Citation Context ...h T decremented in accordance with a user-specified cooling schedule. Simulated annealing was formally introduced as a general combinatorial optimization technique by Kirkpatrick, Gellatt, and Vecchi =-=[87]-=-, and by Cerny [29] indepen-13 Simulated Annealing Algorithm Choose initial point x ∈ ℜ n at random While (T > Tmin) do • Randomly generate a neighbor y of x • If (E(y) < E(x)), then Else End – Set x... |

5119 | Stochastic relaxation, Gibbs Distributions and the Bayesian Restoration of Images
- Geman, Geman
- 1984
(Show Context)
Citation Context ...ng to the cooling schedule) Figure 2.1 A Simulated Annealing Algorithm dently. Much of the theory for combinatorial optimization problems relies on the asymptotic behavior of Markov chains (e.g., see =-=[55]-=-, [57], [102], [111], and [115]). Hajek [66] is generally given credit for first establishing necessary and sufficient conditions for proving that the algorithm converges to a global minimum in probab... |

1664 |
Optimization and Nonsmooth Analysis
- Clarke
- 1983
(Show Context)
Citation Context ...lowing a review of the appropriate literature in Chapter 2, Chapter 3 summarizes important definitions and results from the theory of positive linear dependence [38] and the Clarke nonsmooth calculus =-=[31]-=- that will be used extensively throughout this document. Chapter 4 discusses in greater detail the basic GPS algorithm for NLP problems with bound and simple linear constraints [8, 94, 95, 130], and t... |

1216 |
Nonlinear Programming: Theory and Algorithms,
- Bazaraa, Sherali, et al.
- 2006
(Show Context)
Citation Context ...ker (KKT) first-order necessary conditions for optimality. Originally published in [83] and [92], its proof is omitted, since it can be found in many standard nonlinear programming textbooks, such as =-=[14]-=-. The constraint qualification mentioned in the statement of the theorem is an additional assumption that must be satisfied in order for the theorem to hold. Several different constraint qualification... |

1090 | An Analysis of the Behavior of a Class of Genetic Adaptive Systems - Jong - 1975 |

1047 |
On the Origin of Species by Means of Natural Selection
- Darwin
- 1859
(Show Context)
Citation Context ...Algorithms is given in Figure 2.3, the details of which will define the various classes of algorithms. Genetic Algorithms Genetic Algorithms were developed by Holland [74, 75] as a model of Darwinian =-=[37]-=- theory of genetic evolution, where members of a population with higher fitness values are given a higher probability of reproducing. In most implementations, variables are encoded as binary strings, ... |

907 |
A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output from a
- McKay, Beckman, et al.
- 1979
(Show Context)
Citation Context ...mooth data chosen at the data sites. Examples include kriging, responses surfaces, polynomial interpolants, and neural networks. Data site selection is often done using Latin hypercube sampling (LHS) =-=[107, 126]-=-, orthogonal arrays (OA) [116], OA-based LHS [129], or other probability-based space filling strategy, – or alternatively, by experimental design, with the goal of obtaining a reasonable and rich samp... |

823 |
Artificial Intelligence Through Simulated Evolution
- Fogel, Owens, et al.
- 1966
(Show Context)
Citation Context ...e rate of evolution strategies remain open.21 Evolutionary Programming Evolutionary Programming was first introduced as an artificial intelligence technique applied to finite state machines by Fogel =-=[51]-=-, and it was later extended by Burgin [26, 27] and Atmar [3]. Though developed under entirely different circumstances, it is actually quite similar to evolution strategies. For example, each individua... |

680 | Tabu Search–Part I
- Glover
- 1989
(Show Context)
Citation Context ...ation shows favorable performance results compared to that of the traditional simulated annealing [79] and genetic algorithms [80]. 2.2.2 Tabu Search The tabu search was formally introduced by Glover =-=[59, 60, 61]-=- as a metaheuristic for solving combinatorial optimization problems, although several of its ideas were developed independently by Hansen [67]. The basic idea is that, in searching for a better point ... |

614 |
Convex Analysis and Minimization Algorithms
- Hiriart-Urruty, Lemaréchal
- 1993
(Show Context)
Citation Context ... f is differentiable at x, then ∇f(x) ∈ ∂f(x). • When f is continuously differentiable at x, ∂f(x) reduces to the singleton {∇f(x)}, and when f is convex near x, it coincides with the subdifferential =-=[73, 103]-=-; i.e., ∂f(x) = {η ∈ ℜ n : f(y) ≥ f(x) + η T (y − x) for all y ∈ ℜ n }. • If f is strictly differentiable at x, then f is Lipschitz near x and differentiable at x with ∂f(x) = {∇f(x)}. • If f is Lipsc... |

527 | AMPL: A Modeling Language for Mathematical Programming - Fourer, Kernighan - 2002 |

501 |
A Thermodynamical Approach to the Travelling Salesman Problem: An Ef¢cient Simulation Algorithm,
- Cerny
- 1985
(Show Context)
Citation Context ...accordance with a user-specified cooling schedule. Simulated annealing was formally introduced as a general combinatorial optimization technique by Kirkpatrick, Gellatt, and Vecchi [87], and by Cerny =-=[29]-=- indepen-13 Simulated Annealing Algorithm Choose initial point x ∈ ℜ n at random While (T > Tmin) do • Randomly generate a neighbor y of x • If (E(y) < E(x)), then Else End – Set x = y E(y)−E(x) − – ... |

481 | Nonlinear Programming
- Kuhn, Tucker
- 1951
(Show Context)
Citation Context ... T ◦ Y (x) = {v ∈ ℜ n : v T w ≤ 0 ∀ w ∈ TY (x)}. (1.5)6 The following theorem describes the Karush-Kuhn-Tucker (KKT) first-order necessary conditions for optimality. Originally published in [83] and =-=[92]-=-, its proof is omitted, since it can be found in many standard nonlinear programming textbooks, such as [14]. The constraint qualification mentioned in the statement of the theorem is an additional as... |

478 |
Numerical Methods for Unconstrained Optimization and Nonlinear Equations
- Schnabel
- 1996
(Show Context)
Citation Context ...s” for user-defined search and surrogate functions, so that users can seamlessly attach their own favorite method to the code. • Optional scaling of the mesh directions in a manner similar to that of =-=[41]-=-. • Additional optional termination criteria, including limits on the number of GPS iterations, number of function calls, and CPU time. • Real-time plots of the filter (similar to Figure 4.5) and iter... |

457 |
Direct search” solution of numerical and statistical problems
- Hooke, Jeeves
- 1961
(Show Context)
Citation Context ...ct search algorithms, in which the minimizer of a continuous function is sought without the use of derivatives. Among the first direct search algorithms were the well-known method of Hooke and Jeeves =-=[76]-=- and the simplex algorithm of Nelder and Mead [113]. At the time, these were considered heuristics with no formal convergence theory. 2.3.1 Lewis and Torczon In an award-winning 1997 paper, Torczon [1... |

387 | Tabu Search–Part II
- Glover
- 1990
(Show Context)
Citation Context ...ation shows favorable performance results compared to that of the traditional simulated annealing [79] and genetic algorithms [80]. 2.2.2 Tabu Search The tabu search was formally introduced by Glover =-=[59, 60, 61]-=- as a metaheuristic for solving combinatorial optimization problems, although several of its ideas were developed independently by Hansen [67]. The basic idea is that, in searching for a better point ... |

379 | Future paths for integer programming and links to artificial intelli- gence
- Glover
- 1986
(Show Context)
Citation Context ...ation shows favorable performance results compared to that of the traditional simulated annealing [79] and genetic algorithms [80]. 2.2.2 Tabu Search The tabu search was formally introduced by Glover =-=[59, 60, 61]-=- as a metaheuristic for solving combinatorial optimization problems, although several of its ideas were developed independently by Hansen [67]. The basic idea is that, in searching for a better point ... |

339 |
Extremum problems with inequalities as subsidiary conditions
- John
- 1948
(Show Context)
Citation Context ...first-order KarushKuhn-Tucker (KKT) necessary conditions for optimality with respect to the continuous variables. An alternative to the KKT conditions for optimality is the Fritz John conditions (see =-=[82]-=-). These conditions also have a similar constraint qualification, but it is automatically satisfied for any problem without equality constraints, as is the case7 for (1.1). The Fritz John conditions ... |

277 |
Nonlinear Programming
- Mangasarian
- 1969
(Show Context)
Citation Context ...of the theorem is an additional assumption that must be satisfied in order for the theorem to hold. Several different constraint qualifications appear in the literature, and are summarized in [14] or =-=[104]-=-. Among these is the KKT constraint qualification, which is stated immediately preceding the theorem. It can be shown that a sufficient condition for the first-order KKT constraint qualification to ho... |

272 |
Cooling schedules for optimal annealing
- Hajek
- 1988
(Show Context)
Citation Context ...ulated Annealing Algorithm dently. Much of the theory for combinatorial optimization problems relies on the asymptotic behavior of Markov chains (e.g., see [55], [57], [102], [111], and [115]). Hajek =-=[66]-=- is generally given credit for first establishing necessary and sufficient conditions for proving that the algorithm converges to a global minimum in probability. Specifically, the cooling schedule mu... |

263 | A survey of evolution strategies - Back, Hoffmeister, et al. - 1991 |

262 |
Sur les operations dans les ensembles abstraits et leur application aux equations integrales
- Banach
- 1922
(Show Context)
Citation Context ...ntroduced a new class of so-called contractive mapping genetic algorithms, which, under certain conditions, converge (not just in probability) based on Banach’s well-known contraction mapping theorem =-=[12]-=-. A drawback of this approach is that the contraction mapping requires an improvement of the entire population’s average fitness at each iteration. If an an improved population is not found, then more... |

249 | Nonlinear programming without a penalty function
- Fletcher, Leyffer
(Show Context)
Citation Context ...undation of this work. In the first paper [7], Audet and Dennis implement a filter method into the GPS framework to handle general nonlinear constraints. Originally introduced by Fletcher and Leyffer =-=[47]-=- to conveniently globalize sequential quadratic programming (SQP) and sequential linear programming (SLP), filter methods accept steps if either the objective function or an aggregate constraint viola... |

230 | Very fast simulated re-annealing
- Ingber
- 1989
(Show Context)
Citation Context ... to be exponentially long. Furthermore, Bilbro and Snyder [17] and Rutenbar [121] identify problems for which simulated annealing problems are unable to find the global minimum in finite time. Ingber =-=[78]-=- offers some algorithmic improvements without sacrificing theoretical convergence. His implementation shows favorable performance results compared to that of the traditional simulated annealing [79] a... |

227 |
Generalized Benders decomposition
- Geoffrion
- 1972
(Show Context)
Citation Context ...[88], Leyffer [97], Floudas [50], and Viswanathan and Grossman [135]. 2.1.2 Generalized Benders Decomposition Similar to outer approximation, Generalized Benders Decomposition, developed by Geoffrion =-=[56]-=-, solves the same NLP subproblem, but a different MILP subproblem, which is formed by linearizing the Lagrangian function L(x, λ) = f(x) + λT C(x) around the current point, where λ ∈ ℜp is the vector ... |

223 | Simulated annealing: Practice versus theory
- Ingber
- 1993
(Show Context)
Citation Context ...vergence properties is found in [131]. Although convergence in probability to the global optimum is certainly a desirable result, computational studies show that the rate of convergence is often slow =-=[79]-=-. A theoretical reasoning for this can be found in [102], where convergence on some combinatorial optimization problems is shown to be exponentially long. Furthermore, Bilbro and Snyder [17] and Ruten... |

212 |
An outer-approximation algorithm for a class of mixed integer nonlinear programs
- Duran, Grossman
- 1986
(Show Context)
Citation Context ... Benders Decomposition, Branch and Bound, and the Extended Cutting Plane method. 2.1.1 Outer Approximation Outer Approximation was first introduced for a class of MINLP problems by Duran and Grossman =-=[43]-=- and then extended to more general problems by Fletcher and Leyffer [46]. At each iteration, an upper bound is obtained by solving a restricted10 NLP, in which discrete variables are held constant. T... |

204 | A Rigorous Framework for Optimization of Expensive Functions by Surrogates
- Booker, Dennis, et al.
- 1999
(Show Context)
Citation Context ...blems is to optimize a significantly less costly surrogate function during the search step of each iteration, map the resulting point to a nearby mesh point, and compute its true function value there =-=[21]-=-. While the search step contributes nothing to the convergence theory of GPS (and in fact, an unsuitable search may impede performance), the use of surrogates enables the user to gain significant impr... |

187 | CUTE: Constrained and unconstrained testing environment
- Bongartz, Conn, et al.
- 1995
(Show Context)
Citation Context ...sic GPS algorithm that uses any available derivative information to reduce the number of function evaluations for each GPS iteration. It includes numerical results on some test problems from the cute =-=[18]-=- set. Finally, Chapter 9 offers concluding remarks and several recommendations for future research.9 Chapter 2 Literature Survey In considering the solution of MVP problems, a discussion of the exist... |

179 |
Outline for a logical theory of adaptive systems
- Holland
- 1962
(Show Context)
Citation Context ...framework [138] for Evolutionary Algorithms is given in Figure 2.3, the details of which will define the various classes of algorithms. Genetic Algorithms Genetic Algorithms were developed by Holland =-=[74, 75]-=- as a model of Darwinian [37] theory of genetic evolution, where members of a population with higher fitness values are given a higher probability of reproducing. In most implementations, variables ar... |

171 |
Numerical Analysis
- Kincaid, Cheney
- 1990
(Show Context)
Citation Context ...ntegrations were performed by applying a composite Simpson’s Rule, with nodes matching those of the cubic spline. This eliminates truncation error, since Simpson’s Rule is exact for cubic polynomials =-=[85]-=-. 7.3.2 Choosing Discrete Neighbors Recall from the discussion in Section 5.1, and especially from Definition 5.3, that the neighborhood structure that the user chooses to incorporate determines the d... |

156 |
Minima of Functions of Several Variables with Inequalities as Side Constraints
- Karush
- 1939
(Show Context)
Citation Context ... NY (x) = T ◦ Y (x) = {v ∈ ℜ n : v T w ≤ 0 ∀ w ∈ TY (x)}. (1.5)6 The following theorem describes the Karush-Kuhn-Tucker (KKT) first-order necessary conditions for optimality. Originally published in =-=[83]-=- and [92], its proof is omitted, since it can be found in many standard nonlinear programming textbooks, such as [14]. The constraint qualification mentioned in the statement of the theorem is an addi... |

142 |
The cutting plane method for solving convex programs
- Kelley
- 1960
(Show Context)
Citation Context ...ratic programming (SQP) branch and bound algorithm. 2.1.4 Extended Cutting Plane Method The Extended Cutting Plane method was introduced by Westerlund and Pettersson [136] as an extension of Kelley’s =-=[84]-=- cutting plane algorithm for NLPs. In this algorithm, a non-decreasing sequence of lower bounds is generated by solving a sequence of MILPs, in which each MILP adds as a constraint a linearization of ... |

133 | Direct search methods on parallel machines
- Dennis, Torczon
- 1991
(Show Context)
Citation Context ...s, and showed that it includes coordinate search with fixed step sizes, evolutionary operation using factorial design [23], Hooke and Jeeves’ algorithm [76], and the multidirectional search algorithm =-=[42]-=-. Without ever computing or approximating derivatives, Torczon [130] also show that if all iterates lie in a compact set and the objective function f is continuously differentiable in a neighborhood o... |

121 | What makes a problem hard for a genetic algorithm? Some anomolous results and their explanation
- Forrest, Mitchell
- 1993
(Show Context)
Citation Context ...are not reliable at finding global optima (which is their goal), and when they do converge to a local optima, it is often at a slower rate than traditional gradient or so-called hill-climbing methods =-=[52, 128]-=-. Furthermore, because of their very nature, the requirement to generate new populations rather than single iterates makes them even more costly with respect to the number of function evaluations.20 ... |

121 |
Finite markov chain analysis of genetic algo- rithms
- Goldberg, Segrest
- 1987
(Show Context)
Citation Context ...ted bit to its complement. A brief survey of convergence theory results for genetic algorithms can be found in [110]. A Markov chain analysis of genetic algorithms with finite populations is given in =-=[64]-=-, but only considers reproduction and mutation operators. Kingdon [86] studied and attempted to characterize problems that genetic algorithms have difficulty solving, and provides some convergence res... |

121 | Genetic algorithms and very fast simulated reannealing: A comparison
- Ingber, Rosen
- 1992
(Show Context)
Citation Context ...ic improvements without sacrificing theoretical convergence. His implementation shows favorable performance results compared to that of the traditional simulated annealing [79] and genetic algorithms =-=[80]-=-. 2.2.2 Tabu Search The tabu search was formally introduced by Glover [59, 60, 61] as a metaheuristic for solving combinatorial optimization problems, although several of its ideas were developed inde... |

107 |
Evolutionary operation: A method for increasing industrial productivity
- Box
- 1957
(Show Context)
Citation Context ... of generalized pattern search (GPS) methods for solving unconstrained NLP problems, and showed that it includes coordinate search with fixed step sizes, evolutionary operation using factorial design =-=[23]-=-, Hooke and Jeeves’ algorithm [76], and the multidirectional search algorithm [42]. Without ever computing or approximating derivatives, Torczon [130] also show that if all iterates lie in a compact s... |

106 | Pattern Search Algorithms for Bound Constrained Minimization
- Lewis, Torczon
- 1999
(Show Context)
Citation Context ...nonsmooth calculus [31] that will be used extensively throughout this document. Chapter 4 discusses in greater detail the basic GPS algorithm for NLP problems with bound and simple linear constraints =-=[8, 94, 95, 130]-=-, and the filter GPS algorithm for general NLP problems [7]. Chapter 5 presents the Audet-Dennis GPS algorithm for solving bound constrained MVP problems [8], but with new convergence results (for bou... |

101 |
Convergence of an annealing algorithm
- Lundy, Mees
- 1986
(Show Context)
Citation Context ...oling schedule) Figure 2.1 A Simulated Annealing Algorithm dently. Much of the theory for combinatorial optimization problems relies on the asymptotic behavior of Markov chains (e.g., see [55], [57], =-=[102]-=-, [111], and [115]). Hajek [66] is generally given credit for first establishing necessary and sufficient conditions for proving that the algorithm converges to a global minimum in probability. Specif... |

94 | Analysis of generalized pattern searches
- Audet, Dennis
- 2000
(Show Context)
Citation Context ...gorithm is, for the most part, unknown. An asynchronous parallel version of the GPS algorithm has also been developed by Hough, Kolda and Torczon [77, 90, 91]. 2.3.2 Audet and Dennis Audet and Dennis =-=[6]-=- present a hierarchy of convergence results for a slightly modified, but equivalent version of GPS for bound and linearly constraints, in which the strength of the results depends on local continuity ... |

92 | A Globally Convergent Augmented Lagrangian Algorithm for Optimization With General Constraints and Simple Bounds
- Conn, Gould, et al.
- 1991
(Show Context)
Citation Context ...ts Dk conforms to X for some ɛ > 0. Assumption A1 already is sufficient to guarantee that there are convergent subsequences of the iteration sequence. However, this is, in fact, a standard assumption =-=[6, 7, 8, 32, 35, 47, 94, 95, 130]-=-. A sufficient condition for this to hold is that the level set L(x0) = {x ∈ X : f(x) ≤ f(x0)} is compact. We can assume that L(x0) is bounded, but not closed, since we allow f to be discontinuous and... |

87 | Asynchronous parallel pattern search for nonlinear optimization
- Hough, Kolda, et al.
(Show Context)
Citation Context ...l results are provided. Thus, the efficiency of the algorithm is, for the most part, unknown. An asynchronous parallel version of the GPS algorithm has also been developed by Hough, Kolda and Torczon =-=[77, 90, 91]-=-. 2.3.2 Audet and Dennis Audet and Dennis [6] present a hierarchy of convergence results for a slightly modified, but equivalent version of GPS for bound and linearly constraints, in which the strengt... |

75 |
A tree-search algorithm for mixed integer programming problems
- Dakin
- 1965
(Show Context)
Citation Context ...angian function L(x, λ) = f(x) + λT C(x) around the current point, where λ ∈ ℜp is the vector of Lagrange multipliers. 2.1.3 Branch and Bound Methods Branch and Bound methods were introduced by Dakin =-=[36]-=-, and further developed by Gupta and Ravindran [65], Borchers and Mitchell [22], and Leyffer [98]. In this class of methods, if a continuous NLP relaxation of the MINLP does not produce a feasible sol... |

69 |
The steepest ascent mildest descent heuristic for combinatorial programming
- Hansen
- 1986
(Show Context)
Citation Context ...The tabu search was formally introduced by Glover [59, 60, 61] as a metaheuristic for solving combinatorial optimization problems, although several of its ideas were developed independently by Hansen =-=[67]-=-. The basic idea is that, in searching for a better point among its discrete neighbors (defined by the user), the algorithm may accept a worse point if no better ones are found, and if the candidate p... |

69 | A Globally Convergent Augmented Lagrangian Pattern Search Algorithm for Optimization With General Constraints and Simple Bounds
- Lewis, Torczon
(Show Context)
Citation Context ...≥ 0 for any feasible x. Audet [4] provides several clever examples to show that many of Torczon’s theoretical results cannot be relaxed. For NLP problems with nonlinear constraints, Lewis and Torczon =-=[96]-=- developed a derivative-free augmented Lagrangian version of GPS. The augmented Lagrangian they use, which comes from Conn, Gould, and Toint [32], is given by Φ(x; λ, S, µ) = f(x) + p∑ i=1 λiCi(x) + 1... |

68 | A Pattern Search Filter Method for Nonlinear Programming Without Derivatives
- Audet, Dennis
(Show Context)
Citation Context ... summary of convergence results. We should note that, since this chapter is meant to clarify already existing algorithms and results, the convergence results remain exactly the same as those found in =-=[6, 7]-=-, with the exception of some revised terminology and notation. This is done despite our recent discovery that the key hypothesis of strict differentiability can be slightly weakened in a few of the im... |

63 |
Theory of positive linear dependence
- Davis
- 1954
(Show Context)
Citation Context ...is document is laid out as follows. Following a review of the appropriate literature in Chapter 2, Chapter 3 summarizes important definitions and results from the theory of positive linear dependence =-=[38]-=- and the Clarke nonsmooth calculus [31] that will be used extensively throughout this document. Chapter 4 discusses in greater detail the basic GPS algorithm for NLP problems with bound and simple lin... |

60 | On the global convergence of an SLP- filter algorithm
- Fletcher, Leyffer, et al.
- 1998
(Show Context)
Citation Context ...ming (SQP) and sequential linear programming (SLP), filter methods accept steps if either the objective function or an aggregate constraint violation function is reduced. Fletcher, Leyffer, and Toint =-=[18]-=- show convergence of the SLP-based approach to a limit point satisfying Fritz John [20] optimality conditions; they show convergence of the SQP approach to a KKT point [19], provided a constraint qual... |

60 |
Branch and bound experiments in convex non- linear integer programming
- Gupta, Ravindran
- 1985
(Show Context)
Citation Context ... current point, where λ ∈ ℜp is the vector of Lagrange multipliers. 2.1.3 Branch and Bound Methods Branch and Bound methods were introduced by Dakin [36], and further developed by Gupta and Ravindran =-=[65]-=-, Borchers and Mitchell [22], and Leyffer [98]. In this class of methods, if a continuous NLP relaxation of the MINLP does not produce a feasible solution (i.e., integer variables take on integer valu... |

59 | Tabu search for nonlinear and parametric optimization (with links to genetic algorithms). Discrete Applied Mathematics
- Glover
- 1994
(Show Context)
Citation Context ...N(x) \ T = ∅), then – Set s∗ ∈ arg min{f(s) : s ∈ N(x) \ T } – Set x ∗ ∈ arg min{f(s ∗ ), f(x ∗ )} • If (k > kmax or k > L or N(x) \ T = ∅), STOP • Update T Figure 2.2 A Tabu Search Algorithm Glover =-=[62]-=- offers ideas for extending the tabu search heuristic to nonlinear and parametric optimization problems. He points out that the tabu list must incorporate some notion of distance between the current p... |

56 |
Nonstationary Markov chains and convergence of the annealing al- gorithm
- Gidas
- 1985
(Show Context)
Citation Context ...the cooling schedule) Figure 2.1 A Simulated Annealing Algorithm dently. Much of the theory for combinatorial optimization problems relies on the asymptotic behavior of Markov chains (e.g., see [55], =-=[57]-=-, [102], [111], and [115]). Hajek [66] is generally given credit for first establishing necessary and sufficient conditions for proving that the algorithm converges to a global minimum in probability.... |

53 | Evolutionary programming and evolution strategies: Similarities and differences
- Back, Rudolph, et al.
- 1993
(Show Context)
Citation Context ...ted by a pair of vectors (x, σ) in precisely the same way as evolution strategies. The mutation operator is also essentially identical; however, evolutionary programming has no recombination operator =-=[10]-=-. Other than that, the primary differences between the two approaches involve the selection and competition processes. Rather than selecting the best offspring, evolutionary programming uses a tournam... |

53 |
Convergence theorems for a class of simulated annealing algo- rthms on &d
- Belisle
- 1992
(Show Context)
Citation Context ...In what remains an active area of research, convergence proofs for other classes of annealing algorithms, having various assumptions on cooling schedules and probability distributions can be found in =-=[16]-=-, [54], [100] and [101], among many others. An excellent overview of simulated annealing and description of its convergence properties is found in [131]. Although convergence in probability to the glo... |

48 | An improved branch and bound algorithm for mixed integer nonlinear programs
- Borchers, Mitchell
- 1994
(Show Context)
Citation Context ... is the vector of Lagrange multipliers. 2.1.3 Branch and Bound Methods Branch and Bound methods were introduced by Dakin [36], and further developed by Gupta and Ravindran [65], Borchers and Mitchell =-=[22]-=-, and Leyffer [98]. In this class of methods, if a continuous NLP relaxation of the MINLP does not produce a feasible solution (i.e., integer variables take on integer values), then the NLP solution i... |

47 | Simulated annealing algorithms for continuous global optimiza- tion: Convergence conditions
- Locatelli
- 2000
(Show Context)
Citation Context ...ive area of research, convergence proofs for other classes of annealing algorithms, having various assumptions on cooling schedules and probability distributions can be found in [16], [54], [100] and =-=[101]-=-, among many others. An excellent overview of simulated annealing and description of its convergence properties is found in [131]. Although convergence in probability to the global optimum is certainl... |

45 | Integrating SQP and branch-and-bound for mixed integer nonlinear programming
- Leyffer
- 1998
(Show Context)
Citation Context ...Lagrange multipliers. 2.1.3 Branch and Bound Methods Branch and Bound methods were introduced by Dakin [36], and further developed by Gupta and Ravindran [65], Borchers and Mitchell [22], and Leyffer =-=[98]-=-. In this class of methods, if a continuous NLP relaxation of the MINLP does not produce a feasible solution (i.e., integer variables take on integer values), then the NLP solution is treated as a low... |

39 |
Relaxation Strategy for the Structural Optimization of Process Flow Sheets”,
- Kocis, Grossmann
- 1987
(Show Context)
Citation Context ...rations). At each successive step, the lower and upper bounds approach each other, yielding an approximate solution. Variations and improvements of the method have been proposed by Kocis and Grossman =-=[88]-=-, Leyffer [97], Floudas [50], and Viswanathan and Grossman [135]. 2.1.2 Generalized Benders Decomposition Similar to outer approximation, Generalized Benders Decomposition, developed by Geoffrion [56]... |

35 |
A surrogate-model-based method for constrained optimization
- Audet, Dennis, et al.
- 2000
(Show Context)
Citation Context ...ite heuristic, such as those described in Section 2.2, or perhaps optimize an inexpensive surrogate function, as is common in difficult engineering design problems with expensive function evaluations =-=[5, 19, 20, 21]-=-. In the poll step, a positive spanning set Dk ⊆ D is chosen from which to construct the poll set. Again, we represent Dk also as a matrix whose columns are the members of the set. It is a function of... |

35 |
Cryogenic Systems
- Barron
- 1966
(Show Context)
Citation Context ...tion causes additional stress on the materials and, if excessive, can cause other difficulties, such as deformations in the material. The development of this constraint presented here is adapted from =-=[13]-=-. Since different insulators at different temperatures exhibit different contraction behaviors, we must treat thermal contraction of each insulator separately as a change in its thickness. The constra... |

32 | Metropolis-type annealing algorithms for global optimization
- Gelfand, Mitter
- 1993
(Show Context)
Citation Context ...t remains an active area of research, convergence proofs for other classes of annealing algorithms, having various assumptions on cooling schedules and probability distributions can be found in [16], =-=[54]-=-, [100] and [101], among many others. An excellent overview of simulated annealing and description of its convergence properties is found in [131]. Although convergence in probability to the global op... |

31 | On the convergence of grid-based methods for unconstrained optimization
- Coope, Price
- 2001
(Show Context)
Citation Context ...logous to the original and surrogate models, respectively.28 2.4 Other Relevant Methods A few other papers in related areas are also of interest. First, a recently published paper by Coope and Price =-=[35]-=- introduces what they term a grid-based method for unconstrained NLP problems with continuously differentiable objective functions. It is similar to that of Torczon [130], but with an alternative, mor... |

25 | Mixed variable optimization of the number and composition of heat intercepts in a thermal insulation system
- Kokkolaras, Audet, et al.
(Show Context)
Citation Context ...in the dimensions of the problem. For example, if an engineer wants to build a structure, different materials will have3 different characteristics, such as bounds on thicknesses. Such is the case in =-=[89]-=-, where the number of continuous variables is a function of the number of heat intercepts in a thermal insulation system, the latter being treated as a categorical variable. The constraint set can cha... |

23 | Convergence results for pattern search algorithms are tight
- Audet
- 1998
(Show Context)
Citation Context ...od of the level set {x ∈ ℜn : f(x) ≤ f(x0)}, the algorithm is guaranteed to produce a subsequence of iterates converging to a limit point ˆx satisfying ∇f(ˆx) T (x − ˆx) ≥ 0 for any feasible x. Audet =-=[4]-=- provides several clever examples to show that many of Torczon’s theoretical results cannot be relaxed. For NLP problems with nonlinear constraints, Lewis and Torczon [96] developed a derivative-free ... |

22 |
Optimization of functions with many minima
- Bilbro, Snyder
- 1991
(Show Context)
Citation Context ...to the minimization of real continuous functions over a compact set in ℜn . Actual adaptations of the simulated annealing algorithm for continuous variable problems have been proposed by many authors =-=[17, 127, 134]-=-. For example, Bilbro and Snyder [17] introduce what they term tree annealing, in which they add a tree structure to the the original Metropolis scheme to handle the14 continuous variables. Szu and H... |

21 | Generalized pattern searches with derivative information
- Abramson, Audet, et al.
- 2002
(Show Context)
Citation Context ...class of GPS algorithms discussed here, and presents results when applied to a problem in the design of a load bearing thermal insulation system. Chapter 8, which primarily contains the work found in =-=[2]-=-, presents a version of the basic GPS algorithm that uses any available derivative information to reduce the number of function evaluations for each GPS iteration. It includes numerical results on som... |

19 | Optimization using surrogate objectives on a helicopter test example
- Booker, Jr, et al.
- 1998
(Show Context)
Citation Context ...d regions, or take on infinite value at specific points. In fact, in some engineering simulations, function calls may even result in the function simply not returning a value (e.g., see Booker et al. =-=[20]-=-). This happens, for example, when each function evaluation requires the solution of a system of differential equations, and the method used to numerically solve the system fails to generate a solutio... |

18 |
An Analysis of the Effects of Selection in Genetic Algorithms
- Baker
- 1989
(Show Context)
Citation Context ...tion for individuals with better fitness values. The most common approaches select parents based on proportional values of the fitness functions [63], ordinal rankings of individuals by fitness value =-=[11]-=-, or a combination of the two [63, 110]. In genetic algorithms, since the individuals are usually represented by binary strings, the crossover (reproduction) operator involves sampling bits from the t... |

18 |
Pattern search algorithms for linearly constrained minimization
- Lewis, Torczon
- 2000
(Show Context)
Citation Context ...tisfied, and convergence of ∆k to zero is not guaranteed. A sufficient condition for Assumption A2 to hold is that Gi = I for each i = 1, 2, . . . , imax and that the coefficient matrix A is rational =-=[95]-=-.74 We should note also that the rationality of τ is essential for convergence. In a revised version of [4], Audet gives an example in which an irrational value for τ generates a sequence satisfying ... |

17 |
Speculation on the Evolution of Intelligence and its Possible Realization in Machine Form
- Atmar
- 1976
(Show Context)
Citation Context ...rogramming Evolutionary Programming was first introduced as an artificial intelligence technique applied to finite state machines by Fogel [51], and it was later extended by Burgin [26, 27] and Atmar =-=[3]-=-. Though developed under entirely different circumstances, it is actually quite similar to evolution strategies. For example, each individual is represented by a pair of vectors (x, σ) in precisely th... |

16 |
A coarse-grained parallel variable-complexity multidisciplinary optimization paradigm,
- Burgee, Giunta, et al.
- 1996
(Show Context)
Citation Context ...he original problem. The surrogate can then be recalibrated using the new points that have been evaluated. The literature contains several reasonable approaches for handling the recalibration process =-=[25, 58, 132]-=-. The building of surrogates often involves the use of surfaces, which are functions designed to fit or smooth data chosen at the data sites. Examples include kriging, responses surfaces, polynomial i... |

14 |
Nonsmooth Optimization
- Neittaanmäki
- 1992
(Show Context)
Citation Context ... f is differentiable at x, then ∇f(x) ∈ ∂f(x). • When f is continuously differentiable at x, ∂f(x) reduces to the singleton {∇f(x)}, and when f is convex near x, it coincides with the subdifferential =-=[73, 103]-=-; i.e., ∂f(x) = {η ∈ ℜ n : f(y) ≥ f(x) + η T (y − x) for all y ∈ ℜ n }. • If f is strictly differentiable at x, then f is Lipschitz near x and differentiable at x with ∂f(x) = {∇f(x)}. • If f is Lipsc... |

13 | A direct search conjugate directions algorithm for unconstrained minimization
- Coope, Price
(Show Context)
Citation Context ...onstruction allows them to extend the algorithm by constructing the grid with conjugate directions, thereby achieving the classical result of finite termination on strictly convex quadratic functions =-=[33]-=-. Second, Bünner, Schittkowski, and van de Braak [24] solve problems in the design of surface acoustic wave filters by applying an SQP method, that has been extended to handle non-relaxable integer va... |

13 |
Solving mixed integer programs by outer approxi- mation
- Fletcher, Leyffer
- 1994
(Show Context)
Citation Context ... method. 2.1.1 Outer Approximation Outer Approximation was first introduced for a class of MINLP problems by Duran and Grossman [43] and then extended to more general problems by Fletcher and Leyffer =-=[46]-=-. At each iteration, an upper bound is obtained by solving a restricted10 NLP, in which discrete variables are held constant. Then, if the problem is convex (i.e., the objective is convex and the con... |

12 | A generalized stationary point convergence theory for evolutionary algorithms
- Hart
- 1997
(Show Context)
Citation Context ... handle non-relaxable integer variables by adding a direct search component. However, the algorithm treats the direct search method as a heuristic, and has no proven convergence theory. Finally, Hart =-=[68, 69, 70]-=- introduces an evolutionary pattern search algorithm that combines the ideas of evolutionary algorithms with the Torczon [130] GPS algorithm in a way that allows him to prove probabilistic analogs of ... |

12 | A convergence analysis of unconstrained and bound constrained evolutionary pattern search
- Hart
(Show Context)
Citation Context ... handle non-relaxable integer variables by adding a direct search component. However, the algorithm treats the direct search method as a heuristic, and has no proven convergence theory. Finally, Hart =-=[68, 69, 70]-=- introduces an evolutionary pattern search algorithm that combines the ideas of evolutionary algorithms with the Torczon [130] GPS algorithm in a way that allows him to prove probabilistic analogs of ... |

11 | D.B.: Managing surrogate objectives to optimize helicopter rotor design–further experiments
- Booker, Frank, et al.
- 1998
(Show Context)
Citation Context ...ite heuristic, such as those described in Section 2.2, or perhaps optimize an inexpensive surrogate function, as is common in difficult engineering design problems with expensive function evaluations =-=[5, 19, 20, 21]-=-. In the poll step, a positive spanning set Dk ⊆ D is chosen from which to construct the poll set. Again, we represent Dk also as a matrix whose columns are the members of the set. It is a function of... |

10 |
On playing two-person zero-sum games against npnminimax players
- Burgin
- 1968
(Show Context)
Citation Context ....21 Evolutionary Programming Evolutionary Programming was first introduced as an artificial intelligence technique applied to finite state machines by Fogel [51], and it was later extended by Burgin =-=[26, 27]-=- and Atmar [3]. Though developed under entirely different circumstances, it is actually quite similar to evolution strategies. For example, each individual is represented by a pair of vectors (x, σ) i... |

9 |
Aircraft Multidisciplinary Optimization Using Design of Exper- iments Theory and Response Surface Modeling Methods
- Giunta
- 1997
(Show Context)
Citation Context ...he original problem. The surrogate can then be recalibrated using the new points that have been evaluated. The literature contains several reasonable approaches for handling the recalibration process =-=[25, 58, 132]-=-. The building of surrogates often involves the use of surfaces, which are functions designed to fit or smooth data chosen at the data sites. Examples include kriging, responses surfaces, polynomial i... |

9 |
Rank ordering and positive basis in pattern search algorithms
- Lewis, Torczon
- 1996
(Show Context)
Citation Context ...nction is evaluated on a finite set of neighboring points on a carefully constructed discrete mesh, formed by considering nonnegative integer combinations of vectors that form a positive spanning set =-=[93]-=- (see Definition 3.2). If an improvement is found, then the new iterate is accepted and the mesh is retained or coarsened; otherwise, the mesh is refined and a new set of neighboring mesh points is ev... |

7 |
Systems identification by quasilinear and evolutionary program- ming
- Bürgin
- 1973
(Show Context)
Citation Context ....21 Evolutionary Programming Evolutionary Programming was first introduced as an artificial intelligence technique applied to finite state machines by Fogel [51], and it was later extended by Burgin =-=[26, 27]-=- and Atmar [3]. Though developed under entirely different circumstances, it is actually quite similar to evolution strategies. For example, each individual is represented by a pair of vectors (x, σ) i... |

6 |
A general variational principle for thermal insulation system de- sign
- Bejan
- 1979
(Show Context)
Citation Context ...ectional area for the mechanical supports. In considering systems with more general types of insulation, Chato and Khodadidi [30] sought to minimize entropy, similar to the formulation given by Bejan =-=[15]-=-. Other related attempts to optimize the design of114 these systems are found in [99], [112], and [137]. An actual implementation of these types of systems for the Large Hadron Collider (LHC) project... |

6 |
Floudas. Nonlinear and Mixed Integer Optimization
- A
- 1995
(Show Context)
Citation Context ... step, the lower and upper bounds approach each other, yielding an approximate solution. Variations and improvements of the method have been proposed by Kocis and Grossman [88], Leyffer [97], Floudas =-=[50]-=-, and Viswanathan and Grossman [135]. 2.1.2 Generalized Benders Decomposition Similar to outer approximation, Generalized Benders Decomposition, developed by Geoffrion [56], solves the same NLP subpro... |

6 |
Optimization of mechanical supports for large superconductive magnets
- Hilal, Boom
- 1977
(Show Context)
Citation Context ...f this type of system used in cryogenic engineering applications, such as superconducting magnetic energy storage systems and space borne magnets, have been studied by several authors. Hilal and Boom =-=[71]-=- used a gradient-based optimizer to minimize power for n = 1, 2, and 3 intercepts with two choices of insulators of constant cross-sectional area, but without mixing insulator types within the system.... |

5 |
Nonlinear optimization with mixed variables and derivatives- Matlab® (NOMADm). Software. Available from Mark Abramson's NOMADm
- Abramson
- 2002
(Show Context)
Citation Context ...ence wi → w such that xi + tiwi ∈ Y for all i. The tangent cone TY (x) is the collection of all tangent vectors to Y at x. If Y is a convex set (i.e., if v, w ∈ Y , then αv + (1 − α)w ∈ Y for all α ∈ =-=[0, 1]-=-), then a straightforward application of Definition 1.1 yields TY (x) = cl{t(w − x) : t ≥ 0, w ∈ Y }. (1.3) This latter definition will be used often, since Xc is a convex set. The following definitio... |

4 |
Positive bases in numerical optimization
- Coope, Price
- 2000
(Show Context)
Citation Context ...ctation that, for a sufficiently fine mesh, this new direction will provide a crude estimate for the direction of steepest descent. Similar ideas, but in a slightly different context, can be found in =-=[34]-=-. Lewis and Torczon have extended the GPS method to solve both bound [94] and linearly constrained problems [95]. In doing so, they showed that by choosing the search directions appropriately, and if ... |

4 |
Minimization of refrigeration power for large cryogenic systems
- Hilal, Eyssa
- 1980
(Show Context)
Citation Context ...nt things. The objective function is to minimize power, as measured in Figure 7.2, but the required power shown in Table 7.3 is normalized (hence the ( comparisons with the results of Hilal and Eyssa =-=[72]-=-. P L) notation), so as to allow A Figure 7.3 depicts the progression of the filter during the run of the full model, where the plots in the right column are magnifications of those on the left. Each ... |

3 |
Optimization of cooled shields in insulations
- Chato, Khodadidi
- 1984
(Show Context)
Citation Context ...m. Hilal and Eyssa [72] studied the same problem, but with variable cross sectional area for the mechanical supports. In considering systems with more general types of insulation, Chato and Khodadidi =-=[30]-=- sought to minimize entropy, similar to the formulation given by Bejan [15]. Other related attempts to optimize the design of114 these systems are found in [99], [112], and [137]. An actual implement... |

3 |
meeting on surrogate optimization
- Danish
- 2000
(Show Context)
Citation Context ...rogate functions, in which case, the surrogate would be constructed as the sum or product, respectively, of the surface and surrogate functions. One very interesting and new approach is space mapping =-=[40]-=-, in which the relationship between an expensive fine model and an inexpensive coarse model is explored. The fine and coarse models are analogous to the original and surrogate models, respectively.28... |

3 |
On the global convergence of an SQP- filter algorithm
- Fletcher, Leyffer, et al.
- 2002
(Show Context)
Citation Context ...letcher, Leyffer, and Toint [48] show convergence of the SLP-based approach to a limit point satisfying Fritz John [82] optimality conditions; they show convergence of the SQP approach to a KKT point =-=[49]-=-, provided a constraint qualification is satisfied. However, in both cases, more than a simple decrease in the function values is required for convergence with these properties. Audet and Dennis show ... |

3 |
Characterization of net type thermal insulators at 1.8 k low boundary temperature
- Jenninger, Peon, et al.
- 1996
(Show Context)
Citation Context ...design of114 these systems are found in [99], [112], and [137]. An actual implementation of these types of systems for the Large Hadron Collider (LHC) project is discussed in three technical reports =-=[45, 81, 105]-=-. While all of these studies vary in geometry and fidelity of the underlying models, none of them optimizes with respect to the categorical variables; namely, the number of intercepts and types of ins... |

3 |
Genetic algorithms: Deception, convergence and starting conditions
- Kingdon
- 1992
(Show Context)
Citation Context ...s for genetic algorithms can be found in [110]. A Markov chain analysis of genetic algorithms with finite populations is given in [64], but only considers reproduction and mutation operators. Kingdon =-=[86]-=- studied and attempted to characterize problems that genetic algorithms have difficulty solving, and provides some convergence results. Rudolph [120] proved that the only genetic algorithms that conve... |

3 | Generalized outer approximation
- Leyffer
- 1997
(Show Context)
Citation Context ...ach successive step, the lower and upper bounds approach each other, yielding an approximate solution. Variations and improvements of the method have been proposed by Kocis and Grossman [88], Leyffer =-=[97]-=-, Floudas [50], and Viswanathan and Grossman [135]. 2.1.2 Generalized Benders Decomposition Similar to outer approximation, Generalized Benders Decomposition, developed by Geoffrion [56], solves the s... |

3 |
Minimization of total refrigeration power of liquid neon and nitrogen cooled intercepts for SMES magnets
- Li, Li, et al.
- 1989
(Show Context)
Citation Context ...pes of insulation, Chato and Khodadidi [30] sought to minimize entropy, similar to the formulation given by Bejan [15]. Other related attempts to optimize the design of114 these systems are found in =-=[99]-=-, [112], and [137]. An actual implementation of these types of systems for the Large Hadron Collider (LHC) project is discussed in three technical reports [45, 81, 105]. While all of these studies var... |

3 |
Convergence properties of simulated annealing algorithms for continuous global optimization
- Locatelli
- 1996
(Show Context)
Citation Context ...ins an active area of research, convergence proofs for other classes of annealing algorithms, having various assumptions on cooling schedules and probability distributions can be found in [16], [54], =-=[100]-=- and [101], among many others. An excellent overview of simulated annealing and description of its convergence properties is found in [131]. Although convergence in probability to the global optimum i... |

3 |
293 k - 1.9 k supporting systems for the large hadron collider (lhc) cryo-magnets
- Mathieu, Parma, et al.
- 1998
(Show Context)
Citation Context ...design of114 these systems are found in [99], [112], and [137]. An actual implementation of these types of systems for the Large Hadron Collider (LHC) project is discussed in three technical reports =-=[45, 81, 105]-=-. While all of these studies vary in geometry and fidelity of the underlying models, none of them optimizes with respect to the categorical variables; namely, the number of intercepts and types of ins... |

2 |
de Braak. Optimal design of surface acoustic wave filters for signal processing by mixed-integer nonlinear program- ming
- Bünner, Schittkowski, et al.
- 2002
(Show Context)
Citation Context ...nstructing the grid with conjugate directions, thereby achieving the classical result of finite termination on strictly convex quadratic functions [33]. Second, Bünner, Schittkowski, and van de Braak =-=[24]-=- solve problems in the design of surface acoustic wave filters by applying an SQP method, that has been extended to handle non-relaxable integer variables by adding a direct search component. However,... |

2 |
An extension of Curry's thereom to steepest descent in normed linear spaces
- Byrd, Tapia
- 1975
(Show Context)
Citation Context ...licitly constructed. The notation established earlier becomes clear. For v = −∇f(xk), the vectors d (1) and d (∞) are the negatives of the normalized ℓ1 and ℓ∞ gradients of f at xk, respectively (see =-=[28]-=-), while d (2) is the vector in D that makes the smallest angle with −∇f(xk).141 Theorem 8.9 If ∇f(xk) is available at iteration k of the GPS algorithm with positive spanning set D = D and if Dk = D(... |

2 |
Com- parison of floating and thermalized multilayer insulation systems at low bound- ary temperature
- Ferlin, Jenninger, et al.
- 1996
(Show Context)
Citation Context ...design of114 these systems are found in [99], [112], and [137]. An actual implementation of these types of systems for the Large Hadron Collider (LHC) project is discussed in three technical reports =-=[45, 81, 105]-=-. While all of these studies vary in geometry and fidelity of the underlying models, none of them optimizes with respect to the categorical variables; namely, the number of intercepts and types of ins... |