#### DMCA

## An adaptive radial basis algorithm (ARBF) for expensive black-box global optimization’. (2007)

Venue: | Journal of Global Optimization, DOI 10.1007/s10898-007-9256-8, ISSN 0925-5001 (Print) |

Citations: | 9 - 0 self |

### Citations

906 |
A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output From a Computer Code
- McKay, WJ, et al.
- 1979
(Show Context)
Citation Context ...g Feasible is used to track if the algorithm has found any feasible point or not. Note that while hL1(x) is scale dependent it does not influence the behavior of the algorithm RBF for MINLP given below, nor the ARBF algorithm in the next section. This is due to the fact that hL1(x) is not used in building the interpolation surface, and it is not used in the subproblem solutions. To compute the first RBF interpolation surface, at least n ≥ d + 1 disjunct sample points are needed. This set is normally found by using a statistical experimental design algorithm, e.g. Latin Hypercube (McKay et al. [11]) or by evaluating some or all the corners of the box defined by Ω. For a constrained or mixed-integer nonlinear problem, the RBF interpolation needs to be a good approximation of f(x) in ΩC . Assuming that the constraints c(x) are much less time-consuming to compute than the costly f(x), one can try to find a large number of sample points using Latin Hypercube design, then compute hL1(x) for each of the points and select the first n feasible points found as the initial experimental design. Define this strategy as a Constrained Latin Hypercube (CLH) design. In case enough feasible points can n... |

481 |
Efficient global optimization of expensive black-box functions
- Jones, Schonlau, et al.
- 1998
(Show Context)
Citation Context ...63-5451, USA. 1 K. Holmstrom, N-H. Quttineh & M.M. Edvall 1 Introduction Global optimization of continuous black-box functions that are costly (computationally expensive, CPU-intensive) to evaluate is a challenging problem. Several approaches based on response surface techniques, most of which utilize every computed function value, have been developed over the years. In his excellent paper [9], Jones reviews the most important developments. Many methods have been developed based on statistical approaches, called kriging, see e.g. the Efficient Global Optimization (EGO) method in Jones et al. [10]. In this paper we mainly consider methods based on radial basis function interpolation, RBF methods, first discussed in [4] and [13]. Problems that are costly to evaluate are commonly found in engineering design, industrial and financial applications. A function value could be the result of a complex computer program, an advanced simulation, e.g. computational fluid dynamics (CFD), or design optimization. One function value might require the solution of a large system of partial differential equations, and hence consume anything from a few minutes to many hours. In the application areas discu... |

235 | A taxonomy of global optimization methods based on response surfaces
- Jones
- 2001
(Show Context)
Citation Context ...tion MINLP Mixed-Integer Nonlinear Programming ∗Department of Applied mathematics, Malardalen University, SE-721 23 Vasteras, Sweden. †Tomlab Optimization Inc., 1260 SE Bishop Blvd Ste E, Pullman, WA 99163-5451, USA. 1 K. Holmstrom, N-H. Quttineh & M.M. Edvall 1 Introduction Global optimization of continuous black-box functions that are costly (computationally expensive, CPU-intensive) to evaluate is a challenging problem. Several approaches based on response surface techniques, most of which utilize every computed function value, have been developed over the years. In his excellent paper [9], Jones reviews the most important developments. Many methods have been developed based on statistical approaches, called kriging, see e.g. the Efficient Global Optimization (EGO) method in Jones et al. [10]. In this paper we mainly consider methods based on radial basis function interpolation, RBF methods, first discussed in [4] and [13]. Problems that are costly to evaluate are commonly found in engineering design, industrial and financial applications. A function value could be the result of a complex computer program, an advanced simulation, e.g. computational fluid dynamics (CFD), or desi... |

214 |
The theory of radial basis function approximation in
- Powell
- 1990
(Show Context)
Citation Context ...φ(r) = r 3 or thin plate spline with φ(r) = r2 log r, the radial basis function interpolant sn has the form sn(x) = n∑ i=1 λiφ ( ‖x− xi‖2 ) + bTx+ a, (4) with λ1, . . . , λn ∈ R, b ∈ Rd, a ∈ R. The unknown parameters λi, b, a are obtained as the solution of the linear equations( Φ P PT 0 )( λ c ) = ( F 0 ) , (5) where Φ is the n× n matrix with Φij = φ ( ‖xi − xj‖2 ) and P = xT1 1 ... ... xTn 1 , λ = λ1 ... λn , c = b1 ... bd a , F = f(x1) ... f(xn) . (6) If rank(P ) = d + 1, the matrix ( Φ P PT 0 ) is nonsingular and system (5) has a unique solution [12]. Thus a unique radial basis function interpolant to f at the points x1, . . . , xn is obtained. After this, one has to consider the question of choosing the next point xn+1 to evaluate the objective function for. The idea of the RBF algorithm is to use radial basis function interpolation and a measure of “bumpiness” of a radial function, σ. A target value f∗n is chosen as an estimate of the global minimum of f . For each y /∈ {x1, . . . , xn} there exists a radial basis function sy(x) that satisfies the interpolation conditions sy(xi) = f(xi), i = 1, . . . , n, sy(y) = f ∗ n. (7) The next poi... |

76 | A radial basis function method for global optimization
- Gutmann
- 2001
(Show Context)
Citation Context ...nction for. The idea of the RBF algorithm is to use radial basis function interpolation and a measure of “bumpiness” of a radial function, σ. A target value f∗n is chosen as an estimate of the global minimum of f . For each y /∈ {x1, . . . , xn} there exists a radial basis function sy(x) that satisfies the interpolation conditions sy(xi) = f(xi), i = 1, . . . , n, sy(y) = f ∗ n. (7) The next point xn+1 is then calculated as the value of y in the feasible region that minimizes σ(sy). As a surrogate model is used, the function y 7→ σ(sy) is much cheaper to compute than the original function. In [3], a “bumpiness” measure σ(sn) is defined and it is shown that minimizing σ(sy) subject to the interpolation conditions (7) is equivalent to minimizing a utility function gn(y) defined as gn(y) = (−1)mφ+1µn(y) [sn(y)− f∗n] 2 , y ∈ Ω \ {x1, . . . , xn} . (8) The method of Gutmann and the bumpiness measure is further discussed in the more recent papers [14] and [15] by Regis and Shoemaker. Writing the radial basis function solution to the target value interpolation problem (7) as sy(x) = sn(x) + [f ∗ n − sn(y)] ln(y, x), x ∈ Rd, (9) 4 An adaptive radial basis algorithm (ARBF) for constrained CGO ... |

28 | Global optimization of costly nonconvex functions using radial basis functions
- Björkman, Holmström
- 2000
(Show Context)
Citation Context ...x) is continuous with respect to all variables, even though we demand that some variables only take integer values. Otherwise it would not make sense to do surrogate modeling of f(x). Another assumption is that the nonlinear constraints are 2 An adaptive radial basis algorithm (ARBF) for constrained CGO cheap to compute compared to the costly f(x). All costly constraints can be treated by adding penalty terms to the objective function in the following way: min x p(x) = f(x) + ∑ j wj max ( 0, cj(x)− cjU , c j L − c j(x) ) , (2) where weighting parameters wj have been added. As we have shown in [2] this strategy works in practice for an industrial train set design problem. The idea of the RBF algorithm by Powell and Gutmann [4] is to use radial basis function interpolation to build an approximating surrogate model and define three utility functions. The next point, where the original objective function should be evaluated, is determined by optimizing one or more of these utility functions. Roughly speaking, the utility functions measure the likelihood that the solution to the problem occurs at a given point with the objective function equal to a certain “target value”. Maximizing the ut... |

24 |
Constrained global optimization of expensive black box functions using radial basis functions
- Regis, Shoemaker
(Show Context)
Citation Context ... . . . , n, sy(y) = f ∗ n. (7) The next point xn+1 is then calculated as the value of y in the feasible region that minimizes σ(sy). As a surrogate model is used, the function y 7→ σ(sy) is much cheaper to compute than the original function. In [3], a “bumpiness” measure σ(sn) is defined and it is shown that minimizing σ(sy) subject to the interpolation conditions (7) is equivalent to minimizing a utility function gn(y) defined as gn(y) = (−1)mφ+1µn(y) [sn(y)− f∗n] 2 , y ∈ Ω \ {x1, . . . , xn} . (8) The method of Gutmann and the bumpiness measure is further discussed in the more recent papers [14] and [15] by Regis and Shoemaker. Writing the radial basis function solution to the target value interpolation problem (7) as sy(x) = sn(x) + [f ∗ n − sn(y)] ln(y, x), x ∈ Rd, (9) 4 An adaptive radial basis algorithm (ARBF) for constrained CGO Table 1: Different choices of Radial Basis Functions. RBF φ(r) > 0 p(x) mφ = degree(p(x)) cubic r3 bT · x+ a 1 thin plate spline r2 log r bT · x+ a 1 linear r a 0 multiquadric (r2 + γ2) 1 2 , γ > 0 a 0 inverse multiquadric 1/(r2 + γ2) 1 2 , γ > 0 a 0 Gaussian exp(−γr2), γ > 0 {0} -1 µn(y) is the coefficient corresponding to y of the radial basis interpol... |

18 |
Improved strategies for radial basis function methods for global optimization
- Regis, Shoemaker
(Show Context)
Citation Context ...n, sy(y) = f ∗ n. (7) The next point xn+1 is then calculated as the value of y in the feasible region that minimizes σ(sy). As a surrogate model is used, the function y 7→ σ(sy) is much cheaper to compute than the original function. In [3], a “bumpiness” measure σ(sn) is defined and it is shown that minimizing σ(sy) subject to the interpolation conditions (7) is equivalent to minimizing a utility function gn(y) defined as gn(y) = (−1)mφ+1µn(y) [sn(y)− f∗n] 2 , y ∈ Ω \ {x1, . . . , xn} . (8) The method of Gutmann and the bumpiness measure is further discussed in the more recent papers [14] and [15] by Regis and Shoemaker. Writing the radial basis function solution to the target value interpolation problem (7) as sy(x) = sn(x) + [f ∗ n − sn(y)] ln(y, x), x ∈ Rd, (9) 4 An adaptive radial basis algorithm (ARBF) for constrained CGO Table 1: Different choices of Radial Basis Functions. RBF φ(r) > 0 p(x) mφ = degree(p(x)) cubic r3 bT · x+ a 1 thin plate spline r2 log r bT · x+ a 1 linear r a 0 multiquadric (r2 + γ2) 1 2 , γ > 0 a 0 inverse multiquadric 1/(r2 + γ2) 1 2 , γ > 0 a 0 Gaussian exp(−γr2), γ > 0 {0} -1 µn(y) is the coefficient corresponding to y of the radial basis interpolation fun... |

17 | Review of the Space Mapping Approach to Engineering Optimization and Modeling’.
- Bakr, Bandler, et al.
- 2000
(Show Context)
Citation Context ...luid dynamics (CFD), or design optimization. One function value might require the solution of a large system of partial differential equations, and hence consume anything from a few minutes to many hours. In the application areas discussed, derivatives are most often hard to obtain and the algorithms make no use of such information. The practical functions involved are often noisy and nonsmooth; however, the commonly used approximation methods assume smoothness. Another area illustrating the challenges of optimization with expensive function evaluations is space mapping optimization, see e.g. [1]. Instead of one costly function value, in space mapping a vector valued function is the result of each costly evaluation. Companion ”coarse” (ideal or low-fidelity) and ”fine” (practical or high-fidelity) models of different complexities are intelligently linked together to solve engineering model enhancement and design optimization problems. Our goal is to develop global optimization algorithms that work in practice and produce reasonably good solutions with a very limited number of function evaluations. From an application perspective there are often restrictions on the variables besides th... |

17 | Recent Research at Cambridge on Radial Basis Functions’.
- Powell
- 1999
(Show Context)
Citation Context ...are costly (computationally expensive, CPU-intensive) to evaluate is a challenging problem. Several approaches based on response surface techniques, most of which utilize every computed function value, have been developed over the years. In his excellent paper [9], Jones reviews the most important developments. Many methods have been developed based on statistical approaches, called kriging, see e.g. the Efficient Global Optimization (EGO) method in Jones et al. [10]. In this paper we mainly consider methods based on radial basis function interpolation, RBF methods, first discussed in [4] and [13]. Problems that are costly to evaluate are commonly found in engineering design, industrial and financial applications. A function value could be the result of a complex computer program, an advanced simulation, e.g. computational fluid dynamics (CFD), or design optimization. One function value might require the solution of a large system of partial differential equations, and hence consume anything from a few minutes to many hours. In the application areas discussed, derivatives are most often hard to obtain and the algorithms make no use of such information. The practical functions involved ... |