Citation Context

...aier, 2008). There has been also an interest in designing sample-efficient strategies, only requiring local smoothness around (one) of the global maxima (Kleinberg et al., 2008; Bubeck et al., 2011a; =-=Munos, 2011-=-). However, these approaches still assume the knowledge of this smoothness, i.e., the metric under which the function is smooth, which may not be available to the optimizer. Recently, Munos (2011) pro...

Citation Context

...[Srinivas et al., 2010, Hoffman et al., 2011]. In the realm of optimizing deterministic functions, a few works have proven exponential rates of convergence for simple regret [de Freitas et al., 2012, =-=Munos, 2011-=-]. A stochastic variant of the work of Munos has been recently proposed by Valko et al. [2013]; this approach takes a tree-based structure for expanding areas of the optimization problem in question, ...

Citation Context

...s the necessity of re-starting the auxiliary optimization at each iteration. Recent optimistic optimizationmethods provide a viable alternative to BO (Kocsis & Szepesvári, 2006; Bubeck et al., 2011; =-=Munos, 2011-=-). Instead of estimating a posterior distribution over the unknown objective function, these methods build space partitioning trees by expanding leaves with high function values or upper-bounds. The t...

Citation Context

...lated in practice, as shown in Section 6.1. One potential remedy for dealing with this difficulty is to use correlation information between rankers and make use of continuous armed bandits algorithms =-=[3, 10, 25, 30, 33]-=-, although both the algorithm and the theoretical analysis would be considerably more intricate in this setting, since the goal is not simply to find the maximum of a continuous function. As explained...

Citation Context

...2] and stochastic systems [25], [39], [7], [3], [10], [9], [40] when the system dynamics / transition model is known, and also (iii) optimization of unknown functions only accessible through sampling =-=[27]-=-. The optimistic principle has also been used for addressing the E/E dilemma for MDPs when the transition model is unknown and progressively learned through interactions with the environment. For inst...

Citation Context

...on for introducing a new optimization problem lies in the assumption that the cost of evaluating the objective function f dominates that of solving additional optimization problem. For deterministic function, de Freitas et al. [1] recently presented a theoretical procedure that maintains exponential convergence rate. However, their own paper and the follow-up research [1, 2] point out that this result relies on an impractical sampling procedure, the δ-cover sampling. To overcome this issue, Wang et al. [2] combined GP-UCB with a hierarchical partitioning optimization method, the SOO algorithm [18], providing a regret bound with polynomial dependence on the number of function evaluations. They concluded that creating a GP-based algorithm with an exponential convergence rate without the impractical sampling procedure remained an open problem. 3 Infinite-Metric GP Optimization 3.1 Overview The GP-UCB algorithm can be seen as a member of the class of bound-based search methods, which includes Lipschitz optimization, A* search, and PAC-MDP algorithms with optimism in the 2 face of uncertainty. Bound-based search methods have a common property: the tightness of the bound determines its effec...

Citation Context

...x −1.0 −0.8 −0.6 −0.4 −0.2 0.0 f (x ) 0.0 0.2 0.4 0.6 0.8 1.0 ρ 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 si m pl e re gr et af te r 50 00 ev al ua tio ns Figure 1: Difficult function f : x → s (log2 |x− 0.5|) · ( √ |x− 0.5 |− (x− 0.5)2) − √ |x− 0.5| where, s(x) = 1 if the fractional part of x, that is, x − bxc, is in [0, 0.5] and s(x) = 0, if it is in (0.5, 1). Left: Oscillation between two envelopes of different smoothness leading to a nonzero d for a standard partitioning. Right: Regret of HOO after 5000 evaluations for different values of ρ. Another direction has been followed by Munos [11], where in the deterministic case (the function evaluations are not perturbed by noise), their SOO algorithm performs almost as well as the best known algorithms without the knowledge of the function smoothness. SOO was later extended to StoSOO [15] for the stochastic case. However StoSOO only extends SOO for a limited case of easy instances of functions for which there exists a semi-metric under which d = 0. Also, Bull [6] provided a similar regret bound for the ATB algorithm for a class of functions, called zooming continuous functions, which is related to the class of functions for which th...

Citation Context

...e pour des systèmes interconnectés”. The main ingredient lending this approach its generality is the global optimization algorithm used at the control stage, called sequential optimistic optimization =-=[13]-=-. This algorithm is selected since it guarantees closeness to the reference states for very general dynamics, and crucially, it only uses a black-box simulation model of the dynamics rather than their...