Results 1  10
of
99
NonUniform Random Variate Generation
, 1986
"... This is a survey of the main methods in nonuniform random variate generation, and highlights recent research on the subject. Classical paradigms such as inversion, rejection, guide tables, and transformations are reviewed. We provide information on the expected time complexity of various algorith ..."
Abstract

Cited by 1000 (26 self)
 Add to MetaCart
This is a survey of the main methods in nonuniform random variate generation, and highlights recent research on the subject. Classical paradigms such as inversion, rejection, guide tables, and transformations are reviewed. We provide information on the expected time complexity of various algorithms, before addressing modern topics such as indirectly specified distributions, random processes, and Markov chain methods.
When are QuasiMonte Carlo Algorithms Efficient for High Dimensional Integrals?
 J. Complexity
, 1997
"... Recently quasiMonte Carlo algorithms have been successfully used for multivariate integration of high dimension d, and were significantly more efficient than Monte Carlo algorithms. The existing theory of the worst case error bounds of quasiMonte Carlo algorithms does not explain this phenomenon. ..."
Abstract

Cited by 185 (23 self)
 Add to MetaCart
Recently quasiMonte Carlo algorithms have been successfully used for multivariate integration of high dimension d, and were significantly more efficient than Monte Carlo algorithms. The existing theory of the worst case error bounds of quasiMonte Carlo algorithms does not explain this phenomenon. This paper presents a partial answer to why quasiMonte Carlo algorithms can work well for arbitrarily large d. It is done by identifying classes of functions for which the effect of the dimension d is negligible. These are weighted classes in which the behavior in the successive dimensions is moderated by a sequence of weights. We prove that the minimal worst case error of quasiMonte Carlo algorithms does not depend on the dimension d iff the sum of the weights is finite. We also prove that under this assumption the minimal number of function values in the worst case setting needed to reduce the initial error by " is bounded by C " \Gammap , where the exponent p 2 [1; 2], and C depends ...
Random search for hyperparameter optimization
 In: Journal of Machine Learning Research
"... Grid search and manual search are the most widely used strategies for hyperparameter optimization. This paper shows empirically and theoretically that randomly chosen trials are more efficient for hyperparameter optimization than trials on a grid. Empirical evidence comes from a comparison with a ..."
Abstract

Cited by 111 (16 self)
 Add to MetaCart
(Show Context)
Grid search and manual search are the most widely used strategies for hyperparameter optimization. This paper shows empirically and theoretically that randomly chosen trials are more efficient for hyperparameter optimization than trials on a grid. Empirical evidence comes from a comparison with a large previous study that used grid search and manual search to configure neural networks and deep belief networks. Compared with neural networks configured by a pure grid search, we find that random search over the same domain is able to find models that are as good or better within a small fraction of the computation time. Granting random search the same computational budget, random search finds better models by effectively searching a larger, less promising configuration space. Compared with deep belief networks configured by a thoughtful combination of manual search and grid search, purely random search over the same 32dimensional configuration space found statistically equal performance on four of seven data sets, and superior performance on one of seven. A Gaussian process analysis of the function from hyperparameters to validation set performance reveals that for most data sets only a few of the hyperparameters really matter, but that different hyperparameters are important on different data sets. This phenomenon makes
Numerical Integration using Sparse Grids
 NUMER. ALGORITHMS
, 1998
"... We present and review algorithms for the numerical integration of multivariate functions defined over ddimensional cubes using several variants of the sparse grid method first introduced by Smolyak [51]. In this approach, multivariate quadrature formulas are constructed using combinations of tensor ..."
Abstract

Cited by 83 (16 self)
 Add to MetaCart
(Show Context)
We present and review algorithms for the numerical integration of multivariate functions defined over ddimensional cubes using several variants of the sparse grid method first introduced by Smolyak [51]. In this approach, multivariate quadrature formulas are constructed using combinations of tensor products of suited onedimensional formulas. The computing cost is almost independent of the dimension of the problem if the function under consideration has bounded mixed derivatives. We suggest the usage of extended Gauss (Patterson) quadrature formulas as the onedimensional basis of the construction and show their superiority in comparison to previously used sparse grid approaches based on the trapezoidal, ClenshawCurtis and Gauss rules in several numerical experiments and applications.
Latin Supercube Sampling for Very High Dimensional Simulations
, 1997
"... This paper introduces Latin supercube sampling (LSS) for very high dimensional simulations, such as arise in particle transport, finance and queuing. LSS is developed as a combination of two widely used methods: Latin hypercube sampling (LHS), and QuasiMonte Carlo (QMC). In LSS, the input variables ..."
Abstract

Cited by 80 (8 self)
 Add to MetaCart
This paper introduces Latin supercube sampling (LSS) for very high dimensional simulations, such as arise in particle transport, finance and queuing. LSS is developed as a combination of two widely used methods: Latin hypercube sampling (LHS), and QuasiMonte Carlo (QMC). In LSS, the input variables are grouped into subsets, and a lower dimensional QMC method is used within each subset. The QMC points are presented in random order within subsets. QMC methods have been observed to lose effectiveness in high dimensional problems. This paper shows that LSS can extend the benefits of QMC to much higher dimensions, when one can make a good grouping of input variables. Some suggestions for grouping variables are given for the motivating examples. Even a poor grouping can still be expected to do as well as LHS. The paper also extends LHS and LSS to infinite dimensional problems. The paper includes a survey of QMC methods, randomized versions of them (RQMC) and previous methods for extending Q...
DimensionAdaptive TensorProduct Quadrature
 Computing
, 2003
"... We consider the numerical integration of multivariate functions defined over the unit hypercube. Here, we especially address the highdimensional case, where in general the curse of dimension is encountered. Due to the concentration of measure phenomenon, such functions can often be well approxi ..."
Abstract

Cited by 73 (12 self)
 Add to MetaCart
(Show Context)
We consider the numerical integration of multivariate functions defined over the unit hypercube. Here, we especially address the highdimensional case, where in general the curse of dimension is encountered. Due to the concentration of measure phenomenon, such functions can often be well approximated by sums of lowerdimensional terms. The problem, however, is to find a good expansion given little knowledge of the integrand itself.
Building Robust Simulationbased Filters for Evolving Data Sets
, 1999
"... this paper we will focus on an alternative class of filters in which theoretical distributions on the state space are approximated by simulated random measures. The first goal in filter design is to produce a compact description of the posterior distribution of the state given all the observations a ..."
Abstract

Cited by 36 (0 self)
 Add to MetaCart
this paper we will focus on an alternative class of filters in which theoretical distributions on the state space are approximated by simulated random measures. The first goal in filter design is to produce a compact description of the posterior distribution of the state given all the observations available so far. A basic requirement is that this description should be readily updated as new data become available. A mechanism has therefore to be devised which enables the approximating random measure to evolve and adapt. 3 SIMULATION BASED FILTERS Simulation based filters have a long history in the engineering literature, dating back to the work of Handschin and Mayne (1969); Handschin (1970); Akashi and Kumamoto (1977). Doucet (1998) provides a comprehensive review of the material. Since the Kalman filter is essentially a Bayesian update formula, the theory of Bayesian time series analysis is directly relevant (West and Harrison, 1997). We take as our starting point the filter developed by Gordon (1993); Gordon et al. (1993). The essence of the method is contained in a paper by Rubin (1988) who proposed the Sampling Importance Resampling (SIR) algorithm for obtaining samples from a complex posterior distribution without recourse to MCMC. In the simple nondynamic case described by Rubin (1988), the method consists of sampling n observations from the prior distribution, attaching weights to the sampled points according to their likelihood, and then sampling with replacement from this weighted discrete distribution. As n ! 1, the resulting set of values then approximates a sample from the required posterior (Smith and Gelfand, 1992). In the dynamic version, proposed by Gordon et al. (1993), the SIR algorithm is applied repeatedly as new data are acquired. One can think of...
Extensible Lattice Sequences For QuasiMonte Carlo Quadrature
 SIAM Journal on Scientific Computing
, 1999
"... Integration lattices are one of the main types of low discrepancy sets used in quasiMonte Carlo methods. However, they have the disadvantage of being of fixed size. This article describes the construction of an infinite sequence of points, the first b m of which form a lattice for any nonnegative ..."
Abstract

Cited by 35 (10 self)
 Add to MetaCart
Integration lattices are one of the main types of low discrepancy sets used in quasiMonte Carlo methods. However, they have the disadvantage of being of fixed size. This article describes the construction of an infinite sequence of points, the first b m of which form a lattice for any nonnegative integer m. Thus, if the quadrature error using an initial lattice is too large, the lattice can be extended without discarding the original points. Generating vectors for extensible lattices are found by minimizing a loss function based on some measure of discrepancy or nonuniformity of the lattice. The spectral test used for finding pseudorandom number generators is one important example of such a discrepancy. The performance of the extensible lattices proposed here is compared to that of other methods for some practical quadrature problems.
The MultiElement Probabilistic Collocation Method: Error Analysis and Applications
 J Comp Physics
"... Stochastic spectral methods are numerical techniques for approximating solutions to partial differential equations with random parameters. In this work, we present and examine the multielement probabilistic collocation method (MEPCM), which is a generalized form of the probabilistic collocation me ..."
Abstract

Cited by 34 (3 self)
 Add to MetaCart
(Show Context)
Stochastic spectral methods are numerical techniques for approximating solutions to partial differential equations with random parameters. In this work, we present and examine the multielement probabilistic collocation method (MEPCM), which is a generalized form of the probabilistic collocation method. In the MEPCM, the parametric space is discretized and a collocation/cubature grid is prescribed on each element. Both full and sparse tensor product grids based on Gauss and ClenshawCurtis quadrature rules are considered. We prove analytically and observe in numerical tests that as the parameter space mesh is refined, the convergence rate of the solution depends on the quadrature rule of each element only through its degree of exactness. In addition, the L2 error of the tensor product interpolant is examined and an adaptivity algorithm is provided. Numerical examples demonstrating adaptive MEPCM are shown, including lowregularity problems and longtime integration. We test the MEPCM on twodimensional Navier Stokes examples and a stochastic diffusion problem with various random input distributions and up to 50 dimensions. While the convergence rate of MEPCM deteriorates in 50 dimensions, the error in the mean and variance is two orders of magnitude lower than the error obtained with the Monte Carlo method using only a small number of samples (e.g., 100). The computational cost of MEPCM is found to be favorable when compared to the cost of other methods including stochastic Galerkin, Monte Carlo and quasirandom sequence methods. 1