We apply Stochastic Meta-Descent (SMD), a stochastic gradient optimization method with gain vector adaptation, to the training of Conditional Random Fields (CRFs). On several large data sets, the resulting optimizer converges to the same quality of solution over an order of magnitude faster than limited-memory BFGS, the leading method reported to date. We report results for both exact and inexact inference techniques. 1.

We describe a novel method for controlling physics-based fluid simulations through gradient-based nonlinear optimization. Using a technique known as the adjoint method, derivatives can be computed efficiently, even for large 3D simulations with millions of control parameters. In addition, we introduce the first method for the full control of free-surface liquids. We show how to compute adjoint derivatives through each step of the simulation, including the fast marching algorithm, and describe a new set of control parameters specifically designed for liquids.

Graph coloring has been employed since the 1980s to efficiently compute sparse Jacobian and Hessian matrices using either finite differences or automatic differentiation. Several coloring problems occur in this context, depending on whether the matrix is a Jacobian or a Hessian, and on the specifics of the computational techniques employed. We consider eight variant vertexcoloring problems here. This article begins with a gentle introduction to the problem of computing a sparse Jacobian, followed by an overview of the historical development of the research area. Then we present a unifying framework for the graph models of the variant matrixestimation problems. The framework is based upon the viewpoint that a partition of a matrixinto structurally orthogonal groups of columns corresponds to distance-2 coloring an appropriate graph representation. The unified framework helps integrate earlier work and leads to fresh insights; enables the design of more efficient algorithms for many problems; leads to new algorithms for others; and eases the task of building graph models for new problems. We report computational results on two of the coloring problems to support our claims. Most of the methods for these problems treat a column or a row of a matrixas an atomic entity, and partition the columns or rows (or both). A brief review of methods that do not fit these criteria is provided. We also discuss results in discrete mathematics and theoretical computer science that intersect with the topics considered here.

The proper orthogonal decomposition (POD) is shown an efficiently model reduction technique in simulating physical processes governed by partial differential equations. In this paper we make an initial effort to investigate problems related to POD reduced modeling of a large-scale upper ocean circulation in the tropic Pacific domain. We constructed different POD models with different choices of snapshots and different number of POD basis functions. The results from these different POD models are compared with that of the original model. The main findings are: (1) the large-scale seasonal variability of the tropic Pacific obtained by the original model can be captured well by a low dimensional system of order of 22, which is constructed by 20 snapshots and 7 leading POD basis functions. (2) RMS errors for the upper ocean layer thickness of the POD model of order of 22 is less than 1m that is less than 1from the POD model is around 0.99. (3) The modes that capture 0.99 energy are necessary to construct POD models in order to yield a high accuracy.

We present a modification of Newton’s method to restore quadratic convergence for isolated singular solutions of polynomial systems. Our method is symbolic-numeric: we produce a new polynomial system which has the original multiple solution as a regular root. We show that the number of deflation stages is bounded by the multiplicity of the isolated root. Our implementation performs well on a large class of applications. 2000 Mathematics Subject Classification. Primary 65H10. Secondary 14Q99, 68W30. Key words and phrases. Newton’s method, deflation, numerical homotopy algorithms, symbolic-numeric computations. 1

In many real-life situations, it is very difficult or even impossible to directly measure the quantity y in which we are interested: e.g., we cannot directly measure a distance to a distant galaxy or the amount of oil in a given well. Since we cannot measure such quantities directly, we can measure them indirectly: by first measuring some relating quantities x1 ; : : : ; xn , and then by using the known relation between x i and y to reconstruct the value of the desired quantity y. In practice, it is often very important to estimate the error of the resulting indirect measurement. In this paper, we describe and compare different deterministic and randomized algorithms for solving this problem in the situation when a program for transforming the estimates e x1 ; : : : ; e xn for x i into an estimate for y is only available as a black box (with no source code at hand). We consider this problem in two settings: statistical, when measurements errors \Deltax i = e x i \Gamma x i are inde...

Abstract not found

Uncertainty is very important in risk analysis. A natural way to describe this uncertainty is to describe a set of possible values of each unknown quantity (this set is usually an interval), plus any additional information that we may have about the probability of different values within this set. Traditional statistical techniques deal with the situations in which we have a complete information about the probabilities; in real life, however, we often have only partial information about them. We therefore need to describe methods of handling such partial information in risk analysis. Several such techniques have been presented, often on a heuristic basis. The main goal of this paper is to provide a justification for a general formalism for handling different types of uncertainty, and to describe a new black-box technique for processing this type of uncertainty.

This paper presents a number of algorithm developments for adjoint methods using the `discrete' approach in which the discretisation of the nonlinear equations is linearised and the resulting matrix is then transposed

The automatic generation of adjoints of mathematical models that are implemented as computer programs is receiving increased attention in the scientific and engineering communities. Reverse-mode automatic differentiation is of particular interest for large-scale optimization problems. It allows the computation of gradients at a small constant multiple of the cost for evaluating the objective function itself, independent of the number of input parameters. Source-to-source transformation tools apply simple differentiation rules to generate adjoint codes based on the adjoint version of every statement. In order to guarantee correctness, certain values that are computed and overwritten in the original program must be made available in the adjoint program. For their determination we introduce a static dataflow analysis called “to be recorded ” analysis. Possible overestimation of this set must be kept minimal to get efficient adjoint codes. This efficiency is essential for the applicability of source-to-source transformation tools to real-world applications. 1 Automatically Generated Adjoints We consider a computer program P evaluating a vector function y = F (x), where F: IR n → IR m. Usually, P implements the mathematical model of some underlying real-world application and it is referred to as the original code. The goal of automatic differentiation (AD) [3,7,15] by source transformation is to build automatically a new source program P ′ evaluating some derivatives of F. This is arrow AD in Figure 1. We consider a simplified mathematical model, symbolized by arrow R in Figure 1: Every particular run of P on a particular set of inputs is equivalent to a simple sequence of p scalar assignments vj = ϕj(vk)k≺j, j = 1,..., q, (1)