A hybrid 3-dimensional variational (3D-Var) / ensemble Kalman filter analysis scheme is demonstrated using a quasigeostrophic model under perfect-model assumptions. Four networks with differing observational densities are tested, including one network with a data void. The hybrid scheme operates by computing a set of parallel data assimilation cycles, with each member of the set receiving unique perturbed observations. The perturbed observations are generated by adding random noise consistent with observation error statistics to the control set of observations. Background error statistics for the data assimilation are estimated from a linear combination of time-invariant 3D-Var covariances and flow-dependent covariances developed from the ensemble of short-range forecasts. The hybrid scheme allows the user to weight the relative contributions of the 3D-Var and ensemble-based background covariances. The analysis scheme was cycled for 90 days, with new observations assimilated every 12 h...

this paper is to examine how different sampling strategies and implementations of the analysis scheme influence the quality of the results in the EnKF. It is shown that by selecting the initial ensemble, the model noise and the measurement perturbations wisely, it is possible to achieve a significant improvement in the EnKF results, using the same number of members in the ensemble

This work studies the effect of using Monte Carlo based methods to estimate high-dimensional systems. Recent focus in the geosciences has been on representing the atmospheric state using a probability density function, and, for extremely high-dimensional systems, various sample based Kalman filter techniques have been developed to address the problem of real-time assimilation of system information and observations. As the employed sample sizes are typically several orders of magnitude smaller than the system dimension, such sampling techniques inevitably induces considerable variability into the state estimate, primarily through prior and posterior sample covariance matrices. In this article we quantify this variability with mean squared error measures for two Monte-Carlo based Kalman filter variants, the ensemble Kalman filter and the square-root filter. Under weak assumptions, we derive exact expressions of the error measures. In other cases, we rely on matrix expansions and provide approximations. We show that covariance-shrinking (tapering) based on the Schur product of the prior sample covariance matrix and a positive definite function is a simple, computationally feasible, and very effective technique to reduce sample variability and to address rank-deficient sample covariances. We propose practical rules for obtaining optimally tapered sample covariance matrices. The theoretical results are verified and illustrated with extensive simulations.

Abstract not found

Let us know how access to this document benefits you.

[1] Many geophysical problems are characterized by high-dimensional, nonlinear systems and pose difficult challenges for real-time data assimilation (updating) and forecasting. The present work builds on the ensemble Kalman filter (EnsKF), with the goal of producing ensemble filtering techniques applicable to non-Gaussian densities and highdimensional systems. Three filtering algorithms, based on representing the prior density as a Gaussian mixture, are presented. The first, referred to as a mixture ensemble Kalman filter (XEnsF), models local covariance structures adaptively using nearest neighbors. The XEnsF is effective in a three-dimensional system, but the required ensemble grows rapidly with the dimension and, even in a 40-dimensional system, we find the XEnsF to be unstable and inferior to the EnsKF for all computationally feasible ensemble sizes. A second algorithm, the local-local ensemble filter (LLEnsF), combines localizations in physical as well as phase space, allowing the update step in high-dimensional systems to be decomposed into a sequence of lower-dimensional updates tractable by the XEnsF. Given the same prior forecasts in a 40-dimensional system, the LLEnsF update produces more accurate state estimates than the EnsKF if the forecast distributions are sufficiently

ABSTRACT A truly variance-minimizing filter is introduced and its performance is demonstrated with the KortewegDeVries (KdV) equation and with a multilayer quasigeostrophic model of the ocean area around South Africa. It is recalled that Kalman-like filters are not variance minimizing for nonlinear model dynamics and that fourdimensional variational data assimilation (4DVAR)-like methods relying on perfect model dynamics have difficulty with providing error estimates. The new method does not have these drawbacks. In fact, it combines advantages from both methods in that it does provide error estimates while automatically having balanced states after analysis, without extra computations. It is based on ensemble or Monte Carlo integrations to simulate the probability density of the model evolution. When observations are available, the so-called importance resampling algorithm is applied. From Bayes's theorem it follows that each ensemble member receives a new weight dependent on its ''distance'' to the observations. Because the weights are strongly varying, a resampling of the ensemble is necessary. This resampling is done such that members with high weights are duplicated according to their weights, while low-weight members are largely ignored. In passing, it is noted that data assimilation is not an inverse problem by nature, although it can be formulated that way. Also, it is shown that the posterior variance can be larger than the prior if the usual Gaussian framework is set aside. However, in the examples presented here, the entropy of the probability densities is decreasing. The application to the ocean area around South Africa, governed by strongly nonlinear dynamics, shows that the method is working satisfactorily. The strong and weak points of the method are discussed and possible improvements are proposed.

[1] Hydrologic models use relatively simple mathematical equations to conceptualize and aggregate the complex, spatially distributed, and highly interrelated water, energy, and vegetation processes in a watershed. A consequence of process aggregation is that the model parameters often do not represent directly measurable entities and must therefore be estimated using measurements of the system inputs and outputs. During this process, known as model calibration, the parameters are adjusted so that the behavior of the model approximates, as closely and consistently as possible, the observed response of the hydrologic system over some historical period of time. In practice, however, because of errors in the model structure and the input (forcing) and output data, this has proven to be difficult, leading to considerable uncertainty in the model predictions. This paper surveys the limitations of current model calibration methodologies, which treat the uncertainty in the input-output relationship as being primarily attributable to uncertainty in the parameters and presents a simultaneous optimization and data assimilation (SODA) method, which improves the treatment of uncertainty in hydrologic modeling. The usefulness and applicability of SODA is demonstrated by means of a pilot study using data from the Leaf River watershed in Mississippi and a simple hydrologic model with typical conceptual components.

It has been widely realized that Monte Carlo methods (approximation via a sample ensemble) may fail in large scale systems. This work offers some theoretical insight into this phenomenon in the context of the particle filter. We demonstrate that the maximum of the weights associated with the sample ensemble converges to one as both the sample size and the system dimension tends to infinity. Specifically, under fairly weak assumptions, if the ensemble size grows sub-exponentially in the cube root of the system dimension, the convergence holds for a single update step in state-space models with independent and identically distributed kernels. Further, in an important special case, more refined arguments show (and our simulations suggest) that the convergence to unity occurs unless the ensemble grows super-exponentially in the system dimension. The weight singularity is also established in models with more general multivariate like-lihoods, e.g. Gaussian and Cauchy. Although presented in the context of atmospheric data assimilation for numerical weather prediction, our results are generally valid for high-dimensional particle filters. 1 1

This paper describes the Local Ensemble Transform Kalman Filter (LETKF) data assimilation scheme and its implementa-tion on the National Centers for Environmental Prediction (NCEP) Global Forecast System (GFS) model at the University of Maryland. Numerical results are shown for both simulated observations and observations of the real atmosphere. The role of flow-dependent information in data assimilation is discussed based on the results of the numerical experiments. Preliminary assimilation results with AMSU-A radiance observations are also presented. 1