In this paper we study a new general class of algorithms for the propagation of probabilities on graphical structures based on importance sampling techniques. The idea is to make an approximate and fast propagation in order to obtain a sampling distribution as close as possible to the true one. Our proposal is based on a deletion sequence of the variables to calculate the 'a posteriori' probability in one variable. The deletion procedure is the basis for the exact propagation algorithms. Here the difference is that sometimes, when the cost of the exact deletion exceeds a given limit, an approximated deletion is done. The calculations of the deletion procedure will be used to obtain in a very fast way a sample for the simulation. Some experimental tests are carried out to compare our procedure with other known methods. 1 INTRODUCTION Probability propagation in belief networks consists on updating the probability values of the variables in a dependence graph, given some variables that h...

We present a generalization of dynamic Bayesian networks to concisely describe complex probability distributions such as in problems with multiple interacting variable-length streams of random variables. Our framework incorporates recent graphical model constructs to account for existence uncertainty, value-specific independence, aggregation relationships, and local and global constraints, while still retaining a Bayesian network interpretation and efficient inference and learning techniques. We introduce one such general technique, which is an extension of Value Elimination, a backtracking search inference algorithm. Multi-dynamic Bayesian networks are motivated by our work on Statistical Machine Translation (MT). We present results on MT word alignment in support of our claim that MDBNs are a promising framework for the rapid prototyping of new MT systems. 1

The performance of most vision systems (or subsystems) is significantly dependent on the choice of its various parameters or thresholds. The associated parameter search space is extremely large and nonsmooth; moreover, the optimal choices of the parameters are usually mutually dependent on each other. In this paper we offer a Bayesian network-based probabilistic formalism, which we call the parameter dependence networks (PDNs), to model, abstract, and analyze the parameter space behavior of vision systems. The various algorithm parameters are the nodes of the PDN and are associated with probabilistic beliefs about the optimality of their respective values. The links between the nodes capture the direct dependencies between them and are quantified by conditional belief functions. The PDN structure captures the interdependence among the parameters in a concise and explicit manner. We define information theoretic measures, based on these PDNs, to quantify the global parameter sensitivity and the strength of the interdependence of the parameters. These measures predict the general ease of parameter tuning and performance stability of the system. The PDNs can also be used to stochastically sample the parameter

As an extension of an overview paper [Pan and McMichael, 1997] on information fusion and Causal Probabilistic Networks (CPN), this paper formalizes kernel algorithms for probabilistic inferences upon CPNs. Information fusion is realized through updating joint probabilities of the variables upon the arrival of new evidences or new hypotheses. Kernel algorithms for some dominant methods of inferences are formalized from discontiguous, mathematics-oriented literatures, with gaps lled in with regards to computability and completeness. In particular, possible optimizations on causal tree algorithm, graph triangulation and junction tree algorithm are discussed. Probanet has been designed and developed as a generic shell, or say, mother system for CPN construction and application. The design aspects and current status of Probanet are described. A few directions for research and system development are pointed out, including hierarchical structuring of network, structure decomposition and adaptive inference algorithms. This paper thus has a nature of integration including literature review, algorithm formalization and future perspective.

The chief aim of this paper is to propose mean-field approximations for a broad class of Belief networks, of which sigmoid and noisy-or networks can be seen as special cases. The approximations are based on a powerful mean-field theory suggested by Plefka. We show that Saul, Jaakkola, and Jordan's approach is the first order approximation in Plefka 's approach, via a variational derivation. The application of Plefka's theory to belief networks is not computationally tractable. To tackle this problem we propose new approximations based on Taylor series. Small scale experiments show that the proposed schemes are attractive. 1.

This paper reviews and formalizes algorithms for probabilistic inferences upon causal probabilistic networks (CPN), also known as Bayesian networks, and introduces Probanet - a development environment for CPNs. Information fusion in CPNs is realized through updating joint probabilities of the variables upon the arrival of new evidences or new hypotheses. Kernel algorithms for some dominant methods of inferences are formalized from discontiguous, mathematics-oriented literatures, with gaps filled in with regards to computability and completeness. Probanet has been designed and developed as a generic shell, a development environment for CPN construction and application. The design aspects and current status of Probanet are described. 1 Introduction Digital signal processing has entered the era of multisensor data fusion and multisource information fusion. Whatever the application may be, the process of data and information fusion generally involves multiple data types such as sensor sig...

Multipoint linkage analyses ofgenetic data on extended pedigrees can involve exact computations which are infeasible. Markov chain Monte Carlo methods represent an attractive alternative, greatly extending the range of models and data sets for which analysis is practical. In this paper, several advances in Markov chain Monte Carlo theory, namely joint updates of latent variables across loci and meioses, integrated proposals, Metropolis-Hastings restarts via sequential imputation and Rao Blackwellized estimators, are incorporated into a sampling strategy which mixes well and produces accurate results in real time. The methodology is demonstrated through its application to several data sets originating from a study of early-onset Alzheimer's disease in families of Volga-German ethnic origin.

Factorisation of probability trees is a useful tool for inference in Bayesian networks. Probabilistic potentials some of whose parts are proportional can be decomposed as a product of smaller trees. Some algorithms, like lazy propagation, can take advantage of this fact. Also, the factorisation can be used as a tool for approximating inference, if the decomposition is carried out even if the proportionality is not completely reached. In this paper we propose the use of approximate factorisation as a means of controlling the approximation level in a dynamic importance sampling algorithm. 1

To study the effect of individual genes by segregation or linkage analyses, the likelihood of the model needs to be evaluated. The likelihood can be computed efficiently using the Elston-Stewart algorithm. This algorithm involves summing over the unobserved genotypes in the pedigree, which is called peeling. An important aspect of this algorithm is to determine the order of peeling to maximize efficiency. This paper shows how determining peeling order is related to a problem in solving systems of symmetric sparse linear equations. It also shows how algorithms developed to efficiently solve those systems, can be used to determine the optimal order of peeling in the Elston-Stewart algorithm. (Key words: peeling order, sparse matrices)

A class of Monte Carlo algorithms for probability propagation in belief networks is given. The simulation is based on a two steps procedure. The first one is a node deletion technique to calculate the 'a posteriori' distribution on a variable, with the particularity that when exact computations are too costly, they are carried out in an approximate way. In the second step, the computations done in the first one are used to obtain random configurations for the variables of interest. These configurations are weighted according to the importance sampling methodology. Different particular algorithms are obtained depending on the approximation procedure used in the first step and in the way of obtaining the random configurations. In this last case, a stratified sampling technique is used, which has been adapted to be applied to very large networks without problems with rounding errors.