Friedman, N., Nachman, I., & Pe'er, D. (1999). Learning bayes network structure from massive datasets: The "sparse candidate " algorithm. UAI 15 (p. 206:215).  Home/Search   Document Not in Database   Summary   Related Articles   Check

....a learning curve is fit to the results, and this curve is used to decide when to stop. This may lead to stopping earlier than with our method, but stopping can also occur prematurely, due to the di#culty in reliably extrapolating learning curves. Friedman et al. s Sparse Candidate algorithm [5] alternates between heuristically selecting a small group of potential relatives for each variable and doing a search step limited to considering changing arcs between a variable and its potential relatives. This procedure avoids the quadratic dependency on the number of variables in a domain. ....

N. Friedman, I. Nachman, and D. Peer. Learning Bayesian network structure from massive datasets: The "sparse candidate" algorithm. In Proc. 15th Conf. on Uncertainty in Artificial Intelligence, pp. 206--215, Stockholm, Sweden, 1999.

....of the input data given to our program, indicating that our program reconstructed the levels of activity of these processes. 2 Probabilistic Model In this section we present our probabilistic model. Our approach is based on the language of probabilistic relational models (PRMs) as described in [10, 7]. For lack of space, we do not review the general PRM framework, but focus on the details of the model, which follow the application of PRMs to gene expression in [14] A simplified version of our model is presented in Fig. 1(a) we now describe its elements. The PRM framework represents the ....

....1010 genes, 173 arrays, and 10 processes. This results in a model with M M N hidden variables. Moreover, the nature of the data is such that we cannot treat genes (or arrays) as independent samples. Instead, any two hidden variables are dependent on each other given the observations (see [7] for an elaboration of this point) For example, consider two genes and s assignment to processes influences our estimates of the variables, which in turn influence our membership probabilities for . Due to these dependencies, the exact computation of the E step is intractable for large ....

N. Friedman, I. Nachman, and D. Peer. Learning of Bayesian network structure from massive datasets: The "sparse candidate" algorithm. Submitted, 1999.

....step, MMBN discovers the edges of the BN, and in the second step orients them. However, for the rest of the paper we only focus on the first step of edge discovery. The orientation phase of the PC algorithm or a constrained hill climbing searchand score in the fashion of the Sparse Candidate [3] can then be used for edge orientation. Edge identification is important by itself since an edge between variables X and Y under certain conditions corre sponds to a direct causal relation between the two variables, i.e. X directly causing Y or vice versa [5, 9, t0] Thus, edges may be used to ....

....The details of the method are omitted due to lack of space. 4. Experimental Results Experiment 1: MMBN Outperforms State of theart BN Learning Algorithms. The first set of experiments compares MMBN with state of the art BN algorithms, namely PC [10] TPDA [6] and the Sparse Candidate algorithm [3]. MMBN is implemented using Matlab 6.5, while for the rest of the algorithms we used the publicly avail able versions and default values. One thousand train ing instances were generated by randomly sampling from the distribution of ALARM, a BN used in a medical diagnosis decision support system ....

Friedman, N., I. Nachman, and D. Pe'er. Learning Bayesian Network Structure from Massive Datasets: The "Sparse Candidate" Algorithm. in Fifteenth Conference on Uncertainty in Artificial Intelligence (UAI). 1999.

....considered a guaranteed recipe for bad performance. 1 Introduction Bayes nets are widely used for data modeling. However, the problem of constructing Bayes nets from data remains a hard one, requiring search in a super exponential space of possible graph structures. Despite recent advances [1], learning network structure from big data sets demands huge computational resources. We therefore turn to a simpler model, which is easier to compute while still being expressive enough to be useful. Namely, we look at dependency trees, which are belief networks that satisfy the additional ....

Nir Friedman, Iftach Nachman, and Dana Peer. Learning bayesian network structure from massive datasets: The "sparse candidate" algorithm. In Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence (UAI-99), pages 206--215, Stockholm, Sweden, 1999.

....use each to infer a network structure, and measure the confidence of a feature as the fraction of these networks in which the feature is valid. To allow reasonable learning times for a graph structure with hundreds of nodes, Friedman et al. utilize the sparse candidate algorithm described in [10]. This iterative algorithm restricts the set of candidate parents for each node using a local mutual information criterion. It then searches for an optimal network structure within these restrictions. Next, it uses the chosen network structure to select a new set of candidate parents. ....

N. Friedman, I. Nachman, and D. Pe'er. Learning bayesian network structure from massive datasets: The "sparse candidate" algorithm. In Proc. Fifteenth Conf. on Uncertainty in Artificial Intelligence, 1999.

....restrict the maximum number of parents to be k, instead of n. This is reasonable, since we expect the fan in to be small) This reduces the number of parent sets we need to evaluate from O(2 n ) to n k n k . Some heuristics for choosing the set of k potential parents are given in [FNP99] As long as we give non zero probability to all possible edge changes in our proposal, we are guaranteed to get the correct answer (since we can get from any graph to any other by single edge changes, and hence the chain is irreducible) heuristics 7 merely help us reach the right answer ....

....non zero probability to all possible edge changes in our proposal, we are guaranteed to get the correct answer (since we can get from any graph to any other by single edge changes, and hence the chain is irreducible) heuristics 7 merely help us reach the right answer faster. The heuristics in [FNP99] change with time, since they look for observed dependencies that cannot be explained by the current model. Such adaptive proposal distributions cause no theoretical problems for convergence. 3.2 Computing the marginal likelihood We now discuss how to eciently compute the posterior odds ....

N. Friedman, I. Nachman, and D. Peer. Learning Bayesian network structure from massive datasets: The "sparse candidate" algorithm. In UAI, 1999.

....in data is similar to developing an efficient code for the underlying random process, as efficient coding is analogous to probabilistic modeling. Perhaps the earliest well known work on structure learning in directed graphical models is [7] More recent research on this topic may be found in [17, 5, 16, 25, 10, 20, 23, 13]. 2 In general, the task of learning Bayesian networks can be grouped into four categories [10] depending on 1) if the data is fully observable or if it contains missing values, and 2) if it is assumed that the structure of the model is known or not. The easiest case is when the data is fully ....

....randomly chosen sets are clear: because of the large search space, it is unlikely that good sets of dependencies will be found in a reasonable amount of time. It seems crucial, therefore, to constrain the random search to those that found to be useful in some way, as has been argued in the past [13]. Finally, as argued in [2] an HMM can approximate a distribution arbitrarily well given enough capacity, enough training data, and enough computation. The results in the tables support this claim as increasing parameters leads to improved accuracy. The performance improvement obtained by adding ....

N. Friedman, I. Nachman, and D. Peer. Learning Bayesian network structure from massive datasets: The "sparse candidate" algorithm. 15th Conf. on Uncertainty in Artificial Intelligence, 1999.

....consistent with some total ordering, OE. Then, we introduce a distribution over the orderings. 4.2 Analysis for a Fixed Ordering Let OE be a total ordering of X . We restrict attention to network structures that are consistent with OE, i.e. if there is an edge X Y , then X OE Y . Following [Friedman et al. 1999] , we also assume that each node X i has a set W i of at most k possible candidate parents that is fixed before each query round. In certain domains, we can use prior knowledge to construct W i ; in others, we can use the data itself to point out nodes that are more likely to be directly related ....

N. Friedman, I. Nachman, and D. Pe'er. Learning Bayesian network structure from massive datasets: The "sparse candidate" algorithm. In Proc. UAI, 1999.

....with an extremely large search space. Therefore, we need to use (and develop) efficient search algorithms. To facilitate efficient learning, we need to be able to focus the attention of the search procedure on relevant regions of the search space, giving rise to the Sparse Candidate algorithm [18]. The main idea of this technique is that we can identify a relatively small number of candidate parents for each gene based on simple local statistics (such as correlation) We then restrict our search to networks in which only the candidate parents of a variable can be its parents, resulting in ....

....mutual information, I(X i ; X j jPa Gn Gamma1 (X i ) The score we actually use is an estimator of the conditional mutual information in the underlying distribution, that takes into account also the number of parameters needed to learn X i s conditional probability. We refer the reader to [18] for more details on the algorithm and its complexity, as well as empirical results comparing its performance to traditional search techniques. 3.4 Discretization In order to specify a Bayesian network model, we still need to define the local probability model for each variable. At the current ....

[Article contains additional citation context not shown here]

N. Friedman, I. Nachman, and D. Pe'er. Learning Bayesian network structure from massive datasets: The "sparse candidate " algorithm. In Proc. Fifthteenth Conference on Uncertainty in Artificial Intelligence (UAI '99), pp. 196--205, 1999.

....with an extremely large search space. Therefore, we need to use (and develop) efficient search algorithms. 7 To facilitate efficient learning, we need to be able to focus the attention of the search procedure on relevant regions of the search space, giving rise to the Sparse Candidate algorithm (Friedman, Nachman Pe er 1999). The main idea of this technique is that we can identify a relatively small number of candidate parents for each gene based on simple local statistics (such as correlation) We then restrict our search to networks in which only the candidate parents of a variable can be its parents, resulting in ....

....each X j by computing ScoreContribution BDe (X i ; fX j g [ Pa n 1 (X i ) D) where Pa n 1 (X i ) are the parents of X i in the network found at the end of previous iteration. We then set C n i to consist of Pa n 1 (X i ) and the variables that maximize this score. We refer the reader to (Friedman, Nachman Pe er 1999) for more details on the algorithm and its complexity, as well as empirical results comparing its performance to traditional search techniques. 3.4 Discretization In order to specify a Bayesian network model, we still need to define the local probability model for each variable. At the current ....

[Article contains additional citation context not shown here]

Friedman, N., Nachman, I. & Pe'er, D. (1999), Learning bayesian network structure from massive datasets: The "sparse candidate" algorithm, in Dubios & Laskey (1999).

....construction, together with the decomposability of the score, allows the steps of the search (say, greedy hill climbing) to done very efficiently. The success of this approach depends on the choice of the potential parents. Clearly, a wrong initial choice can result to poor structures. Following [Friedman et al. 1999] , which examines a similar approach in the context of learning Bayesian networks, we propose an iterative approach that starts with some structure (possibly one where each attribute does not have any parents) and select the sets Pot k (X:A) based on this structure. We then apply the search ....

....in large set of potential parents. Thus, we actually use a more refined algorithm that only adds parents to Pot k 1 (X:A) if they seem to add value beyond Pa k (X:A) There are several reasonable ways of evaluating the additional value provided by new parents. Some of these are discussed in [Friedman et al. 1999] in the context of learning Bayesian networks. Their results suggest that we should evaluate a new potential parent by measuring the change of score for the family of X:A if we add the (X: B) to its current parents. We then choose the highest scoring of these, as well as the current parents, to ....

N. Friedman, I. Nachman, and D. Peer. Learning of Bayesian network structure from massive datasets: The "sparse candidate" algorithm. Submitted, 1999.

No context found.

Friedman, N., Nachman, I., & Pe'er, D. (1999). Learning bayes network structure from massive datasets: The "sparse candidate " algorithm. UAI 15 (p. 206:215).

No context found.

N. Friedman, I. Nachman, and D. Pe'er. Learning Bayesian network structure from massive datasets: The "sparse candidate" algorithm. In Proc. UAI, 1999.

No context found.

N. Friedman, I. Nachman, and D. Per (1999). Learning Bayesian network structure from massive datasets: The "sparse candidate" algorithm. In Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence, pp. 206-15.

No context found.

N. Friedman, I. Nachman, and D. Peer. Learning of Bayesian network structure from massive datasets: The "sparse candidate" algorithm. Submitted, 1999.

No context found.

N. Friedman, I. Nachman, and D. Peer. Learning Bayesian network structure from massive datasets: the "sparse candidate" algorithm. In Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence, pages 206--215. Morgan Kaufmann, 1999.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC