Robert E. Schapire. Learning probabilistic read-once formulas on product distributions. In Proceedings of the Fourth Annual Workshop on Computational Learning Theory, August 1991. To appear, Machine Learning.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....the a more permissive setting obtained by requiring successful learning only when D is a member of some predetermined class of distributions over X. One such candidate is the class of product distributions (each variable has a fixed distribution independent of the values of other variables [26]) In some cases we shall even assume that the permissible class contains only one distribution (i.e. the distribution is fixed and known to the learner) Two main types of restrictions on the learner s focus of attention naturally rise, and are defined bellow. The k RFA restriction is a ....

Robert E. Schapire. Learning probabilistic read-once formulas on product distributions. In Proceedings of the 4th Annual Workshop on Computational Learning Theory, pages 184--198, 1991.

.... knowledge of the system and knowledge of the language of the plaintext (which determines the distribution of the examples) While there is some work within the machine learning community relating to learning from known distributions (such as the uniform distribution, or product distributions [40]) and the field of pattern recognition has developed many techniques for this problem [12] most of the modern research in machine learning, based on Valiant s PAC learning formalization of the problem, assumes that random examples are drawn according to an arbitrary but fixed probability ....

....drawn from 0, 1 n according to the uniform distribution. While this assumption seems rather unrealistic and restrictive in most learning applications, it is a perfect match for such a cryptographic scenario. What cryptographic lessons can be drawn from the learning theory research Schapire [40] shows how to e#ciently infer a class of formula he calls probabilistic read once formula against product distributions. A special case of this result implies that a formula f constructed from and, or, and not gates can be exactly identified (with high probability) in polynomial time from ....

Robert E. Schapire. Learning probabilistic read-once formulas on product distributions. In Proceedings of the Fourth Annual Workshop on Computational Learning Theory, pages 184--198, Santa Cruz, California, August 1991.

....because of its statistical nature [7] the proposed algorithm can tolerate a classification noise rate # up to the information theoretic limit of # =1 2. There exist other statistical methods to learn other classes of read once formulas under particular distributions [1] and product distributions [8]. They all basically di#er in the statistical tests they use to identify the gate parameters and the formula s skeleton. Our key novel contribution is to introduce a new test (for discovering the network s connectivity) which exploits the strong unimodality [5] property of sums of independent ....

....whose children are all variables) Then, after finding the exact thresholds (for TG perceptrons) we will consider these bottom level perceptrons as new meta variables (that replace their children) from which we can find their parent perceptrons. In this manner similar to Schapire s algorithm [8], we will build every perceptron of the net until we reach the root. The coinfluence function will enable the learner to determine if certain variables are siblings of a perceptron g and if g is fed by other perceptrons. This is possible because the distribution of a sum of independent random ....

Schapire R., (1994) "Learning probabilistic read-once formulas on product distributions " Machine Learning vol. 14, 47--81. San Mateo, CA: Morgan Kaufman.

.... and uses techniques from, recent positive results for learning other classes of read once formulas (Angluin, Hellerstein, and Karpinski, 1993; Bshouty, Hancock, and Hellerstein, 1992a and 1992b; Goldman, Kearns, and Schapire, 1990; Kearns, Li, Pitt, and Valiant, 1987; Pagallo and Haussler, 1989; Schapire, 1991). In the terminology of that literature, nonoverlapping perceptron networks are read once formulas (or synonymously formulas) over the basis of weighted threshold functions. One can think of this type of architecture as a network of decoupled perceptrons, which in terms of architecture ....

....of the algorithm to nonoverlapping perceptron networks with more than one output is straightforward. A number of problems remain open. One important problem is whether or not nonoverlapping 26 perceptron networks are learnable from examples only under the uniform distribution. By analogy, Schapire (1991) has shown that some some classes of read once formulas are learnable under those conditions, and we have preliminary results for subclasses of NPNs (Golea, Marchand, and Hancock, 1992) Learning general (overlapping) perceptron networks on the uniform distribution remains intractable, given ....

Schapire, R. (1991). Learning probabilistic read-once formulas on product distributions. Proceedings of the Fourth Annual Workshop on Computational Learning Theory (pp. 184--198). San Mateo, CA: Morgan Kaufman.

.... particular, positive learnability results have been obtained for read once formulas in disjunctive normal form ( DNF) on the uniform distribution (Kearns et al. 1987; Pagallo Haussler, 1989) and read once boolean formulas whose gates compute functions in AND,OR,NOT on product distributions (Schapire, 1991). Nonoverlapping perceptron networks seem to be the natural neural version of read once boolean formulas. Two perceptrons are said to be nonoverlapping if they do not share any input variables (Barkai et al. 1990) A (or nonoverlapping) perceptron network is a loop free network in which each ....

.... robust against a large amount of random classification noise (Kearns, 1993) There are a number of previous algorithms for learning read once boolean formulas that exploit the idea of using various statistical estimates to infer the target function (Goldman et al. 1990; Pagallo Haussler, 1989; Schapire, 1991). While the high level structures of these statistical algorithms are similar, they di#er on the the specific statistical tests used. In fact, the main di#culty of applying these techniques is to find the appropriate statistical quantities 2 The intersection is simply the complement of the union ....

[Article contains additional citation context not shown here]

Schapire R.E. (1991). Learning probabilistic read-once formulas on product distributions.

.... to Furst, Jackson and Smith [5] several efficient algorithms for learning restricted forms of DNF with respect to the uniform distribution in the Valiant model [12] and efficient algorithms for learning unbounded depth readonce circuits with respect to product distributions in the Valiant model [21, 7]. For all of these classes we can obtain efficient algorithms for learning with noise by Theorem 3; in this list, only for conjunctions [1] and Schapire s work on read once circuits [21] were there previous noise analyses. As further evidence for the generality of the statistical query model and ....

.... for learning unbounded depth readonce circuits with respect to product distributions in the Valiant model [21, 7] For all of these classes we can obtain efficient algorithms for learning with noise by Theorem 3; in this list, only for conjunctions [1] and Schapire s work on read once circuits [21] were there previous noise analyses. As further evidence for the generality of the statistical query model and to give a flavor for the methods involved, we now spend the remainder of this section describing in high level detail three cases in which new statistical query algorithms can be obtained ....

Robert E. Schapire. Learning probabilistic read-once formulas on product distributions. In Proceedings of the Fourth Annual Workshop on Computational Learning Theory, August 1991. To appear, Machine Learning.

....the a more permissive setting obtained by requiring successful learning only when D is a member of some predetermined class of distributions over X . One such candidate is the class of product distributions (each variable has a fixed distribution independent of the values of other variables [36]) In some cases we shall even assume that the permissible class contains only one distribution (i.e. the distribution is fixed and known to the learner) Two main types of restrictions on the learner s focus of attention naturally rise, and are defined below. The k RFA restriction is a ....

Robert E. Schapire. Learning probabilistic read-once formulas on product distributions. Machine Learning, 14(1):47--81, 1994.

....et al. 15] for learning the class of read once formulas in disjunctive normal form (DNF) against the uniform distribution. A similar result, though based on a different method, was obtained by Pagallo and Haussler [18] These results were extended by Hancock and Mansour [10] and by Schapire [21] as described below. Also, Linial, Mansour and Nisan [17] used a technique based on Fourier spectra to learn the class of constant depth circuits (constructed from gates of unbounded fan in) against the uniform distribution. Furst, Jackson and Smith [7] generalized this result to learn this same ....

....other variables) Verbeurgt [25] gives a different algorithm for learning DNF formulas against the uniform distribution. However, all three of these algorithms require quasi polynomial (n polylog(n) time, though Verbeurgt s procedure only requires a polynomial size sample. Finally, Schapire [21] has recently extended our technique to handle a probabilistic generalization of the class of all read once Boolean formulas constructed from the usual basis fand; or; notg. He shows that an arbitrarily good approximation of such formulas can be inferred in polynomial time against any product ....

[Article contains additional citation context not shown here]

Robert E. Schapire. Learning probabilistic read-once formulas on product distributions. In Proceedings of the Fourth Annual Workshop on Computational Learning Theory, pages 184--198, August 1991.

....et al. 12] for learning the class of read once formulas in disjunctive normal form (DNF) against the uniform distribution. A similar result, though based on a different method, was obtained by Pagallo and Haussler [15] These results were extended by Hancock and Mansour [6] and by Schapire [18] as described below. Also, Linial, Mansour and Nisan [14] used a technique based on Fourier spectra to learn the class of constant depth formulas (constructed from gates of unbounded fan in) against the uniform distribution. Furst, Jackson and Smith [5] generalized this result to learn this same ....

....other variables) Verbeurgt [21] gives a different algorithm for learning DNF formulas against the uniform distribution. However, all three of these algorithms require quasi polynomial (n polylog(n) time, though Verbeurgt s procedure only requires a polynomial size sample. Finally, Schapire [18] has recently applied our technique to a probabilistic generalization of the class of all read once Boolean formulas constructed from the usual basis fand; or; notg. He shows that an arbitrarily good approximation of such formulas can be inferred in polynomial time against any product ....

Robert E. Schapire. Learning probabilistic read-once formulas on product distributions. In Computation Learning Theory: Proceedings of the Fourth Annual Workshop, August 1991.

No context found.

Robert E. Schapire. Learning probabilistic read-once formulas on product distributions. In Proceedings of the Fourth Annual Workshop on Computational Learning Theory, August 1991. To appear, Machine Learning.

No context found.

Robert E. Schapire. Learning probabilistic read-once formulas on product distributions. In Proceedings of the Fourth Annual Workshop on Computational Learning Theory, pages 184-198, Santa Cruz, California, August 1991.

No context found.

Robert E. Schapire. Learning probabilistic read-once formulas on product distributions. Machine Learning, 14(1):47--81, 1994.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC