R. Maclin and J. Shavlik. Refining algorithms with knowledge-based neural networks: improving the Chou-Fasman algorithm for protein folding. In Stephen Hanson, George Drastal, and Ronald Rivest, editors, Computational Learning Theory and Natural Learning Systems, pages 249--286. The MIT Press, 1994.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....of these approaches use the leave one out criterion, to the best of our knowledge, none of them test performance with the leave half out criterion. Various machine learning techniques have been applied to the protein structure prediction problem. The two main approaches are neural nets (e.g. [47, 67, 59]) and hidden Markov models (e.g. 53, 9] Both of these approaches require adequate data on the target motif, since there is a training session on sequences that are known to contain the target motif. Our approach differs from these methods since it does not require well analyzed data on the ....

R. Maclin and J. Shavlik. Refining algorithms with knowledge-based neural networks: improving the Chou-Fasman algorithm for protein folding. In Stephen Hanson, George Drastal, and Ronald Rivest, editors, Computational Learning Theory and Natural Learning Systems, pages 249--286. The MIT Press, 1994.

....will be consistent with the generally admitted knowledge about protein secondary structure and can reveal new biologically interesting dependencies. The first attempt to use an a priori knowledge, namely, the Chou Fasman theory for the neural network based secondary structure prediction made in [8] gave the statistically significant gain in the neural network s prediction performance. As the Chou Fasman theory is of statistical nature, its insertion does not provide a principally new information in comparison with the neural network based technique, it just helps to place a neural network ....

Maclin R., Shavlik J.W. (1991) Refining Algorithms with Knowledgebased Neural Networks: Improving the Chou-Fasman Algorithm for Protein Folding, Machine Learning Research Group Working Paper, CS department, University of Wisconsin, Madison, available via anonymous ftp from steves.cs.wisc.edu

....approach of combining HMMs with RNNs. We discuss an algorithm for directly mapping a trained HMM into a RNN architecture and derive a gradient descent learning algorithm for refining the knowledge. 1 MOTIVATION Recently, there has been a lot of interest in combining symbolic and neural learning [2, 7, 9, 10, 11, 14, 15, 16, 19, 20, 21, 22, 23, 24, 25, 27, 28, 30, 31]. There are different ways in which neural and symbolic learning can be combined to solve a given learning task. An excellent collection of a variety of approaches can be found in [1] The traditional approach to using neural networks is shown in the lower part ( connectionist representation ) ....

R. Maclin and J. Shavlik, "Refining algorithms with knowledge-based neural networks: Improving the Chou-Fasman algorithm for protein folding," in Computational Learning Theory and Natural Learning Systems (S. Hanson, G. Drastal, and R. Rivest, eds.), MIT Press, 1992.

....is used to initialize a new network and simulation results in Section 7. A summary of and potential future work conclude this paper. 2 Knowledge Based Neural Networks Recently, there has been a lot of interest in knowledge based neural networks, i.e. the combining symbolic with neural learning ([4, 14, 20, 29, 40, 44, 41, 43, 39]) There are different ways in which neural and symbolic Initial Domain Theory Knowledge Extraction Knowledge Insertion Symbolic Representation Refined Domain Theory Connectionist Representation Initialized Neural Network Trained Neural Network Training on Data Random Initialization ....

R. Maclin and J. Shavlik, "Refining algorithms with knowledge-based neural networks: Improving the Chou-Fasman algorithm for protein folding," in Computational Learning Theory and Natural Learning Systems (S. Hanson, G. Drastal, and R. Rivest, eds.), MIT Press, 1992.

....the area of computational molecular biology (protein folding) They modeled prior knowledge about protein structures as DFAs, initialized a neural network with that prior knowledge, and trained it on sample data. This approach outperformed the best known traditional algorithm for protein folding [23, 24]. The purpose of this paper is to show that recurrent networks can learn the behavior of FFAs, and that the learned knowledge can be extracted in a symbolic form. The latter objective requires an understanding of how trained networks represent FFAs. We proved in [33] that the computational ....

R. Maclin and J. Shavlik, "Refining algorithms with knowledge-based neural networks: Improving the Chou-Fasman algorithm for protein folding," in Computational Learning Theory and Natural Learning Systems (S. Hanson, G. Drastal, and R. Rivest, eds.), MIT Press, 1992.

....network dynamics. When partial symbolic knowledge is encoded into a network in order to improve training, programming as few weights as possible is desirable because it leaves the network with many unbiased adaptable weights. This is important when a network is used for domain theory revision [15, 21], where the prior knowledge is not only incomplete, but may also be incorrect [10, 17] Methods for constructing DFA s in recurrent networks where neurons have hard limiting discriminant functions have been proposed [1, 14, 16] This paper is concerned with neural network implementations of DFA s ....

R. Maclin and J. Shavlik, "Refining algorithms with knowledge-based neural networks: Improving the chou-fasman algorithm for protein folding," in Computational Learning Theory and Natural Learning Systems (S. Hanson, G. Drastal, and R. Rivest, eds.), MIT Press, 1992.

....indefinitely due to the built in feedback [29] In particular, they can be encoded [20, 18] and trained [8, 11, 16, 26, 31, 33] to behave like deterministic, sequential finite state automata. Methods for inserting prior knowledge into recurrent neural networks have been previously discussed [4, 6, 7, 12, 15, 21]. It has been demonstrated [12, 21] that prior knowledge can significantly reduce the amount of training necessary for a network to correctly classify a training set of temporal sequences. Our interpretation of rule revision consists of three stages: 1) insert all the available prior knowledge by ....

R. Maclin and J. Shavlik, "Refining algorithms with knowledge-based neural networks: Improving the chou-fasman algorithm for protein folding," in Computational Learning Theory and Natural Learning Systems (S. Hanson, G. Drastal, and R. Rivest, eds.), MIT Press, 1992.

....There is an increased interest in implementing neural network architectures in analog VLSI [3, 26, 30, 32, 38, 39, 54, 56] Analog implementations offer the advantage of speed and low power consumption. Several methods for mapping DFAs into recurrent neural network architectures have been proposed [2, 16, 18, 36, 45]. Our algorithm takes advantage of the second order structure of the recurrent network model which allows for a natural mapping of DFA state transitions into network state changes. A. DFA Encoding Algorithm Our encoding algorithm illustrated in figure 2 achieves a nearly orthonormal internal ....

.... GammaH=2 or H are set to zero. B. Learning with Prior Knowledge Empirical studies have shown that partial prior knowledge of a DFA (states and transitions) can significantly improve the training time [23] Recurrent networks can even perform rule revision, i.e. refine incomplete initial rules [17, 36, 37, 57] and correct incorrect prior knowledge through learning from data [48] C. Stability of Designed Networks The following theorem asserts that time discrete, continuous recurrent neural networks can represent DFAs [47] Theorem C. 1 Let L(MDFA ) denote the regular language accepted by some DFA ....

R. Maclin and J. Shavlik, "Refining algorithms with knowledge-based neural networks: Improving the chou-fasman algorithm for protein folding," in Computational Learning Theory and Natural Learning Systems (S. Hanson, G. Drastal, and R. Rivest, eds.), MIT Press, 1992.

....scheme to smaller networks can lead to conflicts in terms of the values for the programmed weights. The resolution of these conflicts is not obvious and is beyond the scope of this paper. The hint insertion method discussed above is not unique. There are potentially many other approaches ( Maclin 92] 4 LEARNING WITH HINTS 4.1 Hints Consider strings over the alphabet f0, 1g. A string is a member of the language 2 parity if the number of both 0 s and 1 s is even. The ideal, minimal DFA which accepts strings in the language is shown in figure 2a. We inserted hints according to figures 2b, ....

R. Maclin, J.W. Shavlik, Refining Algorithms with Knowledge-Based Neural Networks: Improving the Chou-Fasman Algorithm for Protein Folding, S. Hanson, G. Drastal, R. Rivest (Eds), Computational Learning Theory and Natural Learning Systems, MIT Press, to appear, (1992).

....area of computational molecular biology (protein folding) They represented prior knowledge about protein structures as DFAs, initialized a neural network with that prior knowledge, and trained it on sample data. This approach outperformed the best known traditional algorithm for protein folding [33, 34]. to accommodate FFAs. 3 The capability of representing FFAs can be viewed as a foundation for the problem of learning FFAs from example strings (if a network cannot represent FFAs, then it certainly cannot learn them either) Since DFAs are used in high level VLSI design, and recurrent neural ....

R. Maclin and J. Shavlik, "Refining algorithms with knowledge-based neural networks: Improving the Chou-Fasman algorithm for protein folding," in Computational Learning Theory and Natural Learning Systems (S. Hanson, G. Drastal, and R. Rivest, eds.), MIT Press, 1992.

....the training of the resulting networks. However, it does not represent a fundamental limitation on KBANN, as there exist algorithms based upon backpropagation that can be used to train networks with cycles [40] Moreover, others have extended KBANN to handle recursive finite state grammars [23]. In addition to these constraints, the rule sets provided to KBANN are usually hierarchically structured. That is, rules do not commonly map directly from inputs to outputs. Rather, at least some of the rules provide intermediate conclusions that describe useful conjunctions of the input ....

....favor. As tests on small, artificial domains show qualitatively similar results [51, 35, 56] we are optimistic that future tests will continue to show the success of the KBANN algorithm. Indeed, work by us and our colleagues on another biological domain (protein secondary structure prediction) [23] and a process control problem [49] have shown the generality of the approach. Finally, our tests show that KBANN s value is due to both the identification of informative input features and useful derived features (thereby establishing a good network topology) Neither focusing attention nor ....

[Article contains additional citation context not shown here]

R. Maclin and J. W. Shavlik, Refining Algorithms with Knowledge-Based Neural Networks: Improving th Chou-Fasman Algorithm for Protein Folding, Machine Learning 11 (1993) 195--213.

....the training of the resulting networks. However, it does not represent a fundamental limitation on Kbann, as there exist algorithms based upon backpropagation that can be used to train networks with cycles [40] Moreover, others have extended Kbann to handle recursive finite state grammars [23]. In addition to these constraints, the rule sets provided to Kbann are usually hierarchically structured. That is, rules do not commonly map directly from inputs to outputs. Rather, at least some of the rules provide intermediate conclusions that describe useful conjunctions of the input ....

....favor. As tests on small, artificial domains show qualitatively similar results [51, 35, 56] we are optimistic that future tests will continue to show the success of the Kbann algorithm. Indeed, work by us and out colleagues on another biological domain (protein secondary structure prediction) [23] and a process control problem [49] have shown the generality of the approach. Finally, our tests show that Kbann s value is due to both the identification of informative input features and useful derived features (thereby establishing a good network topology) Neither focusing attention nor ....

[Article contains additional citation context not shown here]

R. Maclin and J. W. Shavlik, Refining Algorithms with Knowledge-Based Neural Networks: Improving the Chou-Fasman Algorithm for Protein Folding, Machine Learning 11 (1993) 195--213.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC