R. Maclin and J. Shawlik, "Using knowledge-based neural networks to improve algorithms: Refining the chou-fasman algorithm for protein folding," Machine Learning, . (to appear).  Home/Search   Document Details and Download   Summary   Related Articles   Check

....inasmuch as the number of parameters to be learned explodes. 7.1. 2 Recurrent KBANNs The Knowledge Based Artificial Neural Networks (KBANN) learning system was introduced as a method for mapping hierarchically structured rules into feedforward networks [54] The extension to recurrent network [55][56] is quite straightforward, inasmuch as transition rules for a DFA are used and delayed feedback connections are introduced from the output to the input layer of the network (see fig 7.1.2. Like 21 u q 3 Delta q 2 q 1 1 1 0 0 1 0 q 2 q 3 q 1 Figure 5: Translation of knowledge based ....

R. Maclin and J. Shawlik, "Using knowledge-based neural networks to improve algorithms: Refining the chou-fasman algorithm for protein folding," Machine Learning, . (to appear).

.... this property, i.e. their internal representation of states and transitions may become unstable for sufficiently long input sequences [37] Finally, with the extraction of knowledge from trained neural networks, the methods discussed here could potentially be applied to incorporating and refining [38] fuzzy knowledge previously encoded into recurrent neural networks. 1.2 Background A variety of implementations of FFA have been proposed, some in digital systems [39, 40] However, here we give a proof that such implementations in sigmoid activation RNNs are stable, i.e. guaranteed to converge ....

R. Maclin and J.W. Shavlik, "Using knowledge-based neural networks to improve algorithms: Refining the chou-fasman algorithm for protein folding," Machine Learning, vol. 11, pp. 195--215, 1993.

....artificial neural networks (Towell et al., 1990) to represent expert theories that have been previously written in a simplified form of Horn clause logic. There are a number of systems that use neural network encodings to refine a variety of representation types, including finite state automata (Maclin Shavlik, 1993), certainty factor rule bases (Mahoney, 1996) and first order horn clause logic (Towell et al., 1990) indigent uses the Kbann (Towell et al., 1990) method for encoding domain theories. There has also been work done on genetically 3 refining neural network topologies. Opitz (1995) describes the ....

....bases represented as neural networks can be refined through the addition or removal of rules in the knowledge base. A number of different representations of human knowledge have been encoded in the form of neural networks, including deterministic finite automata (DFAs) Omlin Giles, 1996) (Maclin Shavlik, 1993), certainty factor rule bases (Mahoney, 1996) pushdown automata (Das et al., 1992) and modified first order logic rule bases (Towell et al., 1990) Most of these systems translate the domain knowledge into a network topology and then refine the network s biases and weights through standard neural ....

Maclin, R. and Shavlik, J. (1993). Using knowledge-based neural networks to improve algorithms: Refining the Chou-Fasman algorithm for protein folding. Machine Learning, 11:195--215.

.... 2 Play = Bias = Weight Figure 2.3: The network based on the golf domain. with higher accuracy than standard neural networks, but also converge to a highly accurate network much more quickly [TSN90] They also have been shown to outperform symbolic inductive learning algorithms such as C4.5 [MS93] in a number of domains. Their accuracy is, however, limited by the accuracy of the domain theory that the network is given, and incorrect domain theories can restrict a network s performance [OS93] A number of systems based on these ideas have been developed and are addressed in Section 3.1. ....

....have shown that knowledge bases represented as neural networks can be refined through the addition or removal of rules in the knowledge base. A number of different representations of human knowledge have been encoded in the form of neural networks, including deterministic finite automata (DFAs) [OG96, MS93], certainty factor rule bases [Mah96] pushdown automata [DGS92] and modified first order logic rule bases [TSN90] Most of these systems translate the domain knowledge into a network topology and then refine the network s biases and weights through standard neural network techniques in order to ....

R. Maclin and J. W. Shavlik. Using knowledge-based neural networks to improve algorithms: Refining the Chou-Fasman algorithm for protein folding. Machine Learning, 11:195--215, 1993.

....we have described KRUST s refinement of a propositional KBSs. This approach has been used iteratively on the Chou Fasman Theory which attempts to predict secondary structure in proteins, Craw Hutton, 1995) The accuracy of KRUST s refined KBs is comparable with KBANN s trained neural network (Maclin Shavlik, 1993). We are currently considering other methods to utilise multiple examples to drive the refinement process. We have also been using KRUST to refine corrupted versions of the student loan KB (Pazzani Brunk, 1991) which has a first order representation. KRUST s method of refinement has been ....

Maclin, R., & Shavlik, J. W. (1993). Using knowledge-based neural networks to improve algorithms: Refining the Chou-Fasman algorithm for protein folding. Machine Learning, 11 (2/3), 195--216.

....the possibility of using adaptive recurrent networks to refine partial knowledge of the grammar to be inferred. Methods for mapping knowledge about finite state transitions into the connection weights have been developed for both first and second order recurrent networks (Omlin Giles, to appear; Maclin Shavlik, 1992; Frasconi et al. 1995) A common characteristic of all these methods is that exact FSA injection requires large connection weights (Omlin Giles, to appear) i.e. dynamical units working in the saturated region of the sigmoidal function. Hence, the advantages of prior knowledge are somewhat ....

Maclin, R. & Shavlik, J. (1992). Using knowledge-based neural networks to improve algorithms: Refining the Chou-Fasman algorithm for protein folding.

....based on different protein sets are hard to compare. This fact started a wave of applications of neural networks to the secondary structure prediction problem (Holley Karplus, 1989; Kneller et al. 1990; Stolorz et al. 1992) sometimes in combination with other methods (Zhang et al. 1992; Maclin Shavlik, 1993). The type of neural network used in most of this work were essentially the same as the one used in the study of Qian and Sejnowski, namely a fully connected perceptron with at most one hidden layer. A very serious problem with these networks is the over fitting caused by the huge number of free ....

....1991) In most previous work the over fitting is dealt with by stopping the training of the network before the error on the training set is at a minimum, see e.g. Qian Sejnowski, 1988; Rost Sander, 1993b) and section 2.3 of this paper. A significant exception is the work of Maclin and Shavlik (Maclin Shavlik, 1993) in which the Chou Fasman method (Chou Fasmann, 1978) was built into a neural network before training. This procedure led to a network with much more structure than the fully connected ones. The most successful application of neural networks to secondary structure prediction is probably the ....

[Article contains additional citation context not shown here]

Maclin, R. & Shavlik, J. (1993). Using knowledge-based neural networks to improve algorithms: Refining the Chou-Fasman algorithm for protein folding. Machine Learning, 11, 195--215.

....network dynamics. When partial symbolic knowledge is encoded into a network in order to improve training, programming as few weights as possible is desirable because it leaves the network with many unbiased adaptable weights. This is important when we use a network for domain theory revision [Maclin and Shavlik 1993; Shavlik 1994; Towell et al. 1990] where the prior knowledge is not only incomplete but may also be incorrect [Giles and Omlin 1993; Omlin and Giles 1996a] Methods for constructing DFAs in recurrent networks where neurons have hard limiting discriminant functions have been proposed [Alon et ....

MACLIN, R., AND SHAVLIK, J. 1993. Using knowledge-based neural networks to improve algorithms: Refining the Chou--Fasman algorithm for protein folding. Mach. Learn. 11, 195--215.

....routes and use them to determine the structure of an interconnection network. The possibility that this technique can be useful has been made more likely by automata rule encoding and extraction methods recently developed for recurrent neural networks (Andrews et al. 1995; Giles Omlin, 1993; Maclin Shavlik, 1993) . 2 Self Routing Interconnection Networks In this section we describe SRINs. The purpose of a SRIN is to allow a set of processors to communicate amongst themselves using a store and forward methodology. For store and forward routing, a message travels along the path towards its destination one ....

Maclin, R. & Shavlik, J. (1993). Using knowledge-based neural networks to improve algorithms: Refining the Chou-Fasman algorithm for protein folding. Machine Learning, 11, 195--215.

....network dynamics. When partial symbolic knowledge is encoded into a network in order to improve training, programming as few weights as possible is desirable because it leaves the network with many unbiased adaptable weights. This is important when a network is used for domain theory revision [19, 27, 30], where the prior knowledge is not only incomplete, but may also be incorrect [13, 22] Methods for constructing DFAs in recurrent networks where neurons have hard limiting discriminant functions have been proposed [1, 18, 21] This paper is concerned with neural network implementations of DFAs ....

R. Maclin and J. Shavlik, "Using knowledge-based neural networks to improve algorithms: Refining the Chou-Fasman Algorithm for Protein Folding," Machine Learning, vol. 11, pp. 195--215, 1993.

....until it finds an amino acid that is unlikely to form an ff helix thus terminating the nucleation site. The same approach is used to identify possible fi strands. The algorithm finally resolves overlaps by predicting the more likely of the two structures and fills gaps with coils. Maclin Shavlik [9] have implemented this algorithm as a theory, referred to here as the Chou Fasman Theory. A window moves along the protein revealing 6 amino acids. This, together with the previous secondary structure, allows the theory to predict the secondary structure for the first amino acid in the window, ....

....predictions from the library. Testing has experimented with various distance metrics and window sizes; the best results for the Chou Fasman domain being achieved with Manhattan distance, weighted library cases, a window size of 19 and post processing to remove short helix and strand sites. FSkbann [9] proposes a knowledge based neural network where the topology of the neural network matches the structure of the graph associated with the initial theory. FSkbann also has a feedback loop to transfer the previous prediction to an input unit. The knowledge based links are given a significant ....

[Article contains additional citation context not shown here]

R. Maclin and J. W. Shavlik. Using knowledge-based neural networks to improve algorithms: Refining the Chou-Fasman algorithm for protein folding. Machine Learning, 11(2/3):195--216, 1993.

....and Sejnowski seemed better than those obtained by previous methods, although tests based on different protein sets are hard to compare. This work started a wave of applications of neural networks to the secondary structure prediction problem [2, 9] sometimes in combination with other methods [11, 5]. Our goal has been to get as good predictions as possible from single sequences, ie, only the amino acid sequence of the considered protein is used as input. This work had three stages. Firstly, individual networks were designed for prediction of the three structures. Due to the use of weight ....

R. Maclin and J. Shavlik, "Using knowledgebased neural networks to improve algorithms: Refining the Chou-Fasman algorithm for protein folding," Machine Learning, vol. 11, pp. 195--215, 1993.

....network dynamics. When partial symbolic knowledge is encoded into a network in order to improve training, programming as few weights as possible is desirable because it leaves the network with many unbiased adaptable weights. This is important when we use a network for domain theory revision [20, 28, 31], where the prior knowledge is not only incomplete, but may also be incorrect [13, 23] Methods for constructing DFAs in recurrent networks where neurons have hard limiting discriminant functions have been proposed [1, 19, 22] This paper is concerned with neural network implementations of DFAs ....

R. Maclin and J. Shavlik, "Using knowledge-based neural networks to improve algorithms: Refining the Chou-Fasman Algorithm for Protein Folding," Machine Learning, vol. 11, pp. 195--215, 1993.

....is used or how long it is used. This can lead to robustness that is noise independent. Finally, with the extraction of knowledge from trained neural networks, the methods dicussed here could potentially be applied to incorporating and refining a priori fuzzy knowledge in recurrent neural networks [31]. The computational capabilities of recurrent neural networks (RNNs) are quite powerful [43] RNNs are also capable of modeling and learning state processes and automata [40] This makes RNNs appropriate tools for modeling and learning many diverse dynamical problems from control to signal ....

R. Maclin and J. Shavlik, "Using knowledge-based neural networks to improve algorithms: Refining the chou-fasman algorithm for protein folding," Machine Learning, vol. 11, pp. 195--215, 1993.

....inasmuch as the number of parameters to be learned explodes. 7.1. 2 Recurrent KBANNs The Knowledge Based Artificial Neural Networks (KBANN) learning system was introduced as a method for mapping hierarchically structured rules into feedforward networks [56] The extension to recurrent network [57][58] is quite straightforward, inasmuch as transition rules for a DFA are used and delayed feedback connections are introduced from the output to the input layer of the network (see fig 7.1.2. Like in the approach described in 7.1.1, the state transition function is supposed to be partially known. In ....

R. Maclin and J. Shawlik, "Using knowledge-based neural networks to improve algorithms: Refining the chou-fasman algorithm for protein folding," Machine Learning, . (to appear).

....it finds an amino acid that is unlikely to form an ff helix thus terminating the nucleation site. The same approach is used to identify possible fi strands. The algorithm finally resolves overlaps by predicting the more likely of the two structures and fills gaps with coils. Maclin Shavlik (Maclin and Shavlik, 1993) have implemented this algorithm as a theory, called here the Chou Fasman Theory. A window moves along the protein revealing 6 amino acids. This, together with the previous secondary structure, allows the theory to predict the secondary structure for the first amino acid in the window, see ....

....a slightly different problem in the same domain; ff helix prediction only. Their dataset is therefore more homogeneous, being dominated by ff helix structures. We note that none of the above approaches has made use of the existing Chou Fasman Algorithm or Theory. In contrast, Maclin Shavlik (Maclin and Shavlik, 1993) propose a knowledge based neural network FSkbann where the topology of the neural network matches the structure of the graph associated with the initial theory. FSkbann also has a feedback loop to transfer the previous prediction to an input unit. FSkbann is closer to our approach since it ....

[Article contains additional citation context not shown here]

Maclin, R. and Shavlik, J. W. (1993). Using knowledge-based neural networks to improve algorithms: Refining the Chou-Fasman algorithm for protein folding. Machine Learning, 11(2/3):195--216.

....involving function approximation, especially when the input data is highly complex with a lot of noise. The backpropagation algorithm has been effective in solving problems including character recognition [LeCun et al. 1989] recognizing spoken words [Lang et.al. 1990] and in biological systems [Maclin,1993]. Pomerleau [1993] has provided an example of a neural control system, ALVINN, which uses an artificial neural network to steer an autonomous vehicle driving at normal speeds on public highways. In this section, I outline a feed forward neural network and the backpropagation (BP) algorithm ....

Maclin, R. & Shavlik, J. (1993) Using knowledge-based neural networks to improve algorithms: Refining the Chou-Fasman algorithm for protein folding. Machine Learning.

.... that the best reported error rate for the digit recognition task is 14.9 , using decision lists [ Shen, 1992 ] For this set of proteinfolding data, the best reported error rates are 37.3 using standard neural networks [ Qian and Sejnowski, 1988 ] 36.6 using a knowledge based neural network [ Maclin and Shavlik, 1993 ] and 30.7 using a case based reasoning algorithm (and a somewhat larger data format) Leng et al. 1994 ] Our results are better than all but the case based reasoning results (which also used a different input encoding) In Figure 5 we report error as a function of the number of networks ....

R. Maclin and J. Shavlik. Using knowledge-based neural networks to improve algorithms: Refining the Chou-Fasman algorithm for protein folding. Machine Learning, 11:195--215, 1993.

....to decompose the problem for the network) Suddarth and Holden s work however only deals with hints in the form of useful output signals, and still requires network learning, while ratle incorporates advice immediately. Our work on ratle is similar to our earlier work with the fskbann system (Maclin Shavlik, 1993). Fskbann uses a type of recurrent neural network introduced by Elman (1990) that maintains information from previous activations using the recurrent network links. Fskbann extends kbann to deal with state units, but it does not create new state units. Similarly, other researchers (Frasconi, ....

Maclin, R., & Shavlik, J. (1993). Using knowledge-based neural networks to improve algorithms: Refining the Chou-Fasman algorithm for protein folding. Machine Learning, 11, 195--215.

.... in press) Our work extends knowledge based neural networks to a new task and shows that domain theories can be supplied incrementally (as opposed to providing the domain theory at the start of the learning task) Our work on ratle is similar to our earlier work with the fskbann system (Maclin Shavlik, 1993). Fskbann uses a type of recurrent neural network introduced by Jordan (1989) and Elman (1990) that maintains information from previous activations using the recurrent network links. Fskbann extends kbann to deal with state units, but it does not create new state units. Similarly, Omlin and Giles ....

Maclin, R. & Shavlik, J. (1993). Using knowledge-based neural networks to improve algorithms: Refining the Chou-Fasman algorithm for protein folding. Machine Learning, 11:195--215.

No context found.

Maclin, R. & Shavlik, J. (to appear). Using knowledge-based neural networks to improve algorithms: Refining the ChouFasman algorithm for protein folding. Machine Learning. Muggleton, S. & King, R. (1991). Predicting protein secondary-structure using inductive logic programming.

....examples needed in RL; but, unlike our approach, they do not allow an observer to provide general advice. Our work, which extends knowledge based neural networks to a new task and shows that domain theories can be supplied piecemeal, is similar to our earlier work with the fskbann system (Maclin Shavlik 1993). Fskbann extended kbann to deal with state units, but it does not create new state units. Gordon and Subramanian (1994) developed a system similar to ours. Their agent accepts high level advice of the form if conditions then achieve goal. It operationalizes these rules using its background ....

Maclin, R., & Shavlik, J. 1993. Using knowledge-based neural networks to improve algorithms. Machine Learning 11:195--215.

....that existing knowledge be expressed in the form of nonrecursive, propositional rules. Subsequent work extended the types of representations supported by Kbann. Our FSkbann algorithm allows domain knowledge expressed as generalized finitestate automata to be mapped into recurrent neural networks [6]. This representation enabled us to consider state dependent domain theories. We tested FSkbann by using it to refine the Chou Fasman algorithm for predicting how proteins fold. The resulting networks outperformed both the Chou Fasman algorithm (a standard in the biological community) as well as ....

R. Maclin and J. Shavlik, "Using knowledge-based neural networks to improve algorithms: Refining the ChouFasman algorithm for protein folding," Machine Learning, vol. 11, no. 2,3, pp. 195--215, 1993.

No context found.

Maclin, R. & Shavlik, J. W. (in press). Using knowledge-based neural networks to improve algorithms: Refining the Chou-Fasman algorithm for protein folding. Machine Learning. Mahoney, J. J. & Mooney, R. J. (1993). Combining neural and symbolic learning to revise probabilistic rule bases.

No context found.

R. Maclin and J. W. Shavlik. Using knowledge-based neural networks to improve algorithms: Refining the Chou-Fasman algorithm for protein folding. Machine Learning, 11:195--215, 1993.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC