D.E. Rumelhart, P. Smolensky, J.L. McClelland, and G. E. Hinton. Schemata and sequential thought processes in PDP models. In D.E. Rumelhart and J. L. McClelland, editors, Parallel Distributed Processing, volume 2, pages 7--57. MIT Press, Cambridge, 1986.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....made: the exceptional story is encoded by strengthening or weakening a large number of constraints. It can be argued that scripts and frames are a rough approximation to the nature of the events they represent. The all or none boundaries they impose on the representation of events are artificial (Rumelhart, Smolensky, McClelland, Hinton, 1986). The true nature of common events is better described as a set of prototypes that develop from exposure to numbers of instances (McClelland and Rumelhart, 1985; and Posner, and Keele, 1968) The instances encode all of the salient details. Prototypes emerge from the correlations between details. ....

Rumelhart, D. E., Smolensky, P., McClelland, J. L., & Hinton, G. E. (1986). Schemata and sequential thought processes in PDP models. In D. E. Rumelhart, J. L. McClelland, and the PDP Research Group (Eds.), Parallel distributed processing: Explorations in the microstructure of cognition, Volume 2. Cambridge, MA.: MIT Press.

....and Pylyshyn 1988) where it is argued that the basic units are symbols handled by rule based processes. On the other hand, the connectionist school argues that we should approach cognition at another level and study how neuronlike elements interact to produce collectively emerging effects (e.g. Rumelhart et al. 1986). We believe that it is possible to unify the symbol processing capabilities of the classical theories to the constraint satisfying capabilities of connectionist theories. We want to show that by developing a high level description of the properties of neural networks it is possible to bridge the ....

....to describe a large set of different neural mechanisms. Generally the state transition functions in F have much faster dynamics than the learning functions in G. We will assume that the state in C is fixed while studying the state transitions in S. Example: In an Interactive Activation network (Rumelhart and McClelland 1986) with four nodes, S is the space [min,max] 4 , C is the space of all 4 4 matrices, and F = f c (x) 1 q)x I(c,x) c C, x S . I i (c,x) c i x i (max x i ) if c i x i 0 and I i (c,x) c i x i (x i min) otherwise. Here the constant 1 q dampens the activation levels of the neurons and I i ....

[Article contains additional citation context not shown here]

Rumelhart, D. E., Smolensky, P., McClelland, J. L. and Hinton, G. E (1986): "Schemata and sequential thought processes in PDP models," in Rumelhart, D.E., Parallel Distributed Processing, Vol 2, 7-57, Cambridge, MA: MIT Press.

....between two words is equal to the difference between an excitatory and an inhibitory relation, the latter being reinforced whenever one word appears without the other. The derivation for this formula is given in [8] w ij = f Delta P (i j) P (i)P (j) Gamma 1 (6) Formula 7 was suggested in [7] for the construction of a constraint satisfaction network that relates typical features to different types of objects. w ij = Gamma ln P ( i j)P (i :j) P (i j)P ( i :j) 7) where P (i :j) is the probability that word i but not j occurs in a window. In formulas 1 to 7 the connective ....

Rumelhart, D. E.; Smolensky, P.; McClelland, J. L.; Hinton, G. E.: Schemata and Sequential Thought Processes in PDP Models. In: Rumelhart, D. E.; McClelland, J. L. (Eds.): Parallel Distributed Processing, Vol. 2. Cambridge, MA: The MIT Press, 1986, 7--57

....the solution to the first problem requires addressing the second problem. Although visual schemas have been extensively studied in the symbolic framework (Draper et al. 1989; Hanson and Riseman 1978) there has been very little work in neural networks in this area (see Arbib 1989; Feldman 1986; Rumelhart et al. 1986 for related approaches) Neural networks are not very good at manipulating symbolic structures explicitly. Instead, they are good at feature extraction, association, constraint satisfaction, pattern classification, and making other fuzzy decisions, based on cooperation and competition among units ....

Rumelhart, D. E., Smolensky, P., McClelland, J. L., and Hinton, G. E. (1986). Schemata and sequential thought processes in PDP models. In McClelland, J. L., and Rumelhart, D. E., editors, Parallel Distributed Processing, Volume 2: Psychological and Biological Models, 7--57. Cambridge, MA: MIT Press.

....whatever reason) one does not want to rely on the evidence of introspection, then it would seem that some sort of talk about spin glasses and the like could be produced to make the possibility of this more academically respectable. 1. 2 The PDP Approach In fact, the theorists of connectionism [16] have come up with such an approach. They start with what they call schemata as emergent features of PDP networks. Schemata are not things . There is no representational object which is a schema. Rather, schemata emerge at the moment they are needed from the interaction of large numbers of ....

D.E. Rumelhart, P. Smolensky, J.L. McClelland, and G.E. Hinton. Schemata and sequential thought processes in PDP models. In James L. McClelland, David E. Rumelhart, and the PDP Research Group, editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition, volume 2, pages 7--57. MIT Press, 1986.

....learning model, were unable to classify groups of concepts that are not linearly separable, back propagation can overcome this problem. Back propagation is a gradient descent method that propagates error signals back through a multi layer network. It has been described in many places (e.g. Rumelhart et al. 1986; Rumelhart and McClelland, 1986) and readers interested in a more detailed description should look there. We will use the decision tree algorithm ID3 (Quinlan, 1986) as the basis for comparison with decision tree algorithms. In addition, we have compared the performance our algorithm to other ....

....unable to classify groups of concepts that are not linearly separable, back propagation can overcome this problem. Back propagation is a gradient descent method that propagates error signals back through a multi layer network. It has been described in many places (e.g. Rumelhart et al. 1986; Rumelhart and McClelland, 1986), and readers interested in a more detailed description should look there. We will use the decision tree algorithm ID3 (Quinlan, 1986) as the basis for comparison with decision tree algorithms. In addition, we have compared the performance our algorithm to other methods used on the same data for ....

Rumelhart, D., Smolensky, P., McClelland, J., and Hinton, G. (1986) Schemata and sequential thought processes in PDP models. In Parallel Distributed Processing: explorations in the microstructure of cognition, vol II, J. McClelland, D. Rumelhart, and the PDP Research Group (Eds.), 7-57. Cambridge, MA: MIT Press.

....reasoning models. The TEMPORAL REASONING AND REASONING THEORIES A CASE STUDY IN ANAESTHESIOLOGY 8 second is connectionist and postulates a distributed activation of simple units which, once stable, tend to form a schema which will be activated by a system input pattern (e.g. Rumelhart 1980, Rumelhart, Smolensky, McClelland Hinton 1986). This position is represented within artificial intelligence by the field of artificial neural networks. The third position asserts that reasoning is ensured by domain specific rules (see Cheng Holyoak 1985, Holyoak Thagard 1989 and Smith, Langston Nisbett 1992) Specialized production rule ....

Rumelhart, D. E., Smolensky, P., Mc Clelland, J. L. & Hinton, G. E. (1986). Schemata and Sequential Thought Processes in PDP Models. In Mc Clelland, J. L., Rumelhart, D. E. & The PDP Research Group. Parallel Distributed Processing. Vol 2. Cambridge: MIT Press.

....significant one. It is possible that connectionism, when interpreted simply as an inductive method, will force a re evaluation of this claim. 2.2 Connectionism s relationship with nativism There are currently two common views on connectionism s relationship with nativism. The first arises from Rumelhart and McClelland s (1986a) solution to the perceived problem of connectionism, because of its use of terms such as associative learning , training and inhibition , leaning towards the behaviourist camp. If behaviourism is essentially empirical, connectionism will not then be able to cope with various criticisms ....

....is a need for some degree of nativism by connectionists on the issue of categorial and constituent representational structure. 3. 1 The issue of categories and constituents When discussing distributed representations, especially on how to represent constituent structure, Hinton, McClelland and Rumelhart (1986) state that there are two different kinds of hierarchy which need to be represented by researchers planning to implement the sort of conceptual structures used by people: the ISA hierarchy which relates types to instances, and the part whole hierarchy which relates items to the constituent ....

[Article contains additional citation context not shown here]

Rumelhart, D. E., Smolensky, P., McClelland, J. L. and Hinton, G. E. (1986) Schemata and sequential thought processes in PDP models, in D.E. Rumelhart, J.L. McClelland, and the PDP Research Group, Parallel Distributed Processing: Explorations in the Microstructure of Cognition.

....computation namely connectionism. Difficult tasks, such as arithmetic, need to be turned into pattern matching problems. That is, we succeed in solving logical problems not so much through the use of logic, but by making the problems we wish to solve conform to problems we are good at solving (Rumelhart, Smolensky, McClelland Hinton 1986, p. 44) Exactly how this is done for arithmetic is the topic of this thesis. Two elementary arithmetic skills are considered: adult memory for multiplication facts and children s errors in long (multicolumn) multiplication. 1.1 Part I Mental arithmetic The first part of the thesis considers ....

....is support for the idea that connectionist systems are the appropriate tool for capturing developmental phenomena. 3. More generally, connectionist system exhibit mind like properties, such as automatic generalization and graceful degradation. Connectionist computation is brain style computation (Rumelhart McClelland 1986b) 4. Theory and implementation are never as independent as one would wish. Without an alternative for comparison there is a danger that Sierra is unduly biased by symbolic AI. 5. Connectionism is changing our understanding of notions like symbol . Chapter 5 gives details on the construction of ....

[Article contains additional citation context not shown here]

Rumelhart, D. E., Smolensky, P., McClelland, J. L. & Hinton, G. E. (1986). Schemata and sequential thought processes in PDP models. In Rumelhart, D. E. & McClelland, J. L., eds, Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. 2: Psychological and biological models. MIT Press, Cambridge, MA.

.... temporal difference learning (Sutton, 1988) and to AI methods for planning and search (Korf, 1990) Werbos (1987) has previously argued for the general idea of building AI systems that approximate dynamic programming, and Whitehead (1989) and others (Sutton Barto, 1981; Sutton Pinette, 1985; Rumelhart et al. 1986) have presented results for the specific idea of augmenting a reinforcement learning system with a world model used for planning. 2 Dyna PI: Dyna by Approximating Policy Iteration I call the first Dyna architecture Dyna PI because it is based on approximating a DP method known as policy ....

Rumelhart, D. E., Smolensky, P., McClelland, J. L., & Hinton, G. E. (1986) Schemata and sequential thought processes in PDP models. In Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Volume II, by J. L. McClelland, D. E. Rumelhart, and the PDP research group, 7--57. MIT Press, Cambridge, MA.

....when learning, organizing and forming generalised representations of the input, which makes them attractive as models of the gradual process of language development (Bates Elman, 1993) 6.3. 1 Implicit Schemata Neural networks are also natural candidates for modelling schematic phenomena (see Rumelhart, Smolensky, McClelland Hinton, 1986, for a detailed discussion) In their connectionist guise, schemata can be thought of as emergent properties of the interactions between processing units in a constraint satisfaction network. An individual unit represents the hypothesis that a certain feature is present in the input (Rumelhart ....

....Hinton, 1986, for a detailed discussion) In their connectionist guise, schemata can be thought of as emergent properties of the interactions between processing units in a constraint satisfaction network. An individual unit represents the hypothesis that a certain feature is present in the input (Rumelhart et al. 1986). Links between units are best thought of as statistically encoding the patterns of co occurrence of items in the input. The strengths of these links are continually adjusted, in order to maintain maximal consistency with the range of inputs. When presented with a novel input pattern, or part of a ....

Rumelhart, D. E., Smolensky, P., McClelland, J. L. & Hinton, G. E. (1986). Schemata and sequential thought processes in PDP models. In D. E. Rumelhart & J. L. McClelland (Eds.) Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. 2, Cambridge, MA: MIT Press.

....differs in certain respects from our own. There have been a number of further developments of the idea (Kawato, 1990; Miyata, 1988; Munro, 1987; Nguyen Widrow, 1989; Robinson Fallside, 1989; Schmidhuber, 1990) based either on the work of Werbos or our own unpublished work (Jordan, 1983; Rumelhart, 1986). There are also close ties between our approach and techniques in optimal control theory (Kirk, 1970) and adaptive control theory (Goodwin Sin, 1984; Narendra Parthasarathy, 1990) We discuss several of these relationships in the remainder of the paper, although we do not attempt to be ....

....1 2 (y Gamma y) T (y Gamma y) 5) J is the sum of squared error at the output units of the network) It is generally desired to minimize this cost. Backpropagation is an algorithm for computing gradients of the cost functional. The details of the algorithm can be found elsewhere (e.g. Rumelhart, et al. 1986); our intention here is to develop a simple notation that hides the details. This is achieved formally by using the chain rule to differentiate J with respect to the weight vector w: rwJ = Gamma y w T (y Gamma y) 6) This equation shows that any algorithm that computes the gradient ....

[Article contains additional citation context not shown here]

Rumelhart, D. E., Smolensky, P., McClelland, J. L. & Hinton, G. E. (1986). Schemata and sequential thought processes in PDP models. In D. E. Rumelhart & J. L. McClelland (Eds.), Parallel distributed processing: Volume 2, 7-57.

No context found.

D.E. Rumelhart, P. Smolensky, J.L. McClelland, and G. E. Hinton. Schemata and sequential thought processes in PDP models. In D.E. Rumelhart and J. L. McClelland, editors, Parallel Distributed Processing, volume 2, pages 7--57. MIT Press, Cambridge, 1986.

No context found.

D.E. Rumelhart, P. Smolensky, J. L. McClelland, G. E. Hinton, (1986), Schemata and sequential thought processes in PDP models. In D.E. Rumelhart, J. L. McClelland Eds. Parallel distributed processing: explorations in the microstructure of cognition, Vol. 2: Psychological and biological models, MIT Press, Cambridge MA.

No context found.

D.E. Rumelhart, P. Smolensky, J.L. McClelland, and G. E. Hinton. Schemata and sequential thought processes in PDP models. In D.E. Rumelhart and J. L. McClelland, editors, Parallel Distributed Processing, volume 2, pages 7--57. MIT Press, Cambridge, 1986.

No context found.

Rumelhart D. E., Smolensky P., McClelland J. L. and Hinton G. E.: Schemata and Sequential Thought Processes in PDP Models, in: D.E. Rumelhart, J. L. McClelland, et al., Parallel Distributed Processing: Explorations in The Microstructure of Cognition, Vol. 2: Psychological and Biological Models, Cambridge: M.I.T. Press, pp. 7-58, 1986.

No context found.

Rumelhart, D. E., Smolensky, P., McClelland, J. L., and Hinton, G. E. (1986b). Schemata and sequential thought processes in PDP models. In Rumelhart, D. E., McClelland, J. L., and the PDP Research Group, editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition, chapter 14. MIT Press, Cambridge.

No context found.

Rumelhart, D.E., P. Smolensy, J.L. McClelland, and G. Hinton. (1986). Schemata and sequential thought processes in PDP models. In D.E. Rumelhart and J.L McClelland (eds.), Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. 2: Psychological and Biological Models. Cambridge, MA: MIT Press, 7-57.

No context found.

Rumelhart, D. E., Smolensky, P., McClelland, J. L., and Hinton, G. E. (1986). Schemata and sequential thought processes in PDP models. In McClelland, J. L., and Rumelhart, D. E., editors, Parallel Distributed Processing, Volume 2: Psychological and Biological Models, 7--57. Cambridge, MA: MIT Press.

No context found.

D. E. Rumelhart, P. Smolensky, J. L. McClelland, and G. E. Hinton. Schemata and sequential thought processes in PDP models. In Parallel Distributed Processing: Explorations in the Microstructures of Cognition, Volume 2: Psychological and Biological Models, chapter 14. MIT Press, 1986.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC