McClelland, J. L. and Kawamoto, A. H. Mechanisms of Sentence Processing: Assigning Roles to Constituents. In J. L. McClelland, D. E. Rumelhart, and the PDP Research Group, Parallel Distributed Processing, vol. 2. The MIT Press, Cambridge, Massachusetts, 1986.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....damage, and incomplete or conflicting input [Miikkulainen, 1993; St. John and McClelland, 1990] Second, because computation in these networks is constraint based, the subsymbolic approach naturally combines syntactic, semantic, and thematic constraints on the interpretation of linguistic data [McClelland and Kawamoto, 1986] . Third, subsymbolic systems can be lesioned in various ways and the resulting behavior is often strikingly similar to human impairments [Miikkulainen, 1993; 1996; Plaut, 1991] These properties of subsymbolic systems have attracted many researchers in the hope of accounting for interesting ....

James L. McClelland and Alan H. Kawamoto. Mechanisms of sentence processing: Assigning roles to constituents. In James L. McClelland and David E. Rumelhart, editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Volume 2: Psychological and Biological Models, pages 272--325. MIT Press, Cambridge, MA, 1986.

.... to be distributed, in a sense that one concept is represented along many units of the connectionist architecture [9] 14] while localist representations lack semantic distinctiveness [5] 17] Natural language processing systems that use distributed representations have shown good performance [11] [21] 22] 19] Here, a connectionist NLP system called Bio R is presented to account for thematic role relationships in natural language sentences. The architecture employed is a bi directional (recurrent) artificial neural network. The processors are perceptron like units and the ....

.... The processors are perceptron like units and the connectionist learning algorithm uses a simple reinforcement rule, based only on available information of local synapses [13] The words are presented sequentially to the network and represented by means of distributed semantic microfeature arrays [11] [24] Twenty three valued logic semantic microfeature units account for each noun and verb. The schema on table 1 displays the semantic features for verbs. Table 2 shows the microfeatures for nouns. control of action no control of action direct process triggering indirect triggering direction ....

[Article contains additional citation context not shown here]

J. L. McClelland, and A. H. Kawamoto, "Mechanisms of Sentence Processing: Assigning Roles to Constituents of Sentences", in J. L. McClelland and D. E. Rumelhart (Eds.), Parallel Distributed Processing -- Explorations in the Microstructure of Cognition -- Volume 2: Psychological and Biological Models. A Bradford Book, MIT Press, 1986, pp. 272-325.

....conncctionist net and at a higher level as a set of soft rules) and, finally, to test the explanatory adequacy of the formalism by using it to characterize universal properties of the grammars of human languages. By contrast, the most typical approach to applying connectionism to language [1, 24, 37, 38, 39, 42, 43, 48, 49, 58] is to identify some linguistic phenomenon of interest, construct specific data sets thzt exhibit it, train and test some particular connectionist network on these data, and then try to draw more general linguistic conclusions that go beyond these particular data. Thus ....

James L McClelland and Alan II. Kawamoto. Mechanisms of sentence processing: Assigning roles to constituents. In David E. Rumelhart, James L. McClelland, and the PDP Research Group, editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Volume 2: Psychological and Biological Models, chapter 19, pages 272-325. MIT Press/Bradford Books, Cambridge, MA, 1986.

....for a ball, and BENEFICIARY for the girl. In a system called HTRP [9] designed to reveal the thematic grid of semantically sound sentences, individual words are represented by means of semantic features. Verbs, specially, are represented as a three valued array of semantic microfeatures ( 14] [8]) which are based on relevant features in a thematic frame [2] Semantic microfeatures are arranged in sub arrays along certain semantic dimensions. For verbs, each semantic dimension encompasses two elements e.g. control of action and no control of action and, for thematically unambiguous ....

....the input sentences. As soon as the training is over, symbolic rules can be obtained from the connectionist architecture by running an extraction procedure ( 3] 12] 13] 3 Verb Microfeatures and Complementarity in HTRP The representations used by HTRP are based on McClelland and Kawamoto s [8] and Waltz and Pollack s [14] notion of semantic microfeature. For the verb, the representation is mainly derived from Franchi and Canado [2] Twenty binary semantic microfeature units take care of each noun or verb. For verbs, pairs of microfeatures are grouped together into ten different ....

McClelland, J. L. and Kawamoto, A. H.: Mechanisms of Sentence Processing: Assigning Roles to Constituents of Sentences. In J. L. McClelland, D. E. Rumelhart (Eds.), Parallel Distributed Processing, Volume 2. A Bradford Book, The MIT Press (1986)

....grid, share a common semantic core. Assuming this is regular, one could try to capture such regularity (a) by describing each word in terms of its semantic features, and (b) by generalizing over all such descriptions for each thematic slot. Semantic feature generalization is the hallmark of McClelland and Kawamoto s [1986] pioneering proposal, and of much subsequent work. In a system called CPPro [Rosa, 1997] a connectionist architecture based on an adaptation of McClelland and Kawamoto s [1986] model is proposed. The words are represented by arrays of semantic microfeatures, formed by subsets accounting for ....

....by generalizing over all such descriptions for each thematic slot. Semantic feature generalization is the hallmark of McClelland and Kawamoto s [1986] pioneering proposal, and of much subsequent work. In a system called CPPro [Rosa, 1997] a connectionist architecture based on an adaptation of McClelland and Kawamoto s [1986] model is proposed. The words are represented by arrays of semantic microfeatures, formed by subsets accounting for aspects of word meaning, like human and non human , where only one value in each subset is active. For the verb, these arrays are arranged on the basis of thematic relationships ....

[Article contains additional citation context not shown here]

J. L. McClelland and A. H. Kawamoto. Mechanisms of Sentence Processing: Assigning Roles to Constituents of Sentences. In Parallel Distributed Processing, Vol. 2 . Ed. by J. L. McClelland and D. E. Rumelhart . A Bradford Book, MIT Press, 1986.

....mental states in terms of what they represent. For a neurological version of eliminativism, see P.S. Churchland, 1986; for a behavioral version, see Watson, 1930; for a syntactic version, see Stich, 1983) Connectionists are on the Representationalist side of this issue. As Rumelhart McClelland (1986a) say, PDPs are explicitly concerned with the problem of internal representation (p 121) Correspondingly, the specification of what the states of a network represent is an essential part of a Connectionist model. Consider, for example, the well known Connectionist account of the bistability of ....

....therefore presumably not representational. But this is misleading: Connectionist modeling is consistently Representationalist in practice, and Representationalism is generally endorsed by the very theorists who also like the idea of cognition emerging from the sub symbolic . Thus, Rumelhart McClelland (1986a) insist that PDP models are . strongly committed to the study of representation and process (p. 121) Similarly, though Smolensky (1988) takes Connectionism to articulate regularities at the sub symbolic level of analysis, it turns out that sub symbolic states do have a semantics, though ....

[Article contains additional citation context not shown here]

McClelland, J.L. & Kawamoto, A.H. (1986) Mechanisms of sentence processing: Assigning roles to constituents. In McClelland, Rumelhart and the PDP Research Group, (eds.) Parallel Distributed Processing: Volume 2, Cambridge, Mass. M.I.T. Press, A Bradford Book.

.... some complete di#erent approaches to connectionism for natural language processing that are not based on syntactic analysis, but rather use techniques like strongly interactive distributed processing of word senses, case roles, and semantic markers (from [odAANL89] see e.g. CS84] WP85] and [MK86] Obviously, these approaches are outside the scope of this report. 4.5 Reducing the Parsing Problem to Other Problems 4.5.1 Parsing as Matrix Multiplication [Tho94] rewords the fact that a basic step in a tabular chart parser is functionally equivalent to boolean matrix multiplication. ....

James L. McClelland and A. H. Kawamoto. Mechanisms of sentence processing: assigning roles to constituents of sentences. In David E. Rumelhart James L. McClelland and the PDP Research Group, editors, Parallel Distributed Processing, volume 2, chapter 19, pages 272--325. MIT Press, Cambridge, Ma., 1986. 38

....architecture, and strategy As previously mentioned, our system involves two major phases a training phase and a test phase. Both phases assume the prior existence of representational structure within the output layer (see Fig. 2) Thus, we have assumed (as others have, including Pinker, 1984; McClelland and Kawamoto, 1986; St. John and McClelland, 1990) that the agent possesses at least a primitive ability to conceptualize external situations. In our model, this ability involves concept nodes (that can represent objects or actions) and role nodes (that can represent agent, action, or patient roles) During the ....

McClelland, J. L., & Kawamoto, A. H. (1986). Mechanisms of sentence processing: assigning roles to constituents of sentences. In D. E. Rumelhart & J. L. McClelland & PDP Research Group (Eds.), Parallel Distributed Processing; Explorations in the Microstructure of Cognition, 2. Cambridge, MA: MIT Press.

....has a unique representation in the resulting vectors, and similar words have similar representations. 3.2 Concept representations The semantic concept representations stand for distinct meanings in the language. Although it is possible to encode meanings by hand as feature vectors (see e.g. McClelland and Kawamoto 1986), as was done for symbols, it is difficult to decide what the appropriate semantic features should be. With the FGREP mechanism (Miikkulainen 1993; Miikkulainen and Dyer 1991) it is possible to derive a distributed encoding automatically, based on examples of how the words are used in the ....

....items that are used in similar ways in the training examples become similar, and in this sense, FGREP representations can be claimed to stand for the meanings of the input items. The semantic representations for DISLEX were formed with an FGREP network in the sentence case role assignment task of McClelland and Kawamoto (1986). A number of sentence examples were generated based on a set of templates and semantic categories (listed in appendix C) Only concepts that represented unique meanings among the lexical symbols were used. For example, MAN, WOMAN, BOY, and GIRL were used exactly the same way in the data, and ....

[Article contains additional citation context not shown here]

McClelland, J. L., and Kawamoto, A. H. 1986. Mechanisms of sentence processing: Assigning roles to constituents. In McClelland, J. L., and Rumelhart, D. E. (Eds.), Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Volume 2: Psychological and Biological Models, 272--325. Cambridge, MA: MIT Press.

....(1985) and more recently Bookman(1994) Still in these works a concept is given a fixed definition. A context, though coded and detected distributedly, is used to prime one of the preexisting senses of a concept. Subsymbolic models are potentially capable of dealing with this problem. In fact McClelland and Kawamoto (1986) presented their model with The ball broke the vase and though ball in the previous examples has been always associated with softness, the model strongly activated hard instead of soft. They attributed this to the fact that breakers in their model all were hard and the model has shaded the meaning ....

McClelland, J.L. and Kawamoto, A.H. (1986). Mechanisms of Sentence Processing: Assigning Roles to Constituents of Sentences. In Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Edited by McClelland J.L., Rumelhaurt, D.E. and the PDP Research Group, Vol. 2. MIT press.

....in which several sets of input units are used to represent the input to the network at different points in time. However, this arrangement has the disadvantage that it arbitrarily limits the length of input sequence, so it is not an ideal solution. One feedforward model is Rumelhart and McClelland s (1986) model of past tense learning for English verbs (see figure 2.2) They use a four layer network that takes a phonological representation of the root form of a verb as its input and produces a phonological representation of the past tense as its output. The two interior layers of the network ....

....Wickelphone layers by a pair of fixed weight coding networks, and the two Wickelphone layers are connected by an adjustable weight associative network. During training, the root form is presented to the input layer at the same time that the past tense is presented to 7 Figure 2.2. Rumelhart and McClelland s (1986) model of past tense acquisition. Fixed Encoding Network Modifiable Association Network Fixed Decoding Network Phonological representation of root form Wickelfeature representation of root form Phonological representation of past tense Wickelfeature representation of past ....

[Article contains additional citation context not shown here]

McClelland, James L., and Alan H. Kawamoto (1986). Mechanisms of sentence processing: Assigning roles to constituents of sentences. In Rumelhart, McClelland, and the PDP Research Group (1986b), 272-325.

....often thought, symboloids in real cognitive systems must somehow reflect their meanings in their formal configuration more directly. Some connectionist compositional schemes do utilize semantically non arbitrary symboloids. For example, in the McClelland and Kawamoto model of case role assignment (McClelland and Kawamoto 1986), input sentences are built out of vectors corresponding to distinct words, and those vectors themselves are essentially just lists indicating the presence or absence of microfeatures in the designated objects. Therefore, since the actual vector that is used to represent a given object (e.g. ....

McClelland, J. L.and A. H. Kawamoto (1986). Mechanisms of sentence processing: Assigning roles to constituents of sentences. In McClelland and Rumelhart (1986), pp. 272-326.

....An example of strong systematicity would be to correctly infer that John is the lover of Mary , given the premise John loves Mary , having only been trained on examples where John appeared in the object position of the binary relation loves . Hadley (1993, 1994) found that the models of McClelland and Kawamoto (1986), Chalmers (1990) Elman (1989, 1990, 1991) Smolensky (1990) and St. John and McClelland (1990) did not demonstrate strong systematicity. Essentially, by a statistical analysis of training sets, in all likelihood all components appeared in all of their allowable positions. This result raises ....

....question John does what to Mary . Again, by the property of systematicity, one can also infer dogs from the statement John dogs 5 Mary and the same question. But, in the second case, the same surface representation (or, physical stimulus) 4 See also Hadley (1994) for a similar comment on McClelland and Kawamoto (1986). 5 Chases relentlessly. 7 is supposed to identify a different structural role (i.e. verb) Of course, the reason such inferences are possible is because of the surrounding context. Novel input may derive similar internal representations from context, but they cannot be relied upon to possess ....

McClelland, J. L., & Kawamoto, A. H. (1986). Mechanisms of Sentence Processing: Assigning roles to Constituents of Sentences, Vol. 2 of Computational models of cognition and perception, chap. 19. Cambridge, MA: MIT Press.

....and Pylyshyn 1988) where it is argued that the basic units are symbols manipulated by symbolic processes. On the other hand, the connectionist school argues that we should approach cognition at another level and study how neuronlike elements interact to produce collectively emerging effects (e.g. Rumelhart et al. 1986). I believe that it is possible to marry the symbol processing capabilities of the classical theories to the constraint satisfying capabilities of the connectionist theories. I would like to show that by striving for a high level description of the properties of neural networks it is possible to ....

....networks on levels above the neurons (cf. Balkenius 1990a) Even though the concept seems to have as many definitions as authors, some common core exists in all of them. I will use the term schema as a collective name of the structures as used by Piaget (1952, 1973) Arbib and Hanson (1987) and Rumelhart et al. 1986). I would also like to include concepts usually denoted by other names such as Frames (Minsky 1987) Scripts (Schank Abelsson 1977) etc. Among the different authors we find some common properties of schemata. S1) Schemata are used for the representation of objects, situations, and actions. ....

Rumelhart, D. E. and Kawamoto, A. H. 1986. "Mechanisms of sentence processing: assigning roles to constituent of sentences". in Rumelhart, D. E., Parallel distributed processing , Vol 2, pp. 272-325, MIT Press.

....by noun phrases. The basic mapping problem is to decide which sentence constituents fill which roles. Though case analysis is only a part of the overall task of sentence interpretation, the problem is nontrivial even in simple sentences. Consider these sentence case analysis examples given by McClelland and Kawamoto (1986): 1. The boy hit the window. hit agt:boy pat:window] 2. The hammer hit the window. hit inst:hammer pat:window] 3. The hammer moved. moved pat:hammer] 4. The boy ate the pasta with the cheese. ate agt:boy pat:pasta accomp:cheese] 5. The boy ate the pasta with the fork. ate agt:boy pat:pasta ....

....of a complete database query application. The following subsections discuss several of these experiments. 5. 1 Case Role Mapping Chill was first tested on artificial data for case role mapping previously used to demonstrate certain language processing abilities of artificial neural networks (McClelland Kawamoto, 1986; Miikkulainen Dyer, 1991) The corpus consists of 1475 sentence case structure pairs produced from a set of 19 sentence templates such as The HUMAN ate the FOOD with the UTENSIL , where the capitalized items are replaced with words of the given category. The corpus actually contains 1390 ....

McClelland, J. L., & Kawamoto, A. H. (1986). Mechanisms of sentence processing: Assigning roles to constituents of sentences. In Rumelhart, D. E., & McClelland, J. L. (Eds.), Parallel Distributed Processing, Vol. II, pp. 318--362. MIT Press, Cambridge, MA.

.... (hammer, obj,type:hammer] hit, propel] moved, ptrans] fork, inst,type:fork] cheese, food,type:cheese] bat, animal,type:flying bat] bat, obj,type:baseball bat] 5 Experimental Results Our system has been tested on a corpus based on that of McClelland and Kawamoto (1986). This corpus consists of 1475 sentence case structure pairs, artificially produced from a set of 19 sentence templates. The case structure portion of these pairs was modified to produce deeper semantic representations. Examples of sentences and their representation are given in the small corpus ....

McClelland, J. L., & Kawamoto, A. H. (1986). Mechanisms of sentence processing: Assigning roles to constituents of sentences. In Rumelhart, D. E., & McClelland, J. L. (Eds.), Parallel Distributed Processing, Vol. II, pp. 318--362. MIT Press, Cambridge, MA.

....: p m g; r i ) 2 C. 2 The problem can be restated less formally as follows. The first thing available to the learner is a set of sentences paired with their meanings. There are two motivations for making these pairs available. First is the pragmatic motivation. Corpora such as those used by McClelland and Kawamoto (1986) and Kay and Roescheisen (1993) which pair natural language sentences with some representation of their meaning are already widely available. Second is the cognitive motivation. When learning language, children have access to spoken sentences together with some sensory input, part of which ....

....is that useful and correct meaning representations can be learned by Wolfie. One way to test this is by comparing the learned lexicons to correct lexicons. Another way to test this is to use the learned lexicons to assist a larger learning system. The first corpus used is based on that of McClelland and Kawamoto (1986). This corpus consists of 1475 sentence case structure pairs, artificially produced from a set of 19 sentence templates. Only the case structure portion of these pairs was modified as discussed in Section 4 to produce deeper semantic representations. The resulting corpus is hereafter referred to ....

McClelland, J. L., & Kawamoto, A. H. (1986). Mechanisms of sentence processing: Assigning roles to constituents of sentences. In Rumelhart, D. E., & McClelland, J. L.

....determines the input lines with the most variance among its input vectors, and passes a compressed vector down to its submap for a more detailed mapping. Input representations in a cognitive system often have some discrete structure, such as e.g. role specific assemblies (Hinton, 1981; McClelland and Kawamoto, 1986; Miikkulainen and Dyer, 1989a) If the data is hierarchical, the items belonging to the same category have a number of componets in common. These components can be dropped from the input to the next level map (figure 5) The higher level map acts as a filter, 1) choosing the relevant items for ....

McClelland, J. L. and Kawamoto, A. H. (1986). Mechanisms of sentence processing: Assigning roles to constituents. In McClelland, J. L. and Rumelhart, D. E., editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Volume 2: Psychological and Biological Models. Cambridge, MA: MIT Press.

....way to understand the parser acquisition process in Chill is to consider a concrete example. This chapter details the operation of Chill by way of a simple case role mapping problem that has been previously used to demonstrate certain language processing abilities of artificial neural networks (McClelland Kawamoto, 1986; Miikkulainen Dyer, 1991; Miikkulainen, 1993) 3.1 Overview As described in the introduction, Chill treats parser acquisition as a control rule learning problem within a logic program that implements a suitable parser. The parsers learned by Chill are based on basic a mechanism known as ....

....by noun phrases. The basic mapping problem is to decide which sentence constituents fill which roles. Though case analysis is only a part of the overall task of sentence interpretation, the problem is nontrivial even in simple sentences. Consider these sentence case analysis examples from McClelland and Kawamoto (1986): 1. The boy hit the window. hit agt:boy pat:window] 2. The hammer hit the window. hit inst:hammer pat:window] 3. The hammer moved. moved pat:hammer] 4. The boy ate the pasta with the cheese. ate agt:boy pat: pasta accomp:cheese] 5. The boy ate the pasta with the fork. ate agt:boy ....

[Article contains additional citation context not shown here]

McClelland, J. L., & Kawamoto, A. H. (1986). Mechanisms of sentence processing: Assigning roles to constituents of sentences. In Rumelhart, D. E., & McClelland, J. L.

....Net Storing a Single Pointer 4 Recursive Auto Associative Memory and Grammar A number of connectionist processors have used nets as distributed representations of structure, and such networks can be viewed as encoding the thematic roles of propositions. Early versions of the idea such as McClelland and Kawamoto (1986) were non recursive, but Pollack (1990) showed how recursively embedded structure could be built in such rule like nets, in an architecture called the Recursive Auto Associative Memory (RAAM) This was a more efficient version of an even simpler device called the Associative Net (Willshaw et al. ....

McClelland, James and Kawamoto, Alan, 1986. "Mechanisms of Sentence Processing: Assigning Roles to Constituents." In James McClelland, David Rumelhart, and the PDP REesearch Group (eds.), Parallel Distributed Processing, Cambridge MA: MIT Press.

....Another feature of Selman s system is that the connectionist constraint of limited processing cycles is ignored and a parse may take several thousand cycles of annealing. And even the newer crop of research in this area suffers from the same fixed width problem (Allen, 1987; Hanson Kegl, 1987; McClelland Kawamoto, 1986). 4.5.1.2. Representational Adequacy Closely related to the problem of generative capacity is the problem of representational adequacy. One must be careful that a model being proposed can actually represent the elements of the domain being modeled. One of the major attacks on connectionism has ....

....and the representation of an element of the structure (by features) this type of system cannot represent recursive propositions such as John saw the nurse riding an elephant . Finally, parallel representations of sequences which use implicit sequential coding (such as Rumelhart and McClelland (1986) used in their perceptron model for learning the past tenses of verbs) have limits representing repetitive constituents. So a system, for example, which represented words as as collections of letter triples, would not be able to represent words with duplicate triples such as Banana. 4.5.1.3. Task ....

[Article contains additional citation context not shown here]

McClelland, J. & Kawamoto, A. (1986). Mechanisms of Sentence Processing: Assigning Roles to Constituents. In J. L. McClelland, D. E. Rumelhart & the PDP research Group, (Eds.), Parallel Distributed Processing: Experiments in the Microstructure of Cognition, Vol. 2. Cambridge: MIT Press.

....is a rule, the denotes a conjunctive condition, and the denotes a wildcard symbol. A disjunction is denoted by . This formal string manipulation task can be viewed as an abstraction of several interesting cognitive models in the connectionist literature, including case role assignment (McClelland Kawamoto, 1986), grapheme phoneme mapping (Sejnowski Rosenberg, 1987) and mapping verb stems to the past tense (Rumelhart McClelland, 1986) 3 TASK RuleNet s task is to induce a compact set of rules that accurately characterizes a set of training examples. We generate training examples using a predefined ....

.... manipulation task can be viewed as an abstraction of several interesting cognitive models in the connectionist literature, including case role assignment (McClelland Kawamoto, 1986) grapheme phoneme mapping (Sejnowski Rosenberg, 1987) and mapping verb stems to the past tense (Rumelhart McClelland, 1986). 3 TASK RuleNet s task is to induce a compact set of rules that accurately characterizes a set of training examples. We generate training examples using a predefined rule base. The rules are over strings of length four and alphabets which are subsets of a, b, c, d, e, f, g, h, i, j, k, l . ....

[Article contains additional citation context not shown here]

McClelland, J. & Kawamoto, A. (1986). Mechanisms of sentence processing: assigning roles to constituents.

....analyses such as case role assignments or logical forms is significantly more useful for natural language processing than learning a syntactic recognizer or grammar. There has been some research using both neural networks and symbolic induction to learn parsers that produce case role analyses (McClelland Kawamoto 1986; Miikkulainen 1993; St. John McClelland 1990; Zelle Mooney 1993; Miikkulainen 1996) Even more useful is a parser that can map natural language queries into a logical form or a database query language (e.g. SQL) that can be immediately executed to retrieve an answer to the question. ....

McClelland, J. L., and Kawamoto, A. H. 1986. Mechanisms of sentence processing: Assigning roles to constituents of sentences. In Rumelhart, D. E., and McClelland, J. L., eds., Parallel Distributed Processing, Vol. II. Cambridge, MA: MIT Press. 318--362.

.... in general one could start with [25, 51] Architectural issues of connectionist and hybrid connectionist systems are discussed in [68, 22, 23, 40, 61, 3, 21, 52, 19, 56, 80] Some representative references for semantic and syntactic analysis with connectionist networks can be found in [38, 50, 60, 75, 70, 79]. For references on cognitively oriented connectionist natural language processing some references are [14, 78, 69, 42, 12] 3 Statistical Approaches 3.1 Introduction With the recent trend for learning in natural language processing, statistical methods have gained new popularity, and are being ....

J. L. McClelland and A. H. Kawamoto. Mechanisms of sentence processing: assigning roles to constituents. In J. L. McClelland and D. E. Rumelhart, editors, Parallel Distributed Processing, volume 2, pages 272--326. MIT Press, Cambridge, MA, 1986.

....lexical category of the verb with respect to aspectual properties, and temporal and modal sentential adverbs. There is quite a number of work which is related to the present one. Feature based syntactic representations for pattern association with semantic representations have been used by [McClelland and Kawamoto, 1986] and [John and McClelland, 1990] Their models are used to deal with the association between syntactic cases and semantic roles. The semantic representations achieved were not interfaced with any other module. A combined interpretation and generation approach has been used by [Miikkulainen, 1990] ....

J. McClelland and A. Kawamoto. Mechanisms of sentence processing: Assigning roles to constituents. In D. E. Rumelhart and J. L. McClelland, editors, Parallel distributed processing: Explorations in the microstructure of cognition, pages 77--109. Cambridge, MA: MIT Press, 1986.

....object) lexical category of the verb with respect to aspectual properties, and temporal and modal sentential adverbs. There is quite a number of work which is related to the present one. Featurebased syntactic representations for pattern association with semantic representations have been used by [McClelland and Kawamoto, 1986] and [St. John and McClelland, 1990] Their models are used to deal with the association between syntactic cases and semantic roles. The semantic representations achieved were not interfaced with any other module. A combined interpretation and generation approach has been used by [Miikkulainen, ....

J. L. McClelland and A. Kawamoto. Mechanisms of sentence processing: Assigning roles to constituents. In D. E. Rumelhart and J. L. McClelland, editors, Parallel distributed processing: Explorations in the microstructure of cognition, pages 77--109. Cambridge, MA: MIT Press, 1986.

....fragment of text. The same view can be found in the localist connectionism (e.g. 2] Another kind of approaches are based mainly on statistical methods explicit, within symbolic architectures ( 4] 21] and implicit (derived through learning) within distributed connectionist architectures ([12], 14] end elsewhere) Our approach follows the idea of viewing the linguistic entities as interacting agents providing interaction among different aspects of language description. It differs from the cited works in the methods used for the overall organization of the interaction. While our ....

McClelland, J.L. -- Kawamoto, A.: Mechanisms of sentence processing: Assigning roles to constituents of sentences. In: Rumelhart D.E., McClelland J.L. and the PDP Research Group: Parallel Distributed Processing: Explorations in the microstructure of cognition, Vol.2. Cambridge, MA: MIT Press, 1986.

....Japanese, using translations of the same corpus. We compared these results to the results obtained with lexicons learned by a comparable system developed by Siskind (1996) showing a significant improvement for our system in most circumstances. One of the artificial corpora was based on that of McClelland and Kawamoto (1986), and augmented with Conceptual Dependency (Schank, 1975) information. Finally, we used a set of artificially generated corpora to test the scalability of our algorithm, with encouraging results. While building an annotated corpus is arguably less work than building an entire NLP system, it is ....

....But in the CD like representations used here, the same sentence is represented by [ingest,agt: person,sex:male,age:adult] pat: food,type:cheese] This representation was chosen to demonstrate the learning of complicated structures as word meanings. The corpus used here is based on that of McClelland and Kawamoto (1986). This corpus consists of 1475 sentence case structure pairs containing 30 unique words. The sentences were artificially produced from a set of 19 templates. The case structure portion of the original pairs was modified as illustrated above. The resulting corpus is hereafter referred to as the ....

McClelland, J. L., & Kawamoto, A. H. (1986). Mechanisms of sentence processing: Assigning roles to constituents of sentences. In Rumelhart, D. E., & McClelland, J. L. (Eds.), Parallel Distributed Processing, Vol. II, pp. 318--362. MIT Press, Cambridge, MA.

....processes independent of the identity of components of structured objects (The term systematicity comes from Fodor and Pylyshyn [1988] There have been three broad classes of reactions to the problem of representing complex structure. The first is exemplified by the argument of Rumelhart and McClelland s [1986b] that the importance of the ability to compose and decompose complex structure is overrated. The second is exemplified by the work of Smolensky [1990] on designing connectionist representations for complex structure. The third is exemplified by the work of Pollack [1990] and Elman [1990] on ....

....than 8 7 = 63) are similar to people s performance on the task of learning multiplication tables. These performance characteristics are a direct consequence of the analogical structure of the representations of the numbers. Rumelhart and McClelland: learning past tenses of verbs Rumelhart and McClelland [1986a] describe a distributed connectionist model which learns to produce the past tense forms of regular and irregular verbs. They represent words as phonemic strings in a distributed fashion over about 500 units. Each word is represented by a set of phonemic triples, which in turn are represented by ....

[Article contains additional citation context not shown here]

McClelland, J. L. and Kawamoto, A. H. 1986. Mechanisms of sentence processing: Assigning roles to constituents. In J. L. McClelland, D. E. R. and the PDP research group, editors, Parallel distributed processing: Explorations in the microstructure of cognition, volume 2, pages 272--326. The MIT Press, Cambridge, MA.

....the set of interpretations possible for a given input. It does not simply produce all possible interpretations and let an external agent select the best. Selection (or ambiguity resolution) is a part of the job of language understanding. Many connectionist models of language comprehension (e.g. McClelland and Kawamoto, 1986) as well as parsing algorithms such as Earley s (1970) algorithm (which is considered an efficient combination of bottom up and top down parsing) violate this principle. They simply present a set of possible interpretations and let an external agent pick a suitable interpretation. The principle of ....

McClelland, J. L. and Kawamoto, H. (1986). Mechanisms of sentence processing: Assigning roles to constituents. In McClelland, J. L., Rumelhart, D. E., and the PDP Research Group, editors, Parallel Distributed Processing. Explorations in the microstructure of cognition, Vol.

....processing can be modeled with neural networks. Such models can give us insight into the processes underlying sentence processing, and suggest further hypotheses that can be tested experimentally. Several subsymbolic models of sentence processing have been built around this idea (Elman 1991; McClelland and Kawamoto 1986; Miikkulainen and Dyer 1991; St. John and McClelland 1990) This chapter aims at two goals: 1) providing a clear, concrete example of soft constraints and how they are combined in sentence processing, and (2) showing how systems based on soft constraints can process language productively and ....

....FGREP method (Miikkulainen and Dyer 1991; Miikkulainen 1993) so that similar words will have similar representations. FGREP is a convenient way for forming these representations, but SPEC is not dependent on FGREP. The word representations could have been obtained through semantic feature encoding (McClelland and Kawamoto 1986) as well, or even assigned randomly. The case role assignment is represented at the output of the Parser as a case role vector (CRV) that is, a concatenation of those three word representation vectors that fill the roles of agent, act, and patient in the sentence (the representation was limited ....

McClelland, J. L., and Kawamoto, A. H. (1986). Mechanisms of sentence processing: Assigning roles to constituents. In McClelland, J. L., and Rumelhart, D. E., editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Volume 2: Psychological and Biological Models, 272-- 325. Cambridge, MA: MIT Press.

....in their hidden layers as part of the learning. Such representations are learned essentially in the same process as the FGREP representations, and therefore the analysis below illustrates the properties of internal network representations in general. The network was trained with data designed by McClelland and Kawamoto (1986) for the caserole assignment task. There were 19 sentence templates of the type the human ate the food. The actual sentences were formed by replacing the category words human and food with the actual members of the category, such as man, woman, boy, girl, and chicken, cheese, pasta, carrot. ....

....matches human performance very well. The above architectures demonstrated that subsymbolic networks build meaningful internal representations when exposed to examples of strings in a language. They did not address how such capabilities could be put to use in parsing and understanding language. McClelland and Kawamoto (1986) first identified the sentence case role assigment as a good approach. The approach is particularly well suited for neural networks because the cases can be conveniently represented as assemblies of units that hold distributed representations, and the parsing task becomes that of mapping between ....

McClelland, J. L., and Kawamoto, A. H. (1986). Mechanisms of sentence processing: Assigning roles to constituents. In McClelland, J. L., and Rumelhart, D. E., editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Volume 2: Psychological and Biological Models, 272--325. Cambridge, MA: MIT Press.

....approaches to the building of semantic representations under connectionism. The localist schemes use a separate node for each word (e.g. Cottrell 1989) Another approach is to use microfeatures for forming the representation patterns such as the one in the case role assignment task solved by McClelland and Kawamoto (1986). The problem with these two approach is that they can hardly scale up to deal with large body of words. And, hand coding of word representations can be biased. Statistical method for building word representations also exists (Schutze 1993) in which cooccurrence statistics for words is used to ....

....4 Experimental Evaluation Experimental Methodology Since the approach to building word representations is corpus based. The input training data is a body of lexical items. Two kinds of input data are used in the experiments. One is the artificial data taken from the case role assignment task of McClelland and Kawamoto (1986) with the caserole assignment part of each sentence being neglected since the task is to map a sentence to itself. The original data set consists of a total of 1,476 sentences divided into two groups: one group of d1i Sjd2jw1ij = Drci hd1i = 6 Figure 2: Feature Map for Artificial Data (not ....

[Article contains additional citation context not shown here]

McClelland, J. L., and A. H. Kawamoto. (1986). Mechanisms of sentence processing: Assigning roles to constituents of sentences. In J. L. McClelland, D. E. Rumelhart, and the PDP Research Group (Ends.), Parallel Distributed Processing. Explorations in the Microstructure of Cognition.

....is that the processing knowledge can be extracted automatically from examples. The same architecture can learn to process a wide variety of inputs and take advantage of the implicit statistical regularities in the data, without having to be specifically programmed with particular data in mind (McClelland et al. 1986). The gradual evolution of the system performance as it is learning often resembles human learning in the same task (Rumelhart and McClelland, 1987; Sejnowski and Rosenberg, 1987) Such PDP models typically have very little internal structure or architectural complexity. They produce the ....

....can take place using distributed representations. This is a major advantage of the PDP approach in general. Distributed representations can reflect the meanings of the items they stand for. Similar items have similar representations, which results in several interesting processing enhancements (McClelland et al. 1986; van Gelder, 1989) Information is inherently content addressable, and fuzzy categories can be naturally represented. Performance of a PDP system degrades gracefully in less than perfect conditions, such as with overload, damage, and noisy or incomplete input. PDP networks also spontaneously ....

[Article contains additional citation context not shown here]

McClelland, J. L. and Kawamoto, A. H. (1986). Mechanisms of sentence processing: Assigning roles to constituents. In McClelland, J. L. and Rumelhart, D. E., editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Volume 2: Psychological and Biological Models. Cambridge, MA: MIT Press.

....damage, and incomplete or conflicting input (Miikkulainen 1993; St. John and McClelland 1990) Second, because computation in these networks is constraint based, the subsymbolic approach naturally combines syntactic, semantic, and thematic constraints on the interpretation of linguistic data (McClelland and Kawamoto 1986). Third, subsymbolic systems can be lesioned in various ways and the resulting behavior is often strikingly similar to human impairments (Miikkulainen 1993, 1996; Plaut 1991) These properties of subsymbolic systems have attracted many researchers in the hope of accounting for interesting ....

McClelland, J. L., and Kawamoto, A. H. (1986). Mechanisms of sentence processing: Assigning roles to constituents. In McClelland, J. L., and Rumelhart, D. E., editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Volume 2: Psychological and Biological Models, 272--325.

....the dynamically allocated variable sized symbolic data structures traditionally used in AI. The limitation shows in the fact that pure connectionism has generated somewhat unsatisfying systems in this domain; for example, parsers for fixed length sentences [1 4] without embedded structures [5]. 1 Indeed, some of the recent attacks on connectionism have been aimed precisely at the question of representational adequacy. According to Minsky Papert [10] for example, work on neural network and other learning machines was stopped by the need for AI to focus on knowledge representation in ....

....are needed to represent an ELEPHANT, then the attempt to represent a NURSE RIDING ELEPHANT may come out either as a WHITE ELEPHANT or a rather LARGE NURSE WITH FOUR LEGS. To solve the problem of feature superposition, one might use full size constituent buffers, such as Agent, Action, and Object [5]. In each buffer would reside a feature pattern filling these roles such as NURSE, RIDING, and ELEPHANT. Unfortunately, because of the dichotomy between the representation of a structure (by concatenation) and the representation of an element of the structure (by features) this type of system ....

J. McClelland and A. Kawamoto, Mechanisms of Sentence Processing: Assigning Roles to Constituents, in Parallel Distributed Processing: Experiments in the Microstructure of Cognition, vol. 2, J. L. McClelland, D. E. Rumelhart and the PDP research Group (ed.), MIT Press, Cambridge, 1986.

....Pollack s (1990) recursive auto associative memory correctly process sentences not present in the training set. The question is, does this degree of generalization constitute systematicity Hadley (1994) concludes No. His conclusion was based on a closer examination of six models, which included: McClelland and Kawamoto (1986), Chalmers (1990) Elman (1990) Pollack (1990) Smolensky (1990) and St. John and McClelland (1990) All six models failed to provide clear demonstrations of what Hadley terms, strong systematicity (generalization across syntactic position) 1 , for two reasons. Either, the training sets in all ....

McClelland, J. L., &Kawamoto, A. H. (1986). Mechanisms of Sentence Processing: Assigning roles to Constituents of Sentences, Vol. 2 of Parallel Distributed Processing: Explorations in the Microstructure of Cognition, chap. 19, pp. 272--331. MIT Press, MA.

....nodes and of the phrase structure tree as a whole. Since these predicate units represent a feature decomposition of the types of nodes, and these features are only interpreted within this module, the predicate units are analogous to hidden units in Parallel Distributed Processing (Rumelhardt and McClelland, 1986) networks. The network s input units are just a stand in for the word recognition component of a complete system. There is one input unit per word. 20 When the next word is input, that word s unit becomes active and stays active (across all phases) until a grammar entry for that word is ....

....them from being adequate for parsing natural language. Elman (1991) applies his work on simple recurrent networks to natural language parsing and gets reasonable results for simple sentences. Simple recurrent networks are a form of Parallel Distributed Processing (PDP) model (Rumelhardt and McClelland, 1986). They have a layer of input units, a layer of output units, and a layer of hidden units, but unlike plain PDP models, they have an additional set of input units whose values are the values of the hidden layer units after the last word was processed. An extension of the backpropagation learning ....

[Article contains additional citation context not shown here]

J.L. McClelland and A. Kawamoto. Mechanisms of sentence processing: Assigning roles to constituents. In D. E. Rumelhardt and J. L. McClelland, editors, Parallel Distributed Processing, Vol. 2. MIT Press, Cambridge, MA, 1986.

....tagging and prediction of the next syntactic category in a linear pattern. Chalmers (1992) proposes a model which can generalize a simplified active passive transformation over distributed representations of sentential input, but the syntactic capability of the model is again quite restricted. McClelland Kawamoto (1986) also exploit the generalization properties of distributed representations, but their model is limited to matching phrases to the roles they play in a sentence. Sopena (1992) develops a model in which roles are determined within embedded syntactic structures, but the syntactic capabilities of the ....

....corresponds to a solution to a particular problem. Newer approaches in AI have described intelligent processes as the global behavior that emerges within a massively parallel network of computationally simple processing units (for example, Anderson, 1983; Fahlman, 1981; Feldman Ballard, 1982; McClelland et al. 1986; Reggia Sutton, 1988; Rumelhart et al. 1986; Smolensky, 1988) Each processor can only perform simple computations on numeric values, and communicate the results in parallel to all of its neighboring nodes in the network. The solution to a problem consists of a pattern of numeric activation ....

McClelland, J. and A. Kawamoto (1986). "Mechanisms of Sentence Processing: Assigning Roles to Constituents." In McClelland, J., D. Rumelhart, and the PDP Research Group (Eds.), Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Volume 2 . Cambridge: MIT Press.

.... Previous work on Connectionist NLP ANNs have been extensively used for the following NLP tasks: Speech recognition and generation, e.g. Sejnowski and Rosenberg, 1986] Letter and word perception and generation, e.g. McClelland and Rumelhart, 1981] Sentential deep case identification, e.g. [McClelland and Kawamoto, 1986]; Word disambiguation in context, e.g. Ide and Vernois, 1990] Lately, there have also been considerable advances in the fields of syntax and semantics, with work on: Parsing, which has always posed special challenges to connectionism, e.g. Bennett, 1988] Compositional semantics, e.g. ....

McClelland, J. L. and Kawamoto, A. H. (1986). Mechanisms of sentence processing: Assigning roles to constituents of sentences. In Rumelhart, D. E. and McClelland, J. L., editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Volume 1: Foundations. MIT Press, Cambridge, Mass.

.... In one experiment, Chill and naive ILP were compared on an artificial data set for case role mapping that has been used to demonstrate certain language processing abilities of artificial neural networks [16] This task involves a corpus of 1475 sentence case structure pairs originally presented in [15]. The corpus was produced from a set of 19 sentence templates, generating sentences such as The HUMAN ate the FOOD with the UTENSIL , where the capitalized items are replaced with words of the given category. The sample actually comprises 1390 unique sentences, some of which allow multiple ....

J. L. McClelland and A. H. Kawamoto. Mechanisms of sentence processing: Assigning roles to constituents of sentences. In D. E. Rumelhart and J. L. McClelland, editors, Parallel Distributed Processing, Vol. II, pages 318--362. MIT Press, Cambridge, MA, 1986.

No context found.

McClelland, J. L. and Kawamoto, A. H. Mechanisms of Sentence Processing: Assigning Roles to Constituents. In J. L. McClelland, D. E. Rumelhart, and the PDP Research Group, Parallel Distributed Processing, vol. 2. The MIT Press, Cambridge, Massachusetts, 1986.

No context found.

James L. McClelland and A. H. Kawamoto. Mechanisms of sentence processing: assigning roles to constituents of sentences. In David E. Rumelhart James L. McClelland and the PDP Research Group, editors, Parallel Distributed Processing, volume 2, chapter 19, pages 272--325. MIT Press, Cambridge, Ma., 1986.

No context found.

McClelland, J. & Rumelhart, D. Mechanisms of sentence processing: assigning roles to constituents of sentences. Parallel Distributed Processing: Explorations in the Microstructure of Cognition Vol.2, MIT Press, Cambridge MA, 1986, pp. 272-325.

No context found.

J. L. McClelland and A. H. Kawamoto, "Mechanisms of Sentence Processing: Assigning Roles to Constituents of Sentences", in J. L. McClelland and D. E. Rumelhart (Eds.), Parallel Distributed Processing, Vol. 2, A Bradford Book, MIT Press, 1986.

No context found.

McClelland J. L. and Kawamoto A. H., Mechanisms of Sentence Processing: Assigning Roles to Constituents of Sentences, in Parallel Distributed Processing, Vol. 2, pp. 272-325, 1986.

No context found.

J. L. McClelland and A. H. Kawamoto. Mechanisms of sentence processing: Assigning roles to constituents. In David E. Rumelhart and James L. McClelland, editors, Parallel Distributed Processing Vol. 2. The MIT Press, Cambridge, MA., 1986.

No context found.

McClelland, J.L. and Kawamoto, A.H. (1986). Mechanisms of Sentence Processing: Assigning Roles to Constituents of Sentences. In McClelland et al., (1986), p272-325.

No context found.

Rumelhart, D. E. and Kawamoto, A. H. 1986a. Mechanisms of sentence processing: assigning roles to constituent of sentences. in Rumelhart, D. E. and McClelland, J. L. (eds.) Parallel distributed processing, Vol 2, pp. 272-325. MIT Press.

No context found.

McClelland J.L. and A.H. Kawamoto. "Mechanisms of sentence Processing : Assigning Roles to the constituents of sentences". Parallel Distributed Processing, vol 2:pp 272--325, 1986.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC