Andreas Stolcke. Learning feature-based semantics with simple recurrent networks. Technical Report TR-90-015, International Computer Science Institute, Berkeley, CA, 1990.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....generalization easier. A well known problem with the SRN model is its low memory accuracy. It is difficult for it to remember items that occurred several steps earlier in the input sequence, especially if the network is not required to produce them in the output layer during the intervening steps [Stolcke, 1990; Miikkulainen, 1996] The intervening items are superimposed in the hidden layer, obscuring the traces of earlier items. Nor has simply increasing the size of the hidden layer or lowering the learning rate been found to offer much advantage. As a result, parsing with an SRN has been limited to ....

Andreas Stolcke. Learning feature-based semantics with simple recurrent networks. Technical Report TR-90-015, International Computer Science Institute, Berkeley, CA, 1990.

....et al. 1993] However, the amount of data required is sometimes quite large. Moreover, Neural Networks (NNs) so called Connectionist Models, can also be considered as an encouraging approach to EB MT. On that score, NNs have also shown empirical success dealing with Language Understanding tasks [Stolcke, 1990, Casta no et al. 1995] However, only a few connectionist MT systems have been developed in the literature. PARSEC [Jain, 1991] which was used in the JANUS project [Waibel et al. 1991] follows this approach. Another effective and more simple EB connectionist translator for text to text ....

A. Stolcke: "Learning Feature-based Semantics with Simple Recurrent Networks". Technical Report no. TR-90-015, International Computer Science Institute, Berkeley, California. (1990)

....task that was recently introduced in the general context of Cognitive Science as a touchstone for showing the capabilities of learning systems. This task is the so called Miniature A. Castellanos, E. Vidal, M. A. Var o and J. Oncina 6 Language Acquisition (MLA) task, proposed by Feldman, Lakoff, Stolcke Weber (1990). It presents fundamental challenges to several areas of Cognitive Science including language, inference and learning. Thus, it may easily be reformulated to be a paradigmatic task in the LU framework as well. The task consists of understanding the meaning of pseudo natural English sentences that ....

....of understanding the meaning of pseudo natural English sentences that describe simple visual scenes. These scenes may involve different objects in different relative positions and each object possibly has a different shape, size and or shade. A restricted version of the MLA task was considered by Stolcke (1990) using Recurrent Neural Networks, with fairly good results. A detailed description of the MLA task is given in section 4. In order to frame the MLA task into our transducer learning paradigm, an adequate output language is required to conveniently state the semantic contents of each English input ....

[Article contains additional citation context not shown here]

Stolcke, A. (1990). Learning Feature-based Semantics with Simple Recurrent Networks. Technical Report, TR-90-015. International Computer Science Institute. Berkeley, California, U.S.A.

.... speech recognition [Waibel et al. 1987; Morgan and Bourlard, 1989; Renals et al. 1991; Waibel et al. 1991; Osterholtz et al. 1992] natural language processing and inference [Waltz and Pollack, 1985; Cottrell, 1985; Elman, 1988; Shastri, 1988; Fanty, 1988; Weber, 1989b; Miikkaulainen, 1990; Stolcke, 1990b; Jain et al. 1992] vision [Ballard, 1987b; Sejnowski and Hinton, 1987; Olson, 1989; LeCun, 1989; Hummel and Biederman, 1990; Poggio and Edelman, 1990; Ahmad and Omohundro, 1990; Ahmad, 1991; Mozer et al. 1991; Keeler et al. 1991] and purely theoretical work probing the limits of connectionist ....

Andreas Stolcke, "Learning Feature-Based Semantics with Simple Recurrent Networks," Technical Report TR-90-015, International Computer Science Institute, Berkeley, CA, April 1990.

....enhance the appeal of Connectionism to computational linguists. 5.3 Earlier Work on Connectionist Unification 5.3.1 Stolcke s Unification Network So far there has not been much work done on Connectionist unification. The earliest work the author is aware of is that of Andreas Stolcke [26]. He employs a localist approach to perform unification. To see how his method works, consider representing f structures as directed acyclic graphs (DAGs) Figure 5 2 shows the f structures of figure 5 1 rerepresented as DAGs. The unification of (a) with (b) is represented in (c) Stolcke ....

A. Stolcke. Learning feature based semantics with simple recurrent networks. Technical Report TR-90-015, International Computer Science Institute, Berkeley, CA, 1990.

....they are Turing equivalent (Siegelmann Sontag 1992) However, only recently has any work been successful with moderately large grammars. Recurrent neural networks have been used for several small natural language problems, e.g. papers using the Elman network for natural language tasks include: (Stolcke 1990, Allen 1983, Elman 1984, Harris Elman 1984, John McLelland 1990) 2 Data Our primary data consists of 552 English positive and negative examples taken from an introductory GB linguistics textbook by Lasnik and Uriagereka (Lasnik Uriagereka 1988) Most of these examples are organized into ....

Stolcke, A. (1990), Learning feature-based semantics with simple recurrent networks, Technical Report TR-90-015, International Computer Science Institute, Berkeley, California.

.... Schabes 1992) In the past few years several recurrent neural network (RNN) architectures have emerged which have been used for grammatical inference. RNNs have been used for several smaller natural language problems, e.g. papers using the Elman network for natural language tasks include: (Stolcke 1990, Allen 1983, Elman 1984, Harris Elman 1984, St. John McClelland 1990) Neural network models have been shown to be able to account for a variety of phenomena in phonology (Gasser Lee 1990, Hare 1990, Touretzky 1989a, Touretzky 1989b) morphology (Hare, Corina Cottrell 1989, MacWhinney, ....

Stolcke, A. (1990), Learning feature-based semantics with simple recurrent networks, Technical Report TR-90-015, International Computer Science Institute, Berkeley, California.

.... is currently only practical for relatively small grammars [48] which have been used for grammatical inference [9, 21, 19, 20, 68] Recurrent neural networks have been used for several smaller natural language problems, e.g. papers using the Elman network for natural language tasks include: [1, 12, 24, 58, 59]. Neural network models have been shown to be able to account for a variety of phenomena in phonology [23, 61, 62, 18, 22] morphology [51, 41, 40] and role assignment [42, 58] Induction of simpler grammars has been addressed often e.g. 64, 65, 19] on learning Tomita languages [60] The task ....

Andreas Stolcke. Learning feature-based semantics with simple recurrent networks. Technical Report TR-90015, International Computer Science Institute, Berkeley, California, April 1990.

....generalization easier. A well known problem with the SRN model is its low memory accuracy. It is difficult for it to remember items that occurred several steps earlier in the input sequence, especially if the network is not required to produce them in the output layer during the intervening steps (Stolcke 1990; Miikkulainen 1996) The intervening items are superimposed in the hidden layer, obscuring the traces of earlier items. Nor has simply increasing the size of the hidden layer been found to offer much advantage. As a result, parsing with an SRN has been limited to relatively simple sentences with ....

Stolcke, A. (1990). Learning feature-based semantics with simple recurrent networks. Technical Report TR-90-015, International Computer Science Institute, Berkeley, CA.

.... years several recurrent neural network architectures have emerged which have been used for grammatical inference [8, 21, 18, 19, 20, 59] Recurrent neural networks have been used for several smaller natural language problems, e.g. papers using the Elman network for natural language tasks include: [52, 1, 12, 24, 51]. Neural network models have been shown to be able to account for 1 The inside outside re estimation algorithm is an extension of hidden Markov models intended to be useful for learning hierarchical systems. The algorithm is currently impractical for anything except relatively small grammars ....

Andreas Stolcke. Learning feature-based semantics with simple recurrent networks. Technical Report TR-90015, International Computer Science Institute, Berkeley, California, April 1990.

.... years several recurrent neural network architectures have emerged which have been used for grammatical inference [7] 20] 17] 18] 19] Recurrent neural networks have been used for several smaller natural language problems, e.g. papers using the Elman network for natural language tasks include: [50] [1] 11] 23] 32] Neural network models have been shown to be able to account for a variety of phenomena in phonology [16] 21] 53] 52] morphology [22] 37] and role assignment [38] 32] Induction of simpler grammars has been addressed often e.g. 54] 18] on learning Tomita languages ....

Andreas Stolcke. Learning feature-based semantics with simple recurrent networks. Technical Report TR-90-015, International Computer Science Institute, Berkeley, California, April 1990.

....that could be reprogrammed readily to solve the L 0 problem for an arbitrary natural language. There were a numberofimportant lessons learned, primarily in the subtle semantics of spatial language, but it took well under a year to build up a handadaptable system #Feldman et al. 1990b; Weber Stolcke 1990#. Our wizards, Andreas Stolcke and Susan Weber, got to the point where it took only an hour or so to realize L 0 for a new language, given a sophisticated informant. This test bed project was abandoned, but aspects of it are still appearing in currentwork. The other two pieces of the original ....

....we quickly conceded to ourselves that that was not realistic, at least not given the current state of the #eld. The initial L 0 testbed #Feldman et al. 1990b# relied on traditional symbolic AI methods for purposes of rapid prototyping. The grammar learning project started out connectionist #Stolcke 1990#, but eventually adopted a probabilistic framework whichwas thoughttoprovide more versatile representations while still capturing the soft computation aspects of the problem. The spatial semantics learning project has remained the most purely connectionist. The current projects are informed by, ....

Stolcke, Andreas. 1990. Learning feature-based semantics with simple recurrent networks. Technical Report TR-90-015, International Computer Science Institute, Berkeley, CA.

....with relative ease. For example, the language processing component described in section 3 does not only handle sentence analysis as required by the testbed system, but also handles generation of sentence semantics pairs as training patterns for a connectionist approach to the L 0 task [13]. ffl The high level nature of Prolog is generally supportive of rapid prototyping of various concepts for test purposes. For language and image processing in particular, Prolog s built in control structures allowed implementations which are simple and elegant, if not optimal. Although Prolog is ....

Andreas Stolcke. Learning feature-based semantics with simple recurrent networks. Technical Report TR-90-015, International Computer Science Institute, Berkeley, Ca., April 1990.

No context found.

Andreas Stolcke, "Learning Feature-based Semantics with Simple Recurrent Networks," Technical Report TR-90-015, International Computer Science Institute, Berkeley, Ca., April 1990.

No context found.

A. Stolcke. Learning feature-based semantics with simple recurrent networks. Technical Report TR-90-015, International Computer Science Institute, University of California at Berkeley, 1990.

No context found.

Andreas Stolcke. Learning feature-based semantics with simple recurrent networks. Technical Report TR--90--015, International Computer Science Institute, April 1990.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC