Results 1 
8 of
8
Natural Language Grammatical Inference: A Comparison of Recurrent Neural Networks and Machine Learning Methods
 Symbolic, Connectionist, and Statistical Approaches to Learning for Natural Language Processing, Lecture notes in AI
, 1996
"... We consider the task of training a neural network to classify natural language sentences as grammatical or ungrammatical, thereby exhibiting the same kind of discriminatory power provided by the Principles and Parameters linguistic framework, or Government and Binding theory. We investigate the foll ..."
Abstract

Cited by 13 (2 self)
 Add to MetaCart
We consider the task of training a neural network to classify natural language sentences as grammatical or ungrammatical, thereby exhibiting the same kind of discriminatory power provided by the Principles and Parameters linguistic framework, or Government and Binding theory. We investigate the following models: feedforward neural networks, FrasconiGoriSoda and BackTsoi locally recurrent neural networks, Williams and Zipser and Elman recurrent neural networks, Euclidean and editdistance nearestneighbors, and decision trees. Nonneural network machine learning methods are included primarily for comparison. We find that the Elman and Williams & Zipser recurrent neural networks are able to find a representation for the grammar which we believe is more parsimonious. These models exhibit the best performance. 1 Motivation 1.1 Representational Power of Recurrent Neural Networks Natural language has traditionally been handled using symbolic computation and recursive processes. The most ...
Can Recurrent Neural Networks Learn Natural Language Grammars?
 IN PROCEEDINGS OF THE IEEE INTERNATIONAL CONFERENCE ON NEURAL NETWORKS
, 1996
"... Recurrent neural networks are complex parametric dynamic systems that can exhibit a wide range of different behavior. We consider the task of grammatical inference with recurrent neural networks. Specifically, we consider the task of classifying natural language sentences as grammatical or ungrammat ..."
Abstract

Cited by 7 (2 self)
 Add to MetaCart
(Show Context)
Recurrent neural networks are complex parametric dynamic systems that can exhibit a wide range of different behavior. We consider the task of grammatical inference with recurrent neural networks. Specifically, we consider the task of classifying natural language sentences as grammatical or ungrammatical  can a recurrent neural network be made to exhibit the same kind of discriminatory power which is provided by the Principles and Parameters linguistic framework, or Government and Binding theory? We attempt to train a network, without the bifurcation into learned vs. innate components assumed by Chomsky, to produce the same judgments as native speakers on sharply grammatical/ungrammatical data. We consider how a recurrent neural network could possess linguistic capability, and investigate the properties of Elman, Narendra & Parthasarathy (N&P) and Williams & Zipser (W&Z) recurrent networks, and FrasconiGoriSoda (FGS) locally recurrent networks in this setting. We show that both Elman...
Recurrent Autoassociative Networks: Developing Distributed Representations Of Hierarchically Structured Sequences By Autoassociation
, 261
"... this reportedly improved the learning. And still another important contribution in this work was a method for representing recursive structures  by means of symbolic transformation of any tree structure into a binary tree, which can easily be transformed to a sequence. Those two operations are rev ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
this reportedly improved the learning. And still another important contribution in this work was a method for representing recursive structures  by means of symbolic transformation of any tree structure into a binary tree, which can easily be transformed to a sequence. Those two operations are reversible,
Learning, Representation, and Synthesis of Discrete Dynamical Systems in Continuous Recurrent Neural Networks
 In Proceedings of the IEEE Workshop on Architectures for Semiotic Modeling and Situation Analysis in Large Complex Systems
, 1995
"... This paper gives an overview on learning and representation of discretetime, discretespace dynamical systems in discretetime, continuousspace recurrent neural networks. We limit our discussion to dynamical systems (recurrent neural networks) which can be represented as finitestate machines (e.g. ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
(Show Context)
This paper gives an overview on learning and representation of discretetime, discretespace dynamical systems in discretetime, continuousspace recurrent neural networks. We limit our discussion to dynamical systems (recurrent neural networks) which can be represented as finitestate machines (e.g. discrete event systems [53]). In particular, we discuss how a symbolic representation of the learned states and dynamics can be extracted from trained neural networks, and how (partially) known deterministic finitestate automata (DFAs) can be encoded in recurrent networks. While the DFAs that can be learned exactly with recurrent neural networks are generally small (on the order of 20 states), there exist subclasses of DFAs with on the order of 1000 states that can be learned by small recurrent networks. However, recent work in natural language processing implies that recurrent networks can possibly learn larger state systems [35]. I. Introduction Answering the questions "What are the rul...
Internet economic news gathering and classification: a neural network software agent based approach
 in Proceedings of VII Brazilian Symposium on Neural Networks, p.112
, 2002
"... based approach ..."
Learning Lexical Phonotactics With Simple Recurrent Networks
"... The language phonotactics describes the possible combinations of phonemes in order to form syllables and words, and it is typical for every language. ..."
Abstract
 Add to MetaCart
The language phonotactics describes the possible combinations of phonemes in order to form syllables and words, and it is typical for every language.
Connectionist Learning of Natural Language Lexical Phonotactics
, 1998
"... Connectionist learning of natural language words and their phonetic regularities is presented. The Neural Network (NN) model we employ in this problem is the Simple Recurrent Network, trained with the Backpropagation Through Time (BPTT) learning algorithm. During the training, it was assigned th ..."
Abstract
 Add to MetaCart
Connectionist learning of natural language words and their phonetic regularities is presented. The Neural Network (NN) model we employ in this problem is the Simple Recurrent Network, trained with the Backpropagation Through Time (BPTT) learning algorithm. During the training, it was assigned the task of predicting the next phoneme given one phoneme at each moment and keeping information of the past phonemes from a given word in a few context neurons. The phonotactics of the Dutch language was studied among others. The shortcomings of some similar previous implementations are explained and successfully overcome. Among the techniques we employed to achieve the muchimproved error rate of 1.1% with monosyllabic words and 3.5% with multisyllabic ones are new methods for network response interpretation, an evolutionary approach in training a set of networks, and the exploitation of the word frequencies in training. Finally, an analysis of the phonotactics rules extracted by a ...
Weighted Probability Distribution Voting, an introduction
, 1999
"... This paper introduces a new machine learning technique, Weighted Probability Distribution Voting (WPDV). During learning, WPDV determines the output class probability distribution for each input feature, both atomic and complex. During classification, WPDV takes all input features that occur in the ..."
Abstract
 Add to MetaCart
This paper introduces a new machine learning technique, Weighted Probability Distribution Voting (WPDV). During learning, WPDV determines the output class probability distribution for each input feature, both atomic and complex. During classification, WPDV takes all input features that occur in the new input and adds the corresponding probability distributions, each multiplied by a weight factor which depends on the feature or feature type. The output class with the highest sum is then selected. Apart from the basic mechanism of WPDV, the paper describes some principles for weight selection and feature restriction. Finally, WPDV is shown to produce results which are better than those of other stateoftheart machine learning systems for several NLP tasks. 1 Introduction Many tasks that play a role in natural language processing can be formulated as classification tasks, i.e. tasks in which an output, taken from a finite set of possible values, is calculated on the basis of a specifi...