Christiansen, M.H. and Nick Chatter (1999). Toward a connectionist model of recursion in human linguistic performance. Cognitive Science, (in press).  Home/Search   Document Details and Download   Summary   Related Articles   Check

....modification as determined by its definiteness. Subjects are slower to attach PPs to definite NPs. Britt (1994) also found that preceding the NP with an adjective causes a preference for VP attachment. Section 3. 7 discusses these effects and a number of other studies of discourse influences on PP attachment. It would be informative to redo the Schutze and Gibson (1999) experiment with full crossing of the VP and NP attachments with the argument modifier distinction. This would be the first study to compare VP and NP attachments while controlling for argumenthood. It would also be useful to include conditions with verbs that take obligatory third arguments (I ....

Christiansen, M. H., & Chater, N. (1999b). Toward a connectionist model of recursion in human linguistic performance. Cognitive Science, 23, 157--205.

.... to histories of symbols yielding similar next symbol distributions [17] Yet, several researches issued a note of caution: When training RNNs to process language structures, activations of recurrent units display a considerable amount of structural di erentiation even prior to learning [1] 2] [18]. Following [18] we refer to this phenomenon as the architectural bias of RNNs. In this paper we provide an explanation of this phenomenon and show that, even prior to any training, clusters in the recurrent layer of RNNs re ect Markov prediction states. In this case clustering recurrent ....

.... symbols yielding similar next symbol distributions [17] Yet, several researches issued a note of caution: When training RNNs to process language structures, activations of recurrent units display a considerable amount of structural di erentiation even prior to learning [1] 2] 18] Following [18], we refer to this phenomenon as the architectural bias of RNNs. In this paper we provide an explanation of this phenomenon and show that, even prior to any training, clusters in the recurrent layer of RNNs re ect Markov prediction states. In this case clustering recurrent activations leads to ....

[Article contains additional citation context not shown here]

M.H. Christiansen and N. Chater, \Toward a connectionist model of recursion in human linguistic performance," Cognitive Science, vol. 23, pp. 417-437, 1999.

....This bias has the effect that structural differentiation due to the inherent dynamics can be observed even before training. This observation has been verified experimentally [22, 23, 37] Moreover, a structural bias has counterparts in the way in which humans recognize language as pointed out in [8], for example. This article establishes a thorough mathematical formalization of this architectural bias. Furthermore, the previous exploration of simpler mechanisms, finite memory, in standard neural network training has beneficial effects for the generalization ability: better generalization to ....

M.H. Christiansen and N. Chater. Towards a connectionist model of recursion in human linguistic performance. Cognitive Science, 23:157-205, 1999.

....This bias has the effect that structural differentiation due to the inherent dynamics can be observed even before training. This observation has been verified experimentally [22, 23, 36] Moreover, a structural bias has counterparts in the way in which humans recognize language as pointed out in [8], for example. This article establishes a thorough mathematical formalization of this architectural bias. Furthermore, the previous exploration of simpler mechanisms, finite memory, in standard neural network training has beneficial effects for the generalization ability: better generalization to ....

M.H. Christiansen and N. Chater. Towards a connectionist model of recursion in human linguistic performance. Cognitive Science, 23:157-205, 1999.

....cernans dcs.elf.stuba.sk, http: www.dcs.elf.stuba.sk cernans Institute of Informatics FMFI, Comenius University, Mlynsk a dolina, 842 48 Bratislava 4, Slovakia benus ii.fmph.uniba.sk, http: www.ii.fmph.uniba.sk benus Abstract. Several studies in the cognitive science community (see [1 3]) reported that when training recurrent neural networks (RNNs) to process amount of structural di erentiation even prior to learning. Following [3] we refer to this phenomenon as the architectural bias of RNNs. In this paper, we show, that when initialized with small weights, RNNs produce ....

....Bratislava 4, Slovakia benus ii.fmph.uniba.sk, http: www.ii.fmph.uniba.sk benus Abstract. Several studies in the cognitive science community (see [1 3] reported that when training recurrent neural networks (RNNs) to process amount of structural di erentiation even prior to learning. Following [3], we refer to this phenomenon as the architectural bias of RNNs. In this paper, we show, that when initialized with small weights, RNNs produce recurrent activations that cluster according to a Markovian strategy: i.e. subsequences sharing a long common sux are represented as points in a dense ....

[Article contains additional citation context not shown here]

Christiansen, M.H., Chater, N.: Toward a connectionist model of recursion in human linguistic performance. Cognitive Science, 23 (1999) 157-205

....the structure of the prime. This is possible because the weight changes associated with a particular message sentence mapping are shared with other message sentence combinations. Figure 5 illustrates the general theory behind the model. Like some other connectionist treatments of language (e.g. Christiansen Chater, 1999; Plaut Kello, 1999) Chang et al. propose a close relationship between the comprehension and production. In particular, they suggest that the context representations that guide sequential production arise during comprehension. For example, suppose that a simple recurrent network that maps word ....

Christiansen, M. H., & Chater, N. (1999). Toward a connectionist model of recursion in human linguistic performance. Cognitive Science, 23, 157--206.

....aware of the relevant regularities in AGL tasks where only rule learning is possible. To demonstrate, Shanks et al. exposed subjects to artificial grammar strings generated by a biconditional grammar (see also Mathews et al. 1989) Biconditional grammars involve cross dependency recursion (see Christiansen Chater, 1999) such that letters that appear at each position before and after a central dot depend on each other. An example is given in Figure 1, where letter D is paired with F, G with L, and so on. Figure 1. A biconditional grammar string as used by Shanks et al. 1997) Possible letters in each position ....

....prediction relevant in terms of the statistical distribution of the embedded elements for such structures to be successfully mastered by an SRN. There is also accumulating evidence that the pattern of failures observed with models like the SRN closely mimic that observed with human subjects (e.g. Christiansen Chater, 1999) in the domain of natural language learning. Empirically, we would like to suggest that experiments be carried out on a slightly different basis than used in Shanks et al. since their match group showed no sign at all of having learned the material. One possibility would consist of changing ....

Christiansen, M., & Chater, N. (1999). Toward a connectionist model of recursion in human linguistic performance, Cognitive Science, 23, 157-205.

....of the string (ready for the next a) The dynamics of such networks is usually based on oscillation around fixed points in state space. Previous work on learning formal languages requiring stacks (e.g. context free languages) has primarily focused on the use of Simple Recurrent Networks (SRNs) [18, 17, 11, 12, 2, 4]. For the a n b n prediction task, two main types of solution have been found [17, 12] of which only one generalizes well [2] This paper reports on experiments along the same lines as previous research but with a second order recurrent neural network, the Sequential Cascaded Network (SCN; ....

....precision is not strictly enforced. Weights are not sufficiently precise to exhibit true fractal nature and enable generalization to infinitely deep levels. However, the performance degradation observed in the trained networks for deep input strings is also apparent in human language performance [4]. 7 Conclusion There is a variety of dynamics available for implementing stack like behavior in RNNs. The present study has explored two types of dynamics which are (1) learnable, 2) unlike classical automata, only limited by numerical precision and not memory, and (3) conforming to ....

M. Christiansen and N. Chater. Toward a connectionist model of recursion in human linguistic performance. Cognitive Science, 23:157--205, 1999.

....states towards the opposite limit border (z 2 = 1) while oscillating outwards. 4 CONCLUSIONS It is difficult for neural networks to learn CFLs. However, natural language does contain CFL constructs, suggesting that humans have indeed the necessary prerequisites but with limited precision (cf. Christiansen and Chater, 1999). This work is one step towards resolving if there can be a simple language mechanism for handling CFLs (based on neural networks) and, if so, how, and how well it can be established through evolutionary algorithms compared to conventional learning. It has been demonstrated in this paper that EAs ....

Christiansen, M. and Chater, N. (1999). Toward a connectionist model of recursion in human linguistic performance. Cognitive Science, 23:157--205.

.... regular languages from examples (Cleeremans et al. 1989; Elman, 1990; Pollack, 1991; Giles et al. 1992) However, their ability to learn deeply embedded languages (context free, context sensitive and recursive languages) is limited (Weckerly and Elman, 1992; Elman, 1993; Wiles and Elman, 1995; Christiansen and Chater, 1999). Previous work has demonstrated that an RNN, successfully trained on a simple context free language, does not make use of an explicit counter or stack to process embedded structures. Instead, hidden units develop oscillating dynamics which provide means for a potentially infinite number of ....

Christiansen, M. H. and Chater, N. (1999). Toward a connectionist model of recursion in human linguistic performance Cognitive Science, to appear.

....produce the same highest level static distributed representations and to output sequential associations There are two points which speak in favor of the cascade structure. First, it is difficult for one homogeneous network to learn long distance relationships [Miikkulainen, 1991] for discussion [Christiansen, 1999]. BPTT learning algorithm propagates back error, but the more steps it propagates back, the smaller the influence of those errors is to the earlier steps. Studying different patterns of recursion, Christiansen and Chater found that the depth of embedding (or recursion) SRNs can handle well is in ....

Christiansen, M. H. and Chatter, N., Toward a connectionist model of recursion in human linguistic performance, Cognitive Science, (in press), 1999.

.... model (Elman, 1991) They have also demonstrated competence in learning a wide range of grammatical structures, for example from an introductory linguistics text (Lawrence et al. 1998) Furthermore, they re ect human performance on a number of language tasks (Weckerly and Elman, 1992; Christiansen and Chater, 1998), and can account for historical descriptions of language change (Hare and Elman, 1995) As well as processing constraints, RNNs have learning constraints. That is, they are constrained not only in what they can represent, but in what they can learn. Returning to our original question, we consider ....

Christiansen, M. H. and Chater, N. (1998). Toward a connectionist model of recursion in human linguistic performance. To appear: Cognitive Science.

No context found.

Christiansen, M.H. & Chater, N. (in press). Toward a connectionist model of recursion in human linguistic performance. Cognitive Science. Chomsky, N. & Halle, M. (1968). The Sound Pattern of English.

No context found.

Christiansen, M.H. and Nick Chatter (1999). Toward a connectionist model of recursion in human linguistic performance. Cognitive Science, (in press).

No context found.

Christiansen, M. H. & Chater, N. (1999). Toward a connectionist model of recursion in human linguistic performance. Cognitive Science, 23 (2), 157--205.

No context found.

Christiansen, M. and Chater, N. (1999). Toward a connectionist model of recursion in human linguistic performance. Cognitive Science, 23:157--205.

No context found.

M. Christiansen and N. Chater. Toward a connectionist model of recursion in human linguistic performance. Cognitive Science, 23:157--205, 1999.

No context found.

M. Christiansen and N. Chater. Toward a connectionist model of recursion in human linguistic performance. Cognitive Science, 23:157--205, 1999.

No context found.

Christiansen, M.H. and Chater, N. (1999) Toward a connectionist model of recursion in human linguistic performance. Cogn. Sci. 23,

No context found.

Morten Christiansen and Nick Chater. Toward a connectionist model of recursion in human linguistic performance. Cognitive Science, 23:157--205, 1999.

No context found.

M. Christiansen and N. Chater. Toward a connectionist model of recursion in human linguistic performance. Cognitive Science, 23:157--205, 1999.

No context found.

Morten Christiansen and Nick Chater. Toward a connectionist model of recursion in human linguistic performance. Cognitive Science, 23:157--205, 1999.

No context found.

M.H. Christiansen and N. Chater. Toward a connectionist model of recursion in human linguistic performance. Cognitive Science, in press, 1999.

No context found.

Christiansen, M. H., & Chater, N. (in press). Toward a connectionist model of recursion in human linguistic performance. Cognitive Science. Hopcroft, J. E., & Ullman, J. D. (1979). Introduction to automata theory, languages, and computation. Reading, MA: AddisonWesley Publishing.

No context found.

Christiansen, M. & Nick Chater (1998). Toward a connectionist model of recursion in human linguistic performance. Cognitive Science (in press).

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC