Cleeremans, A. (1993). Mechanisms of implicit learning: Connectionist models of sequence processing. Cambridge, MA, USA: The MIT Press.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....the drift equation (Eq. 3) 5 The essential feature is some form of recurrency, so that the previous state can 5 This is essentially the architecture used in the simple recurrent network of Elman (1990) and applied extensively to implicit serial learning tasks (Cleeremans McCleeland, 1991; Cleeremans, 1993). The di erence is in the speci cation of the weight matrices between layers. The simple recurrent networks include weight matrices modi able by back propagation between t IN and t i , between t i 1 and t i and between t i and an output layer. We have not yet discussed the connections between ....

Cleeremans, A. (1993). Mechanisms of implicit learning: Connectionist models of sequence processing. Cambridge, MA, USA: The MIT Press.

....constraints arising from sequential learning, we present a set of connectionist simulations of our human data. For the simulations, we used simple recurrent networks (SRNs; Elman, 1991) because they have been successfully applied in the modeling of both non linguistic sequential learning (e.g. Cleeremans, 1993) and language processing (e.g. Christiansen, 1994; Elman, 1991) SRNs are standard feed forward neural networks equipped with an extra layer of so called context units. The SRNs used in our simulations had 7 input output units (corresponding to each of the 6 consonants plus an end of sentence ....

Cleeremans, A. (1993). Mechanisms of implicit learning: Connectionist models of sequence processing. Cambridge, MA: MIT Press.

....we present a set of connectionist simulations of our human data. Networks For the simulations, we used simple recurrent networks (SRNs; Elman, 1991) because they have been successfully applied in the modeling of both non linguistic sequential learning (e.g. Christiansen Devlin, 1997; Cleeremans, 1993) and language processing (e.g. Christiansen, 1994; Elman, 1991) SRNs are standard feed forward neural networks equipped with an extra layer of so called context units. The SRNs used in our simulations had 7 input output units (corresponding to each of the 6 letters plus an end of sentence ....

Cleeremans, A. (1993). Mechanisms of implicit learning: Connectionist models of sequence processing. Cambridge, MA: MIT Press.

....to the processing task. To this end, we use Miikkulainen s [1993] FGREP, augmented with an algorithm we call dispersion, to improve distinctness among the set of letter representations. Our goal is to create a more realistic model of how humans might process natural language. 1 Related work Cleeremans [1993] conducted an important series of experiments on sequence learning with neural networks. His Simple Recurrent Networks (SRNs) achieved perfect learning applied to the Reeber grammar, a formal language which is standardly used in machine learning. Tjong Kim Sang [1998] compared statistical, neural ....

Cleeremans, Axel [1993], Mechanisms of Implicit Learning: Connectionist Models of Sequence Processing, Neural Network Modeling and Connectionism, The MIT Press, Cambridge, MA.

....to deal with integrated sequences of input presented successively. Thus, rather than having a linguistic bias, the SRN is biased towards the learning of hierarchically organized sequential structure and has been successfully applied in the modeling of both non linguistic sequential learning (e.g. Cleeremans, 1993) and language processing (e.g. Christiansen, 1994; Elman, 1990, 1991) In the simulations, SRNs were trained to predict the next lexical category in a sentence, using sentences generated by the 32 grammars derived from the grammar skeleton in Figure 4. Each unit in the input outputlayers ....

Cleeremans, A. (1993). Mechanisms of Implicit Learning: Connectionist Models of Sequence Processing. Cambridge, MA: MIT Press.

....functions, in the last, the representations of symbolic data. It can be argued that ANNs are essentially analogous function estimators that are not really capable of dealing with discrete, symbolic data. This argument does not quite hold: there is a large body of research (see for example Cleeremans 1993, Elman 1990) that has applied neural nets successfully to symbolic language data. Neural nets have also been used for modelling probabilistic finite state automata, which are used frequently in natural language processing, e.g. in finite state morphology (Koskenniemi 1983) and phonology (Ellison ....

Cleeremans, A. (1993), Mechanisms for Implicit Learning: Connectionist Modelsof Sequence Processing, MIT Press, Cambridge, Mass.

....context changes gradually. Figure 2 illustrates one possible way to implement the drift equation (Eq. 3) This is essentially the architecture used in the simple recurrent network of Elman (1990) and applied extensively to implicit serial learning tasks (Cleeremans McCleeland, 1991; Cleeremans, 1993). 6 The essential feature is some form of recurrency, so that the previous state can contribute to the new state. Although Figure 2 shows a separate set of units for the prior state and the current state, this long loop solution is not the only way to implement the drift equation (Eq. 3) 7 ....

Cleeremans, A. (1993). Mechanisms of implicit learning: Connectionist models of sequence processing. Cambridge, MA, USA: The MIT Press.

....of the number of inputs that have been presented. Figure 1 illustrates one possible way to implement t. This is essentially the architecture used in the simple recurrent network introduced by Elman (1990) and applied extensively to implicit serial learning tasks (Cleeremans McClelland, 1991; Cleeremans, 1993; Cleeremans, Destrebecqz, Boyer, 1998) 3 The essential feature is some form of recurrency, so that the previous state can contribute to the new state. Although Figure 1 shows a separate set of units for the prior state and the current state, this long loop solution is not the only way to ....

Cleeremans, A. (1993). Mechanisms of implicit learning : connectionist models of sequence processing. Cambridge, Mass: MIT Press.

.... spared are gradually acquired skills that emerge over several sessions of practice, such as the skill of tracing a figure viewed in a mirror (Milner, 1966) reading mirror reversed print (Cohen Squire, 1980) or anticipating subsequent items in a sequence governed by a complex stochastic grammar (Cleeremans, 1993). A second form of spared learning is exhibited in repetition priming tasks: these are tasks that require subjects to emit some response already within their capabilities, such as naming a word or picture (Milner, Corkin, Teuber, 1968) reading aloud a pronounceable nonword (Haist, Musen, ....

Cleeremans, A. (1993). Mechanisms of implicit learning: Connectionist models of sequence processing.

....to the processing task. To this end, we use Miikkulainen s (1993) FGREP, augmented with an algorithm we call dispersion, to improve distinctness among the set of letter representations. Our goal is to create a more realistic model of how humans might process natural language. 1 Related work Cleeremans (1993) conducted an important series of experiments on sequence learning with neural networks. His Simple Recurrent Networks (SRNs) achieved perfect learning applied to the Reeber grammar, a formal language which is standardly used in machine learning. Tjong Kim Sang (1998) compared statistical, neural ....

Cleeremans, A.(1993), Mechanisms of Implicit Learning: Connectionist Models of Sequence Processing, Neural Network Modeling and Connectionism, The MIT Press, Cambridge, MA.

....input, picking up different kinds of information, subject to perceptual and attentional constraints. There is a growing body of evidence that as a result of attending to sequential stimuli, both adults and children incidentally encode statistically salient regularities of the signal (e.g. Cleeremans, 1993; Saffran, Aslin Newport, 1996; Saffran, Newport Aslin, 1996) The child s immediate task, then, is to update its representation of these statistical aspects of language. Our claim is that knowledge of other, more covert aspects of language is derived as a result of how these representations ....

.... 1995; Saffran, Aslin Newport, 1996) but perhaps the most important tie for our purposes is the use of SRNs to model both sequence learning (e.g. Servan Schreiber, Cleeremans McClelland, 1989) and the learning of linguistic structure (e.g. Christiansen, in preparation; Elman, 1991, 1993) Cleeremans (1993) successfully applied SRNs to model the results from a number of sequential learning experiments. Analyses revealed a specific architectural limita40 tion in relation to the prediction task: SRNs tend only to encode information about previous subsequences if this information is locally relevant ....

Cleeremans, A. (1993). Mechanisms of implicit learning: Connectionist models of sequence processing. Cambridge, Mass: MIT Press.

....incompatible. Using simulation studies, I show that abstract knowledge about the stimulus material may emerge through the operation of elementary, associationist learning mechanisms of the kind that operate in connectionist networks. I focus on a sequence learning task first proposed by Kushner, Cleeremans Reber (1991), during which subjects are exposed to random fixed length sequences and are asked to predict the location at which the last element of each sequence will appear. Unknown to them, the location of the last element is determined based on the relationship between specific previous elements. This ....

....appear. Unknown to them, the location of the last element is determined based on the relationship between specific previous elements. This situation is thus quite complex, because the relevant information is relational, and because it is embedded in a large number of irrelevant contexts. Kushner, Cleeremans Reber (1991) showed that human subjects are able to learn this material despite limited ability to verbalize their knowledge. In this paper, I first present simulation studies in which connectionist networks are trained to predict the last event of the sequences in the same conditions as subjects were. I ....

[Article contains additional citation context not shown here]

Cleeremans (In press). Mechanisms of Implicit Learning: Connectionist models of sequence processing. Cambridge: MIT Press Cleeremans, A., & McClelland, J.L. (1991). Learning the structure of event sequences.

....(Reber, 1992: p. 45; my emphasis) It is my contention that language learning may be construed in terms of such implicit processing a point also made by Reber (1992) and Durkin (1989) The simulations presented in the previous two chapters appear to corroborate this view (which the work by Cleeremans, 1993, on connectionist models of sequence processing hints at too) If the learning and processing of sequential information constitutes some of the most basic elements of cognition as I have suggested then we might expect them to have a long phylogenetic past. Indeed, this idea has been advanced ....

Cleeremans, A. (1993) Mechanisms of Implicit Learning: Connectionist Models of Sequence Processing. Cambridge, Mass.: MIT Press.

No context found.

Cleeremans, A. (1993) Mechanisms of Implicit Learning: Connectionist Models of Sequence Processing, MIT Press

No context found.

Cleeremans, A. (1993a). Mechanisms of implicit learning: connectionist models of sequence processing. Cambridge, MA: MIT Press.

....between words. Interestingly, Saffran et al. 1997) rooted their interpretation of their findings in the apparently remote literature dedicated to implicit learning. The connection is obvious as soon as one recognizes that language acquisition, like implicit learning (see Berry Dienes, 1993; Cleeremans, 1993 for reviews) is likely to involve, at least in part, incidental learning of complex information organized at different levels. In particular, research on sequence learning has, over the past decade or so provided a steady stream of relevant evidence suggesting that participants exhibit detailed ....

....abstract rule that describes permissible transitions between successive stimuli. Rule based paradigms can in turn involve either deterministic (e.g. Lewicki, Hill, Bizot, 1988) or probabilistic rules, as when the stimulus material is generated based on the output of finite state grammars (e.g. Cleeremans, 1993). By contrast, in the more common simple repeating sequence paradigm, a single sequence containing fixed regularities is repeated many times to produce the training set (e.g. Nissen Bullemer, 1987) A perennial question in this context is to determine exactly what people learn about when ....

[Article contains additional citation context not shown here]

Cleeremans, A. (1993). Mechanisms of Implicit learning: Connectionist models of sequence processing. Cambridge, MA: MIT Press.

....human participants and connectionist models are capable of learning sequential material that involves complex, disjoint, longdistance contingencies. We show that the popular Simple Recurrent Network model (Elman, 1990) which has otherwise been shown to account for a variety of empirical findings (Cleeremans, 1993), fails to account for human performance in several experimental situations meant to test the model s specific predictions. In previous research (Cleeremans, 1993) briefly described in this paper, the structure of center embedded sequential structures was manipulated to be strictly identical or ....

....the popular Simple Recurrent Network model (Elman, 1990) which has otherwise been shown to account for a variety of empirical findings (Cleeremans, 1993) fails to account for human performance in several experimental situations meant to test the model s specific predictions. In previous research (Cleeremans, 1993) briefly described in this paper, the structure of center embedded sequential structures was manipulated to be strictly identical or probabilistically different as a function of the elements surrounding the embedding. While the SRN could only learn in the second case, human subjects were found to ....

[Article contains additional citation context not shown here]

Cleeremans, A. (1993). Mechanisms of Implicit learning: Connectionist models of sequence processing. Cambridge, MA: MIT Press.

No context found.

Cleeremans, A. (1993a). Mechanisms of implicit learning: connectionist models of sequence processing. Cambridge, MA: MIT Press.

....representing a node of the grammar as abstract a representation as could be. In other cases, however, the network s internal representations tend to be organized in numerous very small clusters that each correspond to one or to a few training instances (see Servan Schreiber et al. 1991; Cleeremans, 1993; for detailed examples) The SRN has often been described as processing fragmentary information. This is descriptively correct, but it is not how things work inside the network. The network does not develop a database of subsequences that it can consult and ponder about as a result of training. ....

....would somehow consciously encode and memorize all these possible subsequences and use this knowledge to explicitly prepare for the next event. Yet, the reaction time data shows exquisitely detailed sensitivity to the ensemble of constraints resulting from an encoding of all the subsequences (see Cleeremans, 1993). There is no evidence whatsoever that subjects have conscious access to this kind of distributional information about the stimulus material. Further, the fact that subjects can consciously retrieve specific instances does not tell us anything about whether these instances are what performance is ....

Cleeremans, A. (1993). Mechanisms of implicit learning: Connectionist models of sequence processing.

....motivation to continue to try to anticipate the successive elements. Hence, to maintain the direct (i.e. generation) and indirect (i.e. SRT) tasks as similar as possible to each other, it seems preferable to use what we could call a continuous version of the generation task (see also Cleeremans McClelland, 1991; Cohen et al. 1990), in which the next stimulus as prescribed by the sequential structure is presented regardless of participants prediction responses, rather than either the standard or the free generation tasks. One concern with using a direct test that incorporates feedback information, however, is that the ....

Cleeremans, A. (1993a). Mechanisms of Implicit learning: Connectionist models of sequence processing. Cambridge, MA: MIT Press.

....and operative definitions of awareness are lacking. This approach was first proposed by Reingold Merikle (1988) in the context of subliminal perception. In this paper, we apply it to a choice reaction task in which the material is generated based on a probabilistic finite state grammar (Cleeremans, 1993). We show (1) that subjects progressively learn about the statistical structure of the stimulus material over training with the choice reaction task, and (2) that they can use some of this knowledge to predict the location of the next stimulus in a subsequent explicit prediction task. However, ....

....are not only equally direct but also require subjects to perform different discriminations in each case. In sequence learning situations, however, subjects are typically placed in a serial CRT task and kept unaware of the existence of sequential regularities. As has been shown repeatedly (see Cleeremans, 1993), the fact that their performance improves over training reflects not only unspecific practice effects, but also an encoding of the sequential constraints present in the material. This developing sensitivity is clearly indirect in that the discriminations between predictable and unpredictable ....

[Article contains additional citation context not shown here]

Cleeremans, A. (1993). Mechanisms of implicit learning: connectionist models of sequence processing. Cambridge: MIT Press.

No context found.

Cleeremans, A. (1993a). Mechanisms of implicit learning: Connectionist models of sequence processing. Cambridge, MA: MIT Press.

....differentially affected by the availability of attentional resources. In this paper, I propose a new information processing model of sequence learning and explore how well it can account for these data. The model is based on the Simple Recurrent Network (Elman, 1990; Cleeremans McClelland, 1991; Cleeremans, 1993), which it extends by allowing additional information to modulate processing. The model implements the notion that awareness of sequence structure changes the task from one of anticipating the next event based on temporal context to one of retrieving the next event from short term memory. This ....

....SRN s output units to represent response strength, and assumed that human subjects prepare implicitly for the next element. With these assumptions in place, the SRN model is able to account in substantial detail for the results of several sequence learning experiments, such as those reported by Cleeremans (1993), by Lewicki, Hill, and Bizot (1988) and by Cohen, Ivry, and Keele (1990) The model implements a series of principles central to implicit learning performance, such as elementary, gradual, associative learning, processing that is local and results in fragmentary knowledge, and sensitivity to ....

[Article contains additional citation context not shown here]

Cleeremans, A. (1993). Mechanisms of implicit learning: Connectionist models of sequence processing. Cambridge, MA: MIT Press.

....develop as a result of processing may or may not be more general than what could be derived from a list of the processed instances, depending on a variety of factors, such as the network s representational resources or the demands of the task. For instance, a simple recurrent network (SRN, see Cleeremans, 1993a; Elman, 1990) trained to process sequences generated from a finite state grammar will sometimes develop internal representations that are organized in clusters, with each cluster representing a node of the grammar as abstract a representation as could be. In other cases, however, the network s internal ....

Cleeremans, A. (1993a). Mechanisms of implicit learning: Connectionist models of sequence processing. Cambridge, MA: MIT Press.

No context found.

Axel Cleeremans. Mechanisms of Implicit Learning - Connectionist Models of Sequence Processing, in the Neural Networks Modeling and Connectionism series. Cambridge, Massachusetts#: MIT Press, 1993.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC