Jordan, M. I. (1990). Serial order: A parallel distributed processing approach. Tech. Rep., MIT, Cambridge, MA.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....making them more difficult to deal with. For the recognition of those single modalities, only few systems make use of connectionist models [3,7,17,27] for they are not considered well suited to completely address the problem of time alignment and segmentation. However, some neural architecture [10,14,29] has been put forward and successfully exploited to partially solve problems involving the generation, learning or recognition of sequence of patterns. Recently, several research groups have more thoroughly addressed the issue of combining verbal and non verbal behavior. In this context, most of ....

....in space, roughly passing through the base and the tip of the index finger. This line does not usually lie in the target plane, but may intersect it at some point. We recognize pointing gestures by means of a hybrid partial recurrent artificial neural network (RNN) consisting of a Jordan network [14] and a static network with buffered input to handle the temporal structure of the movement underlying the gesture. Concurrently, several speech commands can be issued asynchronously. They are recognized using Dragon 4.0, a commercial speech engine. Speech along with gestures is then used to put ....

[Article contains additional citation context not shown here]

Jordan M., Serial Order: A Parallel Distributed Processing Approach Advances in Connectionist Theory, Lawrence Erlbaum, 1989.

....reward, if this delay is bounded above by a known upper bound N. In [20] and [56] look ahead planning based on neural networks is successfully applied to real time control of a robot arm. For example, in [56] the task is to touch a rolling ball with a robot arm. Recurrent neural networks [10, 15, 61] are trained to predict the robot arm behavior as well as the movement of the ball, and look ahead planning with N = 5 allowed for touching the ball in real time in about 70 of the cases. However, planning with larger horizons suffered significantly from local minima due to gradient descent, as ....

Michael I. Jordan. Serial order: A parallel distributed processing approach. Technical Report ICS Report 8604, Institute for Cognitive Science, University of California, 1986.

....JIDM, thus better results are expected if the global maximum can be found. Evaluating the performance of the proposed structure, comparisons with classical network structures were performed. The T CombNET structure designed using Approach 2 is compared with Kohonen s LVQ1 [5] Elman and Jordan [3] RNNs trained with the standard Backpropagation algorithm, single 3 layer feedforward backpropagation trained Multi Layer Perceptron (MLP) and original CombNET II structures. The LVQ1, MLP and CombNET II are not recurrent neural networks, so they needed a pre processing stage by the previously ....

Jordan, M. I.: Serial order: A parallel distributed processing approach. Technical Report Nr. 8604, Institute for Cognitive Science, University of California, San Diego (1986)

....Whereas all of the early models were hand wired networks with local representations, some recent production models make use of distributed representations acquired from learning algorithms. In addition, recent architectures allow for the production of true sequences (Eikmeyer Schade, 1991; Jordan, 1986; Gupta, 1996; Hartley Houghton, 1996; Houghton, 1990; MacKay, 1987) In this article, we examine some connectionist models of production. Our aim is not to review the field, but rather to concentrate on our own recent efforts in two areas, lexical access and grammatical encoding. Lexical ....

....we present, the phonological error model, confronts these limitations. The Phonological Error Model The phonological error model (Dell, Juliano, Govindjee, 1993) is an attempt to apply PDP principles specifically to phonological encoding. The model uses a simple recurrent network (Elman, 1990; Jordan, 1986) to map from a static representation of a word to a sequence of phonological features. Figure 3 shows the architecture. The input layer represented the word to be spoken. In different versions of the model, the input was either a random bit vector (which can be viewed as either a lemma or a ....

Jordan, M. I. (1986). Serial order: A parallel distributed processing approach (ICS Technical Report 8604). University of California at San Diego, La Jolla, CA.

....are presented at the advice layer, and this input activity is used to direct the settling of the attractor network at the plan layer. The configuration of activity levels at the plan is then used to modulate the behavior of a task oriented network, much like the plan layer of a Jordan network [ Jordan, 1986 ] The connection weights may be trained using a standard inductive learning technique, such as backpropagation, with an error signal provided only on actual task performance. Such inductive learning may be used both to learn the instructional language and to shape task execution via experience. ....

Michael I. Jordan. Serial order: A parallel distributed processing approach. Technical report, Institute for Cognitive Science, UCSD, 1986.

....is used, which can use a target at each time step (e.g [10, 2] it is not obvious how to provide these targets. For classification purpose often there is only one target for the whole time series. Thus, this single target is used for all time steps or it is only applied to some time steps [7]. In the former case for some time steps (e.g. the first ones) this target is much too strong, which results in inappropriate weight updates. In the later case one needs a strategy to find out which time steps should be given a target. In many cases finding such a strategy is not possible as one ....

....network to recognize the whole series after processing the first time step. Using the overall target in the first time step seems to be a too strong teacher for the network. To overcome this problem one could provide constraints for the output units instead of actual targets, like it is done in [7]. But it is often hard to find such constraints. Thus, an automatic procedure for creating the targets can support the learning. b) For high classification rates the output vector o does not have to reach the value of either t 1 or t 2 . It only has to be in the proper half of the unit square. ....

M. I. Jordan. Serial order: A parallel distributed processing approach. Technical report, Institute for Cognitive Science, University of California, 1986.

....is used, which can use a target at each time step (e.g [10, 1] it is not obvious how to provide these targets. For classi cation purpose often there is only one target for the whole time series. Thus, this single target is used for all time steps or it is only applied to some time steps [7]. In the former case for some time steps (e.g. the rst ones) this target is much too strong, which results in inappropriate weight updates. In the later case one needs a strategy to nd out which time steps should be given a target. In many cases nding such a strategy is not possible as one does ....

....network to recognize the whole series after processing the rst time step, so using the overall target in the rst time step seems to be a too strong teacher for the network. To overcome this problem one could provide constraints for the output units instead of actual targets like it is done in [7]. But it is often hard to nd such constraints. Thus, an automatic procedure for creating the targets could support the learning. b) For high classi cation rates the output vector o does not have to reach the value of either t 1 or t 2 . It only has to be in the proper half of the unit square. ....

M. I. Jordan, \Serial order: A parallel distributed processing approach," Tech. rep., Institute for Cognitive Science, University of California, 1986.

....of the proposed way of learning is provided in Section 5. Finally, Section 6 concludes the paper. 2. Motivation As already pointed out, the choice of the actual values of the targets is an important part of the learning process. Although some techniques and heuristics for choosing targets exist [3, 11, 6, 13], a more general approach addressing the problem is desired. The class information should only be used implicitly during learning, which means that the class information does not have to be encoded into some numerical values, the targets, at all. The class information should only be used during ....

M. I. Jordan. Serial order: A parallel distributed processing approach. Technical report, Institute for Cognitive Science, University of California, 1986.

....a target is needed. For example in the first time step one cannot expect the network to recognize the whole series, so a target in the first step seems too hard for the network. To overcome this problem one could provide constraints for the output units instead of actual targets like it is done in [5]. But it is often hard to find such constraints. So, an automatic procedure for creating the targets could support the learning. b) It is easy to see that for a high classification rate the output vector o does not have to reach the value of either t 1 or t 2 . It only has to be in the proper ....

M. I. Jordan. Serial order: A parallel distributed processing approach. Technical report, Institute for Cognitive Science, University of California, 1986.

....that make a robot arm to follow a desired trajectory. In other cases one may be interested in modeling the dynamic behavior of a plant and an adaptive network may be trained to emulate the input output temporal response. As a last example a recurrent network may be forced to act as an oscillator [17]. Architectures and training strategies may be substantially different depending on the particular applicative field. A typical difficulty that arises in motor control related tasks is that one wants to use the output of the network as control actions such as forces or torques to be applied to ....

....be generated by dynamic programming. They go bejond the scope of this paper and the reader may refer to [19] 20] 21] 22] A recurrent network can generate trajectories also without applying an external input. In such cases the network behaves like an autonomous oscillator. The Jordan net [17] is a first example of a network with feedback connections trained to oscillate on a periodic attractor. In that case feedback weights were fixed 7 and feedforward weights were trained by backpropagation. Of course the approach is not optimal and a gradient can be computed also for the feedback ....

M. Jordan, "Serial order: a parallel distributed processing approach," Tech. Rep. 8604, ICS (Institute for Cognitive Science, University of California), 1986.

.... grammar (Hanson and Kegl 1987) However, for the purpose of learning regular string grammars it is more natural to use recurrent networks (which are essentially trainable deterministic finite automata) Various architectures have been used: simple first order recurrent networks (Elman 1990, Jordan 1988); more complex first order networks (Williams and Zipser 1989, Fahlman 1991) and second order recurrent networks (Giles et al. 1992) Elman (1992) has also applied recurrent networks to context free grammars and found that they can represent up to about three levels of recursive embedding; other ....

Jordan, M.I., 1988, Serial Order: A Parallel Distributed Processing Approach.

....presented at the advice layer, and this input activity is used to direct the settling of the attractor network at the plan layer. The stable configuration of activity levels at the plan is then used to modulate the behavior of a task oriented network, much like the plan layer of a Jordan network (Jordan, 1986). The connection weights may be trained using a standard inductive learning technique, such as backpropagation, with an error signal provided only on actual task performance. This allows the language of instruction to be learned in the service of a task (St. John, 1992) Such inductive learning ....

Jordan, M. I. (1986). Serial order: A parallel distributed processing approach. Technical report, Institute for Cognitive Science, UCSD.

....Some architectures of recurrent networks. a) backpropagation through time; b) Jordan net; c) Elman net. 11 BPTT is an efficient algorithm, but it is inherently discrete. We discuss a continuous time analog of BPTT (Pearlmutter) in section II.B.6. II.B. 3 Jordan and Elman nets Jordan style [Jor86] and Elman style [Elm90] networks are basically feedforward nets with their output layer and hidden layer, resp. recycled back as input (Figure II.1) These networks provide limited form of recurrence without a need for new learning algorithms: the recurrent links are not trained, but have a ....

M.I. Jordan. Serial order: a parallel distributed processing approach. Technical Report Technical Report 8604, Institute for Cognitive Science, University of California, San Diego, La Jolla, CA, 1986.

....the corner and then initiate a free from corner routine e.g. by backing up or making a U turn until it finds itself in free space again. Various neural network paradigms incorporate memory themselves by using recurrent links that feed back information. For instance in the Jordan network (Jordan, 1986) output information is fed back to state units that are presented together with the new input. The original Jordan network was created to generate sequences of outputs. In the robot domain it could be used for the generation of trajectories. In Elman s simple recurrent network (SRN, Elman, 1990) ....

Jordan, M. (1986). Serial order: A parallel distributed processing approach. Technical Report 8604, Institute for Cognitive Science.

....net s performance regularly during training. Just as phonemic context is often taken into account in rule sets, it can also be used for classification in neural net al..gorithms. In this case, recurrent networks have to be used. There are two main flavours of recurrent networks: Jordan networks ((Jordan 1988), cited after (Zell 1993) and Elman networks (Elman 1990) In both approaches, the previous state of a layer is fed back into the input layer. With Jordan networks, this is the output layer, with Elman networks, the hidden layer. A sample Jordan network is shown in Fig. 6.7. Recurrent nets have ....

Jordan, M. (1988), Serial order: A parallel distributed processing approach, in J. Elman & D. Rumelhart, eds, `Advances in Connectionist Theory: Speech', Erlbaum, Hillsdale, NJ.

No context found.

Jordan, M. I. (1990). Serial order: A parallel distributed processing approach. Tech. Rep., MIT, Cambridge, MA.

No context found.

M. I. Jordan 1989, "Serial order: A parallel, distributed processing approach," in Advances in Connectionist Theory: Speech,eds.J.L.ElmanandD. E. Rumelhart (Erlbaum, Hillsdale).

No context found.

M. I. Jordan. Serial order: A parallel distributed processing approach. Technical Report ICS Report 8604, Institute for Cognitive Science, University of California, 1986.

No context found.

M. I. Jordan, Serial Order: A Parallel Distributed Processing Approach. Technical Report ICS Report 8604, Institute for Cognitive Science, University of California, B1986

No context found.

M. Jordan. Serial Order: A Parallel Distributed Processing Approach. In J. Elman and D. Rumelhart, editors, Advances in Connectionist Theory, San Diego, La Jolla, CA, 1989. Institute for Cognitive Science, University of California.

No context found.

M. I. Jordan. Serial order: a parallel distributed processing approach. Technical Report 86044, Institute for Cognitive Science, University of California San Diego, 1986.

No context found.

Jordan, M. I. [1986], Serial order: A parallel distributed processing approach, ICS Report 8604, Institute for Cognitive Science, University of California, San Diego.

No context found.

M. I. Jordan 1989, "Serial order: A parallel, distributed processing approach," in Advances in Connectionist Theory: Speech,eds.J.L.ElmanandD. E. Rumelhart (Erlbaum, Hillsdale).

No context found.

Michael Jordan. Serial order: a parallel distributed processing approach. Technical Report ICS Report No. 8604, Institute for Cognitive Science; University of California at San Diego, 1986.

No context found.

Jordan, M. (1986). Serial Order: a Parallel Distributed Processing Approach. ICS Report No. 8604, Institute for Cognitive Science, University of California at San Diego.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC