T.J. Sejnowski and C.R. Rosenberg. NETtalk: A parallel network that learns to read aloud. Technical Report JHU/EECS-86/01, John Hopkins University, Baltimore, MD, 1986.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....the sentence carry no mutual information. Sentences like The cat is asleep. and Soon the cat is asleep. have no common feature in 44 the input vector. ffl Process a window that moves along through the sentence, as done by Charniak (1987) Sejnowski (1987) Nakamura (1989) or Benello (1989) [87, 88, 44, 45]. ffl Take relative rather than absolute word position, and use combinations of adjacent words to get order information, as used by Wyard and Nightingale (1990) in Hodyne [18] This is the approach adopted in our work. Similarly, sets of overlapping triples can model phonemic sequences as in ....

....window representation, this method preserves local context. Since the set of all local contexts in a sentence is used, this model can preserve more information for a given tuple length than a single window of the same length, such as Nakamura s 2 or 3 word window. However, windows are often longer [45, 88]. Weighted (adj,noun) prep,det) Inputs (noun,verb,noun) prep,det,prep) Figure 3.6: Architecture of the Hodyne net Operation of Hodyne The input patterns are binary vectors, initialised to zero, each representing one string generated by the CFG. Each element of the vector represents a tag ....

T Sejnowski and C Rosenberg. NETtalk: a parallel network that learns to read aloud. Technical report, John Hopkins University, Electrical Engineering and computer Science Dept., 1986.

....network structure of interconnected neurons in the brain. Especially neural networks have induced great interest, since they are capable of finding rather difficult relations in the real world. A classical example is NETtalk, a neural network which found out how to read aloud written English text [25]. However, even neural networks are not perfect. Several variants have been proposed, which create better approximations in some situations, are faster or do have some other nice properties. Still, little is known about what methods should be used in what situations. This Master s thesis studies ....

T. J. Sejnowski and C. R. Rosenberg. NETtalk: A parallel network that learns to read aloud. Technical Report JHU/EECS-86/1 , John Hopkins University Department of Electrical Engineering and Computer Science, 1986.

....statistical assumptions on which they rest. They are suspect or powerless when assumptions about the underlying distributions cannot be trusted or when the data is scarce. 1.3. 2 Practical Motivation Arti cial Neural Networks (ANN) have been successfully applied in the area of speech generation [44], speech recognition [50] vision and robotics [35, 18] handwritten character recognition [28] medical diagnostics [22] and game playing [47] However their inability to produce plausible explanations render them useless for safety critical applications like Power System Protection where ....

T. J. Sejnowski and C. R. Rosenberg. Nettalk: A parallel network that learns to read aloud. Technical Report JHU/EECS-86/01, John Hopkins University, 1986.

.... prediction task [2] The B OWN corpus English text database [9] was used 2As pointed out by a referee, the idea for predicting a phonologi cal unit, given the previous ones, can be found in the work by Shillcock et al. 7] Additionally, a somehow related work is Sejnowski and Rosenberg s NETtalk [8], in which they employ a feed forward neural network (with the sliding window approach) to convert English text to speech. WORD CATEGORY Output units (89) Hidden units (16) Hidden units (16) i Hidden units (16) Input units (89) Input units (89) Input units (89) 4 G [LA.M TRIGRAM BIGRAM ....

T. J. Sejnowski and C. R. Rosenberg. NETtalk: A parallel network that learns to read aloud. Tech. Report 86-01, Dep. of Electrical Engineering and Computer Science, Johns Hopkins Univ., Balti- more, MD, 1986.

.... Tuning operations are, for example, the choice of appropriate parameters like the learning and momentum rate [Fahlman 1988] annealing schedules for the learning rate (that is progressively reduced) Malferrari et al. 1990] updating schemes based on summing the contributions of related patterns [Sejnowski and Rosenberg 1986], small batches , selective corrections only if the error is larger than a threshold (that may be progressively reduced) Vincent 1991] Allred and Kelly 1990] randomization of the sequence of pattern presentation, etc. The given references are only some ex amples of significant ....

Sejnowski, T.J. and C.R. Rosenberg. NETtalk: a parallel network that learns to read aloud. The Johns Hopkins University EE and CS Technical Report JHU/EECS-86/01.

....model of Donald Duck, I will evaluate the claims made by Churchland and Ramsey et al. 7. 5 The power of subsymbolic computing A standard example of a connectionist model that displays interesting emergent behaviour is NETtalk (the famous parallel network that learns to read aloud ) developed by [Sejnowski and Rosenberg 1986]. Proponents of subsymbolic connectionism assume that the hidden units (the units between the input and output layers) in connectionist models like NETtalk exhibit subsymbolic representations and thus illustrate the power of subsymbolic computing. Although the designers of NETtalk acknowledge the ....

....power of subsymbolic computing. Although the designers of NETtalk acknowledge the differences between the architecture of NETtalk and the brain, they assume that NETtalk can teach us how information (in this case letter to phoneme mappings) could be represented in large populations of neurons [Sejnowski and Rosenberg 1986, p. 670] Churchland and Sejnowski 1989, p. 244] indicate that it yields clues to how the nervous system can embody models of various domains of the world . With NETtalk Sejnowski and Rosenberg have quite successfully modelled the conversion of English text to speech. NETtalk is proposed as a ....

[Article contains additional citation context not shown here]

Sejnowski T.J., Rosenberg C.R.: NETtalk: a parallel network that learns to read aloud, The Johns Hopkins University Electrical Engineering and Computer Science technical report 86/01, 1986.

....the inputs pattern does not contain the feature that the units were trained to recognize, their outputs will be inhibited. Backpropagation networks provide an effective means to examine data patterns that may be incomplete or noisy, and to recognize subtle patterns from the partial input. NETtalk [39] is an example of a neural network that used the backpropagation algorithm to learn to read out loud. The input layer had 7 groups of 29 units. The hidden layer was comprised of 80 units and attempted to improve the feature detection needed for the input output transformation. The output layer had ....

T. J. Sejnowski and C. R. Rosenberg, "NETtalk: a parallel network that learns to read aloud," in Neurocomputing: Foundations of Research (J. Anderson and E. Rosenfield, eds.), MIT Press, 1988.

.... has been an active area of research in artificial intelligence research (e.g. Matheus Rendell, 1989; see Wisniewski Medin, 1994a for a review) Likewise, researchers who develop connectionist models of learning have been concerned with how a model might form internal representations (e.g. Sejnowski Rosenberg, 1986) or develop feature detectors (e.g. Rumelhart Zipser, 1986) Thus, there is good reason to hope that further progress on this issue will be made in the near future. In contrast to this favorable picture of how current models of categorization can and might address influences of specific ....

Sejnowski, T. J., & Rosenberg, C. R. (1986). NETtalk: A parallel network that learns to read aloud (Technical Report No. JHU/EECS-86/01). Johns Hopkins University, Department of Electrical Engineering and Computer Science.

....TM y and CM5 TM z show that our scheme performs better than the other schemes, both for uniform and non uniform networks. 1 Introduction The Backpropagation algorithm (BP) 1] is one of the most popular neural network learning algorithms. It has been used in a large number of applications [2, 3, 4, 5]. This algorithm is computation intensive and as a result there has been a great interest in developing parallel formulations of this algorithm for a variety of parallel computers. BP can be parallelized either by network partitioning or by pattern partitioning. In network partitioning schemes, ....

....b d a W ce W cd W be W bd W ae W ad e d c b a Nodes: Figure 5: The partition for non uniform networks 6 Non uniform Networks Non uniform networks are characterized by layers with different numbers of nodes. Non uniform networks are interesting, since these are used in many applications [2, 25]. Let I 0 ; I 1 ; I J be the number of nodes in the different layers of the network. The network is uniform iff I 0 = I 1 = Delta Delta Delta = I J ; otherwise, the network is called a non uniform network. Our scheme can be extended to non uniform networks. For per pattern training, we ....

T. J. Sejnowski and C. R. Rosenburg. Nettalk: a parallel network that learns to read aloud. Technical Report JHU/EECS-86/01, Johns Hopkins University - EE/Csci, 1986.

....DR with E=U=8, binary units (I=I u =2) and Pmax =I E =256 (Xu and Zheng, 1992) The input representation in the binary to local problem (Jacobs, 1988) is another example of the one to one DR with E=U=3, binary units (I=I u =2) and Pmax =I E =8. The output representation of NETtalk (Sejnowski and Rosenberg, 1986) can be regarded as an example of the one to one DR with E=26, U=26, binary units (I u =I=2) and Pmax =I E 6.7x10 7 . This Pmax is large enough to guarantee that the NETtalk can generate most of the phonemes and punctuations that may encounter in reading texts. The input representation and ....

....we change its name to one to many DR here. From the pattern to unit correspondence viewpoint, one to many DR is a many to many correspondence because each pattern is represented by many units, and each unit participates in representing many patterns. The input representation in NETtalk (Sejnowski and Rosenberg, 1986) is an example of the one to many DR with E=7, I=29, U=E I=203, and binary units. The entity in NETtalk is the seven character window of text. The patterns are all possible strings of seven characters. E=7 because each pattern is decomposed into seven elements (positions of characters) I=29 ....

[Article contains additional citation context not shown here]

Sejnowski, T. J., and Rosenberg, C. R. (1986). NETtalk: a parallel network that learns to read aloud. The Johns Hopkins University Electrical Engineering and Computer Science Technical Report JHU/EECS-86/01, 32pp.

....spellings of a proper noun. The system architecture allows efficient searches for hypotheses (encoded as combinations of the binary states of various network units or neurons) that maximally satisfy the constraints resulting from the input data and the weighted interaction between individual units [3, 4] by capturing the input statistics seen during training. We have extended the basic network in [2] by adding more flexibility in architecture and algorithmic features to create a more powerful system capable of efficiently generating an ordered list of the N most likely pronunciations of the ....

T.J. Sejnowski and C.R. Rosenberg, ""NETtalk: A Parallel Network That Learns To Read Aloud," Tech. Rep. JHU/EECS-86/01, John Hopkins University, Baltimore, MD, 1986.

.... A pronunciation network is trained using a multilayer perceptron to predict alternative pronunciation A(m) from the five phonemes (i.e. quinphone) of canonical pronunciations L(m02) L(m 2) Figure 1 shows the network structure, which has a structure similar to that employed in NETtalk [10]. L(m 0 2) L(m 2) are given for the pronunciation network inputs; A(m) aligned to L(m) are given for the outputs. A total of 130 units (26 Japanese phoneme sets times 5 contexts) are used in the input layer. The representation of alternative pronunciations at the output layer is ....

T. Sejnowski and C. Rosenberg : "NETtalk: a parallel network that learns to read aloud," The Johns Hopkins Univ. Electrical Engineering and Computer Science Tech. Report JHU/EECS-86/01, 1986.

....data can be prepared by generating an alternative pronunciation sequence and mapping it to the canonical pronunciation as follows. 1. Conduct phoneme recognition using speech training data for dictionary generation. The recognized 1 This network structure is similar to that employed in NETtalk [15], which can predict an English word pronunciation from its spelling. Note that the pronunciation network is designed to predict alternative pronunciations, for the purpose of improving the performance in spontaneous speech recognition, while NETtalk is designed to predict canonical pronunciations ....

T. Sejnowski and C. Rosenberg: "NETtalk: a parallel network that learns to read aloud," The Johns Hopkins Univ. Electrical Engineering and Computer Science Tech. Report JHU/EECS-86/01, 1986.

.... task [2] The BROWN corpus English text database [9] was used 2 As pointed out by a referee, the idea for predicting a phonological unit, given the previous ones, can be found in the work by Shillcock et al. 7] Additionally, a somehow related work is Sejnowski and Rosenberg s NETtalk [8], in which they employ a feed forward neural network (with the sliding window approach) to convert English text to speech. PSfrag replacements WORD CATEGORY BIGRAM TRIGRAM 4 GRAM Input units (89) Input units (89) Input units (89) Output units (89) Hidden units (16) Hidden units (16) Hidden units ....

T. J. Sejnowski and C. R. Rosenberg. NETtalk: A parallel network that learns to read aloud. Tech. Report 86-01, Dep. of Electrical Engineering and Computer Science, Johns Hopkins Univ., Baltimore, MD, 1986.

....neural network. We also present some simulation results of sunspot prediction problem. Keywords evolutionary algorithms, neural networks, sunspot prediction 1 INTRODUCTION Multilayer perceptron (MLP) neural networks have been used for solving many interesting problems like speech generation (Sejnowski 1986), classification of sonar signals (Gorman 1988) and identification of genetic facies (Cardon, Hoogstraten and Davies 1991) MLP networks can be trained for solving a particular problem by using a simple learning algorithm like backpropagation (Rumelhart, Hinton and Williams 1986) The training is ....

Sejnowski T. J. & Rosenberg C. R., "NETtalk: a parallel network that learns to read aloud," Technical Report JHU/ EECS-86/01. The Johns Hopkins University Electrical Engineering and Computer Science, 1986.

....representation for the input vectors, in fact, perform data compression. In this case a binary coding of the input vectors emerges in the hidden layer. In the general case the internal representations do not have any interpretation in terms of obvious higher level features of the input. Sejnowski [25] for instance states that it is not possible to give any obvious meaning to the internal representations in his application. Sejnowski used both the Bolzmann algorithm and back propagation to train a feed forward net to translate english written text to phonemes. 6 Representation of Time ....

T.J. Sejnowski and C.R. Rosenberg. Nettalk: a parallel network that learns to read aloud. The John Hopkins University Electrical Engineering and Computer Science Technical Report, JHU/EECS-86/01:32 pp, 1986.

.... such as the spatial relationships between letters in a word. There are various ways of getting around this problem: the triple representation is one example, another is to have 26 units for each letter position in a seven letter window (this method is used in the NETtalk text to speech system (Sejnowski Rosenberg, 1986)) Recent work is investigating the use of time delay networks for representing sequences (Elman, 1988) this is a new approach to representation in neural networks and may provide an answer to these problems. The representation of sequence in SaMc and similar networks is not adequate for modeling ....

Sejnowski, T. J. & Rosenberg, C. R. (1986). NETTALK: a parallel network that learns to read aloud. Technical Report JHU/EECS-86/01, Johns Hopkins University EE and CS.

.... two spirals , proposed by Alexis Wieland. The main emphasis of this benchmark test was to investigate the convergence rate of the learning error (ignoring the performance or the generalization issues) in the conjunction of different pool size. The second benchmark was chosen Net Talk problem [13]. Here main focus is payed to compare the generalization between incrementally and conventionally trained networks. 5.1 Two spirals problem A standard data set for the two spirals problem consists of 194 data points arranged on X Y plane into two interlocking spirals (see fig.3) 8 6 4 2 ....

....too large with respect to the learning set size. But the two spirals benchmark is not a typical example to study the generalization. We address the generalization of the incrementally learned network in the second benchmark. 5. 2 Net Talk problem The second application was Net Talk problem [13]. It is a benchmark of a real word problem when a feedforward network is trained to produce the proper phonemes, given a string of letters as input. The feedforward network with 203 inputs, 26 outputs and 60 hidden nodes has about 14 thousand parameters. We used data files supplied with ....

T. J. Sejnowski. NET Talk: a parallel network that learns to read aloud. Complex Systems, 1:145--168, 1987.

....rules from networks with real valued and distributed representations. 1 Introduction In the last few years artificial neural networks have been applied successfully to a variety of real world problems. For example, neural networks have been successfully applied in the area of speech generation [12] and recognition [18] vision and robotics [8] handwritten character recognition [5] medical diagnostics [11] and game playing [13] While in these and other approaches neural networks have frequently found to outperform more traditional approaches, one of their major shortcomings is their low ....

T. J. Sejnowski and C. R. Rosenberg. Nettalk: A parallel network that learns to read aloud. Technical Report JHU/EECS-86/01, Johns Hopkins University, 1986.

....relations are complex, if training data is noisy and attributes are real valued, neural network approaches have often been reported to yield performance that compares favorably to other methods. For example, artificial neural networks have been successfully applied in the area of speech generation [Sejnowski and Rosenberg, 1986] and recognition [Waibel, 1989] vision and robotics [Pomerleau, 1989] Hsu and Simmons, 1991] handwritten character recognition [LeCun et al. 1990] medical diagnostics [Jabri et al. 1992] and game playing [Tesauro, 1992] Some of these approaches clearly outperformed other, more ....

....the extrac Extracting Symbolic Knowledge from Artificial Neural Networks 39 tion of certain types of symbolic rules (namely m of n rules) from trained networks. Craven and Shavlik managed, most notably, to extract rules from a reduced version of Sejnowski and Rosenberg s NETtalk domain [Sejnowski and Rosenberg, 1986] . Note that the weight regularization term may replace the need for initial knowledge, as reported in [Towell, 1991] and [Towell and Shavlik, 1992] In both of these extraction schemes the effectiveness of the rule extraction mechanism, as well as the degree of correctness of the extracted ....

[Article contains additional citation context not shown here]

T. J. Sejnowski and C. R. Rosenberg. Nettalk: A parallel network that learns to read aloud. Technical Report JHU/EECS-86/01, John Hopkins University, 1986.

....of computer time to run effectively. Furthermore, connectionism has not developed sufficiently to allow high level structure to be formed to the extent present in more conventional models. An example of this limitation is illustrated by the behaviour of the connectionist computer model Net talk [21], 3.3.2 A Brief Review of Net talk Net talk is a three layer neural network using a back propagation learning algorithm and is capable of learning the conjugation, in the past tense, of English verbs both regular and irregular. With extensive training, the program acts to self organise, ....

....domain specific solutions. As a result, placing the pendulum in a situation outside the training domain, for example giving the pivot a velocity, inevitably requires retraining of the system, usually negating previously learned material 1 This is the same limitation shown by Net talk in section[21], when attempting to change accents. A more detailed look at one particular approach illustrates the limitations of contemporary, decentralised approaches to this problem. 6.3.1 Control of an Inverse Pendulum Using Neural Networks Saravanan[44] presents a multi layer neural network to control an ....

Sejnowski, T. J. Rosenberg, C. R. (1986) Net-talk: A Parallel Network That Learns to Read Aloud, Technical Report JHU/EECS-86/01, Johns Hopkins University, Baltimore, MD.

....minimum. It is interesting to note that in order to speed up the optimization, or to help BP find the right solution, either the architecture of the network is designed in a way the designer thinks might be appropriate to the task [12] or the coding of the input patterns is made more favourable [12, 25], or specific significant patterns only are included in the training set [24, 31, 1] Why then not abandon the notion of a previously dumb network discovering mappings and associations, and replace it with one of using a network to solve a mapping problem This is the approach of the network ....

T. J. Sejnowski and C. R. Rosenberg. NETtalk: A parallel network that learns to read aloud. Complex Systems, 1(1), 1987. Constructive Algorithms 33

....models then they clearly outperform the statistical non linear threshold models. Keywords: evolutionary optimization, neural networks, sunspot prediction 1. 1 Introduction Multilayer perceptron (MLP) neural networks have been used for solving numerous different problems like speech generation [12], classification of sonar signals [3] and Larsen effect elimination [2] MLP networks can be trained for solving a particular problem by using a simple learning algorithm like backpropagation [11] The training is an automatic process, but unfortunately the selection of the network structure is ....

T. J. Sejnowski and C. R. Rosenberg. Nettalk: a parallel network that learns to read aloud. Technical Report JHU/EECS-86/01, The Johns Hopkins University, Electricial Engineering and Computer Science, 1986.

No context found.

T.J. Sejnowski and C.R. Rosenberg. NETtalk: A parallel network that learns to read aloud. Technical Report JHU/EECS-86/01, John Hopkins University, Baltimore, MD, 1986.

No context found.

Sejnowski T.J., Rosenberg C.R., NETtalk: A Parallel Network that Learns to Read Aloud, Johns Hopkins University Department of Electrical Engineering and Computer Science Technical Raport 86/01, 1986. 13

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC