We develop a connectionist approach to processing in quasi-regular domains, as exemplified by English word reading. A consideration of the shortcomings of a previous implementation (Seidenberg & McClelland, 1989, Psych. Rev.) in reading nonwords leads to the development of orthographic and phonological representations that capture better the relevant structure among the written and spoken forms of words. In a number of simulation experiments, networks using the new representations learn to read both regular and exception words, including low-frequency exception words, and yet are still able to read pronounceable nonwords as well as skilled readers. A mathematical analysis of the effects of word frequency and spelling-sound consistency in a related but simpler system serves to clarify the close relationship of these factors in influencing naming latencies. These insights are verified in subsequent simulations, including an attractor network that reproduces the naming latency data directly in its time to settle on a response. Further analyses of the network's ability to reproduce data on impaired reading in surface dyslexia support a view of the reading system that incorporates a graded division-of-labor between semantic and phonological processes. Such a view is consistent with the more general Seidenberg and McClelland framework and has some similarities with---but also important differences from---the standard dual-route account.

We propose that ambiguous prepositional phrase attachment can be resolved on the basis of the relative strength of association of the preposition with noun and verb, estimated on the basis of word distribution in a large corpus. This work suggests that a distributional approach can be effective in resolving parsing problems that apparently call for complex reasoning.

This paper proposes a new theory of the relationship between the sentence processing mechanism and the available computational resources. This theory -- the Syntactic Prediction Locality Theory (SPLT) -- has two components: an integration cost component and a component for the memory cost associated with keeping track of obligatory syntactic requirements. Memory cost is

The problems of access -- retrieving linguistic structure from some mental grammar -- and disambiguation -- choosing among these structures to correctly parse ambiguous linguistic input -- are fundamental to language understanding. The literature abounds with psychological results on lexical access, the access of idioms, syntactic rule access, parsing preferences, syntactic disambiguation, and the processing of garden-path sentences. Unfortunately, it has been difficult to combine models which account for these results to build a general, uniform model of access and disambiguation at the lexical, idiomatic, and syntactic levels. For example psycholinguistic theories of lexical access and idiom access and parsing theories of syntactic rule access have almost no commonality in methodology or coverage of psycholinguistic data. This paper presents a single probabilistic algorithm which models both the access and disambiguation of linguistic knowledge. The algorithm is based on a parallel parser which ranks constructions for access, and interpretations for disambiguation, by their conditional probability. Low-ranked constructions and interpretations are pruned through beam-search; this pruning accounts, among other things, for the garden-path effect. I show that this motivated probabilistic treatment accounts for a wide variety of psycholinguistic results, arguing for a more uniform representation of linguistic knowledge and for the use of probabilisticallyenriched grammars and interpreters as models of human knowledge of and processing of language.

This article provides an overview of a probabilistic constraints framework for thinking about language acquisition and processing. The generative approach attempts to characterize knowledge of language (i.e., competence grammar) and then asks how this knowledge is acquired and used. Our approach is performance oriented: the goal is to explain how people comprehend and produce utterances and how children acquire this skill. Use of language involves exploiting multiple probabilistic constraints over various types of linguistic and nonlinguistic information. Acquisition is the process of accumulating this information, which begins in infancy. The constraint satisfaction processes that are central to language use are the same as the bootstrapping processes that provide entry to language for the child. Framing questions about acquisition in terms of models of adult performance unifies the two topics under a set of common principles and has important consequences for arguments concerning lan...

The most predominant language processing theories have, for some time, been based largely on structured knowledge and relatively simple rules. These symbolic models intentionally segregate syntactic information processing from statistical information as well as semantic, pragmatic, and discourse influences, thereby minimizing the importance of these potential constraints in learning and processing language. While such models have the advantage of being relatively simple and explicit, they are inadequate to account for learning and validated ambiguity resolution phenomena. In recent years, interactive constraint-based theories of sentence processing have gained increasing support, as a growing body of empirical evidence demonstrates early influences of various factors on comprehension performance. Connectionist networks are one form of model that naturally reflect many properties of constraint-based theories, and thus provide a form in which those theories may be instantiated. Unfortunately, most of the connectionist language models implemented until now have involved severe limitations, restricting the phenomena they could address. Comprehension and production models have, by and large, been limited to simple sentences with small vocabularies (cf. St. John & McClelland, 1990). Most models that have addressed the problem of complex, multi-clausal sentence processing have been prediction networks (cf. Elman, 1991; Christiansen & Chater, 1999a). Although a useful component of a language processing system, prediction does not get at the heart of language: the interface between syntax and semantics.

This article addresses the question of how the processor decides on its initial strategy for syntactic ambiguity resolution. At a point of ambiguity, more than one analysis is possible. An effective strategy might be to adopt the analysis that has most frequently turned out to be correct in the past. Assuming that the world stays the same in most respects, the analysis that has most frequently been correct in the past should provide a good estimate of which analysis is most likely to be correct again. Hence, by adopting this analysis, the processor should make fewer errors than if it chose any other analysis

Two striking contrasts currently exist in the sentence processing literature. First, whereas adult readers rely heavily on lexical information in the generation of syntactic alternatives, adult listeners in world-situated eye-gaze studies appear to allow referential evidence to override strong countervailing lexical biases (Tanenhaus, Spivey-Knowlton, Eberhard, & Sedivy, 1995). Second, in contrast to adults, children in similar listening studies fail to use this referential information and appear to rely exclusively on verb biases or perhaps syntactically based parsing principles (Trueswell, Sekerina, Hill, & Logrip, 1999). We explore these contrasts by fully crossing verb bias and referential manipulations in a study using the eye-gaze listening technique with adults (Experiment 1) and Wve-year-olds (Experiment 2). Results indicate that adults combine lexical and referential information to determine syntactic choice. Children rely A portion of this work was presented in proceedings to the 23rd Annual Conference of the Cognitive Science Society. The ideas in this paper owe much to our conversations with Lila Gleitman and to the comments of the many audiences who heard preliminary reports of this research. We thank Kirsten Thorpe for her assistance with testing, coding, and participant recruitment and Sylvia Yuan for her assistance in data analysis. We also gratefully acknowledge Tracy Dardick who carried out the norming studies and Jared Novick and David January who assisted in comparisons between head-mounted eye-tracking and our procedure. This work was supported by NIH Grant 1-R01-HD37507 to the second author and a National Science Foundation Science and Technology grant to the Institute for Research in Cognitive Science at the University of Pennsylvania (NSF-STC Coo...

Abstract not found

This paper presents a corpus-based approach for deriving heuristics to locate the antecedents of relative pronouns. The technique duplicates the performance of hand-coded rules and requires human intervention only during the training phase. Because the training instances are built on parser output rather than word coocctnrences, the technique requires a small number of training examples and can be used on small to medium-sized corpora. Our initial results suggest that the approach may provide a general method for the automated acquisition of a variety of disambiguation heuristics for natural language systems, especially for problems that require the assimilation of syntactic and semantic knowledge.