W. Daelemans. Memory-based lex- ical acquisition and processing. In P..Steffens, editor, Machine Translation and the Lexicon, Lecture Notes in Artificial Intelligence, pages 85-98. Springer-Verlag, Berlin, 1995.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....The hope is that providing a coherent view of di erent learning approaches could help in developing better learning methods and a better understanding for the role of learning in natural language inferences. 2 Memory Based Learning Memory based learning (MBL) methods [ Stan ll and Waltz, 1986; Daelemans, 1995 ] are supervised learning methods that are based on the use of similarity metrics in the instance space to support future predictions. When used in natural language processing the idea is that each language experience leaves a memory trace that can be used to guide future processing. When a ....

....is processed, the k nearest neighbors of this target instance are retrieved from memory, according to some metric on the instance space and the target instance is classi ed by extrapolating on the labels of these k neighbors in some way. The key feature of memory based methods use in NLP [ Daelemans, 1995; Argamon et al. 1998 ] is that all examples are stored in memory and no attempt is made to simplify the model by eliminating noise, low frequency events or exceptions. In this way, the view is that rulelike behavior emerges from the linguistics regularities that are present in the patterns of ....

W. Daelemans. Memory-based lexical acquisition and processing. In P. Steens, editor, Machine Translations and the Lexicon, Lecture Notes in Articial Intelligence, pages 85-98. Springer Verlag, Berlin, 1995.

....new examples, they simply calculate a weighted sum of input features (linear combination) and outputs 0 if the result is below the threshold, and 1 otherwise. Wrongly predicted training examples make the weights of the model change, in a multiplicative way, to better fit the training set. 9 [55, 14, 56, 15], PoS tagging [61, 60, 90] PP attachment disambiguation [246] shallow parsing [227] and smoothing of probability estimates [245] The work of other authors include applications to partial parsing (chunking) and context sensitive parsing [210, 7, 33] WSD [159, 157, 84, 73] text ....

....of heuristics, SVM stands for Support Vector Machines, and, finally, LogL stands for Log linear Models. Table 1 contains information about low level NLP tasks, such as speech processing, morphology and PoS tagging. NB DTs HMMs ME TBL NNs Speech recognition and synthesis [8, 9] 97] 187] [55, 56, 15] [206, 121, 155, 113, 229] Morphology [14] PoS tagging [194, 189] 200, 132, 140, 141, 164, 136, 138, 137] 45, 54, 144] 101, 177] 21, 22, 23, 6, 186] 155, 201, 70, 199, 131] IBL LSM EC PoS tagging [61, 60, 90, 58] 188] 90, 24, 139, 2, 136] Table 1: References corresponding ....

W. Daelemans. Memory--based Lexical Acquisition and Processing. Machine Translation and the Lexicon, Lecture Notes in Artificial Intelligence 898. P. Steffens editor. Springer, Berlin, 1995.

....give a complete overview of this eld. Most recent language learning methods are based on some probabilistic or counting theories. Examples of these methods are Memory Based Learning (MBL) which keeps track of the distribution of contexts of words and assigns word types based on that information [Dae95]. MM90] describes a system that can nd constituent boundaries using mutual information of n grams within sentences. In [FC92] and [RCF98] models are proposed that use distributional information to acquire syntactical categories. Systems based on the Minimum Description Length (MDL) principle ....

Walter Daelemans. Memory-based lexical acquisition and processing. In P. Steens, editor, Machine Translation and the Lexicon, volume 898 of Lecture Notes in Articial Intelligence, pages 85-98. Berlin: Springer Verlag, 1995.

....texts. Although we did start with a children s dictionary in which there is no notion of part of speech, we used a tagger and parser to analyze the sentences. This is one area that could be further expanded. There is much exciting research on language acquisition and automatic grammar learning [52]. We transformed our parse trees into conceptual graphs, and applied multiple structural disambiguation heuristics. At that point as well there is much on going research on finding the right prepositional attachment or the right conjunction attachment. But, whatever method is used, we can always ....

....disambiguation heuristics, the process to find covert categories, and the process to build clusters. It is mostly processes and not data that were entered manually, and therefore, once they are in place, they can be used on all different data. Certainly some investigations into machine learning [52] might show some processes as learnable from data, such as morphological rules, part of speech tagging, parsing rules, or parse to CG transformation rules. But that assumes large set of sentences tagged, parsed, transformed into cg so that we can learn the process from them. It is probably ....

Walter Daelemans. Memory-based Lexical Acquisition and Processing. In Lecture Notes in Artificial Intelligence: Machine Translation and the Lexicon. 1994.

....language applications. The hope is that providing a coherent view of di erent learning approaches could help in developing better learning methods and a better understanding for the role of learning in natural language inferences. 2 Memory Based Learning Memory based learning (MBL) methods [15, 5] are supervised learning methods that are based on the use of similarity metrics in the instance space to support future predictions. When used in natural language processing the idea is that each language experience leaves a memory trace that can be used to guide future processing. When a ....

....instance is processed, the k nearest neighbors of this target instance are retrieved from memory, according to some metric on the instance space and the target instance is classi ed by extrapolating on the labels of these k neighbors in some way. The key feature of memory based methods use in NLP [5, 1] is that all examples are stored in memory and no attempt is made to simplify the model by eliminating noise, low frequency events or exceptions. In this way, the view is that rule like behavior emerges from the linguistics regularities that are present in the patterns of usage in memory, in ....

W. Daelemans. Memory-based lexical acquisition and processing. In P. Steens, editor, Machine Translations and the Lexicon, Lecture Notes in Articial Intelligence, pages 85-98. Springer Verlag, Berlin, 1995.

....earlier examples. Variations on this theme are known under names such as Analogy based, Example based, Instance based, Case based, Memory based, Experiencebased, Data Oriented, Usage Based and Exposure based models (see e.g. Scha (1992) Skousen (1989) R. 1991) Mitchell (1994) Cardie (1996) Daelemans (1995), and others) The basic idea is that language processing and learning are fundamentally interwoven. Each language experience leaves a memory trace which can be used to guide later processing. When a new instance of a task is input to the processor, a set of relevant instances are selected from ....

....neighbors. Note that no abstractions, such as grammatical rules are extracted from the examples. Rule like behavior results solely from the linguistic regularities that are present in the patterns of usage in memory in combination with the use of a (intelligent) similarity metric. Following Daelemans (1995) we will also call this approach Memory Based natural language processing (MB NLP) We will now discuss the similarity metrics that we used in the experiments below. 3.1 Similarity metrics Overlap metric. Because the features of a linguistic problem are often symbolic instead of numeric (e.g. ....

[Article contains additional citation context not shown here]

Daelemans, W. 1995. Memory-based lexical acquisition and processing. In P. Steffens, editor, Machine Translation and the Lexicon. Springer Lecture Notes in Artificial Intelligence, pages 85--98.

....approach to memory based syntactic pat tern recognition. we carve up the syntactic anal 53 ysis process into a number of such classification tasks with input vectors representing a focus iten and a dynamically selected surrounding context. As in Natural Language Processing problems in general [Daelemans, 1995], these classification tasks can be segmentation tasks (e.g. decide whether a focus word or tag is the start or end of an NP) or disambiguation tasks (e.g. decide whether a chunk is the subject NP, the object NP or neither) Output of some memorybased modules (e.g. a tagger or a chunker) is used ....

W. Daelemans. Memory-based lex- ical acquisition and processing. In P..Steffens, editor, Machine Translation and the Lexicon, Lecture Notes in Artificial Intelligence, pages 85-98. Springer-Verlag, Berlin, 1995.

....known beforehand. It is our claim that all useful linguistic tasks can be rede ned this way and can thus be taken on in a ML context. All linguistic problems can be described as mappings of two kinds: disambiguation (or identi cation) and segmentation (identi cation of boundaries) see Daelemans, 1995). Disambiguation. Given a set of possible categories and a relevant context in terms of attribute values, determine the correct category for this context. Instances of disambiguation include part of speech tagging (disambiguating the syntactic category of a word) grapheme to phoneme ....

.... 1989; Derwing Skousen, 1989; Chandler, 1992; Scha, 1992) In computational linguistics (apart from incidental computational work of the linguists referred to earlier) the general approach has only recently gained some popularity: e.g. Cardie (1994, syntactic and semantic disambiguation) Daelemans (1995, an overview of work in the early nineties on memory based computational phonology and morphology) Jones (1996, an overview of example based machine translation research) Federici and Pirrelli (1996) 3.1 Similarity Metric Performance of a lazy learning system (accuracy on the test set) ....

[Article contains additional citation context not shown here]

Daelemans, W. (1995). `Memory-based lexical acquisition and processing.' In Steens, P., editor, Machine Translation and the Lexicon, Lecture Notes in Articial Intelligence 898. Berlin: Springer, 85-98.

.... Language Processing, lazy learning techniques are currently being applied by various Japanese groups to parsing and machine translation under the names exemplarbased translation or memory based translation and parsing (Kitano, 1993) In work by Cardie (Cardie, 1993) and by the present authors (Daelemans, 1995; Daelemans et al. 1994) variants of lazy learning are applied to disambiguation tasks at di erent levels of linguistic representation (from phonology to semantics) One lazy learning variant, Analogical Modeling (Skousen, 1989) was explicitly developed as a linguistic model. On the one hand, ....

Daelemans, W. (1995). Memory-based lexical acquisition and processing. In Steens, P., editor, Machine Translation and the Lexicon, Lecture Notes in Articial Intelligence 898, Springer, pages 85-98.

....problem. Inductive learning is fundamentally a classification paradigm: given a description of an input in terms of a number of feature values, a classification of the input is performed. Most linguistic tasks can be described in this paradigm. Two types of classification tasks can be discerned (Daelemans, 1995): ffl Identification: given a set of possible classifications and an input of feature values, 2 determine the correct classification for this input. For example, given a letter surrounded by a number of neighbours (e.g. a in have) determine the phonemic transcription of that letter. ffl ....

....a boundary is associated with the focus position, and if so, which one. For example, determine if the b in table marks the boundary of a syllable. Differences exist in the ways inductive algorithms extract knowledge from the available instances. In lazy learning (such as memory based learning, Daelemans, 1995), there is no abstraction of higher level data structures such as rules or decision trees at learning time; learning consists of simply storing the instances in memory. A new instance of the same problem is solved by retrieving those instances from memory that match the new instance best ....

Daelemans, W. (1995). Memory-based lexical acquisition and processing. In: P. Steffens (Ed.), Machine Translation and the Lexicon, Springer Lecture Notes in Artificial Intelligence 898, 85--98.

....that discussion here, we can safely argue that the spc model is able to learn its knowledge automatically, without being bound to linguistic engineering or handwiring. The storage of knowledge in a tree and the use of a best guess strategy can be further specified as a kind of lazy learning (cf. Daelemans, 1995), with which learning is relatively straightforward storage of examples, and performance is some kind of intelligent, example based similarity matching. For a discussion on the psychological validity of lazy learning, see for example Smith and Medin (1981) Derwing and Skousen (1989) ii) The ....

Daelemans, W. 1995. Memory-based lexical acquisition and processing. In: P. Steffens (Ed.), Machine Translation and the Lexicon, Springer Lecture Notes in Artificial Intelligence 898, 85--98.

....the task dictates the acquisition method, and the acquisition method dictates which information (lexical and contextual) is needed to solve the task. There is therefore a shift from the reusability of the lexical knowledge to the reusability of the acquisition method (e.g. memory based learning, Daelemans (1995)) e.g. in word sense disambiguation, both lexical and contextual information is needed for acceptable performance. By providing a learning algorithm with a sufficient amount of examples of word sense disambiguation instances in context, the learning algorithm extracts the necessary information ....

Daelemans, W.: 1995, `Memory-Based Lexical Acquisition and Processing'. In: P. Steffens (ed.): Machine Translation and the Lexicon, No. 898 in Springer Lecture Notes in Artificial Intelligence. Springer, pp. 85--98.

.... analysis ( 29, 28] part of speech tagging ( 18, 35] PP attachment ( 36] word sense disambiguation( 30] subcategorization ( 4] chunking (partial parsing) 31] and shallow parsing ( 10, 5] Relations to statistical language processing are discussed in [34] A partial overview paper is [9]. The first dissertation length study devoted to the approach is [26] in which the approach is compared to alternative learning methods for NLP tasks related to English word pronunciation (stress assignment, CHAPTER 4. LEARNING ALGORITHMS 19 syllabification, morphological analysis, alignment, ....

W. Daelemans. Memory-based lexical acquisition and processing. In P. Steffens, editor, Machine Translation and the Lexicon, volume 898 of Lecture Notes in Artificial Intelligence, pages 85--98. Springer-Verlag, Berlin, 1995.

....our approach to memory based syntactic pattern recognition, we carve up the syntactic analysis process into a number of such classification tasks with input vectors representing a focus item and a dynam ically selected surrounding context. As in Natural Language Processing problems in general [ Daelemans, 1995 ] these classification tasks can be segmentation tasks (e.g. decide whether a focus word or tag is the start or end of an NP) or disambiguation tasks (e.g. decide whether a chunk is the subject NP, the object NP or neither) Output of some memory based modules (e.g. a tagger or a chunker) is ....

W. Daelemans. Memory-based lexical acquisition and processing. In P. Steffens, editor, Machine Translation and the Lexicon, Lecture Notes in Artificial Intelligence, pages 85--98. Springer-Verlag, Berlin, 1995.

.... Skousen, 1989; Chandler, 1992; Scha, 1992 are salient examples) In computational linguistics (apart from incidental computational work of the linguists referred to earlier) the general approach has recently gained some popularity: e.g. Cardie (1994, syntactic and semantic disambiguation) Daelemans (1995, an overview of work in the early nineties on memory based computational phonology and morphology) Jones (1996, an overview of example based machine translation research) Federici and Pirrelli (1996) 3.1 Similarity Metric Accuracy of an exemplar based system on previously unseen inputs ....

.... structure) grapheme to phoneme conversion (identify the pronunciation of words) stress assignment (identify the stress pattern of words) morphology (both synthesis and analysis) and morphosyntactic disambiguation (identify for each word in a text its morpho syntactic category) See Daelemans (1995) for a discussion of the general approach, and van den Bosch Daelemans, 1992, 1993; Daelemans van den Bosch, 1992ab, 1993, 1994; van den Bosch et al. 1996; Daelemans et al. 1994, 1995, 1996ab for the details. There are some general trends which become clear when analysing the results of all ....

[Article contains additional citation context not shown here]

Daelemans, W. (1995). `Memory-based lexical acquisition and processing.' In Steffens, P., editor, Machine Translation and the Lexicon, Lecture Notes in Artificial Intelligence 898. Berlin: Springer, 85--98.

....learning to morphological analysis Most linguistic problems can be seen as contextsensitive mappings from one representation to another (e.g. from text to speech; from a sequence of spelling words to a parse tree; from a parse tree to logical form, from source language to target language, etc. (Daelemans, 1995). This is also the case for morphological analysis. Memory based learning algorithms can learn mappings (classifications) if a sufficient number of instances of these mappings is presented to them. We drew our instances from the CELEX lexical data base (Baayen et al. 1993) CELEX contains a large ....

W. Daelemans. 1995. Memory-based lexical acquisition and processing. In P. Steffens, editor, Machine Translation and the Lexicon, Lecture Notes in Artificial Intelligence, pages 85--98. Springer-Verlag, Berlin.

....which are useful in producing accurate extrapolations to new data. ib1 ig, through its implicit parallelism and its feature relevance weighting, is better suited than decision tree methods to make available the most specific relevant patterns in memory to extrapolate from. 7. Related research Daelemans (1995) provides an overview of memory based learning work on phonological and morphological tasks (grapheme to phoneme conversion, syllabification, hyphenation, morphological synthesis, word stress assignment) at Tilburg University and the University of Antwerp in the early nineties. The present paper ....

....set on generalization accuracy is noted. In the context of statistical language learning, it is also relevant to note that as far as comparable results are available, statistical techniques, which also abstract from exceptional events, never obtain a higher generalization accuracy than ib1 ig (Daelemans, 1995; Zavrel and Daelemans, 1997; Zavrel, Daelemans, and Veenstra, 1997) Reliable comparisons (in the sense of methods being compared on the same train and test data) with the empirical results reported here cannot be made, however. In the machine learning literature, the problem of small disjuncts ....

Daelemans, W. 1995. Memory-based lexical acquisition and processing. In P. Steffens, editor, Machine Translation and the Lexicon, volume 898 of Lecture Notes in Artificial Intelligence.

....linguistic levels such as syntax and semantics simultaneously (e.g. domain free multilingual translation, or dialogue modeling) it is time consuming, if not plainly impossible to gather and compile knowledge covering the many to many mappings between these levels. On the other hand, we argue [9, 10] that all NLP tasks can be seen as either ffl light NLP tasks, involving disambiguation or segmentation [9] locally at one language level or between two closely related language levels; or as ffl compositions of light NLP tasks, when the task surpasses the complexity of single light NLP tasks. ....

....dialogue modeling) it is time consuming, if not plainly impossible to gather and compile knowledge covering the many to many mappings between these levels. On the other hand, we argue [9, 10] that all NLP tasks can be seen as either ffl light NLP tasks, involving disambiguation or segmentation [9] locally at one language level or between two closely related language levels; or as ffl compositions of light NLP tasks, when the task surpasses the complexity of single light NLP tasks. This research was performed in the context of the Induction of Linguistic Knowledge research programme, ....

W. Daelemans. Memory-based lexical acquisition and processing. In P. Steffens, editor, Machine Translation and the Lexicon, volume 898 of Lecture Notes in Artificial Intelligence, pages 85--98. Springer-Verlag, Berlin, 1995.

.... formation ( 14] morphological analysis ( 26] part of speech tagging ( 16] PP attachment ( 31] word sense disambiguation( 27] subcategorization ( 4] and chunking (partial parsing) 28] Relations to statistical language processing are discussed in [30] A partial overview paper is [8]. The first dissertation length study devoted to the approach is [24] in which the approach is compared to alternative learning methods for NLP tasks related to English word pronunciation (stress assignment, syllabification, morphological analysis, alignment, grapheme to phoneme conversion) All ....

W. Daelemans. Memory-based lexical acquisition and processing. In P. Steffens, editor, Machine Translation and the Lexicon, volume 898 of Lecture Notes in Artificial Intelligence, pages 85--98. Springer-Verlag, Berlin, 1995.

....known beforehand. It is our claim that all useful linguistic tasks can be redefined this way and can thus be taken on in a ML context. All linguistic problems can be described as mappings of two kinds: disambiguation (or identification) and segmentation (identification of boundaries) see Daelemans, 1995). ffl Disambiguation. Given a set of possible categories and a relevant context in terms of attribute values, determine the correct category for this context. Instances of disambiguation include part of speech tagging (disambiguating the syntactic category of a word) grapheme to phoneme ....

.... 1989; Derwing Skousen, 1989; Chandler, 1992; Scha, 1992) In computational linguistics (apart from incidental computational work of the linguists referred to earlier) the general approach has only recently gained some popularity: e.g. Cardie (1994, syntactic and semantic disambiguation) Daelemans (1995, an overview of work in the early nineties on memory based computational phonology and morphology) Jones (1996, an overview of example based machine translation research) Federici and Pirrelli (1996) 3.1 Similarity Metric Performance of a lazy learning system (accuracy on the test set) ....

[Article contains additional citation context not shown here]

Daelemans, W. (1995). `Memory-based lexical acquisition and processing.' In Steffens, P., editor, Machine Translation and the Lexicon, Lecture Notes in Artificial Intelligence 898. Berlin: Springer, 85--98.

....to tribl, a hybrid combination of igtree and ibl. 1 Introduction The formalism presented here originated in research on the application of lazylearning techniques (such as case based learning, instance based learning, and memory based reasoning) to real world problems in language technology, cf. [Dae95] and [Car93] for overviews of the approach and results. The main difference between learning linguistic problems and learning the typical benchmark problems employed in machine learning research, is that for the linguistic problems huge datasets are available. For example, in the problem of ....

....significantly. A feature relevance ordering technique is available which assigns sufficiently differing relevance to individual features to allow a fixed ordering. These properties apply to a large class of real world problems, including almost all disambiguation tasks in language technology [Dae95, Car93]. For this type of task, igtree attains a generalisation accuracy similar to alternative lazylearning techniques and other inductive machine learning techniques, with modest memory space and processing time requirements. Retrieval is especially fast because its complexity is independent from the ....

Daelemans, W. (1995). Memory-based lexical acquisition and processing. In Steffens, P. (Ed.), Machine Translation and the Lexicon, Lecture Notes in Artificial Intelligence 898. Berlin: Springer.

....html home.html (Penn Treebank) and http: www.cogsci.princeton.edu wn (WordNet) cf. http: www.cs.unimaas.nl signll signllwww. html for more links to home pages of corpora. context. All linguistic problems can be described as a mapping of one of two types of classification (Daelemans, 1995): ffl Disambiguation. Given a set of possible categories and a relevant context in terms of attribute values, determine the correct category for this context. An example from text to speech conversion: given a letter in its context (a word) determine its pronunciation. An example from parsing: ....

....similar example(s) is used as a basis for predicting the category of the test example. From the early nineties onwards, lazy learning approaches to NLP tasks have been explored intensively by the partners of the ATILA project (University of Tilburg, Antwerp University, Universiteit Maastricht) Daelemans (1995) provides an overview of early work of this group on phonological and morphological tasks (graphemeto phoneme conversion, syllabification, hyphenation, morphological synthesis, word stress assignment) More recently, the approach has been applied to part of speech tagging (morphosyntactic ....

Daelemans, W. (1995). Memory-based lexical acquisition and processing. In P. Steffens (Ed.), Machine Translation and the Lexicon, pp. 85--98. Berlin: SpringerVerlag.

No context found.

Daelemans W. Memory-based lexical acquisition and processing. In Steffens P (Ed.), Machine translation and the lexicon: 3rd International EAMT Workshop Proceedings, 1993 Apr 26-28; Heidelberg, Germany. Berlin: Springer-Verlag, 85-98.

No context found.

Walter Daelemans. 1995. Memory-based lexical acquisition and processing. In P. Ste#ens, editor, Machine Translation and the Lexicon, volume 898 of Lecture Notes in Arti#cial Intelligence, pages 85#98. Berlin: Springer Verlag.

No context found.

Walter Daelemans. 1995. Memory-based lexical acquisition and processing. In P. Steffens, editor, Machine Translation and the Lexicon, volume 898 of Lecture Notes in Artificial Intelligence, pages 85--98. Berlin: Springer Verlag.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC