Merialdo, Bernardo. 1994. Tagging english text with a probabilistic model. Computational Linguistics, 2(2):155--171.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....tagging The term stochastic tagger can refer to any number of di erent approaches to the problem of POS tagging. Any model that somehow incorporates frequencies or probabilities, i.e. statistics, may be properly labeled stochastic. Example of stochastic taggers are given in [45] 34] [27], 24] The simplest stochastic taggers disambiguate words based solely on the probability that a word occurs with a particular tag. In other words, the tag encountered most frequently in the training set is the one assigned to an ambiguous instance of that word. The problem with this approach is ....

Bernard Merialdo. Tagging english text with a probabilistic model. Computational Linguistics, 20(2):155-172, 1994.

.... area of automatically trained taggers has explored Markov model based part of speech tagging [Jelinek, 1985; Church, 1988; Derose, 1988; DeMarcken, 1990; Cutting et al. 1992; Kupiec, 1992; Chaxniak et al. 1993; Weischedel et al. 1993; Schutze and Singer, 1994; Lin et al. 1994; Elworthy, 1994; Merialdo 1995]. 2 For a Markov model based tagger, training consists of learning both lexical probabilities (P(wordltag) and contextual probabilities (P(tagiltagi 1 . tagi n) Once trained, a sentence can be tagged by searching for the tag sequence that mayAmizes the product of lexical and contextual ....

.... (e.g. Weischedel et al. 1993; Charniak et al. 1993] It is possible to use unsupervised learning to train stochastic taggers without the need for a manually annotated corpus by using the Baum Welch al gorithm [Banm, 1972; Jelinek, 1985; Cutting et al. 1992; Kupiec, 1992; Elworthy, 1994; Merialdo, 1995] This algorithm works by iteratively adjusting the lexical and contextual probabilities to increase the overall probability of the training corpus. If no prior knowledge is available, probabilities are initially either assigned randomly or evenly distributed. Although less accurate than the ....

[Article contains additional citation context not shown here]

Merialdo, B. 1995. Tagging english text with a probabilistic model. Com- putational Linguistics: To Appear.

....implying that either the features need further improvement or that intra annotator inconsistencies exist in the corpus. Comparison With Previous Work Most of the recent corpus based POS taggers in the literature are either statistically based, and use Markov Model(Weischedel et al. 1993, Merialdo, 1994) or Statistical Decision Tree(Jelinek et al. 1994, Magerman, 1995) SDT) techniques, or are primarily rule based, such as Brill s Transformation Based Learner(Brill, 1994) TBL) The Maximum Entropy (MaxEnt) tagger presented in this paper combines the advantages of all these methods. It uses a ....

Bernard Merialdo. 1994. Tagging English Text with a Probabilistic Model. Computational Linguistics, 20 (2):155-172.

....with the corpus is low. However, the output may have little relation to the part of speech assignments we actually want as output. Getting good performance as measured by assignment of the intended tags, not cross entropy may require a fair amount of manually prepared material. Merialdo [62] and Elworthy [29] conduct experiments to evaluate the e#ectiveness of forward backward training, and conclude that the best performance is obtained by providing large amounts of pre tagged text, and that with large amounts of pre tagged text, forward backward training can in fact damage ....

Bernard Merialdo. Tagging English text with a probabilistic model. Computational Linguistics, 20(2):155--172, 1994.

....data, their accuracies do not seem completely satisfactory, and the problem still needs investigation. Manning (1992) for example, proposes extracting case frames by using a finite state parser. His method first uses a statistical tagger (cf. Church, 1988; Kupiec, 1992; Charniak et al. 1993; Merialdo, 1994; Nagata, 1994; Schutze and Singer, 1994; Brill, 1995; Samuelsson, 1995; Ratnaparkhi, 1996; Haruno and Matsumoto, 1997) to assign a part of speech to each word in the sentences of a corpus. It then uses the finite state parser to parse the sentences and note case frames following verbs. Finally, ....

Merialdo, Bernard. 1994. Tagging English text with a probabilistic model. Computational Linguistics, 20(2):155--171.

....POSs and base forms of each valid decomposition. By base forms we mean the usual not inflected entries of a dictionary. POS tagging. POS tagging is based on a Viterbi decoder that computes the best text POS alignment on the basis of a bigram POS language model and a discrete observation model [5]. The employed tagger works with 57 tag classes and has an accuracy around 96 . Base form extraction. Once the POS and the morphological analysis of each word in the text is computed, a base form can be assigned to each word. Stemming. Word stemming is applied at the level of tagged base forms. ....

Merialdo, Bernard, 1994. Tagging English text with a probabilistic model. Computational Linguistics , 20(2):155--172.

....Misleading similarities in graphical form can sometime induce translation mistakes (deceptive cognates) These forbidden pairs normally involve only one of several possible parts of speech, hence the need to disambiguate them. We do this with a first order HMM part ofspeech tagger (Merialdo [13]) In the rest of the paper, we will use deceptive cognate very losely often to refer to normative usage of word in general. 129 4. Translation models. Being robust, the alignment program will align a pair of texts regardless of possible omissions in the target text. To detect such omissions of ....

Merialdo, B. (1994) Tagging English Text with a Probabilistic Model. Computational Linguistics, 20, pp. 155-168.

.... e.g. a bi label model. This model generally performed poorly, although the coreference versions often performed as well as the coreference model reported here. Our hypothesis is that we are seeing problems similar to those that have bedeviled applying EM to tasks like part of speech tagging [7]. In such cases EM typically tries to lower probabilities of the corpus by using the tags to encode common wordword combinations. As the models corresponding to equations 2 and 8 do not include any label label probabilities, this problem does not appear in these models. 6 Other Applications It ....

Merialdo, B. Tagging English text with a probabilistic model. Computational Linguistics 20 (1994), 155--172.

....learning is applied when the model is obtained from annotated corpora. Unsupervised learning is applied when the model is obtained from raw corpora training. The most noticeable examples of automatic tagging approaches are Markov chains [Church, 1989, Charniak et al. 1993, Jelinek, 1985, Merialdo, 1994, Garside et al. 1987, Cutting et al. 1992] neural networks [Schmid, 1994] decision trees IDaclemans et al. 1996, Mtrquez and Rodriguez, 1995] and transformation based error driven learning [Brill, 1994] The most widely used methods for POS tagging are stochastic methods based on fixed ....

Merialdo, B. (1994). Tagging english text with a probabilistic model. Computational Linguistics, 2(20):155-171.

....; t i t Gamma1 ; t i t Gamma2 ) Count(t i t Gamma1 ; t i t Gamma2 ) 2.22) where W t i t is the number of words that have the tag t i t , C T is the number of different tags in T train , w ; 01 ; 11 ; 12 ; 21 ; 22 ; 23 1 and Count(x) is the frequency of an event x in the training text. In [Merialdo, 1994], the author provides a very convincing comparison of the taggers based on Markov models (i.e. with an annotated corpus) and on hidden Markov models (without an annotated corpus) His experiments confirm the assumption that the more annotated texts are available the better training is obtained. ....

....representative corpus based tagging strategies applied to English can be summarized by means of Tab. 2.3; it shows that the tagging accuracy of the individual approaches applied to English falls within a narrow range. 21 strategy tagger training tagging ID data (size) accuracy ( Trigram MM ([Merialdo, 1994]) MM EN Associated 97.0 Press (955Kw) ME ( Ratnaparkhi, 1996] ME EN WSJ (962Kw) 96.6 EXP ( Hajic, Hladk a, 1998b] EXP EN WSJ (1.2Mw) 96.8 MB ( Daelemans, Zavrel, 1996] MB EN WSJ (2Mw) 96.4 RB ( Brill, 1998] RB EN WSJ (600Kw) 96.9 NE ( Schmid, 1994] NE EN WSJ (2Mw) 96.2 Table 2.3: ....

B. Merialdo. Tagging English Text with a Probabilistic Model. In Computational Linguistics, 20(2), pp. 155-171, 1994.

....each ambiguous word in a sentence in the context of the sentence. It is believed that POS tagging is one of the basic techniques in natural language processing. In the last several years, various POS tagging systems for different languages [2, 10, 17] have been developed by using statistical model [13], rule based approach [1] machine learning technique [15] and neural networks [9, 11, 12, 17] POS tagging techniques have been applied to pre processing for speech synthesis, postprocessing for continuous speech recognition, machine translation, and information retrieval. However, most of the ....

B. Merialdo, "Tagging English text with a probabilistic model," Computational Linguistics, Vol. 20, No. 2, pp. 155-171, 1994.

....does not change substancially if we condition it on preceding subsequences of length greater than L. This feature can be nd in may applications related with natural language procesing such as speech recognition [Jelinek, 1985] Nadas, 1984] and part of speech tagging [Brill, 1994] [Merialdo, 1994]. An improved model of Markov chains has been developed by Dana Ron [Ron, 1996] in 1996. This model is a subclass of PFAs (Probabilistic Finite Automatas) called PSAs (Probabilistic Sux Automatas) A PSA is hence a variant order L Markov chain, in which the order, of equivalently, the memory, is ....

Merialdo, B. (1994). Tagging english text with a probabilistic model. Computational Linguistics, 2(20):155-171.

.... two main groups, depending on the tendencies followed for establishing the Language Model: the linguistic approach, which is based on hand coded linguistic rules [6] 22] and the learning approach which is derived from a corpora (labelled or non labelled) using di erent formalisms: HMM [8] [15], Decision Trees [11] 13] Maximum Entropy [20] Other approximations that use hybrid methods have also been proposed [23] Shallow parsing techniques can also be classi ed into the same 2 F.Pla, A.Molina and N.Prieto two groups as above. These approaches have a common characteristic: they take ....

B. Merialdo. Tagging English Text with a Probabilistic Model. Computational Linguistics, 20(2):155-171, 1994.

....which gives the percentage of correctly tagged word tokens TA(A) Correctly Tagged Words by A Total Tagged Words) 100( Several approaches to the automatic tagging of texts have been proposed. The so called stochastic strategies use various statistical models, namely Markov models (MM) [Merialdo, 1994]) the exponential (EXP) model and the maximum entropy (ME) model ( Ratnaparkhi, 1996] A memory based (MB) strategy represents a kind of supervised learning based on similarity based reasoning ( Daelemans, Zavrel, 1996] In a rule based (RB) strategy, a set of meaningful rules is automatically ....

B. Merialdo. Tagging English Text with a Probabilistic Model. In Computational Linguistics, 20(2), pp. 155-171, 1994.

....get more reliable estimates for contextual probabilities. In the closely related field of statistical language modeling [Jelinek and Mercer, 1980] interpolation of trigram, bigram and unigram probabilities has been used. The same method has been applied to part of speech tagging e.g. by Merialdo [Merialdo, 1994]. Another method is Katz s back off model [Katz, 1987] which redistributes a small percentage of the probability of all observed events to unseen events. At the IMS, two methods have been developed to tackle the estimation problem. One method [Schmid, 1994a] uses a binary decision tree to get ....

....is tagged again and so on, until the accuracy does not improve anymore. The forward backward algorithm [Baum, 1972] is used for this reestimation procedure. Merialdo observed, however, that the reestimation of parameters . degrades the performance unless the initial model is already very bad [Merialdo, 1994]. 4.2.3 The use of Neural Networks for Tagging There has also been some research on neural networks for part of speech tagging. Benello, Mackie and Anderson [Benello et al. 1989] implemented a feed forward network with one hidden layer and an input context of 4 preceding tags. At the IMS, a ....

Merialdo, B. (1994). Tagging english text with an probabilistic model. Computational Linguistics, 20(2):155--171.

....probability of an imperfect model will increase classification accuracy. Will our mod4 els be representative enough for the purposes of text classification There is some reason to be initially pessimistic. Similar generative model approaches for related text tasks, such as part of speech tagging (Merialdo, 1994; Elworthy, 1994) and information extraction (Seymore et al. 1999) have shown that incorporating unlabeled data into supervised learning decreases performance of these systems. However, text classification is not as dependent as these tasks on model correctness at a local level. We demonstrate ....

....class labels, and are therefore detrimental to classification accuracy. For 32 other text tasks, posterior probability maximization has also been detrimental when the amount of labeled data is reasonable. While EM increased the likelihood of the parameters, the accuracy of part of speech taggers (Merialdo, 1994; Elworthy, 1994) and information extractors (McCallum et al. 2000) went down. For these tasks, each example is defined by just a small amount of local context. Here, the correctness of the model is much more important, because there are only a few, very correlated features. In text ....

Merialdo, B. (1994). Tagging English text with a probabilistic model. Computational Linguistics, 20(2), 155--171.

No context found.

Merialdo, Bernardo. 1994. Tagging english text with a probabilistic model. Computational Linguistics, 2(2):155--171.

No context found.

Merialdo, B. (1994). Tagging English Text with a Probabilistic Model. Computational Linguistics 20(2), 155--172.

No context found.

B. Merialdo. Tagging english text with a probabilistic model. Computational Linguistics, 20(2):155--172 , 1994.

No context found.

B. Merialdo. 1994. Tagging english text with a probabilistic model. Computational Linguistics, 20(2):155--172.

No context found.

B. Merialdo, "Tagging English Text with a Probabilistic Model," Computational Linguistics, vol. 20, no. 2, pp. 155--171, June, 1994.

No context found.

B. Merialdo. Tagging english text with a probabilistic model. Computational Linguistics, 20(2):155--172, 1994.

No context found.

B. Merialdo. Tagging english text with a probabilistic model. Computational Linguistics, 20(2):155--172, 1994.

No context found.

Merialdo, B. Tagging english text with a probabilistic model. Computational Linguistics 20, 2 (1994), 155-172.

No context found.

Merialdo, B. (1994) Tagging English Text with a Probabilistic Model. Computational Linguistics 20(2), 155--172.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC