C. Vogt, G. Cottrell, R.K. Belew, B. Bartell, Using relevance to train a linear mixture of experts, in: E. Voorhees, D.K. Harman (Eds.), The Fifth Text REtrieval Conference (TREC-5), NIST, vol. 500-238, 1997, pp. 503 -- 516.  Home/Search   Document Details and Download   Summary   Related Articles   Check

.... of this paper is: How to fuse these ranked lists together to achieve the best ranked list which is returned as a list of the top matches to q The linear combination model is extensively investigated for the fusion of information retrieval systems [6, 89] especially for text retrieval systems [45, 77, 100]. These models have proposed mechanisms to assign weights to the different types of scores to obtain a summary score for each document via training or relevance feedback. However, the heterogeneous property of the scores from different feature exaction techniques was not taken into account. For ....

C. Vogt, G. W. Cottrell, R. K. Belew, and B. T. Bartell. Using relevance to train a linear mixture of experts. The Fifth Text Retrieval Conference (TREC-5), pages 503--515, 1997.

....of the systems are given half credit. They nd that a model trained on ecacy and dissimilarity alone is able to predict when fusion will improve performance with about 70 accuracy. Note that Ng and Kantor aim at predicting when metasearch will yield improvement, not high performance. Vogt et al. [77] simulate the problem by generating random ranked lists, and seeing which pairs fuse well (using their own fusion techniques that we describe below) They nd that improvements are often obtained when 25 1. both constituent systems perform well, and 2. both rank relevant documents similarly. ....

....the training regimen, he achieves 47 improvement over the best single system. Bartell also experiments with more complicated neural networks for fusion (note that the linear combination formula can be viewed as a simple neural net) but nds no improvement over linear combination. Vogt et al. [77, 76, 74, 75] pick up where Bartell left o , performing more experiments with linear combination and neural net fusion methods. Overall, the performance of their linear combinations as tested on TREC adhoc and routing runs is mixed. Sometimes the best input system can be improved upon, especially if only ....

[Article contains additional citation context not shown here]

Christopher C. Vogt, Garrison W. Cottrell, Richard K. Belew, and Brian T. Bartell. Using relevance to train a linear mixture of experts. In Voorhees and Harman [81], pages 503-515. 129

....over a general sports article. 2. 3 Combination Filter Vogt et al. showed that a simple linear combination of scores returned by different Information Retrieval agents can improve the performance of those individual systems on new documents, achieving a better performance than any individual agent [23]. We build upon their work by combining our collaborative filtering prediction with our content based prediction using a weighted average. The trick is to come up with the weights that result in the most accurate prediction. The collaborative filtering predictions are more inaccurate in cases ....

Christopher C. Vogt, Garrison W. Cottrell, Richard K. Belew, and B.T. Bartell. Using Relevance to Train a Linear Mixture of Experts. In Proceedings of the Fifth Text Retrieval Conference, 1996.

....and Gazette Online (Tango) 2. 3 Combination Filter Vogt et al. showed that a simple linear combination of scores returned by different Information Retrieval agents can improve the performance of those individual systems on new documents, achieving a better performance than any individual agent [VCBB96] We build upon their work by combining our collaborative filtering prediction with our content based prediction using a weighted average. The trick is to come up with the weights that result in the most accurate prediction. The collaborative filtering predictions are more inaccurate in cases ....

Christopher C. Vogt, Garrison W. Cottrell, Richard K. Belew, and B.T. Bartell. Using relevance to train a linear mixture of experts. In Proceedings of the Fifth Text REtrieval Conference, 1996.

....the UCSD Active Web will also provide a serious test for the various system and security issues our other faculty are investigating. G.3.4. 5 Adaptive Lenses Gary Cottrell and Rik Belew are developing a methodology for automatically adapting the parameters of an information retrieval system [12][138][13] We have applied this technique to the construction of user lenses that are a user community centered view of the information available. They focus the user s query and the representation of the documents based on relevance feedback from the user community. The approach is based on prior ....

C.C. Vogt, G.W. Cottrell, R.K. Belew, and B.T. Bartell, "Using Relevance to Train a Linear Mixture of Experts," In The Fifth Text Retrieval Conference, Ed. D. Harman, Gaitherberg, MD, 1996.

....raise a number of interesting and important points. First, we reconfirm Bartell s results: Optimization according to the J criterion results in improved performance as measured by precision. In related work, we have found that optimizing J also worked well in some larger scale experiments (Vogt et al. 1997). Together with the results here, it seems to 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 Precision Recall CISI Training Data Baseline No Lens 200 Iters 1000 Iters 2000 Iters 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 Precision Recall CISI Test Data Baseline No Lens 200 Iters 1000 ....

Vogt, C., Cottrell, G., Belew, R., and Bartell, B. (1997). Using relevance to train a linear mixture of experts. In Harman, D. K., ed., The Fifth Text REtrieval Conference (TREC5), 503--515. Gaithersberg, MD: National Institute of Standards and Technology. NIST Special Publication 500-238.

....1993] Boughanem et al. 1993] Unfortunately, the resulting large networks generally require large numbers of training examples, a rare commodity in the IR setting. Although work using Latent Semantic Indexing [Deerwester et al. 1990] to reduce the number of features has met with some success [Vogt et al. 1997a] LSI is itself computationally expensive. Perhaps a wiser approach can be found in fusion, where the results from multiple IR systems are combined to generate a single (hopefully better) list of potentially relevant documents in response to presentation of a single query to a number of ....

.... this article, we examine in detail one such fusion model: the linear combination of scores (LC) The LC model has been used by many IR researchers with varying degrees of success: Bartell et al. 1994] Kantor, 1995] Knaus et al. 1995] Selberg and Etzioni, 1996] Shaw and Fox, 1995] and [Vogt et al. 1997b] Our analysis of the model reveals what types of systems the model works best with, and explores techniques for training the model. This article has the following format. First we describe the model and the specific problems we are examining, along with the data we use and assumptions. Next, ....

Vogt, C., Cottrell, G., Belew, R., and Bartell, B. (1997b). Using relevance to train a linear mixture of experts. In [Harman, 1997], pages 503--515.

....1993] Boughanem et al. 1993] Unfortunately, the resulting large networks generally require large numbers of training examples, a rare commodity in the IR setting. Although work using Latent Semantic Indexing [Deerwester et al. 1990] to reduce the number of features has met with some success [Vogt et al. 1997a] LSI is itself computationally expensive. Perhaps a wiser approach can be found in fusion, where the results from multiple IR systems are combined to generate a single (hopefully better) list of potentially relevant documents in response to presentation of a single query to a number of ....

.... this article, we examine in detail one such fusion model: the linear combination of scores (LC) The LC model has been used by many IR researchers with varying degrees of success: Bartell et al. 1994] Kantor, 1995] Knaus et al. 1995] Selberg and Etzioni, 1996] Shaw and Fox, 1995] and [Vogt et al. 1997b] Our analysis of the model reveals what types of systems the model works best with, and explores techniques for training the model. This article has the following format. First we describe the model and the specific problems we are examining, along with the data we use and assumptions. Next, we ....

Vogt, C., Cottrell, G., Belew, R., and Bartell, B. (1997b). Using relevance to train a linear mixture of experts. In [Harman, 1997], pages 503--515.

....would have to be low in order to account for the remainder (and presumably the majority) of the documents the ones for which the score was inaccurate. Nevertheless, this approach has been used with varying degrees of success by a number of researchers (e.g. 2] 8] 9] 13] 14] and [15]) However, consistent, significant improvement has been elusive. An interesting question is: when is it even possible to improve the performance of two IR systems by linearly combining their estimates of relevance One study by Lee [10] has attempted to answer this question. Lee used five ....

....and denominator are the same (i.e. the IR system ranks documents exactly as the user would) and a minimum value of 1 when the opposite is true. J is a rank order statistic that measures how close an individual IR system is to the user s ranking and is correlated with average precision ( 1] [15]) Note that J is simply the Guttman s Point Alienation (GPA, defined below) between an IR system and a user s relevance judgments. 2.2 Pairwise Measures Additionally, we make a number of measures which are meant to reveal how similar the two systems are to each other, to test the hypothesis ....

C.C. Vogt, G.W. Cottrell, R.K. Belew, and B.T. Bartell. Using relevance to train a linear mixture of experts. In Harman [7]. NIST Special Publication.

....two experiments raise a number of interesting and important points. First, we reconfirm Bartell s results: Optimization according to the J criterion results in improved performance as measured by precision. In related work, we have found that optimizing J also worked well in some TREC experiments [Vogt et al. 1996]. Together with the results here, it seems to be a robust approach to determining IR system model parameters. This makes sense, since users typically care most about, and can provide reliable relevance feedback concerning, the rank order of retrieved documents rather than their absolute ranking ....

Vogt, C., Cottrell, G., Belew, R., and Bartell, B. (1996). Using relevance to train a linear mixture of experts. In Harman, D., editor, The Fifth Text REtrieval Conference, Gaitherberg, MD. To appear. National Institute of Standards and Technology Special Publication.

....June 29, 1998, 4:28pm D R A F T 2 Vogt and Cottrell resulting large networks generally require large numbers of training examples, a rare commodity in the IR setting. Although work using Latent Semantic Indexing [Deerwester et al. 1990] to reduce the number of features has met with some success [Vogt et al. 1997a] LSI is itself computationally expensive. Perhaps a wiser approach can be found in fusion, where the results from multiple IR systems are combined to generate a single (hopefully better) list of potentially relevant documents in response to presentation of a single query to a number of ....

.... (LC) The LC model has been used by many IR researchers with varying degrees of success [Bartell et al. 1994] Kantor, 1995] Knaus et al. 1995] Selberg and Etzioni, 1996] Shaw and Fox, 1995] and D R A F T June 29, 1998, 4:28pm D R A F T Fusion Via a Linear Combination of Scores 3 [Vogt et al. 1997b] Our analysis of the model reveals what types of systems the model works best with and explores techniques for training the model. This article has the following format. First we describe the model and the specific problems we are examining, along with the data we use and assumptions. Next, we ....

Vogt, C., Cottrell, G., Belew, R., and Bartell, B. (1997b). Using relevance to train a linear mixture of experts. In [Harman, 1997], pages 503--515.

No context found.

C. Vogt, G. Cottrell, R.K. Belew, B. Bartell, Using relevance to train a linear mixture of experts, in: E. Voorhees, D.K. Harman (Eds.), The Fifth Text REtrieval Conference (TREC-5), NIST, vol. 500-238, 1997, pp. 503 -- 516.

No context found.

Vogt, Christopher C., Garrison W. Cottrell, Richard K. Belew, Brian T. Bartell. 1997. Using Relevance to Train a Linear Mixture of Experts. In Voorhees Ellen M. and Donna K. Harman (editors). The Fifth Text REtrieval Conference (TREC-5). NIST Special Publication 500-238, National Institute of Standards and Technology, Gaithersburg, MD, (http://trec.nist.gov/pubs.html), pp.503-516.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC