Brian Theodore Bartell. Optimizing Ranking Functions: A Connectionist Approach to Adaptive Information Retrieval. PhD thesis, University of California, San Diego, 1994.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....new metasearch algorithms, and (3) new evaluation techniques for metasearch. 1.3.1 Improvements Performance Weighting Training data is not always readily available. But when it is available, how can our metasearch algorithms take advantage of it Dissertations have been devoted to this topic [4, 74], but we take a simple, pragmatic approach that has been used, but has not been thoroughly tested in the metasearch literature: weight by individual performance. That is, test each input system on the training data, evaluate the result with a standard IR metric, and in future, weight each system ....

....details of our algorithms, however, we must review the relevant literature and describe our experimental apparatus. 19 Related Work The combination of document retrieval results has received considerable attention in the past few years: it has been the subject of several doctoral dissertations [4, 53, 74, 67], journal articles [70, 26, 76] and conference papers [23, 7, 42, 35, 43, 54, 75] being especially used by systems competing in the TREC competitions [25, 68, 56] This chapter reviews the results of previous studies as they relate to our work. 2.1 Preliminaries Before we begin looking at ....

[Article contains additional citation context not shown here]

Brian Theodore Bartell. Optimizing Ranking Functions: A Connectionist Approach to Adaptive Information Retrieval. PhD thesis, University of California, San Diego, 1994.

....system; however, choosing an appropriate threshold in a ranked list of documents is in itself a difficult problem. The hypothesis that learning and optimization can be used to improve rankedretrieval effectiveness and classification accuracy has been suggested by many researchers. Bartell [7] adds significant evidence that suggests this hypothesis is true. Earlier contributions include a widely used query expansion technique introduced by Rocchio [70] This method has been improved using machine learning approaches [48, 49, 56, 80, 78] and heuristic optimization techniques [10] on ....

B.T. Bartell, "Optimizing Ranking Functions: A Connectionist Approach to Adaptive Information Retrieval, " Ph.D. Thesis, University of California - San Diego, 1994.

.... systems w i R i (d; q) Or, for only two IR systems: R(w 1 ; w 2 ; d; q) w 1 R 1 (d; q) w 2 R 2 (d; q) 1) This straightforward approach has been attempted with varying degrees of success by a number of researchers ( Shaw and Fox, 1995] Knaus et al. 1995] Kantor, 1995] in [Harman, 1995] [Bartell et al. 1994] and [Selberg and Etzioni, 1996] and [Vogt et al. 1996] in [Harman, 1997] However, consistent, significant improvement has been elusive. An interesting question is: when is it even possible to improve the performance of two IR systems by linearly combining their estimates of relevance Two ....

.... 2 , and a modified GPA wherein only relevant documents are used (GPA r ) Note that GPA and GPA r are measures of how similar the two IR systems are to each other, whereas J measures how close an individual IR system is to the correct ranking (J is highly correlated with average precision [Bartell, 1994, Vogt et al. 1996] By convention, IR system #1 is always the one with higher 1 If R is the ranking function implemented by an IR system, then J = 1 jQj X q2Q P d q d 0 (R(W; d; q) Gamma R(W; d 0 ; q) P d q d 0 jR(W; d; q) Gamma R(W; d 0 ; q)j where d q d 0 indicates the ....

Bartell, B. T. (1994). Optimizing Ranking Functions: A Connectionist Approach to Adaptive Information Retrieval. thesis, Department of Computer Science and Engineering, The University of California, San Diego, CSE 0114.

....dilemma, as it offers a method for evaluating the performance of experts in combination with others rather than in isolation. In the next section, we outline our method for automatically combining experts. The approach is based on the general rank optimization method developed in previous work [1] [2] Sections 3 and 4 present the results of two applications, Section 5 compares the method to an alternative, and Section 6 discusses the method s strengths and limitations. 2 Expert Combination Algorithm A number of methods for combining retrieval experts have been proposed. Perhaps the most ....

....parameters in the model. The goal of our method is to automatically determine values for these parameters so that the overall estimates result in the best ranking of documents possible. Though we emphasize the linear model here, non linear models (neural networks) have yielded positive results [1]. 2.2 Optimizing Ranking Performance To find values for the parameters, we use a criterion which measures how well the model is ranking documents for a set of training queries. We numerically optimize the criterion in order to heuristically search for parameter values which result in superior ....

[Article contains additional citation context not shown here]

Brian T. Bartell. Optimizing Ranking Functions: A Connectionist Approach to Adaptive Information Retrieval. PhD thesis, Department of Computer Science & Engineering, The University of California, San Diego, 1994.

....(IR) systems. The experiments presented in [3] use connectionist learning models to improve the retrieval of relevant documents from a large collection of text. Here, we present further analysis of those experiments. Previous work in the area employs various techniques for improving retrieval [6, 7, 14]. The experiments presented here show that EG works significantly better than widely used ad hoc methods for finding a good set of query term weights. The retrieval processes being considered operate on a collection of documents, a natural language query, and a training set of documents judged ....

B.T. Bartell, "Optimizing Ranking Functions: A Connectionist Approach to Adaptive Information Retrieval", Ph.D. Theis, UCSD 1994.

....systems leads to higher achievable recall, although this does not account for all of the improvement (Saracevic Kantor 1988) 2 Approach The criterion used in this paper is a variation on Guttman s Point Alienation (Guttman, 1978) a statistical measure of rank correlation. Previous work (Bartell et al. 1994) has demonstrated that this criterion can be highly correlated with average precision 1 , a more typical measure of performance in Information Retrieval. Thus, optimization of this criterion is likely to lead to optimized average precision performance. The technique applies to ranked retrieval ....

....functionality of the system does not appear overly restrictive. 1 Precision is the fraction of documents retrieved that are relevant. Recall is the fraction of relevant documents that are retrieved. Average precision is precision averaged over a number of recall levels. Using the notation in (Bartell et al. 1994), we let R Theta;q (d) be a ranked retrieval function which generates a score indicating the relevance of document d to query q. R is must be differentiable in its parameters Theta. The criterion is: J(R Theta ) Gamma1 j Q j X q2Q P d qd 0 (R Theta;q (d) Gamma R Theta;q (d 0 ) ....

[Article contains additional citation context not shown here]

Bartell, B.T. Optimizing Ranking Functions: A Connectionist Approach to Adaptive Information Retrieval. Ph.D. Thesis, UCSD, 1994.

....parameters which maximize the performance of the system. Bartell, Cottrell, and Belew have explored both of these issues in some depth focusing primarily on both linear and nonlinear neural net models coupled with the use of optimization of rank order statistics to determine model parameters [Bartell et al. 1994b, Bartell et al. 1994a] Even with the simplest linear combination of experts, they achieved some impressive improvements up to 47 higher average precision than the best individual expert. All of their experiments, however, are on relatively small collections. Others have successfully used ....

....the performance of the system. Bartell, Cottrell, and Belew have explored both of these issues in some depth focusing primarily on both linear and nonlinear neural net models coupled with the use of optimization of rank order statistics to determine model parameters [Bartell et al. 1994b, Bartell et al. 1994a] Even with the simplest linear combination of experts, they achieved some impressive improvements up to 47 higher average precision than the best individual expert. All of their experiments, however, are on relatively small collections. Others have successfully used mixture (a.k.a. fusion) ....

[Article contains additional citation context not shown here]

Bartell, B. T. (1994). Optimizing Ranking Functions: A Connectionist Approach to Adaptive Information Retrieval. PhD thesis, Department of Computer Science & Engineering, The University of California, San Diego, CSE 0114.

....before being combined into a ranking score R Since there are no restrictions on the lens matrices, any linear transformation can be used to reweight the input vectors. Our technique for adjusting the entries in the lens matrices is based on work done by Bartell, et al. Bartell et al. 1993] [Bartell, 1994]. Bartell defines a criterion (hereafter called J) based on Guttman s Point Alienation statistic as follows: Defn: the ranking function implemented by an IR system is R Theta : Theta Theta D Theta Q where Theta =the set of system parameters D=the set of document vectors Q=the set of ....

Bartell, B. T. (1994). Optimizing Ranking Functions: A Connectionist Approach to Adaptive Information Retrieval. PhD thesis, Department of Computer Science & Engineering, The University of California, San Diego, CSE 0114.

....which matches the target ordering q as well as possible. 3 Method Our approach is to optimize a criterion derived from Guttman s Point Alienation [6] a measure of rank correlation between two variables. An overview of the approach is provided here; further details can be found in other work [1]. The criterion which is optimized is: J(R Theta ) Gamma1 j Q j X q2Q P d q d 0 (R Theta;q (d) Gamma R Theta;q (d 0 ) P d q d 0 j R Theta;q (d) Gamma R Theta;q (d 0 ) j (3) where Q is the set of training queries, and d; d 0 are from the document set D. This function is ....

Brian T. Bartell. Optimizing Ranking Functions: A Connectionist Approach to Adaptive Information Retrieval. PhD thesis, Department of Computer Science & Engineering, The University of California, San Diego, 1994.

No context found.

Bartell, Brian T. 1994. Optimizing Ranking Functions: A Connectionist Approach to Adaptive Information Retrieval. PhD thesis, Department of Computer Science & Engineering, The University of California, San Diego.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC