H. Turtle. Natural language vs. boolean query evaluation: a comparison of retrieval performance. In Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 212--220, Dublin, Ireland, 1994.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....of the inference network retrieval model. While based on the same retrieval model as the INQUERY system [3] WIN has evolved separately and focused on the retrieval of legal material in large collections in a commercial environment that supports both Boolean and natural language searches [11]. In addition, WIN has shifted from supporting mostly English content to supporting a large number of Western European languages as well. This was performed by localizing tokenization rules and adopting morphological stemming. Moreover, WIN adopted Unicode as its internal character encoding. As a ....

H. Turtle. Natural language vs. boolean query evaluation: a comparison of retrieval performance. In Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 212--220, Dublin, Ireland, 1994.

....our actual distributed production resources where hundreds of thousands of users access over 15,000 document collections daily. We use a collection retrieval inference network (CORI) 5] run against a set of physical database rep resentations. The bedrock of our system is the WIN search engine 2 [30, 32, 31], a close relative to the IN QUERY engine developed at the Center for Intelligent Information Retrieval at the University of Mas sachusetts [1, 2] The performance of our system has led us to conclude that there is a role for automatic collection selection in the act of simplifying a user s ....

H. Turtle. Natural language vs. Boolean query evaluation: A comparison of retrieval performance. In Proc. of SIGIR '94, pages 212-221. Springer-Verlag, July 1994.

....queries improve the effectiveness of the natural language queries [Fujii Croft, 1993] Unlike the previous studies, this study demonstrated that applying proximity operators requiring the co occurence of words in a document decreases the performance of the phrase queries. A recent study by Turtle [Turtle, 1994] produced a different conclusion, in that natural language queries perform better than Boolean queries in retrieving English legal documents. Yet, another study found that both natural language and Boolean types of 3 queries, generated by real user, are equally effective in retrieving medical ....

Turtle, H. Natural language vs. Boolean query evaluation: A comparison of retrieval performance. In W. Bruce Croft & C. J. van Rijsbergen ed. Proceedings of the 17th Annual International ACM Special Interest Group in Information Retrieval. Springer-Verlag: London, 212-220, 1994.

....center of the figure. Consider first the goal of retrieval effectiveness in the upper right hand corner. For small document collections, a simple Boolean model might suffice. On large document collections (shown in the upper left hand corner) however, simple boolean retrieval will perform poorly [72, 1, 85]. To resolve the conflict between these two goals and provide better retrieval effectiveness on large document collections, we turn to more sophisticated retrieval models. Sophisticated retrieval models place additional requirements on the inverted file implementation, such as storage of term ....

Turtle, H. R. Natural language vs. boolean query evaluation: A comparison of retrieval performance. In Proc. of the 17th Inter. ACM SIGIR Conf. on Research and Development in Information Retrieval, pages 212--220, Dublin, July 1994.

....creates a small coherent space that is arbitrarily connected to other such spaces. The number of nodes in each such space rarely exceeds a few hundred. In any case, the large web sites tend to use links for structural rather than for semantic purposes. other hand, have been shown to be effective [16]. Belkin and Croft, for example, reported increased recall and precision of passage based queries compared with Boolean queries [1] Query mediated hypertext Syntactic problems associated with query formulation may be removed by accepting natural language text and converting it to a query ....

.... for example, reported increased recall and precision of passage based queries compared with Boolean queries [1] Query mediated hypertext Syntactic problems associated with query formulation may be removed by accepting natural language text and converting it to a query automatically (e.g. [16]) Interaction may be streamlined further if users can select passages from displayed text to specify their information seeking intent. Thus, a query may be reduced to a drag selection of some passage that would cause other, similar, passages to be displayed. Users could then hop from one ....

Turtle, H. Natural Language vs. Boolean query evaluation: A comparison of retrieval performance. In Proceedings of SIGIR '94 (Dublin, Ireland, 1994), Springer Verlag, 212--220.

....boolean (or relational) queries that return result sets of manageable size. Such queries typically suffer from the problems of either returning too many results, or returning no results at all. Furthermore, the difficulty of formulating effective queries grows with the size of the data set [23]. For text based data items, profiles based on natural language techniques from Information Retrieval (IR) have been shown to be reasonably effective at representing user information needs. Even assuming a good profile representation, however, it is still quite likely that a user s profile will ....

H. Turtle. Natural language vs. boolean query evaluation: A comparison of retrieval performance. In Proc. of the ACM SIGIR Conf., pages 212--220, 1994.

....of success) the Boolean and term weighting models to produce more general models that are flexible and, ideally, more effective. There have been many theoretical and implementation oriented discussions relating term weighting systems and Boolean like retrieval, with representative works including [BR84, Cro86, Eva94, EC92, FW90, Lee94, Los94b, LB88, LBY86, Nie89, Rad82, Rad79, RT90, Sal84, Sme84, Tur94]. These models and systems emphasize combining Boolean and probabilistic or vector based systems into a single system. Our discussion below differs in that it trys to keep separate Boolean and term weighting systems that assumes term 2 independence. Given this knowledge, the better search engine ....

Howard Turtle. Natural language vs. Boolean query evaluation: A comparison of retrieval performance. In ACM Annual Conference on Research and Development in Information Retrieval, pages 212--220, New York, 1994. ACM Press.

....Although a number of studies have been done on the types of queries submitted to information services [9] there is not a large amount of data on what happens in systems with free form or natural language queries. Examples of natural language searching of legal material at West Publishing [10] suggest that, in that environment,queries tend to be somewhat longer than those seen in THOMAS.Some of the common test collections used in information retrieval research, together with the average number of words in the natural language queries, are Cranfield (9.2) CACM (13) Time (8.9) NPL ....

Howard Turtle. Natural language vs. Boolean query evaluation: A comparison of retrieval performance. In Proceedings ACM SIGIR 94, pages 212--220, 1994.

.... use of available operators and in mapping their intent to the appropriate logical query structure; it seems that systems which allow queries to be put in unstructured form allow easier query formulation than those which require Boolean structure, and are, at least for end users, more effective [3, 10]. We therefore restricted queries to simple lists of terms. The only (implicit) operator allowed was the concatenation of terms to form multi term phrases such as automobile recall . One particularly interesting and promising tool to support (or even replace) query formulation in the context of ....

Howard Turtle. Natural language vs. boolean query evaluation: A comparison of retrieval performance. In SIGIR'94. Proc. of the Seventeenth Annual ACM SIGIR Conference on Research and Development in Information Retrieval, pages 212--220, London, 1994. Springer Verlag.

....times will depend on the degree of query expansion. In the next section, we present statistics for some corpora. 4 Corpora and Conflation Classes The corpora that we use in these experiments are the West collection of law documents and the Wall St. Journal collection of newspaper articles [7, 2]. The statistics for these corpora and the associated queries and relevance judgements are shown in Table 1. Two sets of queries are used for the West collection. The first is where the queries are treated as a collection of individual words. The second uses INQUERY operators (such as #PHRASE) to ....

Howard Turtle. Natural language vs. Boolean query evaluation: A comparison of retrieval performance. In Proceedings ACM SIGIR 94, pages 212--220, 1994.

....in which user needs were specified as boolean combinations of search terms. Likewise, existing database systems provide similar exact match query semantics (e.g. SQL) Current opinion in the IR community is that formulating effective boolean retrieval queries requires significant expertise [Tur94] As in the IR case, the limitations of the exact match approach are quickly becoming apparent in emerging data retrieval applications and environments. In such environments queries are posed by hordes of novice users, who may have only a vague notion of what they are looking for. Such users are ....

....quickly find interesting documents, and 2) If there are no good matches, the returned documents may assist the user in figuring out what query to pose next. Extensive studies have shown that the ranked retrieval model gives better results than the boolean model, in terms of precision and recall [Tur94] We believe that ranked retrieval is also a better query model for many emerging database applications, for exactly the same reasons as in the IR case. There has been some work related to this, for example the query interface of [Bro97] where the system uses summary screens to lead the user ....

Howard Turtle. Natural language vs. boolean query evaluation: A comparison of retrieval performance. In Proc. ACM SIGIR Conf., Dublin, Ireland, 1994.

....the sub issue of good faith from Section 1325(a) 3) The HOD corpus contains cases addressing a great many legal questions. It was built by adding approximately 200 cases to another already existing, nearly 12,000 document collection, called the West or FSupp collection [Haines and Croft, 1993] [Turtle, 1994] . The additional texts are for cases contained in the CABARET CKB and cases found when the query home office was posed to the on line WestLaw Federal Taxation Case Law database. We restricted the query to cases decided between January 1986and November 1993. We added in these cases (with redundant ....

Howard Turtle. Natural Language vs. Boolean Query Evaluation: A Comparison of Retrieval Performance. In Proceedings of the 17th Annual International ACM/SIGIR Conference on Research and Development in Information Retrieval, pp 212--220, Dublin, Ireland, July 1994. ACM.

....(for example, see [Coo94] for an interesting exposition of the issues) that in the long run, models founded in probability theory will prove to be more fruitful in this regard. The results obtained here also suggest that some previous experimental results might merit a second look. For example, [Tur94] reports that natural language queries consistently outperform Boolean queries for experiments run on legal collections. The difference observed, however, is within the range of the improvements we have been able to obtain, suggesting that more effective implementations of the Boolean query ....

Howard Turtle. Natural language vs. boolean query evaluation: A comparison of retrieval performance. In W. Bruce Croft and C. J. van Rijsbergen, editors, Proceedings of the 17th Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval, pages 212--220, Dublin, Ireland, July 1994.

....1325(a) the good faith sub issue is discussed in Section 1325(a) 3) The HOD corpus contains cases addressing a great many legal questions. It was built by adding approximately 200 cases to another already existing, nearly 12,000 document collection, called the West or FSupp collection, 8] [18]. The additional texts came from the cases found in the CABARETCKB and those found when the query home office was posed to the on line WestLaw R fl Federal Taxation Case Law database. We restricted the query cases to be between January 1986 and November 1993 and removed all redundant cases. ....

Howard Turtle. Natural Language vs. Boolean Query Evaluation: A Comparison of Retrieval Performance. In Proceedings of the 17th Annual International ACM/SIGIR Conference on Research and Development in Information Retrieval, pages 212--220, Dublin, Ireland, July 1994. ACM.

....that each execute the search using different timeout values: 5 second search, 30 second search, or 5 minute search. The longer the system has to search, the more documents it will be able to retrieve. It has been shown that unstructured queries tend to be more effective for average users[32], thus we use a rudimentary search syntax of space separated keywords, using quotes or parentheses to denote adjacency. The syntax also uses a menu option that defines the logic behind word spacing, e.g. users can choose to have foo bar mean foo AND bar, foo OR bar, foo ADJ bar , or Mr. or ....

Howard Turtle. Natural Language vs. Boolean Query Evaluation: A Comparison of Retrieval Performance. In Proc. of the 1994 SIGIR Conference, pages 212--220, Dublin, Ireland, July 1994.

....IR systems must cope with the ambiguity and inconsistencies that are inherent in the use of such language. Benchmarksand studies for IR query processing have therefore focused on improving retrieval effectiveness, which is a measure of user satisfaction with the returned answers (e.g. see [Tur94, VH97] Of course, efficiency is also a concern for IR systems, and there has been significant research on indexing techniques and query evaluation heuristics that improve the query response time while maintaining a constant level of effectiveness (e.g. see [Fal85, ZMSD92, WL93, Per94, Bro95, ....

....The boolean query model is similar to the query model used by relational languages such as SQL. Formulating boolean queries that return result sets of manageable size has been shown to require significant expertise; as the size of the collection grows, formulating queries becomes even harder [Tur94] Natural language (or vector space model) techniques were developed concurrently with the boolean query model, but have only recently been adopted by commercial IR systems [Tur94] In a document retrieval context, a natural language query consists of a list of terms (implicitly connected by ....

[Article contains additional citation context not shown here]

H. Turtle. Natural language vs. boolean query evaluation: A comparison of retrieval performance. Proc. ACM SIGIR Conf., Dublin, Ireland, 1994.

....return result sets of manageable size. Such queries typically suffer from the problems of either returning too many results, or returning no results at all. Furthermore, it has been shown that as the size of the data set grows, formulating effective queries with such languages becomes even harder [Tur94]. For text based data items, profiles based on natural language techniques from Information Retrieval (IR) have been shown to be reasonably effective at representing user information needs. Even assuming a good profile representation, however, with existing approaches it is still quite likely that ....

H. Turtle. Natural language vs. boolean query evaluation: A comparison of retrieval performance. In Proc. ACM SIGIR Conf., 1994.

....in which user needs were specified as boolean combinations of search terms. Likewise, existing database systems provide similar exact match query semantics (e.g. SQL) Current opinion in the IR community is that formulating effective boolean retrieval queries requires significant expertise [Tur94] As in the IR case, the limitations of the exact match approach are quickly becoming apparent in emerging data retrieval applications and environments. In such environments queries are posed by hordes of novice users, who may have only a vague notion of what they are looking for. Such users are ....

....are seen first, and so the user can quickly find interesting documents, and 2) If there are no good matches, the returned documents may assist the user in figuring out what query to pose next. Extensive studies have shown that the ranked retrieval model gives better results than the boolean model [Tur94] We believe that ranked retrieval is also a better query model for many emerging database applications, for exactly the same reasons as in the IR case. There has been some work related to this, for example the query interface of [Bro97] where the system uses summary screens to lead the user ....

Howard Turtle. Natural language vs. boolean query evaluation: A comparison of retrieval performance. In Proc. ACM SIGIR Conf., Dublin, Ireland, 1994.

.... TREC competitions [BCC93, Cro93, CCB94] The two retrieval engines have common roots but have evolved separately WIN has focused on the retrieval of legal materials from large ( 50 gigabyte) collections in a commercial online environment that supports both Boolean and natural language retrieval [Tur94]. For TREC 3 we decided to run an essentially unmodified version of WIN to see how well a state of the art commercial system compares to state of the art research systems. Some modifications to WIN were required to handle the TREC topics, which bear little resemblance to queries entered by online ....

Howard Turtle. Natural language vs. Boolean query evaluation: a comparison of retrieval performance. In W. Bruce Croft and C. J. van Rijsbergen, editors, Proceedings of the Seventeenth Annual International Conference on Research and Development in Information Retrieval, pages 212--221, Springer-Verlag, London, July 1994.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC