Results 1 - 10
of
655
Accurately interpreting clickthrough data as implicit feedback
- In Proceedings of SIGIR
, 2005
"... This paper examines the reliability of implicit feedback generated from clickthrough data in WWW search. Analyzing the users ’ decision process using eyetracking and comparing implicit feedback against manual relevance judgments, we conclude that clicks are informative but biased. While this makes t ..."
Abstract
-
Cited by 434 (7 self)
- Add to MetaCart
(Show Context)
This paper examines the reliability of implicit feedback generated from clickthrough data in WWW search. Analyzing the users ’ decision process using eyetracking and comparing implicit feedback against manual relevance judgments, we conclude that clicks are informative but biased. While this makes the interpretation of clicks as absolute relevance judgments difficult, we show that relative preferences derived from clicks are reasonably accurate on average. Categories and Subject Descriptors
The perfect search engine is not enough: A study of orienteering behavior in directed search
, 2004
"... This paper presents a modified diary study that investigated how people performed personally motivated searches in their email, in their files, and on the Web. Although earlier studies of directed search focused on keyword search, most of the search behavior we observed did not involve keyword searc ..."
Abstract
-
Cited by 241 (18 self)
- Add to MetaCart
(Show Context)
This paper presents a modified diary study that investigated how people performed personally motivated searches in their email, in their files, and on the Web. Although earlier studies of directed search focused on keyword search, most of the search behavior we observed did not involve keyword search. Instead of jumping directly to their
Query chains: Learning to rank from implicit feedback
- In ACM SIGKDD International Conference On Knowledge Discovery and Data Mining (KDD
, 2005
"... This paper presents a novel approach for using clickthrough data to learn ranked retrieval functions for web search results. We observe that users searching the web often perform a sequence, or chain, of queries with a similar information need. Using query chains, we generate new types of preference ..."
Abstract
-
Cited by 240 (10 self)
- Add to MetaCart
(Show Context)
This paper presents a novel approach for using clickthrough data to learn ranked retrieval functions for web search results. We observe that users searching the web often perform a sequence, or chain, of queries with a similar information need. Using query chains, we generate new types of preference judgments from search engine logs, thus taking advantage of user intelligence in reformulating queries. To validate our method we perform a controlled user study comparing generated preference judgments to explicit relevance judgments. We also implemented a real-world search engine to test our approach, using a modified ranking SVM to learn an improved ranking function from preference data. Our results demonstrate significant improvements in the ranking given by the search engine. The learned rankings outperform both a static ranking function, as well as one trained without considering query chains.
Evaluating the accuracy of implicit feedback from clicks and query reformulations in web search
- ACM TRANSACTIONS ON INFORMATION SCIENCE (TOIS
, 2007
"... This paper examines the reliability of implicit feedback generated from clickthrough data and query reformulations in WWW search. Analyzing the users ’ decision process using eyetracking and comparing implicit feedback against manual relevance judgments, we conclude that clicks are informative but b ..."
Abstract
-
Cited by 161 (21 self)
- Add to MetaCart
This paper examines the reliability of implicit feedback generated from clickthrough data and query reformulations in WWW search. Analyzing the users ’ decision process using eyetracking and comparing implicit feedback against manual relevance judgments, we conclude that clicks are informative but biased. While this makes the interpretation of clicks as absolute relevance judgments difficult, we show that relative preferences derived from clicks are reasonably accurate on average. We find that such relative preferences are accurate not only between results from an individual query, but across multiple sets of results within chains of query reformulations.
Automatic identification of user goals in web search
, 2004
"... There have been recent interests in studying the “goal ” behind a user’s Web query, so that this goal can be used to improve the quality of a search engine’s results. Previous studies have mainly focused on using manual query-log investigation to identify Web query goals. In this paper we study whet ..."
Abstract
-
Cited by 149 (3 self)
- Add to MetaCart
(Show Context)
There have been recent interests in studying the “goal ” behind a user’s Web query, so that this goal can be used to improve the quality of a search engine’s results. Previous studies have mainly focused on using manual query-log investigation to identify Web query goals. In this paper we study whether and how we can automate this goal-identification process. We first present our results from a human subject study that strongly indicate the feasibility of automatic query-goal identification. We then propose two types of features for the goal-identification task: user-click behavior and anchor-link distribution. Our experimental evaluation shows that by combining these features we can correctly identify the goals for 90 % of the queries studied.
A large-scale evaluation and analysis of personalized search strategies
- In WWW
, 2007
"... Although personalized search has been proposed for many years and many personalization strategies have been inves-tigated, it is still unclear whether personalization is consis-tently effective on different queries for different users, and under different search contexts. In this paper, we study thi ..."
Abstract
-
Cited by 142 (2 self)
- Add to MetaCart
(Show Context)
Although personalized search has been proposed for many years and many personalization strategies have been inves-tigated, it is still unclear whether personalization is consis-tently effective on different queries for different users, and under different search contexts. In this paper, we study this problem and provide some preliminary conclusions. We present a large-scale evaluation framework for personalized search based on query logs, and then evaluate five person-alized search strategies (including two click-based and three profile-based ones) using 12-day MSN query logs. By an-alyzing the results, we reveal that personalized search has significant improvement over common web search on some queries but it has little effect on other queries (e.g., queries with small click entropy). It even harms search accuracy under some situations. Furthermore, we show that straight-forward click-based personalization strategies perform con-sistently and considerably well, while profile-based ones are unstable in our experiments. We also reveal that both long-term and short-term contexts are very important in improv-ing search performance for profile-based personalized search strategies.
Eye-Tracking Analysis of User Behavior in WWW-Search
, 2004
"... We investigate how users interact with the results page of a WWW search engine using eye-tracking. The goal is to gain insight into how users browse the presented abstracts and how they select links for further exploration. Such understanding is valuable for improved interface design, as well as for ..."
Abstract
-
Cited by 138 (8 self)
- Add to MetaCart
We investigate how users interact with the results page of a WWW search engine using eye-tracking. The goal is to gain insight into how users browse the presented abstracts and how they select links for further exploration. Such understanding is valuable for improved interface design, as well as for more accurate interpretations of implicit feedback (e.g. clickthrough) for machine learning. The following presents initial results, focusing on the amount of time spent viewing the presented abstracts, the total number of abstract viewed, as well as measures of how thoroughly searchers evaluate their results set.
SearchTogether: An Interface for Collaborative Web Search
- UIST
, 2007
"... Studies of search habits reveal that people engage in many search tasks involving collaboration with others, such as travel planning, organizing social events, or working on a homework assignment. However, current Web search tools are designed for a single user, working alone. We introduce SearchTog ..."
Abstract
-
Cited by 133 (15 self)
- Add to MetaCart
(Show Context)
Studies of search habits reveal that people engage in many search tasks involving collaboration with others, such as travel planning, organizing social events, or working on a homework assignment. However, current Web search tools are designed for a single user, working alone. We introduce SearchTogether, a prototype that enables groups of remote users to synchronously or asynchronously collaborate when searching the Web. We describe an example usage scenario, and discuss the ways SearchTogether facilitates collaboration by supporting awareness, division of labor, and persistence. We then discuss the findings of our evaluation of SearchTogether, analyzing which aspects of its design enabled successful collaboration among study participants. ACM Classification: H5.3 [Information interfaces and
Sindice.com: A document-oriented lookup index for open linked data
- International Journal of Metadata, Semantics and Ontologies
"... Developers of Semantic Web applications face a challenge with respect to the decentralised publication model: how and where to find statements about encountered resources. The “linked data” approach mandates that resource URIs should be de-referenced to return resource metadata. But for data discove ..."
Abstract
-
Cited by 130 (12 self)
- Add to MetaCart
Developers of Semantic Web applications face a challenge with respect to the decentralised publication model: how and where to find statements about encountered resources. The “linked data” approach mandates that resource URIs should be de-referenced to return resource metadata. But for data discovery linkage itself is not enough, and crawling and indexing of data is necessary. Existing Semantic Web search engines are focused on database-like functionality, compromising on index size, query performance and live updates. We present Sindice, a lookup index over resources crawled on the Semantic Web. Our index allows applications to automatically locate documents containing information about a given resource. In addition, we allow resource retrieval through uniquely identifying inverse-functional properties, offer a full-text search and index SPARQL endpoints. Finally we introduce an extension to the sitemap protocol which allows us to efficiently index large Semantic Web datasets with minimal impact on the data providers.
Hourly analysis of a very large topically categorized web query log
- In SIGIR ’04: Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval
, 2004
"... We review a query log of hundreds of millions of queries that constitute the total query traffic for an entire week of a generalpurpose commercial web search service. Previously, query logs have been studied from a single, cumulative view. In contrast, our analysis shows changes in popularity and un ..."
Abstract
-
Cited by 128 (9 self)
- Add to MetaCart
(Show Context)
We review a query log of hundreds of millions of queries that constitute the total query traffic for an entire week of a generalpurpose commercial web search service. Previously, query logs have been studied from a single, cumulative view. In contrast, our analysis shows changes in popularity and uniqueness of topically categorized queries across the hours of the day. We examine query traffic on an hourly basis by matching it against lists of queries that have been topically pre-categorized by human editors. This represents 13 % of the query traffic. We show that query traffic from particular topical categories differs both from the query stream as a whole and from other categories. This analysis provides valuable insight for improving retrieval effectiveness and efficiency. It is also relevant to the development of enhanced query disambiguation, routing, and caching algorithms.
