When a query is submitted to a metasearch engine, decisions are made with respect to the underlying search engines to be used, what modifications will be made to the query, and how to score the results. These decisions are typically made by considering only the user’s keyword query, neglecting the larger information need. Users with specific needs, such as “research papers ” or “homepages,” are not able to express these needs in a way that affects the decisions made by the metasearch engine. In this paper, we describe a metasearch engine architecture that considers the user’s information need for each decision. Users with

Abstract not found

Making recommendations requires treating users as individuals. In this paper, we describe a metasearch engine available at NEC Research Institute that allows individual search strategies to be used. Each search strategy consists of a different set of sources, different query modification rules and a personalized ordering policy. We combine these three features with a dynamic interface that allows users to see the "current best" recommendations displayed at all times, and allows results to be displayed immediately upon retrieval. We present several examples where a single query produces different results, ordered based on different factors, accomplished without the use of training, or a local database.

The Web, containing a large amount of useful information and resources, is expanding rapidly. Collecting domain-specific documents/information from the Web is one of the most important methods to build digital libraries for the scientific community. Focused Crawlers can selectively retrieve Web documents relevant to a specific domain to build collections for domain-specific search engines or digital libraries. Traditional focused crawlers normally adopting the simple Vector Space Model and local Web search algorithms typically only find relevant Web pages with low precision. Recall also often is low, since they explore a limited sub-graph of the Web that surrounds the starting URL set, and will ignore relevant pages outside this sub-graph. In this work, we investigated how to apply an inductive machine learning algorithm and meta-search technique, to the traditional focused crawling process, to overcome the above mentioned problems and to improve performance. We proposed a novel hybrid focused crawling framework based on Genetic Programming (GP) and meta-search. We showed that our novel hybrid framework can be applied to traditional focused crawlers to accurately find more relevant