Improving the Precision of Third-Party Results by Monitoring Browsing Behaviour and Evolution in Internet Search Engine Caches

Abstract

Due to the extremely rapid growth of Web, search engines cannot index the new information at the same time or with the same priority. Besides, web search services index the disseminated information with different algorithms, having as result different response time in updating their directories. In this paper, we propose a meta-search algorithm, which is capable of self-adapting over the continuous changes that occur on the indexed Web, using a web evolution adaptation mechanism. This mechanism carries out statistical results according to the rationale of the capture-recapture experiments conducted in wildlife biological studies. The meta-search algorithm also records the user's browsing behaviour during his search activities. The paper provides the implementation details of the proposed meta-search ranking algorithm along with its initial assessment. We prove in our approach, that when user's browsing behaviour is jointly used with a dynamic survey mechanism, which scores the ability of each search service to adapt in the incessant evolution of the web, a more effective meta-search is provided. Experimental results showed that the precision of the third-party results in several recall levels were increased within a six-month period. For acquiring third-party results we used five known web search services, namely AltaVista, Google, Lycos, MSN, and Yahoo!