DMCA
BlogForever Crawler: Techniques and Algorithms to Harvest Modern Weblogs
| Citations: | 1 - 1 self |
BibTeX
@MISC{Blanvillain_blogforevercrawler:,
author = {Olivier Blanvillain and École Polytechnique Fédérale and Nikos Kasioumis and Vangelis Banos},
title = {BlogForever Crawler: Techniques and Algorithms to Harvest Modern Weblogs},
year = {}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Blogs are a dynamic communication medium which has been widely established on the web. The BlogForever project has developed an innovative system to harvest, preserve, manage and reuse blog content. This paper presents a key compo-nent of the BlogForever platform, the web crawler. More precisely, our work concentrates on techniques to automat-ically extract content such as articles, authors, dates and comments from blog posts. To achieve this goal, we intro-duce a simple and robust algorithm to generate extraction rules based on string matching using the blog’s web feed in conjunction with blog hypertext. This approach leads to a scalable blog data extraction process. Furthermore, we show how we integrate a web browser into the web harvesting pro-cess in order to support the data extraction from blogs with JavaScript generated content.
Keyphrases
blogforever crawler harvest modern weblogs extraction rule data extraction blogforever platform blog content blog hypertext robust algorithm blogforever project web browser web crawler blog post dynamic communication medium scalable blog data extraction process work concentrate key compo-nent automat-ically extract content innovative system
