Junk web pages hinder search say Microsoft, UCD researchers

Stephen Withers
Monday, 26 March 2007 15:04

Your IT - Home IT

Have you ever followed one of the top links from a search engine, only to find an ad-heavy page with little or no content relevant to your search? Researchers at Microsoft and the University of California Davis think they have figured out what's going on and how to beat the search-engine spammers.
In a research paper, Yi-Min Wang and Ming Ma from Microsoft Research, and Yuan Niu and Hao Chen from UCD, describe a "double funnel" used to deliver redirection spam. In essence, this comprises doorway pages that are optimised to attract search engines spiders but that fetch and present spam content when visited by normal users.

The good news is that only two blocks of IP addresses dominate the neck of the double funnel, providing a useful point of attack for the search engine industry and others wishing to stamp out redirection spam. Furthermore, a mere three ad syndicators were responsible for around two-thirds of the spam ads served through redirection chains.

"Ultimately, it is advertisers' money that is funding the search spam industry, which is increasingly cluttering the web with low-quality content and reducing web users' productivity," the report concludes. "By exposing the end-to-end search spamming activities, we hope to educate users not to click spam links and spam ads, and to encourage advertisers to scrutinize those syndicators and traffic affiliates who are profiting from spam traffic at the expense of the long-term health of the web."

The research paper Spam Double-Funnel: Connecting Web Spammers with Advertisers is available from UCD  and you may read more about the Strider Search Ranger used in the project at the Microsoft Research web site.