Google hosts and de-duplicates news wire stories

Stephen Withers
Monday, 03 September 2007 02:34

Your IT - Home IT

One of my major frustrations with Google News has been the way it indexed the same wire service story under every newspaper site that republished it. At last, Google has done something to improve matters.

"Our goal has always been to offer users as many different perspectives on a story from as many different sources as possible, which is why we include thousands of sources from around the world in Google News. However, if many of those stories are actually the exact same article, it can end up burying those different perspectives," said Josh Cohen, business product manager at Google.

Exactly. When I want to read more about an issue, I don't want to be presented with a list of all the papers in the English-speaking world that have run the same wire story. What I want is the chance to quickly find coverage of the issue from different viewpoints.

The change means Google News results will omit links to individual newspapers' copies of wire stories until the user requests a full listing.

Publications that run wire stories probably aren't so happy about the change, as it will almost certainly reduce traffic from Google News to their sites. Less traffic, generally speaking, means less revenue.

One part of Google's announcement doesn't seem to quite square up with reality: "Because the Associated Press, Agence France-Presse, UK Press Association and the Canadian Press don't have a consumer website where they publish their content, they have not been able to benefit from the traffic that Google News drives to other publishers. As a result, we're hosting it on Google News," said Cohen.

Of those four, PA appears to be the only agency that does not offer at least some of its stories directly to the public via its web site. Presumably Google will be hosting the agencies' entire output.

Google announced the change on Friday, but it looks like there are still a few wrinkles to be ironed out. For example, the top link generated for news of the announcement itself was to the Associated Press story republished on the Forbes site, not to Google's 'original' AP copy.

What hasn't been disclosed are the financial terms - it's hard to see the agencies agreeing that Google's hosting is as valuable as their content - or how Google intends to generate additional revenue to cover its costs. At present, Google-hosted agency stories are presented without advertising.