Does Google's power threaten the web? (part two)

Google software engineer Matt Cutts said the search engine business is healthier and more competitive than ever, with many niche providers and international forces. "What we worry about is providing the best results to users; we don't worry about market share," he said. "That will all work itself out."

Exactly how the system works itself out, however, is what worries people in the industry. Boser and others are concerned that paid advertisers get crucial advice that maximizes their editorial listings. "As you're spending money with them as an advertiser, that spending does not buy you position; that spending buys you advice on how to get position - a byproduct of the advertising sport," said Data Recovery's Ahern, a client of WebGuerrilla.

Ahern learned that the term "data recovery" was a valuable commodity in paid search, worth as much as $10 per click. As a result, he was forced to spend several thousand dollars a month on pay-per-click advertising on Google and rival Overture to compensate for losing up to 85 percent of his traffic earlier this year. Ahern no longer buys Google ads because his top position has been restored, but it has been an expensive process.

The depth of concern about the search industry's practices was made clear at a conference earlier this year, where participants stressed the need for policies to protect websites dependent on the engines.

Among the proposals were calls for fair and consistent spam-reporting policies in which Google and others reply to all complaints, not just those from advertisers. Also suggested were standards for a formal review system that helped sites understand why they're not listed and thereby give them information necessary to improve their chances.

Craig Silverstein, Google's chief technology officer, denies that advertisers get preferential treatment in its editorial listings. On the issue of information about changes in rankings, he said it is impractical to provide support for everyone, considering that the company indexes nearly 2.1 billion pages. But Google is examining its system, well aware that growing criticism could damage its credibility with the public at large.

In an effort to beat Google at its own game, some marketers have adopted guerrilla tactics that try to manipulate search rankings. This has led to a proliferation of "link farms" - elaborate linking schemes designed to manipulate one of Google's only publicised algorithms, PageRank, which factors a site's popularity based on the number of web pages that link to that site.

Many industry executives speculate that Google changed its search algorithm in September to combat link farms, which caused a shift in listings and a lower number of documents served up for any given query. The changes have thrown web masters into a frenzy, as shown in thousands of messages on industry sites such as SearchEngineForums.

Daniel Brandt, who publishes the watchdog site Google-watch.org, complains that PageRank is the equivalent of a popularity contest that favours major, established websites. "We don't know which comes first - whether Google is reflecting popularity or if it's creating popularity," he said.

Silverstein asserts that Google does not provide preferential treatment to advertisers in text results or target smaller sites for delisting. Google alters its algorithms all the time to improve search quality, he said, and sites might have seen changes recently because of a "fresh crawl" that updates up to three million documents more regularly.

As for tactics such as spam and link farms, Google says those just make accurate searches more difficult for surfers. That, in turn, could affect Google's credibility at a time when trust is more important to the company than ever.

"There's a constant battle between abusive technologies and providing relevant search results," Silverstein said. "When you use something as a tool and you don't have control over it, that's an issue, an issue of trust. You need to be able to trust us that we are acting in the best interests of our user."

Others dismiss much of the complaints about Google as conspiracy theory, saying it should not be required to baby-sit sites that have simply fallen victim to the internet's evolution.

"Google is a great scapegoat when people get mad. But it's about protecting relevancy," said Jessie Stricciola, a consultant who helps companies improve traffic to their websites. "The cannibalisation of search results has the potential to become problematic, because you have one chance with 10 distribution channels. But that's the nature of how the web is evolving - that's not Google's fault."

Stefanie Olsen writes for News.com