404 frustration drives cartographer to map entire web

LinkGuard is to create one of the world's largest near real-time distributed databases, using up to 40TB of storage. The company claims the system, which will be based on Dell PowerEdge and PowerVault hardware, will be able to search the whole web in ten days.

The idea of the map is to find broken links between sites and fix them, according to Frank Jeannin, founder and CEO of Maidenhead-based LinkGuard.

"Like so many other people, I got fed up with getting a '404 error' message and looked for a way of dealing with it."

Now Jeannin, a self-confessed geek who began programming in France at the age of ten, intends to turn the idea into a business by charging one cent for every page link that is checked each week.

The LinkMap software will scan the web at an average rate of 2,200 pages a second. It takes search engines six months to index approximately 16 per cent of the content on the web.

"With the internet currently numbering two billion pages with an average of 52 links a page, then add to that the fact the internet will double in size over the next year to 18 months, and you have a lot of links that can get moved," says Jeannin.

The award of the hardware contract to Dell was in part due to Michael Dell's own involvement in the bid. He personally emailed senior executives saying that LinkGuard was the kind of company he wanted to partner with.

"We sent out a specification of what we needed," says Jeannin. "Dell delivered above the spec in ten days. IBM took five weeks and performed below what we needed."

Hardware and software testing is still underway and an ISP contract is due to be awarded soon. The project is due to go live in mid-2001.

By Geoffrey Davies