Eighteen years after it was launched, the AltaVista search engine that first revolutionized search, is finally being put out of its misery. Author Steven Levy tweeted: “Yahoo shutting down Alta Vista like getting invitation to funeral of someone who you thought died 10 years ago.” So why should we bother about it?
For those too young to remember, and that’s the vast majority of the Internet user base, AltaVista was Google when Larry Page and Sergey Brin were still brainstorming about their famous search engine. Not that it was the first. There were others like Lycos, OpenText, Magellan, Excite, InfoSeek but their search base was way too narrow. By indexing around 20 million web pages, when a 10th of that was considered an achievement, it was the forerunner to the billions of pages that Google now indexes. But with search not yet seen as a revenue generator and the fact that it came out of the labs at Digital Equipment Corp., meant that AltaVista came to grief early. Indeed, it wasn’t till 1998 that it got its own URL. Till then it had to be accessed through www.altavista.digital.com, owing allegiance to the company that had developed it but didn’t know what to do with it. In 1998, Digital was sold to Compaq and in 1999, Compaq redesigned AltaVista as a web portal, hoping to compete with Yahoo and with that AltaVista lost its mojo.
Early users though swore by its magical properties. You had to learn its advanced syntax to search in page titles, URLs and link texts, but once you did that, what unfolded was like a magic kingdom. A friend, who was one of its early users, puts it succinctly: “With AltaVista, if something was there on the Net and you couldn’t find it, it was your problem and not the Internet’s.” It looked like the sky was the limit until of course Google came along and spoilt the party by showing us what AltaVista had missed. The older search engine’s text-analysis approach just couldn’t cope with the kind of growth and the kind of deliberate spamming in the name of optimization that came later. If Google hadn’t been invented and then evolved the backrub approach (the heart of the new search system as explained by Brin and Page in this academic paper http://stanford.io/KWiv), the Net would have been unusable today.
But the fact is, AltaVista’s original creators, Paul Flaherty, who was responsible for the original idea, Louis Monier, who wrote the crawler, and Michael Burrows, who wrote the indexer, were the first to realise that simple and effective search was the core around which the Internet would exist. They knew that search was not a mere programming feature but a fundamental and major science problem that needed to be solved. It needed serious work in math and information theory and not just engineering man hours. But it was Google that actually did it.
Indeed the history of technology is riddled with examples of prematurely developed products that became the forerunners to eventual leaders. Such cutting-edge products eventually become commonplace but for that early inventor who failed to find a market, this can be little consolation.
AltaVista might not have seen in 1995 that in the not too distant future, a search engine would eventually be as important to the world as the steam engine or the internal combustion engine had been. But for setting us on the path to that dramatic discovery we owe the web’s first “super spider”.