AI-augmented web search is at our fingertips already

Google’s search engine answered the question with its usual list of internet links, led by a link to “AI based literature review tools" and including an article titled, ‘How do you know a human wrote this.’ Microsoft’s Copilot, embedded in the Bing search engine, came back with an emphatic “certainly." Both Copilot and Perplexity.ai, a new search engine that I like, returned responses that laid out the advantages of AI-based search and concluded with some limitations. Perplexity began by asking me about the purpose of writing the column: education, opinion or news. Armed with the “opinion" nature of the column, it went on to suggest that the main purpose of such an exercise was to discuss the benefits and challenges of implementing AI-based search. It also helpfully suggested that AI-based search can help optimize long-tail search results by providing more relevant results for less common queries.

Many moons ago, Google revolutionized search with its unique ‘page rank’ algorithm. The cleanliness of its landing page and the general quality of its algorithms used to display relevant links on the internet made Google the world’s de facto web-search monopoly. In its early days, Google toyed with the idea of a subscription fee for a certain number of searches, but quickly pivoted to an advertising-based revenue model. With that switch, the ‘clean’ page-rank output got mixed with sponsored links, and with each click, its users were unwittingly losing a bit of their soul to Google. Additionally, the number of URLs on the internet is now approaching 1 trillion, more than three-fourths of which are inactive. Google (and to a less extent Meta) dominate the global supply chain of online advertisements today, from consumer data to a bustling online marketplace at the back-end that auctions each click of ours. Google is estimated to have a 92% share of internet searches.

With the advent of big data and massive computing power, the AI revolution was on ‘quiet boil’ for a few years until the ChatGPT public-launch late in 2022 sparked a race to include AI, particularly Large Language Models (LLMs), in every aspect of our lives. Discussions about LLMs and the potentially destructive power of AI have dominated the discourse over the past couple of years. Yet, not much has been written about the fast-evolving world of AI-based or AI-augmented search. These AI tools can handle complex questions, interact conversationally and improve over time with machine learning.

New search engines combine the benefits of traditional search, which rank orders the relevance of existing data, with the presentation, synthesis and generative advantages of AI. Perplexity’s website says this, “You can ask Perplexity questions in natural, everyday language. It will seek to understand your intent and provide answers in a way that resonates with you. To give you the most relevant information, Perplexity may have a back-and-forth conversation and ask questions to clarify your needs. Perplexity achieves this with the help of our advanced answer engine to process your questions and tasks, considering the entire conversation history for context. It then uses predictive text capabilities to generate useful responses, choosing the best one from multiple sources, and summarizes the results in a concise way." The ability to seek answers in natural language without the limitation of keywords, coupled with its anchor in real data ranked for relevance, makes this a system that is subject to minimum hallucinations (producing content that is not real). Given its memory feature, you can go on to request a different form of presentation, for instance as a table rather than as a list. Of course, it makes mistakes and is not comprehensive just yet, but you have a surprisingly good product that you can work with. It is a real productivity enhancer.

I predict that the biggest early use of AI technology for consumers will be in search. Most internet users already use search. While they will have to retrain themselves to query AI-augmented search engines, this is a simple task in a natural language. The results are much richer, synthesized for use, with citations available for validation. Several new engines like Andi Search (offers video and synthesis), Metaphor (considers metaphorical nuances), Brave (good at summaries), Bing/Copilot (which embeds ChatGPT4), Google Gemini (that embeds Bard) and many others have emerged in the last few months. While ChatGPT stole the initial oxygen, the playfield is getting more competitive and it is not clear who will win the long race. Google and Bing have a readily available customer base, but the quality and speed of service (latency) is still a wide-open arena for competitors and Google has more to lose than other players from a migration away from regular search.

Download one or two of these apps on your phone and try some of your regular searches in parallel. It takes a bit of getting used to, but once you see some results, I am sure you will want more. According to Perplexity, you should “try AI-based search for a more advanced, personalized and efficient search, especially for complex queries or a desire for tailored results."

Don’t just search anymore, seek.

P.S: “The real voyage of discovery consists not in seeking new landscapes, but in having new eyes," said Marcel Proust.