Home / Opinion / Columns /  Generative AI models like ChatGPT have competition

Some weeks ago in IT Matters I wrote about detractors of Foundation Models, a wholly new approach to artificial intelligence (AI). These are also called ‘generative AI models’. Generative models have become popular since they blow through the traditional methods of training AI programs with smaller datasets. The most salient feature of generative models for AI is that they scour almost every shred of information that is available on the web, a data store that is doubling in size every two years, and then use the outputs of these to train AI programs to generate output.

Open AI, heavily backed by Microsoft, has two such models: one called GPT-3, which is mainly for documents, and another called DALL-E, which focuses on images. GPT-3 analysed thousands of digital books and nearly a trillion words posted on blogs, social media and the rest of the internet. Its competitor is Google, whose own offering in Generative AI is called BERT. Most industry watchers expected generative AI to move on to newer models such as a potential GPT-4 in 2023. However, it seems as if Open AI and other firms aren’t done tinkering with their old models yet. Early in December, the San Francisco company released a demo of a new model called ChatGPT, a spin-off of GPT-3 that is geared to answer questions in back-and-forth dialogue. This can power industry applications such as chatbots, widely used in customer service applications.

You might also like 

Shikhar Dhawan turns investor, scouts for sports tech firms

ICE cinemas are cool, but will they succeed?

Returning covid casts a shadow over commercial realty sector

Why 33-year-old Tamilnadu Investors’ Association matters so much


What is striking is that ChatGPT is capable of producing short pieces of text that are remarkable in how seemingly coherent and eloquent they are. It has a variety of everyday use cases and its versatility is staggering. It gained 1 million users in just five days.

A scientist friend sent over a sample last week in which he asked ChatGPT to create a short explanation of itself. He was rewarded with what seems like a well written essay by a Class 6 student. It did an adequate job of describing itself, and to me, was clearly an advertorial for the product. This second part is what is problematic about generative models. They regurgitate either the filth or hyperbole that has been fed to them.

In contrast, the more focused cognitive models have smaller data sets (some of them even filled with dummy data) that are used to train AI programs on specific use cases. For instance, a medico-radiological system would limit itself to X-rays, magnetic resonance imaging (MRI) scans and other such medical images, it would probably not be training itself on poetry or music or other such information that has no relevance to the task at hand.

Interestingly however, BigTech companies are not the only players at the cutting edge of generative AI. There has been an open-source revolution to match, and sometimes surpass, what the best funded labs are doing. 2022 saw the first community-built, multilingual large language model, called BLOOM (BigScience Large Open-science Open-access Multilingual Language Model). We also saw an explosion of innovation around the open-source text-to-image AI model Stable Diffusion, which rivalled OpenAI’s DALL-E.

Earlier this year, the MIT Technology Review (bit.ly/3FSTkh6) reported that a group of over 1,000 AI researchers is working on a multilingual large language model that is bigger than GPT-3, and that this community of researchers plans to give out its model for free. This was BLOOM, which is designed to be as transparent as possible, with researchers sharing details about the data it was trained on, challenges in its development, and the way they evaluated its performance. In contrast, Open AI and Google have not shared their code or made their models available to the public, and external parties have very little understanding of how these models are trained. While you can sign up to use any of these models, you don’t get to peek under their hood. This includes not understanding their in-built biases, so that you can correct for them if need be while building your own system that uses generative AI to solve, say, a functional business problem.

The reason that this new open-source community is important is that Big Tech companies which have historically been the ones spending the most money on AI research are now suddenly facing what promises to be a tough 2023. Many are implementing massive layoffs and hiring freezes as the global economic outlook looks headed for a recession. AI research is undoubtedly expensive. As Big Tech companies look to save money, they will have to be very careful about picking which projects to invest in. It stands to reason that they are likely to choose whichever have the potential to make them the most money, rather than the most innovative or experimental ones.

Meta, owner of Facebook, has already made clear that it intends to cut back. In a post on ai.facebook.com (bit.ly/3Gj39q3), the firm says that it is reorganizing its AI research team and is dividing it up and moving it to teams that actually build products. Meta and Facebook have already been hit hard this year, with advertising revenues dropping off, so this move is not a surprise. 2023 will likely see other Big Tech firms tightening their belts as far as AI research is concerned.

The good news for a venture capitalist like me is that some of this work will likely move to startup firms. The old argument against, say, vernacular language models for India, that Google Translate or a similar Big Tech online service will kill a startup in that space, may now be less valid. Ergo, startups may still stand a chance.

Siddharth Pai is co-founder of Siana Capital, a venture fund manager.

Elsewhere in Mint

In Opinion, Raghu Raman says India's projected high GDP growth faces a challenge: a steady supply of corporate leaders. Vivek Agarwal analyses how a House panel wants reining in of Big Tech. Long Story finds young audience dwindling at Chennai's famous music festival.

Catch all the Business News, Market News, Breaking News Events and Latest News Updates on Live Mint. Download The Mint News App to get Daily Market Updates.
More Less
Recommended For You
Get alerts on WhatsApp
Set Preferences My ReadsWatchlistFeedbackRedeem a Gift CardLogout