India’s diversity can provide lots of fodder for conversational AI products
India's large base of internet users is generating data that can be used to fine-tune conversational AI solutions
{{^adFree}}
Business News/ Technology / Gadgets/ India’s diversity can provide lots of fodder for conversational AI products
Conversational artificial intelligence (AI) today is hardly interactive. It’s transactional at best. The reason that the two major conversational AI solutions—text-based (chatbots) and voice-based (personal assistants)—are yet to deliver is that the techniques for effective and efficient human-machine conversations are still evolving as opposed to what the technology industry made us believe.
{{^adFree}}
Human-machine conversations comprise natural language understanding (NLU), which is an understanding what the user said. Natural language generation (NLG) is all about formulating a reasonable and on-topic response to the user. However, response generation is not simply a product of collecting and analysing lots of data.
Natural language processing (NLP), of which NLU is a subset, is difficult because the tasks regarding linguistics need to include variations in human psychology, cultures and linguistic diversities. Further, most conversational experiences today are either very broad but shallow (for example, “What’s the time?" = “The time is 10.00 am") or very narrow but deep (for example, a multi-turn conversation in a quiz game).
{{^adFree}}
To advance beyond these limited experiences, we will need to get to a world of both wide and deep conversations. This will require the machine’s ability to scale beyond the current technical limitations of recognizing between only a few hundred intents at a time. Another limitation of machine conversations is personalization. In a natural conversation between two people, each will normally draw on previous experiences with the other person and tailor the responses accordingly. Computer conversations that don’t do this tend to feel unnatural and even annoying. Addressing this in the long term will require solving challenges such as speaker identification, so that the computer knows who you are and can respond differently to you versus someone else.
Many enterprises and conversational AI solution developers, in the last couple of years, have been busy developing chatbots of primarily two kinds. The first is enterprise chatbots that are built to solve enterprise use cases such as customer support, lead generation, etc. The second category is direct-to-consumer bots—chatbots that reach the consumer directly for specific applications. However, most enterprise bots currently available are automated versions of FAQs (frequently asked questions) and are unable to hold conversations beyond a few interactive dialogues.
Consumer interest will materialize when machine intelligence gets near human intelligence. Having said that, the diversity, scale and capabilities available in the Indian market could be leveraged to make an early mark.
{{^adFree}}
Consider India’s internet subscriber base, which has grown rapidly in recent times to reach close to 500 million subscribers, from 84 million in 2012. India also reportedly consumed 22% of world’s mobile data between April and June 2018, and Indian telcos handle more data traffic than their Chinese and US counterparts, combined, according to the Cellular Operators Association of India. This large and rapidly increasing base of internet users are generating data that can be used to build and fine-tune conversational AI products.
Further, India has 22 official languages, and of its 1.35 billion population, less than 10% can understand English. Of the current internet subscriber base, majority are non-English speakers. Fourth, the latest NLP technique of embedded language models (ELMos), wherein the NLP algorithm trained on the data set of one language can adapt itself for other (similar, adjacent) languages without training, can work best in Indian scenario. For example, an ELMo-based solution trained on Hindi language data set can adapt itself for Marathi and Punjabi as well, thereby scaling faster.
Many global and domestic companies are focusing on India, both as a market and innovation hub. For instance, a significant number of Microsoft’s Cognitive Services’ 300,000 developers globally, are from India. Amazon has grown the number of developers (specific to Alexa skills) from less than 10,000 to 40,000 in just a year of its presence in the country.
{{^adFree}}
For Amazon, India is the second-biggest market (after US) for Alexa skills. Further, according to Tracxn, a startup database, there were over 100 chatbot startups in India as of 2017. While there are a number of companies such as Haptik and Active.ai that help develop conversational AI products for their enterprise clients, some are adopting and creating solutions in niche areas such as Niki.ai (digital assistant), Fynd (Fashion apparel chatbot), Myprivatetutor (tutor finder), Lawrato (legal searches) and Wysa (mental health tracker) to name a few.
Apart from chatbots development, a couple of other Indian startups such as Liv.ai (acquired by Flipkart) and Reverie Language Technologies (disclosure: I am an adviser) are tackling the Indic languages digitization issue with development of voice engines for Indian languages, with advanced language and acoustic models as compared to global technology giants.
For now, bots can continue to help us with automated, repetitive, low-level tasks and queries; as cogs in a larger, more complex system. We did them, and ourselves, a disservice by expecting too much, too soon. Tapping the skills and language markets in India could rectify the situation.
{{^adFree}}
Jayanth Kolla is founder and partner at Convergence Catalyst.
Unlock a world of Benefits! From insightful newsletters to real-time stock tracking, breaking news and a personalized newsfeed – it's all here, just a click away! Login Now!
{{^adFree}}
Catch all the Technology News and Updates on Live Mint. Download The Mint News App to get Daily Market Updates & Live Business News.