Monetizing generative AI: Smaller models aim for wider accessibility

Industry insiders see this as a natural progression to monetizing generative AI technologies, making them more widely accessible, instead of targeting only enterprises through very expensive cloud platforms.

For instance, on 23 February, Qualcomm demonstrated an image-generating AI model running on a device, with Stable Diffusion showcased on an Android smartphone. Subsequently, on 18 July, Meta and Qualcomm jointly introduced Llama-2, an LLM which will be available for real-world use by next year.

Experts said this approach is a way forward to address concerns over costs, environmental impact and practicality, and would pave the way for further adoption of AI technology.

Speaking at a media roundtable, Ziad Asghar, senior vice-president of product management, Qualcomm, said: “When an AI model is being trained, it takes all the information from the cloud. This training aspect will happen on the cloud—but now, it is extremely intensive to use it for inference. This is why we’re looking at making AI models smaller and more compressed for running them on-device."

This evolution in generative AI is natural, Jaspreet Bindra, founder of Tech Whisperer, a technology consultancy firm, said. “This should be the way for our future, and we can be very confident about it. Capability of AI moving to the edge is already happening, including for offline use cases in various industries like networking base stations, without going back to the cloud. Naturally, the evolution of generative AI, too, will happen in this case, and we will need special chips for these use cases—which is also where Qualcomm enters the equation."

According to Qualcomm, such on-device models, which can run offline without a constant connection to host cloud platforms, can be deployed across smartphones, laptops, extended reality (XR) headsets and devices, as well as cars and industrial internet-of-thing (IoT) devices.

On XR devices, it can assist in generating immersive visual models locally. It can also tailor driving rules for connected cars, equipped with advanced driver assistance system while they are in motion by aligning real-time on road data with the pretrained instructions.

Asghar did not mention specific names, but said that Qualcomm is working with multiple partners, including most brands in the smartphone industry. “We’re also working with multiple automotive brands for ADAS experiences, as well as with many extended reality brand partners. We will make more specific announcements at our upcoming Qualcomm Summit," he added.

An executive at a top smartphone brand in India, seeking anonymity, said teams of most smartphone brands are working with Qualcomm, Microsoft, and Google, to explore developing generative AI use cases directly on devices.

“Work on such technologies is at an early stage, but we are already exploring developer implementations of applications such as photo editing and emails, which make for some of the most-used mobile applications in general," he added.

One use case can be during video conference, said Bindra. “During a remote conference call, generative AI can listen to a sentence and replace missing words on a transcript, mitigating audio quality issues."

“Local AI models can also be updated from cloud platforms just like it is with app updates," the executive added.

Another reason for such development would be the environmental cost of running LLMs on cloud. “You cannot have massive models just sitting in the cloud, and every user query gets routed to the cloud. This will be important environmentally as well—running cloud AI operations is a massive environmental cost," Bindra said.