Apple’s ambition: It wants AI to stand for Apple Intelligence

When Apple answers a query using its smaller on-device AI, it’ll do so with a latency of 0.6 milliseconds per prompt token.  (AFP)
When Apple answers a query using its smaller on-device AI, it’ll do so with a latency of 0.6 milliseconds per prompt token. (AFP)

Summary

  • While sophisticated AI tools process queries on cloud servers that need an internet link, Apple will run some AI queries via a small language model loaded on iPhones to make AI usage private and quick. Privacy is good, but will low speed deter consumer adoption?

You have to hand it to Apple Inc. After a tone-deaf ad last month that made the company look oblivious to AI’s impact on the world, its marketing department has now rebranded AI as ‘Apple Intelligence.’ It’s a feat only the company could pull off. Customers of Macs and the latest iPhones will use it to rewrite emails, transcribe and summarize calls, generate images and cross-reference information from Apple apps. “Will I get to my daughter’s play performance on time?" 

Apple Software Chief Craig Federighi asked in a demo at its developers conference on Monday. Apple Intelligence would consult his iPhone’s proprietary Calendar, Maps, Mail and iMessage apps to answer.

After a string of mundane updates to its operating systems, these are the most exciting features in years, but will Apple Intelligence really work as seamlessly as it did in pre-recorded demos when it rolls out this fall? I’m inclined to believe we’ll see glitches and latency issues that will make it a tough sell to consumers, at least initially.

The most sophisticated AI tools today process your queries on cloud servers that need an internet link. Apple’s iPhone has a fraction of the power of those servers, but to make its AI service private and quick, it will run some AI queries via Siri ‘on device,’ on a small language model Apple built for iPhones. No net connection needed.

Also read: Apple's AI push intensifies Big Tech race. Gen AI commercialization next?

Apple Intelligence will also decide, on the fly, if a query like “Will I get to my daughter’s play performance on time?" requires extra computing power. If it does, it’ll access a bigger AI model that Apple made, via its ‘Private Cloud Compute’ (its own servers). 

Anything more complex will request a query to ChatGPT, via a partnership with OpenAI. Apple, admirably, has gone to great lengths to keep this process private, with query requests being end-to-end encrypted and inaccessible to others.

The price for all of this could be speed. When Apple answers a query using its smaller on-device AI, it’ll do so with a latency of 0.6 milliseconds per prompt token, according to an Apple blog post on the subject, or faster than the blink of an eye. 

But Apple didn’t offer corresponding latency times for when the phone has to access its Private Cloud Compute for heavier queries, and that’s a noteworthy omission. It’ll likely be slower, but by how much? Apple doesn’t say.

As shallow as this sounds, consumers hate having to wait a few extra seconds for things they can do themselves, and if it’s simply quicker to look something up in their calendar or mapping apps, they might decide to avoid using Apple Intelligence.

On the other hand, while the on-device AI will be faster, it will also be more prone to errors. According to Apple, the smaller AI model is about as capable as GPT-3.5 Turbo, which OpenAI launched more than a year ago, and which doesn’t have a stellar reputation for accuracy. 

Also read: Apple introduces Game Mode for iPhone with iOS 18; enhanced gaming in AAA titles

The hallucination rate for GPT 3.5 has ranged from 3.5% to more than 15%, according to different estimates, and that points to another omission from Apple. It didn’t offer accuracy rates for either of its new AI models.

Remember that even the most sophisticated models coming from leading players like Google—think about the new AI Overviews—are still laughably error-prone. Apple could have disclosed some decent hallucination rates. Instead, it offered vague stats about how human graders “preferred" its AI over others.

Little wonder then, that Bloomberg’s senior Apple reporter Mark Gurman has said the new AI features will be buggy and in ‘beta’ when they launch this fall.

Apple’s role at the centre of the lives of its customers and its access to reams of their personal information could end up being both a blessing and a curse. It’s better placed than most tech companies to make AI useful because it is plugged so deeply into our daily existence. But that also means it can’t afford to make too many mistakes.

Federighi’s example of making it to his daughter’s play is a case in point. Were Apple Intelligence to inadvertently make its user 30 minutes late, it wouldn’t be easily forgiven. Much the same happened to Apple Maps, where a flurry of mistakes in its early days sent people to competitors like Waze and Google Maps for years afterwards. 

Also read: AI may not be what sells Apple's next iPhone

It doesn’t help that Apple has been slow on the uptake with Generative AI, thanks in part to a smaller AI team compared with other large tech firms, and a history of management troubles at its Siri division.

Apple’s more private approach is a welcome stance, and the company should be commended for sticking to its guns on using private encrypted servers while others are playing fast and loose with AI in a battle for supremacy. But getting consumers to embrace its ambitious new features won’t be easy when the glitzy demos are suddenly made real. ©bloomberg

Catch all the Business News, Market News, Breaking News Events and Latest News Updates on Live Mint. Download The Mint News App to get Daily Market Updates.
more

MINT SPECIALS