Apple has just shown its new lineup of iPads including Pro models with “outrageously powerful AI chips.” The company said the M4’s neural processing unit is more powerful than any “AI PC today.” The chip’s video and photo-editing capabilities were demonstrated. Apple is also reportedly expected soon to offer generative-AI software tools for developers and is working on its own chips for use in data centers.
Meanwhile, Qualcomm has recently launched its AI Hub—”a gateway for developers to enable at-scale commercialization of on-device AI applications.” Qualcomm also believes it is well positioned to capitalize on growth in AI as that expands from the cloud to devices employing its Snapdragon 8 series processors. More than a year ago it demoed Stable Diffusion on an Android phone with this text-to-image generative AI foundation model creating photorealistic images of characterized cats.
Apple’s and Qualcomm’s chips, in tandem with iOS and Android, respectively, cover the majority of high-end smartphones and tablets, as well as other upcoming AI platforms including PCs, cars, XR headsets and additional IoT devices.
Nevertheless, how and when all that neural processing horsepower will actually be employed on devices remains unclear. AI processing—including most compute-intensive training, as well as inferencing—is already very entrenched in cloud data centers.
One more thing
A cynical reader commented in response to a Wall Street Journal article on Apple’s announcement:
“I’m a tech professional for 40+ years and wrote an article that we have hit a quality/price wall whereby the “next thing” is a micro improvement over the past product. No one will print it, least of all marketers. For example, the Apple M1 chip is three times fast enough for 98% of the world, so the CONSUMER world does not need an M2, M3, or M4 chip. There is simply no need. Put an Apple iPhone 7 next to an iPhone 14 and perform the 6 most common tasks. Speed difference? Almost NONE! (Shhhh, don’t tell anyone).”
We’ve been here before: What was once gimmicky and superfluous in mobile phones has become essential and universal in adoption and use. For example, video telephony was a key selling feature following 3 UK’s handset service launch with, for example, a Samsung’s 2004 3G flip phone; but that feature was scarcely used because there was hardly anyone to call and service quality was poor. Camera phone capabilities, even in the iconic and hugely popular 2004 2G Motorola RAZR V3, were also rather pointless, with photos of poor resolution and limited color palette largely remaining trapped inside those devices. It was only much later phone models that could easily and cheaply send photos to one’s PC or to others. Now, most people use only smartphones for recording video and taking pictures. Location-fixing capabilities, as required for e911 emergency services since the millennium, also sat largely dormant until navigation and ride sharing applications took off a decade later.
Revolutionary ensembles
While those video, photo and location-fixing innovations were really important precursors to what has ensued, they were only pieces in a larger puzzle that was still incomplete. It was only when those product features were also included in later-released devices, along with complementary improvements such as much quicker cellular data, faster processors, the addition of WiFi and an iOS or Android operating system, together with some widely-adopted apps like Facetime, Facebook, WhatsApp and Uber that widespread and frequent usage of those precursors really took off.
The first mobile communications revolution brought voice calling and texting to around half the world’s population by 2010. I was also impressed how various vendors and commentators had faith and commitment—during the somewhat disappointing times with 3G in the mid-2000s— to deliver a second personal communications revolution. That made mobile devices the primary or only means for most of those people also to access the Internet. By 2012, most phone sales were of iPhones and Android smartphones and 4G was being rapidly adopted including the introduction of the first 4G iPhone model at year.
AI today and tomorrow
Smartphone users already widely benefit from some mundane machine-learning automation that invisibly optimizes radio network and camera performance. More prominent AI-based capabilities using large language models and generative AI include simultaneous translation and a niche application that provides audio descriptions of the street scene and passers-by to blind pedestrians. We await a “killer app” for smartphone AI, as we got several times over for smartphones in general a decade ago with the likes of Uber, YouTube and Instagram.
It is also unclear how smartphone application AI workloads will be allocated between central data centers, edge computers and devices. Data centers have superlative processing power. The benefits of doing AI processing on devices versus in cloud data centers include privacy, lower latency and lower costs —there is no marginal monetary cost for on-device processing once you have the silicon to do that.
On the other hand, on-device processing saps scarce battery capacity. More strategically, app providers might well be willing to pay for central processing and also offer freebies to those who let their usage be data-mined there for monetization through targeted advertising.
On-device neural processing will be somewhat limited in the next year or two. Nevertheless, demand for that on-device AI will likely increase enormously as the rest of the mobile supply ecosystem including software developers recognizes and is tooled up to harnesses its benefits. With over-the-air software updates and downloads now the norm, and with rich and powerful complementary hardware capabilities in the latest devices, perhaps this time we might not need to wait for much later-released devices before enjoying more extensive on-device AI.
