"On-device AI" gets used as a blanket marketing term, but Apple, Google, and Samsung each implement it differently, and each one quietly sends more to the cloud than the marketing suggests. Here is what is actually happening on your phone right now.
Apple Intelligence: on-device first, cloud as a documented fallback
Apple's system routes requests either to a local model or, for harder tasks, to Private Cloud Compute, Apple-designed servers built so that "personal data is not stored nor made accessible to Apple or anyone else," according to Apple's own security documentation. The on-device model works fully offline with no daily limit but a smaller 4K token context window; the cloud tier requires connectivity, has a daily usage cap, and offers a much larger 32K token context for more complex requests. Notably, Apple has opened Private Cloud Compute to outside audit, publishing the actual server software images and running a public bug bounty specifically for researchers to verify the privacy claims independently, a real, unusual level of transparency for a consumer AI feature. Hardware requirements are strict: 8GB of RAM minimum, which is why the iPhone 15 and 15 Plus, with only 6GB, do not support it at all.
Gemini Nano: built for the sensitive stuff specifically
Google's on-device model runs through a system-level service called Android AICore, and Google has been explicit that it is used for privacy-sensitive workloads specifically, call transcription, scam detection, message summarization. On Pixel 9-series devices, Google reports the on-device model handles roughly 68% of common assistant queries locally without ever reaching the cloud. One concrete performance data point: Gemini Nano can produce an offline, timestamped summary of a full one-hour meeting in about 45 seconds, entirely on-device. The catch is hardware fragmentation, the newest version, Nano v3, is confirmed to require 12GB of RAM and a 2026-generation flagship chip, meaning even fairly recent phones may not get the newest on-device capability.
Galaxy AI: an explicit on-device toggle, with real trade-offs
Samsung is the most transparent about the split. Its own documentation explicitly separates features by where they run: Call Screening, Now Nudge, and Scam Detection process entirely on-device through a secure component called Knox Vault. Creative Studio's generative image editing and Gemini integration require the cloud. Circle to Search, despite feeling like a simple on-device gesture, always requires an internet connection according to Samsung's own support documentation. Samsung also offers a direct privacy toggle, "process data only on device," which trades away cloud-dependent features for guaranteed local processing.
The real benefit, backed by real numbers
Independent testing backs up the core privacy and speed claims. On-device processing measured on Apple silicon showed latency in the 25 to 55 millisecond range, compared to 180 to 600 milliseconds for equivalent cloud calls, almost entirely due to network round-trip time. On the power side, cross-platform testing found that running a task on a dedicated NPU instead of a GPU cut power draw from 30 to 40 watts down to 5 to 10 watts for the same workload, translating to a documented 15 to 20% improvement in battery life under heavy AI use.
The failure Apple had to publicly walk back
On-device does not mean error-free. Apple's on-device notification summary feature produced a string of real, embarrassing hallucinations in production: it summarized a BBC story as "Luigi Mangione shoots himself" when he was in fact in custody and alive, falsely reported that a tennis player had come out as gay, and misreported an ICC arrest warrant as an actual arrest. After the BBC and the National Union of Journalists formally complained, Apple suspended the feature for news and entertainment apps and now displays AI summaries in italics with an explicit error disclaimer. It is a clear, documented example that running locally solves a privacy problem, not an accuracy one.
The takeaway
On-device AI is a real architectural shift, not a marketing buzzword, faster, more private, and increasingly the default for sensitive, everyday tasks. But every major implementation is genuinely hybrid, and the split between what stays local and what goes to the cloud is more specific, and sometimes more surprising, like Circle to Search always needing a connection, than the general "on-device" label suggests.
