Mobile teams now have a real choice about where an AI feature runs: on the phone, using the increasingly capable on-device models, or in the cloud, calling a provider or a hosted model. Both work in a demo. They fail differently in the field, and the failures are about connectivity, battery, privacy and cost rather than model quality. This post is about the feature that taught us to ask the placement question early, and the questions we ask now.
The feature that failed in the field
A field-service app needed to read a serial-number plate from a photo and look up the equipment record. We built it cloud-first: photo up, extraction in the cloud, record back. It was accurate and fast on our phones in the office. In the field, technicians were in basements and plant rooms with no signal. The feature that was supposed to save them typing made them wait, then fail, then type.
We moved the extraction on-device with a small vision model, kept the record lookup for when the signal returned, and cached the equipment records they were likely to need for the day. The accuracy was slightly lower on-device. The feature became useful, which it had not been.
A mobile feature is evaluated where the user is standing, not where the developer is sitting.
The four questions
Does it need to work offline, or under a bad connection? If yes, the core of the feature runs on-device, whatever that costs in capability. Cloud can enhance the result later.
How sensitive is the input? Photos of documents, health data, location, private messages. On-device processing means the data never leaves the phone, which is the strongest privacy story there is and sometimes a regulatory requirement. Cloud processing means a transfer, a provider, retention terms and consent.
What does the task need? Small classification, extraction, transcription and image tasks run well on-device. Long-context reasoning, large knowledge, and anything that must be current needs the cloud, or the cloud plus retrieval.
What are the volume and cost? A feature used many times a day by many users is expensive in the cloud and free on-device once the model is shipped. A feature used occasionally, with a large model, is cheap in the cloud and impossible on-device.
The hybrid most apps end up with
Few features are purely one or the other. The pattern that has worked across our mobile projects is: the fast, private, always-available part on-device; the capable, current part in the cloud; and an explicit design for what happens in between. The on-device model gives an immediate result. If the network is available and the task benefits, the cloud refines it. If not, the on-device result stands and the app says so. Records the user will need are prefetched when the signal is good. Work that needs the cloud queues and syncs when it can.
Engineering realities on the device
On-device models bring constraints that the cloud hides. Model size affects download and update size; a model that doubles the app's install size will be noticed. Battery and thermal limits mean inference must be short and infrequent, or scheduled. Device fragmentation means the model that runs in a second on a flagship phone runs in ten on the device the field team actually carries. We test on the low end of the fleet, not the high end, and we ship model updates separately from app updates so a better model does not wait for a release.
Security on both sides
On-device, the model file and any cached data are assets on a device that can be lost or rooted. Model files are stored with platform encryption, cached records are encrypted and expire, and nothing sensitive persists longer than the workflow needs. In the cloud, the usual controls apply: authenticated endpoints, minimal payloads, no unnecessary retention, and a clear statement in the app's privacy notice about what is sent. For regulated data we default to on-device even when the cloud would be more accurate, and we treat that trade-off as a product decision made with the client, not an engineering one made quietly.
Purposeful placement
The question is not which is better. It is which the feature needs, given where and how it will be used. Deciding that in discovery, with the field team's phones in hand, is far cheaper than discovering it in the field. The feature that finally worked for the technicians was less accurate than the one we built first. It was also the only one they used.
- Decide placement in discovery, on the devices users actually carry.
- Offline need and sensitive input push on-device; complexity and freshness push cloud.
- Design the hybrid: on-device first, cloud refinement, prefetch, sync queue.
- Test on the low end of the fleet; ship model updates independently.
- Encrypt on-device assets; minimise and disclose what leaves the phone.