Can an app use AI without sending your data to the cloud?

Can an app use AI without sending data to a server?

Yes. An app can run a language model locally, on the device’s own processor, so the text or image being analysed never leaves the hardware. This is called on-device inference, and it removes the network round trip completely: there is no API endpoint, no request body carrying user data, no provider-side log, and no third-party breach that can expose something a user typed into your app. For a feature touching health, financial or private-message data, that is a materially different privacy position from a cloud call rather than a difference in wording.

The distinction worth holding onto is that this is an architectural property, not a policy one. A cloud provider promising not to train on your data is making a promise; a model running on the device is not in a position to break one, because the data never reaches anybody who could.

What does on-device AI mean in practice?

There are two routes, and they have very different costs. The first is to call a model the operating system already provides, which costs nothing in app size and nothing per use, but only works on devices whose OS ships one. The second is to bundle your own model weights inside the app and run them with a local inference runtime, which works on far more devices but makes the app substantially larger.

Most products that ship this well use both: the OS model where it exists, your own small bundled model or a cloud call where it does not. Deciding which fallback applies to which device class is the actual engineering decision, and it is better made before the feature is designed than after.

Which on-device model APIs exist on iOS and Android today?

Both platforms now expose a built-in model to third-party apps, and a handful of cross-platform runtimes exist for shipping your own.

What can a small on-device model actually do well?

A 3-billion-parameter model is a genuinely useful tool for bounded, text-shaped work, and a poor substitute for a frontier model on anything open-ended. The tasks it handles reliably:

• Summarising something the user already has — a note, a message thread, an article.
• Classifying and tagging: routing a captured note to the right list, labelling an expense, detecting intent.
• Extracting structure from loose text, which is the quiet workhorse — turning “dentist tuesday half four” into a dated, titled task.
• Rewriting and proofreading, where tone and grammar matter and facts are supplied by the user.
• Short, grounded question answering over content already on the device.

What it does badly is equally specific, and worth stating plainly: multi-step reasoning, long documents that exceed a small context window, anything depending on broad world knowledge, and anything where a confidently wrong answer is expensive. A small local model does not know much; it is good at operating on text you give it. Features designed around that constraint work well, and features that ignore it produce the demo that impresses once and annoys thereafter.

What does on-device AI cost?

There is no per-token bill, which is the headline and genuinely changes the economics of a feature used constantly by every user. The costs move elsewhere instead, and the one that surprises people is app size.

The arithmetic is easy to do and worth doing before committing. A 3-billion-parameter model quantised to 4 bits is about 1.5 GB of weights — three billion values at half a byte each. That is far too large to bundle into a typical mobile app, which is the single best reason to prefer the operating system’s model where it exists: calling it costs zero bytes of download, because the model is already on the device as part of the OS.

The remaining costs are engineering ones. Two code paths instead of one, a fallback that has to be tested on hardware that lacks the built-in model, and output evaluation that is harder than with a cloud model because you cannot change the model under a deployed app — on the OS route, the model updates when the OS does, on someone else’s schedule.

When is on-device AI the wrong choice?

Often, and it is worth being direct about it. Send the work to a cloud model when the task needs real reasoning or breadth of knowledge, when inputs are long, when you need every user on identical model behaviour regardless of their hardware, when you must change model behaviour quickly after launch without shipping an app update, or when the data is not sensitive and the privacy argument is therefore not buying anything.

The expensive mistake is choosing on-device because it sounds better rather than because the data demands it. It constrains what the feature can do, doubles the paths you maintain, and ties you to a model you cannot upgrade on your own timetable. Where the data genuinely cannot leave the device, every one of those costs is worth paying. Where it can, you have taken on all of them for a line in a press release.

How does Percuno approach on-device AI?

On-device AI is the specialism Percuno is built around, and the claim rests on employment history rather than on Percuno’s own client work, which does not exist yet. Kip Sarapas — Percuno’s founder and only engineer — works on on-device and local LLM integration in mobile products in his current role as a mobile engineering lead, which is where the experience behind this article comes from. Percuno is a new company with no clients and no case studies, and this article is not one.

It is also why Superset, Percuno’s own AI workout and food-prep planning app, is being built by someone who already knows how to keep that data off a server. An app that knows what you eat and how you train is a clear case where the privacy argument is real rather than decorative. Superset is in development and not yet released.

For client work, this is the piece most teams cannot staff: a product manager can ask for “AI, but this data cannot leave the device”, and comparatively few people can deliver that sentence. If that is the problem you have, the services page describes how projects work, and hello@percuno.com reaches the engineer directly.