Edge AI and on-device models: AI that runs without the cloud
Edge AI means running AI models on the device where data is created, such as a phone, laptop, camera, car or factory sensor, instead of sending everything to the cloud. On-device models are smaller, compressed models designed for this. They offer lower latency, offline use, better privacy and lower running costs, but they are less capable than the largest cloud models. Many products now use a hybrid: small models on the device for quick, private tasks and cloud models for harder ones.
What is edge AI?
Most generative AI today runs in large data centres: you send a prompt over the internet and a model on remote servers replies. Edge AI moves some of that work to the device itself. A phone can transcribe speech, a laptop can summarise a document, a security camera can recognise a parked car and a factory sensor can detect a fault, all without a round trip to the cloud. Edge AI is not new in industry, where machine vision and sensors have used local models for years, but generative models small enough for phones and laptops have made it mainstream.
Why run AI on the device?
- Speed: No network round trip, so responses feel instant for short tasks.
- Offline use: Works on planes, in basements, on construction sites and in remote areas.
- Privacy: Sensitive data such as health, voice or photos can stay on the device.
- Cost: No per-request cloud bill for tasks the device can handle.
- Reliability: Keeps working when connections or cloud services fail.
- Bandwidth: Cameras and sensors send results, not hours of raw video.
How do models fit on a device?
Models are made smaller through distillation (training a small model to imitate a large one), quantisation (storing numbers with less precision) and pruning (removing parts that contribute little). Phones and laptops now include neural processing units built to run these models efficiently. Many open-weight models come in small sizes designed for local use; our guide to open-weight AI models covers licences and options.
What is hybrid AI?
Hybrid systems route each task to the right place. Quick, private or frequent tasks run on the device; harder tasks go to a larger cloud model, ideally with the user’s knowledge. Some platforms run cloud requests in specially secured environments to limit what the provider can see. For product teams, the routing logic is a design decision: decide which tasks must stay local, which may go to the cloud and how users are told.
Where is edge AI used?
- Phones and laptops: transcription, writing help, photo editing and search
- Cars: driver assistance and voice control
- Retail: shelf monitoring and checkout cameras
- Manufacturing: visual inspection and predictive maintenance
- Healthcare devices: monitoring and alerts without constant connectivity
- Field work: offline assistants for engineers and inspectors
Industrial uses overlap with physical AI, where models on cameras and robots act in the real world.
What are the limits?
Small models know less and reason less well than the largest cloud models, so they make more mistakes on complex tasks. They use battery and memory, and older devices may not support them. Updating models across millions of devices is harder than updating one cloud service. Security matters too: a model on a device can be copied or tampered with. Test carefully on the range of devices your users actually have.
Does edge AI really save money?
For high-volume, simple tasks it often does. If an app transcribes millions of short voice notes or classifies every photo a user takes, running that on the device removes a per-request cloud charge that grows with every user. The trade-off is engineering effort: teams must package models, test them on many devices, manage updates and support fallbacks when a device cannot run the model. For low-volume or complex tasks, cloud models are usually cheaper to build and maintain. A sensible approach is to start in the cloud, measure which tasks are frequent and simple, then move those to the device once the volume justifies the work. Industrial users see a different saving: cameras and sensors that process locally send only results, which cuts bandwidth and storage costs for video and sensor data.
How should product teams design for edge AI?
- Map tasks to locations: Decide what runs on the device, in the cloud or both.
- Design for offline: Show what still works without a connection.
- Explain privacy clearly: Tell users what stays on the device and what is sent.
- Respect battery and storage: Download models on Wi-Fi and let users manage them.
- Measure on real devices: Test speed and quality on older and cheaper hardware.
Our principles for designing AI products apply, and reducing LLM API costs explains how moving some work to devices can cut cloud bills.
What does this look like in practice?
A field-service company in the UAE builds an app for engineers who inspect cooling systems in basements and plant rooms with poor signal. A small on-device model transcribes voice notes, fills in inspection forms and flags missing fields offline. When the engineer is back online, the app syncs and sends longer reports to a cloud model for a summary the customer receives. Customer site data stays on the device until sync, and the app shows clearly what is processed where.
How UIUX Studio helps
We design and build mobile and device experiences with on-device and hybrid AI. Talk to our Product Engineering or Technology Advisory teams.
Frequently asked questions
What is edge AI in simple terms?
AI that runs on the device where data is created rather than in the cloud.
Is on-device AI more private?
It can be, because data does not have to leave the device.
Are on-device models as good as cloud models?
Not for complex tasks, but they are good for quick, common ones.
What is hybrid AI?
Splitting tasks between device and cloud models.
Do I need special hardware?
Recent phones and laptops include AI chips; older ones may be slower.