What is local AI?

Local AI: a definition

Local AI means running an AI model on hardware you control, rather than sending your requests to a provider's data centre. The model's weights sit on your device, the computation happens there, and your prompts and data never travel over the internet to a third party.

The term covers two setups. On-device AI runs directly on a personal machine - a laptop, smartphone or workstation - often for a single user. Self-hosted AI runs on a server or cluster you operate, serving a team, an application or a whole organisation. Both are 'local' in the sense that you, not a vendor, hold the infrastructure and the data.

Local AI vs. cloud AI: the differences

• Data location: Local - data stays on your hardware; Cloud - data is sent to and processed on the provider's servers.
• Model size and quality: Cloud gives access to the largest frontier models; local runs smaller open-weight models tuned to your hardware.
• Cost model: Local - one-off hardware plus electricity; Cloud - pay-per-use or subscription that scales with volume.
• Connectivity: Local works fully offline; cloud needs a reliable internet connection.
• Maintenance: Cloud is managed for you; local means you handle updates, security and scaling.
• Latency and limits: Local has no rate limits or per-token fees; cloud may throttle and meter usage.
• Control and compliance: Local gives full data sovereignty; cloud depends on the provider's terms, region and certifications.

When local AI makes sense

• Sensitive data: health, legal, financial or HR information that must not leave your premises under revDSG/nFADP or professional secrecy.
• Offline or edge settings: factories, clinics, field work or secure sites without reliable connectivity.
• High, predictable volume: constant workloads where per-token cloud fees would dominate costs.
• Data sovereignty requirements: public sector, regulated industries or contracts that mandate Swiss/EU data residency.
• Product features: embedding AI in an app or device that must work without a cloud dependency or recurring API bill.
• Experimentation and learning: running open models freely without usage caps.

Limits and trade-offs

• Model ceiling: the best local open-weight models are strong but generally trail the largest cloud systems on the hardest tasks.
• Hardware cost and complexity: capable models need a modern GPU or NPU and enough RAM/VRAM; setup and tuning take expertise.
• Maintenance burden: you own updates, security patches, monitoring and scaling.
• Slower iteration: you upgrade models manually instead of automatically getting the provider's newest version.
• Not automatically compliant: local reduces data-transfer risk but you still owe governance, access control and documentation.

What you need to run local AI

You don't need a data centre to start. Many capable open-weight models run on a recent laptop, and the tooling has become genuinely accessible.

• Hardware: a modern CPU works for small models; a GPU or NPU with sufficient VRAM unlocks larger, faster ones.
• Open-weight models: families such as Llama, Mistral, Gemma, Qwen or DeepSeek, and the Swiss Apertus model from ETH Zurich and EPFL.
• Runtimes: tools like Ollama, LM Studio, llama.cpp or vLLM that download, quantise and serve models locally.
• Quantisation: compressing a model so it fits on consumer hardware with a modest quality trade-off.
• Optional interface: a local chat UI or an API endpoint your own applications call.

The Swiss lens: data protection and sovereignty

For Swiss and DACH organisations, local AI is often a data-protection and sovereignty decision as much as a technical one. Under the revised Swiss Data Protection Act (revDSG/nFADP), overseen by the FDPIC (EDOEB), keeping personal data on your own infrastructure removes the cross-border transfer and third-party-processing questions that cloud AI raises.

The extraterritorial EU AI Act may also apply to Swiss firms serving the EU market. Running models locally does not exempt you from these rules, but it simplifies data governance and audit trails. Swiss-developed open models such as Apertus, built by ETH Zurich and EPFL, show that sovereign, transparent AI is a realistic option - see the provider's current documentation for capabilities and licensing.