Local and Open-Source AI: What Does Data Sovereignty Mean for Swiss Businesses?

What do local and open-source AI mean?

Two axes are often confused. The first is where it runs: is an AI model used through a provider's API in that provider's cloud, or does it run locally – on a server in your own data centre, on a workstation, or in a private cloud instance you control? The second axis is how open the model is: are the weights publicly downloadable, or do they stay closed behind a service?

Local open-source AI combines both: a model with openly available weights that runs within your own zone of control. Processed data never leaves the controlled environment – the core difference from a pure cloud service.

Why data sovereignty matters for Swiss businesses

Data sovereignty means always knowing and deciding where data is processed, who can access it and under which rules. With closed cloud services, part of that control sits with the provider and often falls under foreign law. Running locally shifts control back to the company.

• revFADP: Switzerland's revised Data Protection Act requires transparency, purpose limitation and documented processing of personal data; the FDPIC (EDOEB) oversees compliance.
• Professional and official secrecy: lawyers, doctors, banks and fiduciaries are bound by special confidentiality duties (e.g. Art. 321 of the Criminal Code, banking secrecy) – leaking data to third parties can be a criminal offence.
• EU AI Act: the EU regulation applies extraterritorially and can affect Swiss firms serving the EU market; it adds risk-based obligations on top of data-protection law.
• Data residency and processing agreements: cloud services require contracts, adequate safeguards for cross-border transfers and clarity on sub-processors – local operation largely removes those transfers.

Local, cloud or hybrid – the comparison

There is no universally correct choice. Cloud AI wins on time-to-start and access to the strongest frontier models; local AI wins on data sovereignty, predictable cost and independence. Many Swiss companies run hybrid: sensitive cases locally, non-critical tasks in the cloud.

• Data sovereignty: highest locally (data stays internal), cloud depends on contract and location, hybrid steerable by data class.
• Cost profile: cloud is variable per usage, local needs an upfront investment with predictable running costs afterwards – economics depend on volume.
• Capability: the strongest closed frontier models often lead on complex tasks; strong open models are already sufficient for many domain tasks.
• Effort: cloud minimises operational overhead, local requires competence for hardware, operations and updates.

Open source vs. open weight – the licensing difference

The term open source is often used loosely for AI. Fully open means a model whose weights, training code and ideally training data are under a free licence. More common are open-weight models: the weights are downloadable, but training data or code stay closed, and the licence may carry restrictions – such as usage limits or conditions for very large providers.

• Permissive licences (e.g. Apache 2.0, MIT): broad commercial use, few conditions – legally the simplest for businesses.
• Community/custom licences: usable commercially but with conditions – read the licence text before a model goes into production.
• Full openness increases auditability and traceability – relevant for regulated sectors and the EU AI Act's documentation duties.

Models and tools for running AI locally

The open-model ecosystem is large and moves fast. Well-known open-weight families include Llama (Meta), Mistral and Mixtral (Mistral AI), Qwen (Alibaba), Gemma (Google) and DeepSeek. From Switzerland comes Apertus, an openly developed multilingual language model from ETH Zurich, EPFL and the CSCS supercomputing centre – an example of sovereign, transparently built AI.

Mature open-source tools handle operation. For getting started and single seats, Ollama, LM Studio or GPT4All fit well; for production, scalable services, llama.cpp and vLLM are used. Models are usually obtained via Hugging Face; quantization reduces memory needs so smaller hardware suffices.

Assessing hardware, cost and effort realistically

Resource needs scale with model size and desired speed. Smaller models already run on a well-equipped workstation; larger ones need dedicated GPUs with plenty of video memory (VRAM). The use case decides: an assistant for a handful of specialists has very different requirements than a company-wide service.

For economics, take a total-cost view over several years: hardware or private cloud, power, operations, maintenance and staff versus a cloud service's ongoing usage fees. Concrete prices are volatile – check the provider's current figures and model against your own volume.

How to decide – and where to go next

Start with the data, not the technology: classify use cases by sensitivity and regulatory load. Cases involving personal data, professional secrecy or trade secrets favour local operation; non-critical, public tasks can run cheaply in the cloud. Start small with a clearly scoped pilot and a measurable expectation.

• Spoke: self-hosting in practice – Ollama, llama.cpp and vLLM compared for production use.
• Spoke: choosing open models – licences, sizes and language capability for German, French and Italian.
• Spoke: revFADP-compliant AI architecture – RAG over internal documents without data leaving.