What Are the Best Open Source LLMs - and How Do You Choose?

Open Source vs. Open Weight: the Key Distinction

Many models marketed as open source only release the trained weights, not the training data or code. The more precise term is open-weight: you can download, run and fine-tune them, but not fully reproduce them. Truly open, OSI-conformant releases with open data and code are rarer.

In practice, the licence matters most. It determines whether you may use a model commercially, redistribute it and use its outputs. Open weights are the key to data sovereignty and local operation - regardless of how fully open the rest of the model is.

The Most Notable Open Model Families

The overview below places well-known families in context. Capabilities and versions change quickly - always check the current generation and licence with the provider.

• Llama (Meta): large ecosystem, open-weight under a custom community licence with usage conditions - not classic open source.
• Mistral / Mixtral (Mistral AI, France): several models under Apache 2.0, a European provider, some using a Mixture-of-Experts architecture.
• Qwen (Alibaba): a broad range of sizes, many variants under Apache 2.0, generally strong multilingual coverage.
• DeepSeek: weights partly under the MIT licence, with variants geared toward reasoning.
• Gemma (Google): compact, efficient open-weight models under Google's own Gemma licence.
• Phi (Microsoft): small models under the MIT licence, often strong for their size.
• OLMo (Allen Institute for AI): fully open including training data, under Apache 2.0 - a reference point for genuine openness.
• Apertus (ETH Zurich, EPFL, CSCS): the fully open, multilingual Swiss model - more on this below.
• Others: Falcon (TII), Command (Cohere), GLM (Zhipu) and a steady stream of new models keep expanding the field.

Reading Licences Correctly

The term open source is used inconsistently. Before any production use, always check the specific licence text - especially for commercial deployment.

• Permissive OSI licences (Apache 2.0, MIT): broad commercial use, redistribution and adaptation are usually allowed.
• Custom community/open-weight licences (e.g. Llama, Gemma): use is allowed but with conditions and some usage restrictions.
• Research-only licences: no commercial use permitted.
• Always clarify: commercial use, redistribution, rights to the outputs, attribution and trademarks.
• Open weights do not mean open training data - the two must be assessed separately.

How to Choose the Right Model

Instead of hunting for the supposedly best model, start from the use case. The following criteria support a sound, neutral choice.

• Use case: chat, coding, extraction, summarisation or RAG each place different demands on a model.
• Language quality: test German, French, Italian and, if needed, Swiss German with your own examples.
• Model size and hardware: larger models need more VRAM; quantisation lowers the requirement.
• Licence and commercial terms: do they fit your business model?
• Operation: local/on-premise for data sovereignty, or managed hosting for less overhead.
• Adaptability: room for fine-tuning, prompting and tool integration.
• Governance and maintenance: update cadence, safety, documentation and community support.

Running Locally: Hardware, Tools and Data Protection

A core advantage of open weights is local operation. Small, quantised models already run on a well-equipped laptop; larger models need a GPU or a server.

• Tools: Ollama, llama.cpp, LM Studio, vLLM and the Hugging Face libraries simplify download and operation.
• Quantisation (e.g. 4-bit, GGUF format) reduces memory needs with usually modest quality loss.
• Data sovereignty: with local operation, data never leaves your infrastructure - a strong argument for protecting sensitive information.
• Compliance: on-premise or Swiss hosting eases revDSG compliance, since no unnecessary transfer abroad takes place.

The Swiss and European Perspective

Apertus is a fully open, multilingual language model from Switzerland, developed by ETH Zurich, EPFL and the CSCS supercomputing centre and trained on the Alps supercomputer. It stands for digital sovereignty and transparency - weights, training data and methodology are disclosed.

• EU AI Act: freely licensed, open-source GPAI models get some relief, but not where systemic risk applies; duties such as a training-data summary and respecting copyright remain. The act applies extraterritorially.
• revDSG and the EDOEB: open weights help keep personal data in Switzerland and document processing transparently.
• Vendor independence: open models reduce lock-in risk and increase control over roadmap and costs.