What is AI security?

AI security in brief

AI security means protecting AI systems – and the data they process – from misuse, manipulation and failure. It combines classic information security (confidentiality, integrity, availability) with new, AI-specific threats that arise from large language models and autonomous agents.

It is worth distinguishing security of AI (hardening the system against attacks) from AI for security (using AI to defend against cyberattacks). This article focuses on the first perspective: how organisations run generative AI responsibly and robustly.

The biggest risks of generative AI

• Data leakage: confidential inputs, trade secrets or personal data escape via prompts, training data or log files.
• Prompt injection: manipulated inputs or hidden instructions in documents and web pages make the model ignore safety rules or perform unwanted actions.
• Model misuse: attackers use AI for phishing, malicious code or disinformation, or bypass safety filters through jailbreaks.
• Hallucinations: the model produces convincing but false statements – a risk for decisions made without review.
• Shadow AI: employees use unauthorised tools and feed sensitive data into uncontrolled services.
• Supply-chain and model risks: compromised models, plug-ins or training data (data poisoning) introduce vulnerabilities.

Safeguards: technical and organisational

Effective AI security is layered; no single measure is enough. Technical controls and organisational rules work together (defence in depth).

• Data minimisation and classification: share only necessary data, block sensitive categories technically.
• Access and permission model (least privilege, zero-trust) for models, data and agent tools.
• Input and output filters (guardrails) against injection, data leakage and toxic content.
• Retrieval-augmented generation with vetted sources instead of blind model knowledge, plus citations.
• Logging, monitoring and anomaly detection; regular red teaming and penetration testing.
• Contractual assurances on data processing, location and no-training-on-customer-data; EU/Switzerland hosting where needed.

Human oversight as the final layer

Human oversight is the final and most important layer of security. The EU AI Act explicitly requires it for high-risk systems, and it is good practice even without an EU link. The key is that people can understand, question and override AI outputs.

• Human-in-the-loop: critical actions (payments, legal advice, medical guidance) are approved before execution.
• Clear accountability: who reviews, who is liable, who may shut it down (kill switch).
• Training: employees recognise hallucinations, injection attempts and data-protection limits.
• Transparency towards individuals when AI interacts with them or decides about them.

Legal framework and standards: Switzerland and EU

Switzerland has no dedicated AI law (yet), but existing rules apply: the revised Federal Act on Data Protection (revised FADP) requires data security, transparency and, where relevant, a data protection impact assessment. The FDPIC monitors compliance. Organisations serving people in the EU are additionally subject to the extraterritorial EU AI Act, which sets risk-based obligations up to outright bans.

• EU AI Act: risk-based, with duties on robustness, transparency and human oversight.
• Revised FADP and FDPIC guidance: data security and data-subject rights in Switzerland.
• ISO/IEC 42001: a management system for responsible AI.
• NIST AI Risk Management Framework: a voluntary, widely recognised governance framework.
• OWASP Top 10 for LLM applications: a concrete list of attacks and defences for developers.

Introducing AI security: a pragmatic roadmap

AI security can be built step by step – even in SMEs without a large security team.

• Build an inventory: which AI tools are in use, with what data?
• Adopt an AI policy: permitted tools, prohibited data types, reporting channels.
• Define data classes and keep sensitive data out of public services.
• Vet providers: location, certifications, training opt-out, contractual terms.
• Start a pilot with guardrails and logging, then roll out in a controlled way.
• Test, train and adapt to new threats on a regular basis.