AI Security Checklist for CISOs

AI security is the discipline of protecting AI systems as assets, not just using AI to defend other assets. This checklist gives CISOs a six-layer map of the AI attack surface, the controls that belong to each layer, and how they cross-reference the OWASP Top 10 for LLM Applications, the NIST AI Risk Management Framework and ISO/IEC 42001, so a security team can turn a board mandate into a working programme.

A fill in the blank version of this checklist is available to download as a PDF at https://mickai.co.uk/downloads/ai-security-checklist-for-cisos.pdf.

What is AI security, and why is it a distinct discipline?

An AI system carries every attack surface of the application underneath it (APIs, infrastructure, identity, data stores) plus a new surface unique to the model itself: training data, embeddings, prompts, model weights, output handling and, increasingly, autonomous action. Traditional AppSec and DevSecOps controls do not disappear. SAST, WAF, RBAC and network segmentation stay necessary. They are simply no longer sufficient on their own, because none of them were designed to catch a prompt injection, a poisoned fine-tuning set or an agent that calls a tool it should never have been trusted with.

It also helps to separate two things CISOs commonly conflate. Securing AI means treating the AI system as an asset that can be attacked, stolen from or manipulated: that is the subject of this guide. AI-enabled attack means adversaries using AI as a tool against other targets, for example AI-generated phishing or deepfake social engineering. Both matter to a security programme, but they need different controls and different owners. This checklist is scoped to the former.

The AI attack surface: six layers to secure

The cleanest way to structure an AI security programme is by layer, because each layer has a different owner, a different threat model and a different set of controls. The six layers below map directly onto the OWASP Top 10 for LLM Applications and the NIST AI RMF core functions, so a control gap in one layer can be traced straight back to a named framework requirement rather than argued from first principles.

1. Data layer

Training data, fine-tuning sets and the retrieval corpora behind a RAG system are all attack surface. Risks include training-data poisoning, mislabelled or adversarial samples, undocumented lineage, and personal or sensitive data leaking into a training or retrieval set without proper classification or consent status.

• Data provenance logging and lineage tracking for every dataset feeding a model
• Anomaly detection on incoming training and fine-tuning data
• Held-out validation sets to catch drift or poisoning before it reaches production
• Strict access control on data pipelines and data lakes, separate from general IT access
• Data minimisation and, where the use case warrants it, differential privacy

2. Model layer

The model itself is now a protected asset. Attackers query inference endpoints to extract or reconstruct model behaviour, attempt model inversion to recover training data from outputs, craft adversarial inputs to force misclassification, or exploit backdoors introduced during training or fine-tuning. Model weights sitting as an unprotected file on a shared drive are as serious a finding as an exposed credential.

• Rate limiting and query-pattern monitoring on every inference endpoint
• Output confidence and anomaly monitoring to catch extraction attempts early
• Signed and versioned model artefacts, with access control on the model registry
• Red-teaming and adversarial robustness testing before any deployment, not just at launch
• Model watermarking or fingerprinting where theft risk is material

3. Pipeline and MLOps layer

This is the software supply chain that produces the model: third-party datasets, pretrained models, open-source libraries, plugins and vector databases, plus the CI/CD that moves code and configuration into training and deployment. It maps directly to OWASP's LLM03 Supply Chain risk. Container and cloud misconfiguration on training or inference clusters, and dependency or package integrity issues such as typosquatting on model hubs, both live here.

• A software bill of materials extended to cover models and datasets (an AI-BOM or ML-BOM), not just code libraries
• Provenance attestation for build artefacts, following the SLSA pattern
• Vetted sourcing for third-party models and datasets, with pinned and scanned dependencies
• Isolated training environments and signed pipeline artefacts

4. Prompt and interaction layer

This is where OWASP's top-ranked risk lives. Prompt injection, both direct (a user typing malicious instructions) and indirect (malicious instructions hidden in a document, web page or email the model later ingests), is LLM01 and remains the most exploited AI vulnerability in production systems today. Alongside it sit jailbreaking and system prompt leakage (LLM07), sensitive information disclosure in model responses (LLM02), and improper output handling, where a model's output is passed unsanitised into a downstream system such as SQL, a shell or a browser (LLM05).

• Input and output filtering and validation on every model-facing interface
• Strict separation of trusted instructions from untrusted retrieved or ingested content
• Output encoding before any downstream system consumes a model response
• Least privilege on what a model's output is ever allowed to trigger
• Human review gates for high-impact outputs, and a system prompt treated as sensitive but never relied on as the sole control

5. Supply chain layer

Supply chain touches data, model and pipeline, but it earns its own checklist column because the accountable party often sits outside the security team's direct control: the model provider, the plugin marketplace, the licence terms on a third-party dataset. A vendor's SOC 2 report says nothing about whether their own model evaluation covers prompt injection resistance or training-data provenance.

• Vendor risk questionnaires updated with AI-specific questions, not general cloud boilerplate
• Contractual right-to-audit and incident notification clauses specific to model behaviour changes
• Dependency scanning extended to model artefacts, not only application code
• Licence and provenance tracking for every external model, dataset and plugin in use

6. Agentic actions layer

This is the newest and fastest-growing category, and the one boards are asking about most. Excessive agency (LLM06) is an AI system granted more autonomy, tools or permissions than the task in front of it actually requires. Unbounded consumption (LLM10) is resource exhaustion or cost-based denial of service from an uncontrolled agent loop. The sharpest risk is tool and function-calling: an agent that can send email, execute code, move money or call an internal API based on manipulated input is no longer a text-generation risk, it is an operational one. Multi-agent systems compound this, since one compromised agent can influence others in a chain.

• Least-privilege scoping of every tool or function an agent is permitted to call
• Human-in-the-loop approval for irreversible or high-value actions
• Hard rate and cost limits, plus sandboxed execution environments
• Full action-level audit logging: who or what asked the agent to act, what it did, and what changed as a result
• Kill-switch and circuit-breaker patterns that can halt an agent's tool access without taking down the whole system

Where the organisation genuinely needs a tamper-evident record of what an autonomous action actually did, the audit trail should be cryptographically verifiable independently of the AI vendor, not just a log entry the vendor could edit.

Cross-cutting controls that do not belong to a single layer

A handful of controls sit above the six layers and apply to the whole programme.

• Identity and access management extended to AI agents and service accounts, which are a new class of identity and need their own lifecycle, not a shared credential borrowed from a human account
• Logging, monitoring and detection tuned to AI-specific indicators: prompt injection attempts, anomalous query volume, output drift over time
• Incident response playbooks updated for AI-specific incidents, such as a model behaviour change, data leakage through an output, or an agent taking an unauthorised action
• Red-teaming and continuous evaluation, run on a cadence, not as a one-off gate before launch
• Governance: a named owner for AI risk, a complete inventory of AI systems in use including shadow AI and unsanctioned tools, and a risk-tiering method so a low-stakes internal summariser is not held to the same control depth as a customer-facing agent with payment access

How does this map to OWASP, NIST, MITRE ATLAS and ISO?

No single framework covers the whole picture, and none of them are formally equivalent to one another. Each answers a different question, so a mature programme uses more than one.

The OWASP Top 10 for LLM Applications (2025 edition) is the practitioner-level, technical risk taxonomy: ten named vulnerability classes, LLM01 through LLM10, ranked by prevalence and impact. It tells a security engineer what to test for. LLM08 (Vector and Embedding Weaknesses) and LLM09 (Misinformation) sit outside the six-layer table above but are worth tracking directly if the system uses retrieval-augmented generation or produces content a user might rely on as fact.

The NIST AI Risk Management Framework (AI RMF 1.0), published January 2023, is the governance-level framework a CISO reports against. Its four core functions, Govern, Map, Measure and Manage, align to seven trustworthiness characteristics including secure and resilient, accountable and transparent, and privacy-enhanced. It is voluntary US guidance, not a certification, and it does not replace a technical control checklist. It tells a board what a mature AI risk process looks like.

MITRE ATLAS (Adversarial Threat Landscape for Artificial-Intelligence Systems) is a knowledge base of adversary tactics and techniques against AI systems, built on the same model as MITRE ATT&CK. It is the right reference for a red team wanting attacker-technique detail beyond OWASP's risk-category framing.

ISO/IEC 42001 is the emerging AI management system standard, the AI-governance equivalent of ISO 27001. It is genuinely certifiable, and genuinely still uncommon to hold, so treat it as a differentiator to work towards rather than a box already ticked across the market. ISO/IEC 27001 and SOC 2 remain table stakes: holding either says nothing specific about AI-layer controls, which is precisely why the AI-specific frameworks above exist alongside them. In the UK, the National Cyber Security Centre has published guidance on secure AI system development, produced jointly with CISA, and is a useful complement to the frameworks above for organisations working to UK government expectations.

Before you deploy: a six-question gate

A CISO can answer these in one sitting for any AI system going live. Any "no" is a finding, not necessarily a blocker, but it needs a named owner and a date.

• Do we have a current inventory entry for this system, including every model, dataset and plugin it depends on?
• Has the prompt and interaction layer been tested for injection, both direct and indirect, before go-live?
• Does every tool or function this system (or its agents) can call operate on least privilege, with human approval on irreversible or high-value actions?
• Is there an action-level audit trail we can verify independently of the vendor, for anything the system does autonomously?
• Has this use case been risk-tiered, and does its control depth match that tier rather than a default template?
• Does the incident response playbook cover an AI-specific scenario for this system by name, not just a generic "security incident" entry?

Who owns this? Governance and accountability

AI risk needs a named owner, not a committee that meets quarterly. In most organisations that owner sits in security or risk, reporting into the CISO, with a dotted line to whoever owns the data platform and whoever owns the product using the model. The owner's job is the inventory, the risk tiering, the deployment gate above, and making sure every new AI system enters through the same door rather than being adopted informally by a business unit that never told security it existed. Shadow AI, meaning tools adopted without security's knowledge, is consistently the largest gap between a paper policy and what is actually running in production.

Data residency and vendor lock-in questions, which sit under NIST's Govern function, come up early in this conversation. Build, buy, rent, on-prem and cloud-hosted deployment are all legitimate answers, and each carries real trade-offs in cost, operational burden, patch velocity and access to a vendor's own security research. On-prem or self-hosted deployment reduces, though it does not eliminate, exposure to third-party model-provider supply chain risk, which is a fair generic point regardless of which vendor an organisation eventually chooses. Where an organisation genuinely needs a per-action audit trail it can verify without trusting the AI vendor, an owned, offline-capable platform with a cryptographically verifiable action log, of which Mickai's SIOS is one example, is one credible answer among several, not the only one.

How an offline verifiable audit chain closes the identity and logging layers above is set out at /sovereign-ai, and the film at /film shows the interface in operation.

Frequently asked questions

What is the AI attack surface, in plain terms?

It is every point an attacker could reach to manipulate, steal from or misuse an AI system: the training data, the model and its weights, the pipeline that builds and deploys it, the prompts and outputs it exchanges with users, its supply chain of third-party models and plugins, and any autonomous action it takes. It sits on top of, not instead of, the ordinary application attack surface.

Is the OWASP Top 10 for LLM Applications the same as the OWASP Top 10 for web applications?

No. They are separate lists maintained by separate OWASP working groups. The web application Top 10 covers classic risks such as injection and broken access control. The LLM Top 10 covers risks unique to language model applications, such as prompt injection and excessive agency, which do not map cleanly onto the older list.

What does the NIST AI Risk Management Framework actually require organisations to do?

Nothing by law. It is voluntary guidance, not a certification or a regulation. It structures an AI risk programme around four functions, Govern, Map, Measure and Manage, and gives organisations a common vocabulary and process to report against, which is why many CISOs use it as the governance layer that sits above technical checklists like this one.

Is the EU AI Act's high-risk deadline still 2 August 2026?

No. That was the original date, but the EU's Digital Omnibus on AI deferred it. Stand-alone high-risk obligations under Annex III are now due 2 December 2027, and high-risk AI embedded in regulated products under Annex I is due 2 August 2028. Article 50 transparency obligations largely stayed on the original schedule. Treat the deferral as a build window, not a reprieve, since the underlying proof requirements survive the move intact.

What is the single most exploited AI vulnerability today?

Prompt injection, ranked LLM01 in the OWASP Top 10 for LLM Applications 2025 edition. It covers both direct injection, where a user types malicious instructions, and indirect injection, where those instructions are hidden inside a document, web page or email the model later reads. It is difficult to fully close because the same channel that carries legitimate instructions also carries untrusted content.

How does agentic AI change the attack surface compared with a simple chatbot?

A chatbot's worst-case failure is a bad answer. An agent that can call tools, execute code or move data can take a real-world action based on manipulated input, which turns a text-generation risk into an operational one. That is why excessive agency and unbounded consumption are treated as their own OWASP risk categories, and why least-privilege tool scoping and human approval on high-value actions matter more here than anywhere else in the stack.

Do ISO 27001 or SOC 2 already cover AI security?

Not specifically. Both are table-stakes general security certifications and a reasonable baseline, but neither was written with model-specific risks such as prompt injection, training-data poisoning or excessive agency in mind. That gap is exactly why AI-specific frameworks such as the OWASP LLM Top 10, the NIST AI RMF and ISO/IEC 42001 exist alongside them, not instead of them.