Your Vendor SOC 2 Says Nothing About the Model

The report on the wall proves the wrong thing

I have lost count of the procurement decks that put a vendor's System and Organization Controls (SOC) 2 badge on the second slide as if it settles the question. It does not settle the question. It does not even ask the question. A SOC 2 Type II report is an attestation, by an independent auditor, that a service organisation operated a defined set of controls over a defined period. Access was provisioned and revoked. Backups ran. Changes went through review. Encryption was enabled. All of that is genuine and worth having. None of it tells you a single thing about what the artificial intelligence (AI) model inside that environment actually decided, on what basis, or whether it will decide the same thing tomorrow.

This is the attestation gap, and it is widening fast. We have spent the better part of two decades building a procurement reflex around infrastructure assurance, and we are now pointing that reflex at systems whose risk does not live in the infrastructure. It lives in the model. The badge tells you the perimeter held. It is mute on the thing inside the perimeter that you are actually paying for, the part that reasons, generates, and decides.

Controls attest to the building, not the tenant

Think about what a SOC 2 examination scopes. It looks at the Trust Services Criteria: security, availability, processing integrity, confidentiality, privacy. The auditor samples evidence that controls were designed appropriately and operated effectively. The unit of analysis is the organisation and its processes. The model is, at most, an asset sitting inside that scope, a file on a disk, a service behind an endpoint. The examination was never built to interrogate what that asset does when it runs.

Processing integrity is the criterion people point to when I press them, and it is the most misleading. Processing integrity asks whether a system processes data completely, accurately, and in a timely way, to meet the entity's objectives. For a payroll engine that is a meaningful test, because correct is definable. For a generative model, correct is not a fixed target. The same prompt can yield different outputs. A model can be perfectly available, perfectly confidential, fully encrypted, change-managed to the letter, and still fabricate a citation, leak a pattern from its training data, or quietly drift after a fine-tune. Every control passed. The decision was still wrong, and nothing in the report would tell you. A security realist learns to ask what a certificate is silent about, not what it shouts. SOC 2 shouts about the perimeter. It is silent about cognition.

Why the gap is dangerous now and not just untidy

For most of the cloud era the gap was tolerable, because the thing inside the building was deterministic software you could reason about. AI broke that. The behaviour you are buying is now emergent, probabilistic, and version-sensitive, and it changes underneath you. A vendor can swap the underlying foundation model, adjust a system prompt, re-weight a retrieval index, or roll a silent fine-tune, and their SOC 2 report does not move a millimetre, because none of those are infrastructure events. The controls are unchanged. The product is not. You re-procured nothing, signed nothing, and yet the system that decides things about your customers is no longer the system you assessed.

The regulatory floor is rising to meet exactly this point. From August 2026 the European Union (EU) AI Act brings its high-risk obligations into force, and those obligations are about the system's behaviour: risk management, data governance, logging that enables traceability, human oversight, and transparency about how outputs are produced. Liability regimes across jurisdictions are tilting the same way, toward holding deployers accountable for what their AI did, not merely for whether their vendor kept the servers patched. When a regulator or a claimant asks you to show, after the fact, why your system produced a particular decision for a particular person, a SOC 2 letter is not an answer. It was never built to be one, and no amount of pointing at it will make it become one.

What procurement should actually demand

Keep the SOC 2. It is necessary. It is just radically insufficient on its own. Alongside it, procurement should require evidence about the model layer, and should treat the absence of that evidence as a finding, not a footnote. The right posture is simple to state and uncomfortable to enforce: if a vendor cannot produce a durable, checkable account of what their model did, then for assurance purposes the model is a black box you are agreeing to be liable for. In practice I would not sign without the following.

• Model provenance. Which foundation model, which version, what fine-tuning was applied, and on what data. If the vendor cannot name what is running, they cannot govern it, and neither can you.
• Change attestation for the model, not just the infrastructure. A signed, dated record every time the model, the weights, the system prompt, or the retrieval corpus changes, so a silent swap is not silent.
• Decision-level logging. A durable, tamper-evident record of what the system was asked, what context it used, and what it returned, sufficient to reconstruct a specific output months later.
• Independent verifiability. The ability for you, or a regulator, to check those records without taking the vendor's word for it and without the vendor being able to rewrite history after the fact.
• Data residency and sovereignty terms that survive the vendor changing their own subprocessors.

Notice the shift. Every one of those demands is about behaviour over time and about who has to be trusted. That is the real test of an AI assurance story. When the record, the auditor, and the vendor are the same party, you do not have assurance, you have a brand promise with a logo on it. Real assurance is the property that you can check the claim yourself, and that the answer does not depend on the vendor's goodwill, solvency, or continued cooperation.

The record has to be signed before the act, and verifiable by a stranger

This is the gap we set out to close when we built Mickai, our Sovereign Intelligence Operating System (SIOS), which is built and running in production. The mechanism is the Open Audit Record (OAR). Every action an AI takes inside the system is signed before it executes, not logged afterward when it is convenient and editable. Each record is hash-chained to the one before it and appended to a ledger that cannot be reordered or quietly amended. The signatures are post-quantum, using the United States National Institute of Standards and Technology (NIST) FIPS 204 standard (ML-DSA-65), because a record meant to hold up for years has to survive the cryptography of those years. Signing before the act is the part that matters most, because a log written after a decision is a story the system tells about itself, and a signature committed before the decision is a fact the system cannot later edit.

The property that closes the attestation gap is the last one. The record is verifiable offline, in an ordinary web browser, with no trust placed in us. You do not call an application programming interface (API) we control. You do not accept a dashboard we render. You check the chain yourself, and the mathematics either holds or it does not. That is the inversion procurement actually needs. Instead of trusting that the building had locks, you can prove what the tenant did, on your own machine, after the fact, even if we have gone out of business or turned hostile. Mickai runs fifty specialised models, twenty-five domain and twenty-five operational, on the Poseidon silicon substrate, and the audit root is anchored through Pantheon, a sovereign Layer 1 network, down to Bitcoin, so the history has a settlement point no single vendor owns. We are training our own models now, fine-tuning and specialising open foundations such as Llama 3.2 and Qwen 2.5 into a sealed corpus, and the same record covers them. This is not theory. It is the architecture, and it sits behind a portfolio of 101 filed United Kingdom patent applications, around 2,234 claims, owned by Mickai LTD.

None of that replaces a SOC 2. It answers the question a SOC 2 was never asked: not whether the controls around the model were sound, but whether you can prove, to a stranger, what the model itself actually did. Until your vendor can hand you a record like that, the badge on slide two is telling you the doors were locked. It is saying nothing about the model. Start demanding the thing that does.