What CIOs Should Ask Before Buying AI

Ask five questions, and insist on an architectural answer to every one: where is our data processed, who owns the outputs, can it run offline, who controls when the model changes, and what evidence does it leave behind. The answers that should worry you are not the hostile ones. They are the fluent ones that substitute a policy commitment for an architectural fact.

A policy is a promise about behaviour; an architecture is a statement about what is possible. A policy survives until the vendor is acquired, rewrites its terms, or receives a lawful order in a jurisdiction you do not sit in. An architecture survives all three. So when you ask about possibility, refuse an answer about intention.

Where is our data processed, and who can compel access to it?

The evasive answer reassures you that data is encrypted in transit and at rest, and that the vendor does not train on customer data. Both are usually true and neither answers the question. Encryption at rest protects against a stolen disk, not against the party that holds the keys and reads plaintext at inference.

Press hardest on the second half. Name every location where our plaintext exists during inference, including subprocessors and abuse-monitoring retention. Then name every legal entity that could be compelled to produce it, and who owns that entity. That is a corporate structure question, and it decides which courts can reach your data. The best answer removes the question: plaintext never leaves hardware you own.

Who owns the outputs, and what happens to them if we leave?

The evasive answer confirms that you own your data. Nobody asked about input data. The valuable material is what the system derives: the indices built from your corpus, the adapted weights, and the record of what agents did on your behalf. Many terms grant ownership of inputs while licensing everything derived from them, and silence on the point gets resolved later by whoever has more lawyers.

Three sub-questions do the work. Do we own the derived artefacts, or licence them? On termination, do we get them in a defined format within a defined window? And can we run what comes back somewhere else? Portability that returns a file no other system can load is a receipt.

Can it operate with the network cable pulled?

The evasive answer claims private deployment, or air-gap capability, with no detail behind it. Many private deployments still call out for licence checks, telemetry, model updates or content filtering, and the vendor has never measured what happens when those calls fail.

This is the sharpest test of dependency available, and dependency is what nobody writes into the business case. A system that runs indefinitely offline has no hidden leash. One that degrades within the hour has shown you where the leash is attached.

Who decides when the model changes?

The evasive answer promises continuous improvement, so that you always have the latest version. Read plainly, that says the system you validated is not the system running when the decision goes wrong. A silent upgrade changes a control you assured a regulator about, from outside your governance. Four things must be true, contractual as much as technical.

• We can pin a version, supported for a stated number of months, not until further notice.
• We are told before a change, and can test our workload against it first.
• We can refuse an update and remain refused, without losing support.
• If behaviour changes on our workload, there is a rollback path and a clear answer on who pays to revalidate.

This does not require a supplier who never updates: freezing a model forever is how you inherit a vulnerability. It requires that you decide when, and that either party can prove what ran on any date.

What evidence does it produce, and who can verify it?

The evasive answer offers full audit logs and comprehensive dashboards. Look at what that describes: records written by the system being audited, held by the vendor being questioned, in a format only their console reads. That is telemetry: fine for operations. Evidence is for a tribunal, and it must survive a hostile reader.

Three tests separate the two. Is the record written before the action executes, or after? A record written afterwards is a record of what the system chose to admit. Is its integrity a property of the record itself, through signing and hash chaining, or does it rest on trusting the custodian? And can a third party verify it without the vendor's software or cooperation? If not, your evidence is only as available as your commercial relationship.

When is this checklist the wrong checklist?

Often, and most frameworks refuse to say so. Applied to everything, these questions are theatre. For a drafting assistant over marketing copy, a reputable supplier's policy commitments are adequate. Cloud is the right answer for a great many workloads, and a CIO who treats every purchase as a sovereignty crisis gets routed around by their own business.

So classify first. Run the five questions hard on the decisions you would have to explain to a regulator, a court or a select committee, and on data that would trigger a notifiable event abroad. Let the rest move quickly.

The honest counter to our own argument: architectural claims are still claims, and a vendor can describe an architecture as loosely as a policy. That is why each question here ends in a demonstration rather than a document. Pull the cable. Export the artefacts. Verify the record with the vendor out of the room.

Frequently asked questions

What is the single most useful question to ask an AI vendor?

Pull the network cable and show us what still works. It is the only question here that cannot be answered with words. It exposes every hidden dependency at once: licensing call-homes, remote filtering, telemetry gates, model fetches. It also shows how long the system lasts without its supplier.

Is a commitment not to train on our data a sufficient assurance?

No. It is a promise about a future decision, and it answers a narrower question than the one you should ask. It says nothing about where plaintext exists during inference, which subprocessors touch it, or which entity could be compelled to produce it. Ask for the locations and the entities.

Should every AI purchase go through this process?

No, and insisting otherwise will cost you the argument on the purchases that matter. Sort the estate first: workloads whose failures you would defend to a regulator, a court or the board get the full five questions, and the rest get a fast commercial review. A uniform process gets bypassed.

What does a good audit answer actually look like?

A record sealed before the action executes, signed with a post-quantum signature, hash-chained so any edit breaks the chain, and verifiable by a third party offline without the vendor's software or consent. Anything less is a log file whose integrity depends on the goodwill of the party being audited. Ask to verify one with the vendor's tooling switched off.

Mickai is a British Sovereign Intelligence Operating System, built and live today. It runs offline on hardware the organisation owns, in its own jurisdiction, with every consequential action sealed in the Open Audit Record before it executes, signed with FIPS 204 ML-DSA-65 and hash-chained so a regulator or a court can verify it offline, without us. The architecture is protected by 104 filed UK patent applications carrying 2,340 claims, held by Mickai LTD. If these questions are going into a procurement pack, start at /sovereign-ai, read /oar, and use /ai-readiness to sort the workloads that deserve them.