How To Write An AI Requirements Specification

The difference between an AI project that ends in a working system and one that ends in a well-received demonstration is usually decided in the specification, before anybody writes code. Specifications that describe a capability produce demonstrations. Specifications that describe a decision produce systems.

Capability language and decision language

A capability specification says the system will analyse contracts and identify risks. It sounds complete and it is almost impossible to fail against, which is exactly the problem. Nobody can say afterwards whether it worked.

A decision specification says something narrower and far more useful. It names who currently makes the decision, what they look at, how long it takes them, how often they are wrong, and what an acceptable answer looks like. It then states what the system must produce for that person to act on it without repeating their work.

What belongs in it

• The decision itself, in one sentence, phrased as the question a person currently answers.
• The inputs that person actually consults, including the ones not in any system, because those are the ones that will be missing.
• The current baseline: time per case, volume per week, and the current error rate if anyone has measured it. If nobody has, say so rather than guessing.
• What the output must contain for the decision to be made from it, rather than checked against the source again.
• The threshold for acting without a human, and the explicit statement that it is not crossed at launch.
• What must never happen, which is usually more precise and more useful than what should.

The part most specifications omit

Almost no specification says how the result will be measured, which means the measurement gets designed after the outcome is known. That is not measurement, it is justification.

Agreeing the instrument before the build costs an afternoon and it is the difference between a result you can defend to a board and a number the supplier produced. It also protects the supplier, because it removes the moving target.

“If the specification cannot be failed, it cannot be passed either.”

Where sovereignty enters

For a regulated operation there is one more clause, and it belongs in the specification rather than in the security review at the end: where the data may sit while the system is working on it. Discovering that constraint after the architecture is chosen is the most common way these projects lose a quarter.

Written properly, a specification is a short document. Most of the length in a bad one comes from describing features nobody asked for, because the decision was never pinned down.