What hardware do you need to run enterprise AI on-premise?

Enterprise AI on-premise runs in three classes of hardware. A single workstation-class machine with a capable GPU serves a small team. A multi-GPU server serves a department. A small cluster serves an organisation-wide deployment. The binding constraint in every class is memory rather than raw compute, and the design decision that matters most is keeping inference selectable between CPU and GPU.

The question matters in 2026 because regulated organisations are moving inference inside their own perimeter, and sizing errors are expensive in both directions: hardware that cannot hold the model, or accelerators bought for workloads a modern CPU could have carried.

What can a single workstation-class machine run?

More than most buyers expect. One machine with a capable GPU runs current-generation sovereign models for drafting, extraction, summarisation and retrieval at production quality, serving a small team with interactive response times. This is the honest starting point for a pilot: one machine, one team, one workflow, with the audit ledger sealed locally from day one. Mickai, a Sovereign Intelligence Operating System, is designed to run whole on exactly this class of hardware, offline, on machines the operator owns.

When do you need a multi-GPU server or a cluster?

When concurrency, model size or availability demands it. A department asking questions all day needs a multi-GPU server so that inference queues stay short and larger models stay resident in memory. An organisation-wide deployment adds a small cluster for redundancy and load, and for separation of duties: one node can serve inference while another verifies and archives the ledger. The progression is driven by simultaneous users at peak and by the size of the largest model kept hot, not by prestige.

Why should inference be selectable between CPU and GPU?

Because not every workload needs an accelerator, and not every site can host one. Drafting, extraction and retrieval run acceptably on modern server CPUs for many team sizes, at lower power and with simpler procurement. GPUs earn their place on large models, long contexts and high concurrency. A deployment locked to GPU-only inference fails at exactly the sites that most need sovereignty: branch offices, vessels, secure rooms and facilities where an accelerator cannot be installed or cooled. We build inference selectable between CPU and GPU as a matter of architecture, so the workload decides, not the hardware order book.

Why is memory the real constraint?

Because a model must fit where it runs: in VRAM for GPU inference, in RAM for CPU inference. Model weights consume memory in proportion to parameter count and numerical precision, and quantisation shrinks the footprint at some cost in fidelity. Context adds more: long documents and long conversations consume memory beyond the weights themselves. The sizing question is therefore not how fast the chip is, but whether the largest model you intend to run, at the precision you intend to run it, with the context you intend to give it, fits in memory with room to spare. Compute determines how long you wait. Memory determines whether it runs at all.

How much storage do models and the audit ledger need?

Two budgets with different shapes. Model storage is large but static: a library of sovereign models in several sizes, retained across versions, because deterministic versioning means old versions are kept rather than overwritten, so past decisions can be rerun exactly as they first ran. The audit ledger is the opposite: modest in size, append-only and permanent, growing for the life of the deployment and retained for as long as regulation demands. Fast local storage serves the models. Replicated, integrity-checked storage serves the ledger, because the ledger is the one artefact the deployment cannot lose.

How should the network be designed?

Around a zero-egress perimeter. The inference segment has no outbound routes: no telemetry, no update callbacks, nothing that phones home. Updates, models and curated corpora arrive through a controlled inbound path as signed artefacts, so material can enter without any route existing for data to leave. Identity is anchored in hardware: operator actions are bound to the audit chain through hardware-attested identity, so the record shows not only what happened but on which machine and under whose credentials. Egress rules should be enforced physically and verified independently, because a policy is a promise and an absent route is a fact.

What should a site survey check before you buy anything?

• Power and cooling: whether the room can feed and cool the class of machine intended, since accelerators are dense loads.
• Physical security: who can touch the machine, because hardware-attested identity assumes the hardware itself is controlled.
• Placement: which jurisdiction the machine sits in, and which data is permitted to sit with it.
• Concurrency: how many simultaneous users at peak, measured rather than guessed.
• Growth: whether the chassis and the network can take a second GPU or a second node without redesign.

“Buy for the memory the model needs and the evidence you must keep, and the compute will follow.”

How the whole system fits together is set out at /sovereign-ai, and the film at /film shows the interface in operation.

Frequently asked questions

Can I run enterprise AI without any GPU at all?

Yes, for the right workloads. Drafting, extraction, classification and retrieval run acceptably on modern server CPUs for small and medium teams. GPU hardware becomes necessary as models grow, contexts lengthen and concurrent users multiply, which is why inference should be selectable between CPU and GPU rather than fixed at design time.

What happens when we outgrow our first machine?

The deployment scales by class: workstation to multi-GPU server to small cluster. With deterministic versioning, the models and the sealed ledger move with the deployment, so history survives the migration. The common path is a single-machine pilot that proves one workflow, followed by department hardware sized on measured concurrency rather than estimates.

Do I need special hardware for the audit ledger?

Not exotic hardware, but dedicated treatment. The ledger is append-only, sealed with post-quantum signatures, and must survive for the retention period regulation requires, so it belongs on replicated storage with its integrity verified on a schedule. Its size is modest next to model storage; its importance is the reverse.

How do we apply security updates on a zero-egress network?

Through the inbound path. Updates arrive as signed, versioned artefacts through a controlled import point, are verified before installation, and leave a sealed record of what changed and when. Zero egress restricts what leaves the perimeter, not what may be deliberately and verifiably brought in.