How Much GPU and Server Hardware Do I Need for 500 Staff?

For 500 staff running a sovereign AI assistant on your own hardware, plan on one inference server carrying eight data-centre GPUs for mixed daily use, plus a second identical node for high availability and burst, so roughly two servers and eight to sixteen accelerators. The reason is concurrency: your 500 people never all query at once, so peak demand is set by simultaneous token generation, not by headcount. At a 10% peak-active fraction and a 20% generation duty cycle, 500 staff produce about ten concurrent generating streams, which a single modern GPU node absorbs comfortably. Light chat-only use fits a single four-GPU node, while heavy agentic automation pushes to three or four nodes.

This question matters in 2026 because the market answer has been a hand-waved "it depends", and vagueness costs money both ways. Buyers either over-buy a rack of accelerators to feel safe, or under-provision and watch response times collapse at 10am. A sizing method that maps headcount to a defensible box count lets procurement sign off with a figure it can defend, while keeping the whole workload on operator-owned hardware rather than a metered public endpoint.

How do you turn 500 staff into a number of servers?

Size to peak simultaneous token generation, not to the payroll. Four named quantities drive the calculation, each a planning default you can adjust with real telemetry once live.

• Peak-active fraction: the share of staff with a live session at the busiest minute. For general knowledge workers, 10% is a safe planning figure.
• Generation duty cycle: the share of a session where the model is actually emitting tokens. Humans spend most of a session reading and typing, so 20% is realistic.
• Streams per GPU: concurrent interactive streams a single accelerator sustains for a mid-size sovereign model with batching, roughly 10 to 20 at a comfortable reading speed.
• GPUs per server: accelerators per node, typically four or eight.

The formula is simple. Peak sessions equal headcount multiplied by the peak-active fraction. Active generating streams equal peak sessions multiplied by the generation duty cycle. GPUs needed equal active streams divided by streams per GPU, rounded up. Servers needed equal GPUs divided by GPUs per node, then add one node for redundancy.

What is the worked example for 500 staff?

Run 500 through the method. Peak sessions come to 50. Active generating streams come to about 10. Dividing by 15 streams per GPU gives one accelerator for the core assistant, which we round to two for headroom. Real deployments also carry document analysis, retrieval and coding, so the figure lands higher. We size by workload class:

• Light (chat, drafting, search): two to four GPUs, a single node.
• Mixed (chat plus document analysis, retrieval, some coding): four to eight GPUs, one eight-GPU node.
• Heavy (large-context coding, always-on background agents, vision): eight to sixteen or more GPUs, three to four nodes.

For most 500-person organisations the mixed profile is correct: one eight-GPU inference node as the primary, and a second identical node for high availability and burst.

“Headcount buys the licence to use AI, but simultaneous token generation buys the silicon, and the two numbers differ by more than an order of magnitude.”

Why does sovereign, on-premises deployment change the maths?

Public cloud AI services hide sizing behind a per-token bill, so you never see the box count. Owning the hardware makes concurrency visible and gives you a fixed ceiling instead of a variable meter. Mickai is a Sovereign Intelligence Operating System, a SIOS, that runs offline on operator-owned hardware behind a zero-egress inbound perimeter, so every stream is served locally and every action is cryptographically sealed. That architecture removes the cloud escape valve, so the sizing must be right, not roughly right.

How much headroom and redundancy should you buy?

Buy for the peak minute, then add one node. Two failure modes justify the extra hardware. The first is availability: if a single node dies, staff must still work, so an N plus one posture means one server can go down without an outage. The second is burst: real demand is spiky around meeting boundaries and deadlines, and a warm second node absorbs those spikes at interactive speed. Size the primary for the mixed daily load, keep the secondary hot, and treat sustained utilisation above 70% as the trigger to plan the next node.

What can procurement and auditors actually check?

Every quantity above is measurable, not asserted. Peak-active fraction and duty cycle are readable from session logs within the first fortnight. Streams per GPU is a benchmark you can run on the actual accelerator with your model, and node utilisation is a live dashboard. On the assurance side, an auditor can verify that no traffic leaves the perimeter, that identity is hardware-attested and bound to the audit chain, and that the audit ledger is signed under the post-quantum standards: FIPS 204 (ML-DSA) as the primary signature scheme, with FIPS 205 (SLH-DSA) as a hash-based alternative. Note that FIPS 203 (ML-KEM) is key encapsulation and never signs anything. The underlying design is covered by 104 filed UK patent applications with approximately 2,340 claims, owned by Mickai LTD and patent pending.

Which rules make owning the hardware necessary?

For many regulated buyers the box count is not the hard part, the location is. DORA, in force since 17 January 2025, holds financial entities accountable for operational resilience and third-party risk, and NIS2 extends security obligations across essential and important entities. GDPR governs personal data wherever it is processed, and the US CLOUD Act means data on a US-linked provider can be reached by US legal process. ISO/IEC 42001 sets the management-system baseline for AI. On the EU AI Act, the high-risk Annex III obligations once due on 2 August 2026 were deferred by the Digital Omnibus to 2 December 2027, with embedded high-risk under Annex I moving to 2 August 2028 and the Article 50 transparency duties largely unchanged. Owning the hardware is the cleanest way to address all of these at once.

Frequently asked questions

How many GPUs are needed per user?

You do not size per user, because staff do not query simultaneously. Size for peak concurrent token generation. For 500 staff at a 10% peak-active fraction and a 20% generation duty cycle, that is about ten concurrent streams, which one modern GPU node handles. Per-user thinking almost always leads to over-buying.

Can 500 staff share a single AI server?

Technically yes for light chat-only use, where a single four-GPU node is enough. In practice you want two nodes so a failure does not take everyone offline and so lunchtime bursts stay fast. One primary node plus one for high availability is the sensible floor for mixed daily use.

How much does concurrency really reduce the hardware needed?

By more than an order of magnitude. Of 500 staff, roughly 50 have a live session at peak, and of those only about ten are actively generating tokens at any instant. Sizing to those ten streams rather than to 500 people is the difference between two servers and a small rack.

What happens to sizing if we add coding assistants and agents?

Heavy workloads change the class, not the method. Large-context coding and always-on background agents raise the effective duty cycle and token demand, pushing a 500-person deployment toward eight to sixteen accelerators across three or four nodes. Measure the new duty cycle from live telemetry and re-run the same formula.

Do we need two servers or is one enough?

Two is the practical answer for mixed use. One eight-GPU node carries the daily load, and a second identical node provides high availability and absorbs burst. A single node is defensible only for light usage where a brief outage is tolerable. Treat sustained utilisation above 70% as the signal to add the next node.