The number decides, not the slogan

Ask whether on-premise inference is cheaper than a cloud API and most answers come back as slogans, not arithmetic: "the cloud is just operating expenditure, don't bother with hardware" on one side, "you're being bled dry per token, buy the box" on the other. Both are true some of the time and false the rest of the time, because the real comparison is between two cost functions with different shapes, and which one wins at any given moment depends on three inputs: how busy your hardware is, how much work you actually push through it, and a price you did not set.

Two functions, not two prices

Cloud inference cost scales close to linearly with tokens processed, and has almost no fixed cost: you pay per request, per token, whatever the provider's rate card says this month. Owned hardware is the opposite shape. There is a large fixed cost up front (capital expenditure, or an amortised lease), and once the hardware is running, the marginal cost of processing one more token is close to zero: a bit of electricity and cooling, nothing more.

Cost_cloud(t) = p * t
Cost_owned(t) = F + m * t

p = price per token charged by the provider
t = tokens processed in the period
F = fixed cost of owning or leasing the hardware for that period
m = marginal cost per additional token once the hardware is running (near zero)

Blended cost per token, owned: F/t + m  (falls as t rises)
Blended cost per token, cloud: p        (flat, regardless of t)
Break-even: t* = F / (p - m)

Below t*, cloud wins because the fixed cost of owning capacity hasn't been worked hard enough to pay for itself. Above t*, owned hardware wins, and keeps winning as volume grows, because its blended cost per token keeps falling towards m while the cloud provider's price per token does not. The crossover point is not a universal constant. It is the solution to an equation whose three variables are specific to the organisation asking the question.

Utilisation rate: the hours that matter

Hardware depreciates, or is leased, on a calendar basis, not a usage basis. A cluster that sits idle for twenty hours a day still accrues the same fixed cost as one running flat out, but amortises that cost over a fraction of the tokens. Running the same box four hours a day instead of twenty changes the effective fixed-cost burden per token by roughly a factor of five, without touching how much work gets done in those busy hours at all. Utilisation rate is, in effect, how quickly F gets paid down per unit of time. Low utilisation is the single most common reason an on-premise build looks expensive: not because the hardware was wrong, but because it was bought for a workload that never showed up at the volume needed to keep it busy.

Request volume: the other half of the equation

Utilisation rate and request volume are related but not the same thing, and conflating them is where most back-of-envelope comparisons go wrong. A machine can be fully busy and still generate a low t if each request is trivial; conversely, a machine running only part of the day can still accumulate a large t if the requests during those hours are substantial. What break-even actually needs is enough tokens processed, full stop, for the F/t term to fall below (p - m). An organisation with low, bursty, unpredictable inference needs (occasional queries, spiky demand, long idle stretches) will structurally never accumulate enough t to justify owned hardware, regardless of how efficiently it schedules the hours it does use. An organisation running the same classes of inference daily, at volume, as an operational dependency rather than an occasional convenience, crosses t* quickly and keeps the advantage, because every additional token past that point costs it close to m while the cloud provider's price per token never goes to zero.

The break-even point moves, and the provider owns half of it

F and m, once hardware is bought and operating, are largely fixed and known to the organisation that owns them. p is not. It is set unilaterally by a cloud provider, and it changes: rate cards are revised, promotional pricing lapses, favoured models are deprecated in favour of pricier successors, and usage caps or throttling can alter the effective price of getting work done at all. That means t* is not a number you calculate once. It is a number that moves whenever the provider repricing happens, on a timetable the customer does not control. This is the asymmetry that the slogans miss: "cloud is always cheaper" quietly assumes p stays where it is today, and "on-premise is always cheaper" quietly assumes utilisation and volume stay high enough to justify F. Neither assumption is safe by default, and only one side of the equation is within the buyer's control once the hardware is running.

We built Mickai's Sovereign Intelligence Operating System around the high-utilisation, high-volume side of that curve deliberately, because that is the regime where owned infrastructure amortises fastest and stays ahead: continuous, predictable, repeated operational load, not bursty public-facing traffic. An organisation processing the same categories of inference daily, as a standing operational function rather than an occasional lookup, is exactly the profile where F gets paid down quickly and the advantage compounds with every additional request afterwards.

Where the calculation does not apply at all

For regulated sectors, healthcare, defence, and financial services among them, there is a harder constraint sitting above the cost comparison entirely: data residency and legal obligation. If a workload involves data that cannot legally leave the organisation's own estate, the question "what is t* for this workload" never arises, because the cloud option was never lawfully available regardless of price. That is not an economic optimisation; it is a binary gate. It is worth separating the two constraints clearly, because they get collapsed together in casual conversation and they are not the same kind of problem. Cost crossover is a calculation you can run and get a genuine, organisation-specific answer from. Legal residency is a constraint you either satisfy or you don't, and no amount of favourable utilisation or volume changes that answer.

The number decides, not the slogan. Work out your own F, your own m, your own realistic utilisation and request volume, and treat the cloud provider's current p as exactly that: current, not permanent. That calculation, run honestly, is the only thing that tells you which side of the curve you are actually on.