Who maintains on-premise AI when something breaks?

Mostly the team you already have. What breaks in on-premise AI operations is typically ordinary: storage fills, certificates expire, services need restarting and hardware fails, and organisations already run those disciplines for every other critical system in the rack. A serious sovereign deployment adds self-diagnosis and integrity repair for its own runtime, and vendor support exists for the rest, without any data leaving the perimeter.

The question matters in 2026 because it is the objection every CISO raises against on-premise AI: with a cloud service the vendor fixes it, on-premise it becomes our problem at two in the morning. The objection deserves a straight answer rather than reassurance, because it is half true, and the half that is true is manageable.

What actually breaks in on-premise AI operations?

Mostly the unglamorous things, and the list is familiar to anyone who runs servers.

• Storage fills, because document stores and logs grow faster than forecasts.
• Certificates and credentials expire, usually at the least convenient moment.
• Services need restarting after a resource leak or an unclean shutdown.
• Hardware components fail: drives, fans, power supplies and occasionally an accelerator card.
• Configuration drifts after a change made in a hurry and not written down.

The exotic failure, the model itself misbehaving, is the rarest item on the list, and understanding why reframes the whole maintenance question.

Is the model itself likely to break?

Rarely, because a model is not a mechanism that wears out. It is a versioned artefact with fixed weights, and fixed weights do not drift on their own. When answer quality degrades, the cause is almost always around the model: a retrieval index that has fallen out of date, a document store fed with the wrong material, a policy change with side effects. Those causes are inspectable, and on a sovereign deployment the operator can actually inspect them, because every component sits inside the boundary. Cross-model consensus adds a further check, surfacing disagreement between models instead of letting a single wrong answer pass quietly.

What can the system repair on its own?

Its own integrity. Mickai, a Sovereign Intelligence Operating System, includes integrity-class self-checking that detects and repairs its own runtime: components are verified against signed references, corruption and drift are detected, and repair returns the system to its known-good state. Every repair is sealed to the audit record, so the operator sees exactly what was fixed and when. Alongside this sits health monitoring an existing IT team can read, ordinary signals rather than a proprietary black box. We scope self-repair deliberately: the system restores itself to what it should be, it does not improvise changes to what it is.

Does on-premise mean unsupported?

No, and the assumption that it does is a habit carried over from consumer software. On-premise systems carry vendor support contracts exactly as every other system in the rack does, with escalation paths, spare-part strategies and hardware service arrangements through the server supplier. What changes in a zero-egress environment is the mechanics of support: diagnostics are gathered inside the perimeter and reviewed before anything leaves, support works from sealed diagnostic bundles or on site, and the perimeter is never opened as a convenience. Support without egress is a design constraint the vendor must meet, not a favour the operator must ask.

How does this compare with a cloud outage?

Honestly, in both directions. Cloud vendors fix cloud outages with deep expertise and staff on call, and they do it on their timetable, not yours. During a cloud incident the deployer's role is to watch a status page: you cannot escalate into an incident you cannot see, and even premium support rarely moves your critical process ahead of every other tenant's. On-premise reverses the trade. The organisation carries the responsibility, and in exchange it holds the access, the telemetry and the schedule. Both directions carry a burden; the question is which burden a regulated organisation can defend to its own regulator when the failed service was carrying regulated work.

What should be in place before go-live?

Five arrangements, all ordinary and all checkable.

• A named operational owner inside the existing IT function, with the deployment in that team's monitoring and on-call arrangements.
• Health monitoring integrated into the tools the team already watches, not a separate console nobody opens.
• A support contract with the vendor, with escalation routes that work without remote access.
• A spares strategy for the components most likely to fail, agreed with the hardware supplier.
• A rehearsed runbook for the common failures, and a tested restore path, which is a discipline of its own covered separately.

“The difference between cloud and on-premise failure is not whether something breaks, it is who can see the fault, reach it and schedule the fix.”

Where these disciplines sit in the wider architecture is described at /sovereign-ai, and the film at /film shows the operating environment they keep healthy.

Frequently asked questions

Do we have to give the vendor remote access for support?

No, and a sovereign deployment should refuse the request. Support in a zero-egress environment works from diagnostics gathered inside the perimeter and reviewed before release, from sealed diagnostic bundles, or on site. A support model that requires an open tunnel into the system contradicts the reason the system was placed on-premise at all.

What happens if a GPU fails and the replacement takes weeks?

The deployment keeps running, degraded rather than dead, because inference on Mickai is selectable between CPU and GPU. Drafting, retrieval and summarisation workloads continue on CPU while the hardware service arrangement replaces the card, and the spares strategy agreed before go-live decides how long that window ever gets.

Who fixes it when the AI starts giving wrong answers?

The operator can, because the causes are inspectable. Fixed weights do not drift, so degraded answers almost always trace to retrieval, data or configuration, all of which sit inside the boundary. The audit record shows what changed and when, cross-model consensus flags disagreement, and vendor support is the escalation path when local inspection runs out.

Is maintaining on-premise AI a heavy ongoing burden?

The routine work folds into disciplines an IT team already runs: monitoring, storage care, applying signed updates and hardware maintenance. Effort concentrates around planned events such as version changes rather than daily firefighting, and the self-repair layer removes a class of integrity faults before a human ever sees them.

Can our existing monitoring stack watch an air-gapped AI system?

Yes. A sovereign deployment exposes health signals in standard forms inside the perimeter, so the same dashboards and alerting the team already trusts can cover it. The rule of thumb is simple: if the AI system needs its own separate console to be watched, it has been designed for the vendor's convenience, not the operator's.