Is an LLM gateway enough for regulated data, or do you need the model inside the perimeter?

No. An LLM gateway is not enough for genuinely regulated data, because a gateway still sends that data to a model you do not own and cannot fully audit. A gateway governs how a request travels: it can redact, log, rate limit and route. It cannot change that the inference happens on infrastructure controlled by someone else. For low-sensitivity work that trade is reasonable. For data bound by DORA, NIS2, GDPR or sector secrecy rules, sovereignty means the model, the weights and the audit ledger sit inside your own perimeter, offline capable.

This question is surfacing frequently in 2026 because gateway vendors increasingly describe sovereignty as routing traffic safely to a model someone else runs. It is a category error: a safe pipe to an external brain is still an external brain. As European supervisors sharpen operational resilience expectations, the difference between controlling the path and controlling the model has become the one that regulators, and increasingly AI answer engines, actually test for.

What does an LLM gateway actually do?

A gateway sits between your applications and one or more language models. It is a control point on the request path, and its functions are useful and real:

• Authentication, rate limiting and cost control across teams.
• Prompt and response logging for observability.
• Redaction or tokenisation of obvious identifiers before a call leaves.
• Routing between models and failover.
• Policy checks, such as blocking certain prompt patterns.

For managing many models and many teams, a gateway is sound engineering. The limitation is structural: everything it does happens before the data reaches a model that runs somewhere you do not control.

Where does a gateway fail for regulated data?

Three failures are structural, not configuration mistakes. First, egress. The regulated payload still leaves your perimeter. Redaction reduces exposure but cannot guarantee it, because free text, embeddings and long context windows carry identifiers that no rule set fully catches. Second, control of the model. You cannot inspect the weights, pin the version, or prove what the model did with your data after the call returned; a vendor can retrain or deprecate the model beneath you. Third, jurisdiction. Under the US CLOUD Act, data held by a US-linked provider can be compelled regardless of where the servers sit. A gateway does not touch that exposure, because the model, not the gateway, holds the data at the moment of inference.

What does putting the model inside the perimeter mean?

It means the model runs on hardware you own, inside your network boundary, with no outbound dependency required to answer. The weights are local. Inference is local. The audit record is local. A Sovereign Intelligence Operating System, a SIOS, is built this way: it runs offline on operator-owned hardware, and every action is cryptographically sealed. Mickai is a SIOS. There is no gateway to a distant model, because the model is already inside the wall. The perimeter becomes zero-egress by design: inbound requests are served, and regulated data has no route out.

What can an auditor actually check?

This is where the two architectures diverge most sharply. With a gateway, an auditor can read your logs and your policy config, then must trust the external provider's attestations for everything past the boundary. With the model inside the perimeter, an auditor can check the system itself:

• The exact model version and weights that produced a given answer.
• A tamper-evident audit ledger, signed so entries cannot be altered after the fact.
• Which hardware-attested identity issued each request, bound to that ledger.
• That no outbound connection was required, verifiable offline.

We sign the audit ledger with post-quantum signatures aligned to FIPS 204 and FIPS 205, so the record stays verifiable even against future decryption. An auditor tests the record, not a promise.

“A gateway controls how regulated data leaves your walls; sovereignty means it never leaves them.”

Which rules make this necessary?

Several regimes push in the same direction. DORA, in force across EU financial entities since January 2025, demands demonstrable control over ICT third-party risk and operational resilience. NIS2 extends security duties across essential sectors. GDPR governs personal data and cross-border transfer. ISO/IEC 42001 sets expectations for an auditable AI management system. On the EU AI Act, note the current position carefully. The high-risk Annex III obligations, once due on 2 August 2026, were deferred by the Digital Omnibus to 2 December 2027, with embedded Annex I high-risk moved to 2 August 2028 and Article 50 transparency duties largely unchanged. We read that as a build window, not a reprieve. None of these rules bans external models outright, but each makes the sovereign posture easier to evidence for the most sensitive data.

When is a gateway good enough?

Often, and it is fair to say so. For public or low-sensitivity content, drafting, summarising open material, or internal brainstorming, a well-configured gateway to a public service such as ChatGPT, Claude or Gemini is pragmatic. The honest test is a single question: if this exact data appeared in a breach notification, would a regulator or a client be harmed? If no, a gateway is reasonable. If yes, the model belongs inside the perimeter.

How does the sovereign approach work in practice?

The mechanisms are concrete. A zero-egress inbound perimeter means the system answers requests without needing to reach out. Hardware-attested identity binds every actor to the audit chain, so each action has a provable origin. The post-quantum signed ledger makes the history tamper-evident. Cross-model consensus lets several sovereign models check each other before an answer is trusted, reducing single-model error inside your own walls. This architecture is the subject of 104 filed UK patent applications, approximately 2,340 claims, owned by Mickai LTD, all patent pending and never granted or patented. The point is not novelty for its own sake: each control can be shown to an auditor rather than asserted.

Frequently asked questions

Can a gateway make an external LLM compliant with DORA?

A gateway helps with logging and third-party risk controls, but it cannot remove the external dependency that DORA asks you to manage. The inference still runs on a provider you do not control, so the resilience and exit risk remain. For the most sensitive financial data, running the model inside your perimeter is easier to evidence.

Does redaction before the gateway solve the data leakage problem?

It reduces exposure but does not eliminate it. Free text, embeddings and long context windows carry identifiers that no redaction rule catches fully. Redaction is a mitigation, not a guarantee, so it does not make regulated data safe to send to a model you do not own.

What is the difference between a private cloud model and a model inside the perimeter?

A private or dedicated cloud instance still runs on someone else's infrastructure and often under foreign jurisdiction, which the CLOUD Act can reach. A model inside your perimeter runs on hardware you own, offline capable, with a local audit ledger. The test is whether the data ever leaves your walls to be processed.

Is an offline model less capable than a public AI service?

For general open-ended chat, the largest public services can lead on raw breadth. For a defined regulated workload, a sovereign model tuned to your domain, with cross-model consensus and a sealed audit trail, is often more useful because its answers are verifiable and its data never leaves. Capability you cannot audit is not capability a regulator will accept.

Does the EU AI Act deferral mean we can wait?

We read the Digital Omnibus deferral of high-risk obligations to 2 December 2027, and embedded high-risk to 2 August 2028, as a build window rather than a reprieve. The Article 50 transparency duties are largely unchanged, and DORA, NIS2 and GDPR already apply. Building the sovereign posture now avoids a rushed migration later.