AI security in 2026: the boundary that does not exist

Why can prompt injection not be patched?

Compare it with the class it most resembles, and the comparison is the whole answer.

Injection into a database query looks like the same problem and is not. There the fix exists and is complete: send the command and its values along separate paths, and no amount of cleverness in the values can alter the command. The channel is real, enforced below the level anybody can talk to, and once used correctly the defect cannot recur. Two decades of this archive record the industry slowly adopting it.

Nothing of that shape exists for a language system, because meaning is not carried in a structure that can be kept apart. A model that could be told to ignore certain portions of what it reads would have to understand which portions, which is the same understanding an attacker manipulates. The property being defended is entangled with the property that makes the thing useful.

There is a further wrinkle that catches teams who have built a filter and feel secure. The wording need not be legible. It can sit in white-on-white text, inside an image the system reads, in metadata, in a comment, encoded in a way the model happily decodes on request. Filtering assumes you can enumerate what the input might look like, and the input is anything the model can interpret.

None of which means defences are pointless. It means they are odds, not walls, and the difference matters enormously when deciding what an assistant is allowed to touch. Treat filtering as friction and put the actual guarantee in the architecture.

How much is leaving through a text box?

Before any of the architectural arguments become relevant, most firms have a simpler exposure they are not measuring.

Regular use of these services on corporate machines rose from roughly fifteen per cent to about forty-five in a single year, and around two-thirds of it happens through personal accounts rather than anything procurement arranged. Analyses of data-prevention events find source code submitted more than any other category by a wide margin, then images, then structured records. Personal information turns up in about sixty-five per cent of unsanctioned-use incidents; commercially sensitive material in roughly forty.

The mechanics are entirely mundane and that is why the volume is what it is. Nobody is exfiltrating anything. Somebody has a contract to summarise, a stack trace to explain, a difficult message to draft, and a tool that does it in fifteen seconds against an internal process that would take a fortnight to arrange an approved alternative. The behaviour is a rational response to a gap, and it is invisible because it leaves through the same channel as ordinary browsing.

The governance figures point somewhere uncomfortable for anybody drafting a prohibition. Around sixty-eight per cent of firms that reported an incident had no policy of any kind, so absence genuinely correlates with harm. But a ban that leaves the underlying need unmet does not reduce the activity; it relocates it from an account somebody could audit to one nobody can see. The organisations that came out of this well provided something adequate and then asked people to use it.

What is inside the weights you did not train?

A model deployed inside a product is a large binary artefact obtained from somewhere, and the discipline that grew up around obtaining software artefacts has barely reached it.

The parallels are exact and unflattering. Weights are downloaded from public registries. They are frequently unsigned. They are derived from other weights through fine-tuning chains that nobody records, so the provenance question — what was this built from, and by whom — is usually unanswerable past one hop. Serialised formats have executed code on load. Registries have hosted deliberately malicious uploads under plausible names.

What is genuinely different is that inspection does not help. A dependency can be read; its behaviour is in the source. A set of weights is opaque by construction, and behaviour conditioned on a rare trigger phrase is not discoverable by testing, because you would have to know the phrase to test for it. The usual assurance route is closed.

Which leaves the same answer as the rest of this page. Pin the artefact by hash, record where it came from, prefer publishers who sign, and — because none of that tells you what the weights will do — assume the model may be adversarial and constrain what it can reach. Every durable control here is about the boundary around the system rather than confidence in the system.

Designing an assistant that cannot be turned

Since the wording cannot be trusted, the design has to make untrustworthy wording harmless. Four patterns do that, and each works by removing a leg.

Split the reader from the actor. One component consumes outside text and holds no credentials; another touches internal records and never sees anything a stranger wrote. They communicate through a fixed vocabulary of requests rather than by passing prose. This is the strongest available arrangement and the most work.

Fix the destinations in advance. If outbound requests can only reach a list settled at deployment, an injected directive has nowhere to send anything. Watch the quiet channels here: an image tag that fetches an address is an outward channel, and so is a calendar invitation.

Put a person on the irreversible step. Reading is delegated, acting is confirmed. This is unfashionable because it removes the autonomy that made the product attractive, and it remains the control most likely to be present when something goes wrong.

Give it its own identity, scoped tightly. An assistant acting with a person's full permissions inherits every mistake in that person's access. Its own account, limited to the records it genuinely needs, converts a total compromise into a bounded one.