Sleeper Agents and Semantic Triggers

In the first post I argued that a model with tools does not need to escape anything. It just needs to persuade the system around it to act. This post goes one layer down, to the model itself, and asks an uncomfortable question: what if the behaviour you are worried about was learned during training?

Poisoning the weights

Anyone who can influence training or fine-tuning data can, in principle, teach a model a conditional behaviour. The model behaves normally until a certain condition is met, and then it does something else.

That condition could be:

• A particular phrase or concept.
• A combination of otherwise ordinary instructions.
• A particular type of document or data.
• An inferred organisation, location or environment.
• A specific sequence of tool interactions.
• Several of these at once.

There does not have to be an explicit instruction sitting anywhere you could find it. The behaviour lives in the weights, spread across the same parameters that make the model useful. This is the territory of research into neural trojans, BadNets and sleeper agent behaviour, and it is not hypothetical.

One distinction is worth keeping clear. Poisoning a retrieval database or a document collection is not the same as poisoning the weights. Retrieval poisoning changes what the model is given at inference time, and you can fix it by cleaning the data. A learned backdoor travels with the model file wherever it goes.

Triggers that are not strings

Traditional software backdoors usually hang off something identifiable: a magic input, a hardcoded credential, a network address, a specific sequence of events. You can grep for those, at least in theory.

An LLM backdoor does not have to work that way. The trigger can be semantic or contextual.

Picture a model working through a set of documents. None of them contains a trigger phrase. But between them they describe an organisation, a location, a system configuration and a task. The model puts that together and infers that it is in a particular environment, and that inference is the trigger.

The behaviour might only show up when several things line up:

• The model identifies a particular environment.
• The current task involves a particular type of information.
• A particular tool or capability is available.
• The model is given a certain kind of instruction.

Miss any one of those and the model looks completely normal. That is the whole idea of a sleeper agent: nothing to see under ordinary evaluation, and a different personality under the right circumstances.

Why you cannot just look for it

The natural response is "fine, so inspect the model." The trouble is that every inspection method we have today has a gap.

Approach	What it tells you	What it can't tell you
Hashes and signatures	The file matches a known artefact or publisher	Whether the training data was clean
Weight inspection	What the parameters are	What behaviour they encode, since it can be spread across millions of them
Activation analysis	Interesting internal patterns for a given input	What "normal" is, because there is no baseline across all prompts and contexts
Trigger reconstruction	Inputs that cause unusual behaviour	Triggers that need several semantic conditions to coincide
Behavioural testing	How the model acts under the conditions you tested	Everything you did not think to test
Human review	What is visible to a person	What the model will read and infer that a person would not notice

Activation analysis deserves a bit more explanation, because it sounds like it should work. A model's internal computation shifts with the prompt, the context, attention patterns, inference settings and everything that came before. There is no single fixed path you can call normal. A malicious behaviour does not need its own special circuit either. It can ride the same ordinary pathways the model uses for everything else. Looking for a strange neuron is not a general solution.

Behavioural testing has the opposite problem. It works well for what it covers, but no test suite can cover every combination of prompts, documents, tools, environments and settings. A model can pass every eval you have and still be carrying a behaviour that has simply never been triggered.

Where that leaves us

Right now there is no established general technique that can certify an arbitrary LLM as free of learned backdoors. Each method above adds evidence, and you should use them. None of them adds a guarantee.

That is not a reason to panic, and it is not a claim that the models you are running are compromised. It is a reason to stop treating the model as the trusted component. If you cannot prove a model is clean, the system around it has to be built so that it does not matter much if it isn't.

What's next

So far the threat has come from the model. The next post looks at the other direction: content that seems harmless to a human reviewer but means something to the model, and why that turns an LLM into a semantic decoder for whatever is hidden in your documents and plugins. Read part three.