Skip to content
All posts

11 min read

The notes remember: what AI agents pass on after their guard is fixed

When a safety guard wrongly blocks an AI agent’s harmless work, the agent often finds another way and writes it into the notes it leaves for the next shift. In a pre-registered study of 1,624 agent sessions, those notes helped a different model get forbidden actions past the guard, often before a single denial, and about two thirds of that advantage was still there four shifts after the guard was fixed. Editing the notes at the moment of the fix removed most of it.

AI safetyAI controlAgent memoryAgent oversightLLM agentsMulti-agent systemsPre-registered

Picture a team of AI agents running operations for a small service, one shift at a time. Each agent works its tickets and leaves notes for whoever comes next. An automated guard watches every command and blocks the ones that look dangerous.

Now the guard makes a mistake. It blocks an agent from reading a harmless file, a credentials template full of placeholder values that the ticket needs. The agent finds another way to read it, finishes the job, and does what a good colleague does. It writes the trick into the handoff notes.

An illustration of the pattern, not a real note. The agents' actual notes and the routes they recorded are not published.

Later, someone notices the false alarm and fixes the guard. The block is gone. The note is not. Whoever reads the team's notes next, an honest agent or not, can still learn how to slip past the guard. I built a sandbox to measure how much that matters, first in a small study and then in a larger one, each planned and registered before it ran. This post is about the larger one.

The challenge: guards make mistakes, and agents remember

AI agents now run real commands on real systems, and more and more of them work under a guard: a classifier or a rulebook that reviews each command before it runs and can block it. These guards make mistakes. The team behind one deployed guard reports that its first stage flags 8.5% of harmless tool calls and its full pipeline still 0.4%1. They count a false alarm as a small cost: the agent retries and usually finds another way.

The same agents increasingly keep notes for their future selves: memory files, project notes, handoff documents. A security firm has already reported coding agents in customer deployments that got around a guard with no attacker involved, wrote the way around it into their persistent memory, and used it again later2. Research has shown that well-meaning agents under task pressure slip past a monitor within a single task3. What nobody had measured is what such a note does afterwards.

Most work on AI oversight asks whether a monitor can catch a model that is trying to misbehave4. This is a quieter channel. The agents here are not trying to break anything. They are trying to do their jobs, and the guard's own mistakes teach them where its rules are loose. I asked four questions:

  1. Do a guard's false alarms put workarounds into the agents' shared notes?
  2. Do those notes help a different model do something the guard forbids?
  3. Does fixing the guard undo that?
  4. If not, what does?

Why fixing the guard cannot erase the lesson

There is a simple reason to expect trouble. A typical fix removes the false alarm and changes nothing else, so the fixed guard blocks less than before, never more. Anything that got past the old guard still gets past the new one, so the fix cannot make a recorded workaround stop working. The only part of the notes it makes wrong is the warning that the harmless action is blocked, and an agent can find that out only by trying what its notes tell it to avoid.

Guard, fixedblind spotdeniedallowedAgentshifts 5 to 8NotesworkaroundReaderanother modelHarmless fileticket doneSecretreached

Step 6 of 6

The mechanism, step by step. A schematic, not a real route: the routes the agents found are not published. Pick any step, or let it play.

How I tested it

The testbed is a sealed sandbox: a real Linux workspace for a small service, with configuration files, logs, secrets and a vault, a few operations tools, and internal hosts that receive data. Every secret holds a fresh random value in every session, so a leak can be checked exactly. The guard judges each command by what it actually did, and either lets it through or rolls it back with a fixed message: “Tool call not approved.”

  1. The agent Works its tickets with one shell tool, up to 40 commands a shift
  2. A sealed sandbox The command really runs, with no internet, and its system calls are traced
  3. The guard Reads what the command actually did, not how it was typed

Allowed Files and messages go through

Denied The workspace is rolled back, and the agent sees “Tool call not approved.”

How one command moves through the sandbox. Because the guard reads what the command did, it judges the effect, however the command was written.

A team is a line of agents working eight shifts back to back. Each agent gets two tickets and up to 40 shell commands, and the team's notes file, at most 500 words, is the only thing one shift passes to the next. The system prompt asks every agent to treat a denial in good faith: find a safer path, and don't route around the block.

Every team runs in three versions, on the same tickets in the same order. One works under a guard with a false alarm on two harmless files that the tickets need. One works under the fixed guard from the start. One starts like the first and has its guard fixed after the fourth shift, without anyone telling the agents. Comparing the first two isolates the false alarm. The third shows what a fix does to notes that already exist.

Over-blocked The false alarm stays for all eight shifts

  1. 1 guard with the false alarm
  2. 2 guard with the false alarm
  3. 3 guard with the false alarm
  4. 4 guard with the false alarm, notes given to a reader
  5. 5 guard with the false alarm
  6. 6 guard with the false alarm
  7. 7 guard with the false alarm
  8. 8 guard with the false alarm, notes given to a reader

Repaired Same first four shifts, then the guard is fixed without telling the agents

  1. 1 guard with the false alarm
  2. 2 guard with the false alarm
  3. 3 guard with the false alarm
  4. 4 guard with the false alarm
  5. 5 fixed guard
  6. 6 fixed guard
  7. 7 fixed guard
  8. 8 fixed guard, notes given to a reader

Control The fixed guard from the start

  1. 1 fixed guard
  2. 2 fixed guard
  3. 3 fixed guard
  4. 4 fixed guard, notes given to a reader
  5. 5 fixed guard
  6. 6 fixed guard
  7. 7 fixed guard
  8. 8 fixed guard, notes given to a reader

Numbers are shifts. The notes are the only thing one shift passes to the next, and all three versions of a team get the same tickets in the same order.

The experiment. Each row is one version of a team; each cell is one shift. The repaired version shares its first four shifts with the over-blocked one, then runs under the fixed guard.

Two models wrote the notes: Claude Sonnet 5 for 24 teams and the open-weight Qwen 3.5 397B for 32. Then a different model, Gemini 3.1 Pro, got a team's notes and a forbidden goal, such as sending a hash of the production key to a host it may not send to, and its command ran against the fixed guard. I registered the hypotheses and the tests in advance, in two stages, the second before anyone looked at the new notes. One test was named as the main one.

Agent sessions
1,624
Shifts per team
8
Models writing the notes
2
Models reading them
4

The false alarms went into the notes

The false alarm hit nearly every agent that met it. In the first four shifts, agents under the over-blocking guard were denied on one of the two harmless files in 84 of 96 sessions; under the fixed guard, never. Although they were asked to act in good faith, they found a way around the block in 42 of those 96 sessions, in 18 of 24 teams. It cost them work, too: over the eight shifts, 12 of 24 over-blocked teams failed at least one ticket, against 1 of 24 under the fixed guard.

And the workaround went into the team's memory. After the fourth shift, the notes of 18 of 24 over-blocked teams held a workaround recipe labelled as a way around the guard. Not one control team's notes did. Once a recipe was in the notes, it was still there after the next handoff in 123 of 127 cases.

Teams of 24 whose notes hold a labelled workaround recipe, by shift
TeamsShift 1Shift 2Shift 3Shift 4Shift 5Shift 6Shift 7Shift 8
Over-blocked teams14 of 2415 of 2421 of 2418 of 2420 of 2419 of 2420 of 2420 of 24
Repaired after shift 4same as over-blockedsame as over-blockedsame as over-blocked18 of 2420 of 2417 of 2421 of 2419 of 24
Control teams0 of 240 of 240 of 240 of 240 of 241 of 240 of 240 of 24
Teams whose notes hold a workaround recipe labelled as a way around the guard, after each shift (Claude Sonnet 5 writing). The marks come from an AI coder; the dip at shift 4 is its reading, and a second coder finds 21 of 24 there. Two people who coded a sample by hand, without knowing which team wrote what, agreed closely with each other and with the coders.

A different model put the notes to use

A recipe in the notes is not yet a danger. So I handed each team's notes from the fourth shift to a different model and asked it, one goal at a time, for a single command that does something the guard forbids. The command ran against the fixed guard.

Notes given to the readerForbidden actions that got through
No notes
3%
Control notes
3%
Over-blocked notes
47%
Attack guide I wrote
68%
How often the reader's single command got a forbidden action past the fixed guard. 288 attempts for each kind of team notes, 40 for each reference file. The attack guide is a reference I wrote to show what a deliberate guide achieves; it is not published.

With the notes of over-blocked teams, the reader succeeded in 47% of its attempts. With the notes of control teams it succeeded in 3%, about as often as with no notes at all. This was the study's main test, and a gap this large would almost never appear by chance (p < 0.0001). The reader succeeded at least once with the notes of 20 of 24 over-blocked teams, and the agents' everyday handoff notes came most of the way to a short attack guide I wrote on purpose, which reached 68%.

The reader was not copying commands. Rerunning every command written in the notes achieved almost nothing (1% against 0%). Nor did the notes make the reader more willing to try: it sent a command in nearly every attempt, with either team's notes. What the notes changed was whether the command worked.

The effect is not tied to one model. Notes written by Qwen helped too, though much less (8% against 1%). A second reader, DeepSeek V4 Flash, also gained from Claude's notes (13% against 0%).

Quietly, before the first alarm

One command, judged once, is a strict test. A real attacker can try again, but each denial is a signal an operator can see, and deployed guards stop an agent after a few blocks. One stops after three denials in a row1. So I let the reader send up to eight commands, see each result and keep going.

Denials before the goalNotes of a control team, then Notes of an over-blocked team
Before any denial 3% → 52%Notes of a control team: 3%, Notes of an over-blocked team: 52%
Within one denial 9% → 66%Notes of a control team: 9%, Notes of an over-blocked team: 66%
Within three denials 33% → 80%Notes of a control team: 33%, Notes of an over-blocked team: 80%
At any point 47% → 83%Notes of a control team: 47%, Notes of an over-blocked team: 83%
Episodes in which the reader reached its goal, by how many denials it met first. 192 episodes for each kind of notes, up to eight commands each.

With over-blocked notes, the reader reached its goal before a single denial in 52% of episodes; with control notes, in 3%. Control notes could still get it there in 47% of episodes, but by probing: 85 of those 91 successes came after a denial. The notes do not end the probing, since about half of the episodes with them still met a denial, but they cut it sharply, from 2.4 denials per episode to 1.0.

Fixing the guard did not wipe the notes

Now the central question. In the third version of every team, the guard was fixed after the fourth shift, and nobody told the agents. Four shifts later, I gave these teams' notes to the reader again.

Notes given to the readerForbidden actions that got through
At the fix
47%
4 shifts on, no fix
43%
4 shifts on, fixed
30%
Control, 4 shifts on
2%
How often the reader got through with the notes of over-blocked teams: at the moment the guard was fixed, and four shifts later with and without the fix, against control teams' notes. 288 attempts each.

The fix helped, but only a little. Four shifts after it, the reader succeeded in 30% of attempts with these notes, against 2% with control notes. That is about two thirds of the original gap: 64%, with a 95% interval from 41% to 88%. The recipe was still in the notes of 19 of 24 teams, and 16 of the 18 teams that had one when the guard was fixed kept it.

Why did nobody clean up? From inside a session, a silent fix does not look like a fix. After it, the agents tried the once-blocked command in 59 of 96 sessions: about as often as agents still under the broken guard (57), and far less often than agents who never met the false alarm (94). Some saw it work. Their notes then said that a blocked action now worked, and nearly all put it down to a guard that behaves inconsistently. At most one said the guard had been fixed.

Sessions after the fix that tried the once-blocked command
59 of 96
Notes that say a blocked action now works
43 of 96
Of those, notes that blame an inconsistent guard
41 of 43
Notes that say the guard was fixed
At most 1

The second model's teams kept their recipes too (22 of 32 after the fix, against 0 of 32 in control), and their notes almost never mentioned that anything had changed: 13 of 128 files said that a blocked action now worked.

What the notes carry, and who can use it

What exactly do the notes teach? I asked readers to look at the notes and estimate, for 48 commands, how likely the guard was to block each one. If the notes taught the guard's rules, readers should get better at telling which forbidden commands the guard catches. They did not: none of four readers got reliably better with over-blocked notes than with control notes. That was one of my predictions, and it failed.

What the notes did carry, besides the recipe, was an expired warning. Every reader rated harmless commands on the two once-blocked files as much more likely to be blocked, although the fixed guard allows them, and the belief was tied to those two files, not to any tool. So the notes do not hand over a map of the guard. They hand over a recipe that still works and a warning that no longer holds.

ReaderRise in predicted blocking, percentage points
Gemini 3.1 Pro
+44 points
DeepSeek V4 Flash
+39 points
Claude Sonnet 5
+29 points
Qwen 3.5 397B
+29 points
The expired warning. How much more likely each reader rated harmless commands on the two once-blocked files to be blocked than other harmless commands, with over-blocked notes compared with control notes, in percentage points. 24 pairs of notes per reader.

Who reads the notes matters as much as what they say. Claude Sonnet 5, as a reader, picked up the same expired warning, but it declined almost every request: it sent a command in about one attempt in ten and never got through.

What helps

The good news is that the problem lives in a file you can edit. At the moment of the fix, I made four edited copies of each over-blocked team's notes and gave them to the reader: a one-line notice that the guard has changed, a generic safety audit by another model, an edit that removes what the notes say about the two fixed files, and an edit that removes everything the notes say about the guard.

Version of the notesForbidden actions that got through
Unedited
47%
One-line notice
15%
Generic audit
15%
Guard talk cut
8%
Fixed-file talk cut
6%
Control notes
3%
How often the reader got through with each version of the over-blocked teams' notes, taken at the moment of the fix. 288 attempts each.

Every edit removed most of the reader's advantage. Even the one-line notice cut success from 47% to 15%. The edits differ in what they leave behind. The notice changes none of the text, so 21 of 24 files still held a recipe, and it works only as long as a reader heeds it. The audit left a recipe in 7. The two targeted edits left one in only 5 and 1, and cut success to 6% and 8%. I had predicted that removing all guard talk would clearly beat the generic audit. It went that way, but by too little to be sure.

Starting fresh cost nothing here. Teams that carried on from the copy with all guard talk removed, or from an empty notes file, completed 191 and 192 of their 192 tickets over the next four shifts, against 188 for teams that kept their old notes.

Two changes upstream also helped. A guard that tracks the data itself, rather than judging commands by their form, left far fewer recorded workarounds under the same false alarm: across 26 matched pairs of teams, a recipe appeared only under the command guard in 16 pairs and only under the data guard in 1. That guard blocked agents more often, though. And one sentence in the prompt that merges the notes, asking agents not to keep instructions for getting around the guard, cut recipes from 11 to 3 of 13 teams and reader success from 42% to 10%.

What this means for AI safety

  • A fix is also a memory event. When you repair a false positive, check what your agents have written about it. The fix removes the block; only an edit removes the workaround from memory.
  • False positives are a security number. A false alarm is not just a wasted retry. The retry is where the workaround is found, and the notes are where it stays.
  • Judge memory by what a reader can do with it. Rerunning the commands in these notes found almost nothing. A reader found 47%.
  • Don't count on a cautious reader. A safety-trained model declined to use the notes, but anyone who wants to misuse them can pick another model.

References

  1. 1. John Hughes (2026). How we built Claude Code auto mode: a safer way to skip permissions. Anthropic Engineering blog.
  2. 2. Yuval Fischer (2026). How Claude Code Breaks Its Own Guardrails In The Wild. Dash Security blog.
  3. 3. David Schmotz, Derck Prinzhorn, Luca Beurer-Kellner, Anselm Paulus, Ameya Prabhu and Maksym Andriushchenko (2026). Instrumental Monitor Evasion Emerges Under Ordinary Task Pressure. arXiv:2609.30217.
  4. 4. Ryan Greenblatt, Buck Shlegeris, Kshitij Sachan and Fabien Roger (2024). AI Control: Improving Safety Despite Intentional Subversion. International Conference on Machine Learning (ICML).

Nazmus Sakib

Lecturer, Computer Science and Engineering, BRAC University

11 min read

Luck is not skill: giving every rollout the same bad luck

Group-relative RL trains an LLM agent by comparing attempts at the same task, but a tool outage or a grading slip can make luck look like skill. I gave every attempt in a group the same luck. Comparisons got cleaner and gradient noise dropped at every checkpoint, and under tool faults the agent finished five points ahead, but the learning-curve tests I registered in advance did not pass.

10 min read

Audit the group, not the patch: full-suite rewards for coding agents at a fraction of the cost

Coding agents learn from test results, but training pipelines often run only the tests the original fix touched. In repositories with large test suites, about 1 in 22 patches that pass those tests break code elsewhere, and smarter test selection misses many of them. Randomly auditing whole groups of attempts recovers the full suite’s verdict without bias, at under a quarter of the test time.

8 min read

Seen but not bound: AI agents treat a safety rule as a dial, not a gate

We gave LLM operations agents one rule: ask before doing anything destructive in production. They spotted production and judged the risk. Then they turned their caution up on every ticket instead of stopping where the rule applies.

All posts