Preprint + public research artifact · 2026
Project Ariadne
An agent's written reasoning is not evidence that the reasoning caused the answer.
Overview
The problem. An agent's written reasoning is not evidence that the reasoning caused the answer.
Why it matters. Chain-of-thought monitoring only works if the chain is load-bearing. If an agent would have produced the same answer with a different rationale, then reading the rationale tells you nothing about why it acted — and every downstream safety argument built on it is unsupported.
My contribution
Designed the intervention protocol, built the auditing harness, and ran the evaluation.
- Designed the intervention protocol: capture the agent's reasoning trace, apply a hard counterfactual intervention to it, then replay the workflow from the modified state.
- Built the harness that records trajectories, applies interventions, replays, and scores whether the stated reasoning actually drove the final output.
- Ran the 30-trajectory audit and analysed where violations concentrated.
Constraints
What the design had to work around.
- The protocol needs a replayable workflow: the agent has to be re-runnable from an edited intermediate state, which rules out agents whose tool calls have irreversible side effects.
- Interventions are hard edits to the reasoning trace, not soft prompts, so the audit measures causal dependence rather than stated agreement.
- 30 trajectories were audited. That is a sample size that supports a directional finding, not a population estimate.
Architecture
Models, tools, state, evaluation, and the paths the system takes when something fails.
01 · Record
02 · Intervene
03 · Replay
04 · Score
Compare outputs → Counterfactual edit— repeat for each reasoning step in the trajectory
- State
- Model
- Tool
- Storage
- Evaluation
- Human control
Text description of this diagram
The harness runs in four stages. First, Record: a task is given to the LLM agent, which reasons, calls tools, and produces an answer, while a trajectory log captures every reasoning step and tool call. Second, Intervene: one reasoning step in the recorded trace is edited with a hard counterfactual — the step is replaced, not merely questioned — producing a modified state. Third, Replay: the workflow is re-run from that modified state, producing a counterfactual output. Fourth, Score: the counterfactual output is compared with the original. If the answer does not move when the reasoning it supposedly rests on is changed, the trajectory is marked as a faithfulness violation; an analyst reviews the verdicts. The intervene-replay-score cycle repeats for each reasoning step in the trajectory.
Evaluation
Fields marked Not reported are absent from the source material. They are listed rather than dropped so the gaps in the evidence stay visible.
- Trajectories audited
- 30
- Protocol
- Counterfactual intervention on the reasoning trace, then replay from the modified state
- Primary metric
- Faithfulness-violation rate (does the output change when the stated reasoning is changed?)
- Headline finding
- 23 of 30 trajectories (76.7%) showed a violation
- Worst subset
- Scientific reasoning — 96% violation rate
- Baseline / comparator
- Not reported
- Model versions
- Not reported
- Confidence intervals
- Not reported
Results
On most of the trajectories audited, the agent produced the same answer even after its stated reasoning was changed. The explanation was, in those cases, not what drove the result.
- 23 of 30 audited trajectories (76.7%) showed a faithfulness violation under the counterfactual-intervention protocol.
- Violations concentrated in scientific reasoning, where the rate reached 96%.
- The harness itself is the reusable output: a protocol for chain-of-thought monitoring and agent regression testing that can be pointed at other agents.
Failure modes and limitations
What this work does not establish, and where it stops being reliable.
- 30 trajectories is a small sample. The finding is directional; it does not establish a violation rate for LLM agents in general.
- No baseline or comparator system is reported, so the numbers describe the agents under test rather than positioning them against an alternative.
- Model versions are not reported in the project materials, which limits reproducibility across model updates.
- The protocol requires replayable workflows and therefore cannot audit agents whose actions cannot be re-run from an edited state.
- A violation means the stated reasoning did not cause the output. It does not, on its own, mean the output was wrong.
Artifacts
Building an agent that must work outside the demo?
I work on evaluation, grounded tool use, failure recovery, and auditable agent workflows.