First-hand information for everyone
A new controlled study on arXiv highlights a surprising vulnerability in how some large language models (LLMs) handle intermediate reasoning steps. The paper, “Prefilling the Reasoning Channel: Output-Prefix Attacks on Reasoning LLMs,” shows that a relatively simple prompt-injection trick — appending a small block of text at the start of a model’s response, called an output prefix — can dramatically amplify the effect of malicious content placed in a model’s intermediate “scratchpad” reasoning channel.
Why this matters
Modern LLMs often perform multi-step calculations, chain-of-thought reasoning, or other internal work in a scratchpad (also called a reasoning channel) before emitting a final answer. Some APIs and interfaces expose or allow editing of that intermediate text. Because an LLM conditions its later tokens on everything that came before, an attacker who can add text to the beginning of the model’s output can influence all subsequent tokens. Prior work had shown that output-prefix injection can jailbreak non-reasoning models; this study isolates and compares the effect on reasoning models specifically.
What the researchers did
The team ran a systematic, factorial experiment across 1,800 test cases drawn from the adversarial benchmark AdvBench. They tested three prefix types (static, contextual, and none) crossed with two kinds of reasoning injections (present or absent), for a total design of 3 × 2 conditions. The targets were three current-generation models in the paper: Gemini 3 Flash Preview, DeepSeek V4 Flash, and Claude Haiku 4.5. The comparison included both models where the reasoning channel is exposed to users and models where it is hidden.
Key findings — simple but striking
- Malicious content placed only in the reasoning scratchpad but not combined with an output prefix produced essentially no successful jailbreaks (about 0% attack success). - The same malicious reasoning, when paired with even a trivial output prefix, produced strikingly large effects: attack success rose to as high as 99% for some models. - Contextual prefixes (those that adapt to the ongoing interaction) were more effective than static, one-size-fits-all prefixes. - Susceptibility varied by model: some models were far more affected than others.
How to think about the mechanics (high level)
Adding an output prefix is a way of “preloading” the model’s generation with text the model treats as part of its own response. If that prefix contains or points to malicious directives, the model’s later tokens are generated under that influence. When that manipulation is paired with an editable or exposed reasoning scratchpad, the combination can steer both intermediate reasoning and the final answer.
Caveats and limits
- The study is a laboratory-style investigation: results depend on the specific models, versions, interfaces, and test-suite used. Models and APIs evolve rapidly, so these exact numbers may change as providers patch or redesign systems. - The experiments assume attackers have the ability to insert or prepend text in the output stream or to edit an exposed reasoning channel — that capability is not universally available across all deployments. - The paper intentionally analyzes the vulnerability and does not provide operational step-by-step attack recipes; it focuses on comparative, controlled measurement.
Implications and practical suggestions
For model developers and API designers - Avoid exposing editable reasoning channels to untrusted actors. If a scratchpad must be available, treat it as a private, non-editable internal state by default. - Block or canonicalize any ability for users to insert text into the assistant’s output stream before the assistant finalizes a reply. - Use integrity checks, provenance flags, or signed metadata so downstream components can detect injected content in intermediate channels. - Include adversarial testing for combined-vector attacks (reasoning manipulation plus output prefixes) as part of safety evaluations.
For product teams and deployers - Minimize interfaces that allow users to append or prepend assistant output, and scrutinize plugins or third-party integrations that modify response text. - Monitor for anomalous patterns that suggest prompt injection attempts, especially when a system exposes internal traces or logs to users.
For policymakers and risk assessors - The study is a reminder that seemingly small interface choices — whether a reasoning channel is editable, and whether output text can be prefilled — can have outsized safety consequences. Regulatory approaches and standards should consider these design details in addition to core model behavior.
Bottom line
The paper documents a clear and replicable amplification effect: when a model’s intermediate reasoning is combined with an editable or prependable output prefix, small injections can become highly effective jailbreaks. The work does not imply that all models are equally vulnerable, nor that the vulnerability is unfixable. Instead, it flags a concrete interaction between interface design and model internals that system builders can and should address through design choices, access controls, and targeted robustness testing.