First-hand information for everyone
The team shows that listening to an idealized, injection‑invariant oracle during training lets models learn a safer response plan, called the SecPO loss. Replacing the traditional DPO regularizer with a KL divergence to the oracle encourages the model to ignore harmful prompts.
To harden the model, they combine SecPO with efficient adversarial training (AT). AT uses a search‑based method (PAIR) to generate strong prompt injections that the model then learns to reject. The resulting security recipe was tested on three adaptive attacks.
When applied to a 27‑billion‑parameter Qwen model, the secure version achieved an agentic attack‑success rate of 8.8 % versus 78.8 % for the prior Meta‑SecAlign baseline. For Llama‑3.1‑8B‑Instruct, the average adaptive ASR fell from about 89 % to 10 % – an 8.5‑fold reduction – while benchmarks for tool‑calling, instruction following, science, and math stayed near the undefended model.
| Metric | Undef. | [14] | Ours |
|---|---|---|---|
| MMLU-Pro (↑) | 46.5% | 46.7% | 46.3% |
| GPQA Diamond (↑) | 23.7% | 24.2% | 24.2% |
| GSM8K (↑) | 82.9% | 84.4% | 81.3% |
| Minerva Math (↑) | 49.2% | 48.8% | 50.1% |
| AlpacaEval2 (↑) | 23.7% | 24.0% | 23.7% |
| SEP Utility (↑) | 55.4% | 55.9% | 59.4% |
| SEP ASR (↓) | 98.1% | 1.8% | 1.4% |
| Genetic Adpt. ASR@800 (↓) | 100.0% | 89.1% | 12.3% |
| TAP Adpt. ASR@800 (↓) | 100.0% | 84.6% | 12.2% |
| PISmith Adpt. ASR@10 (↓) | 98.7% | 95.1% | 7.0% |
These figures illustrate that SecPO + AT delivers strong protection against unseen prompt‑injection attacks without harming everyday use.