First-hand information for everyone
Main discovery GRADUS decouples what a task should teach from the concrete task and solution. It treats a fixed executable environment as a latent syllabus and extracts guarded behavioral operators that capture action conditions and their effects.
Method — guarded operators and composition A guarded operator is an LLM-readable description of an atomic action. It records the action’s purpose, parameters, relevant state fields, guarded-condition branches, and conditional consequences. Formally this is written g(a) = { r_a,i ⇒ q_a,i } where each guard r_a,i describes conditions and q_a,i describes corresponding effects or required behavior.
GRADUS composes these guarded operators into symbolic scenario blueprints. A global coverage ledger prioritizes underrepresented (policy, outcome) pairs during atomic generation and structural evolution. Structural kernels (CAUSAL, PARALLEL, REPETITION) and interaction kernels (indirect references, ambiguity, social pressure, reasoning) evolve scenarios and vary how requirements are surfaced.
Joint task–solution refinement loop GRADUS runs an initialization → simulation → verification loop. Initialization grounds blueprints to concrete environment states. Simulation uses parallel search and evaluators to produce candidate demonstrations. Verification replays reference and demonstration outcomes to decide acceptance and to attribute failures. Verification is used as a recovery signal: failures can prompt task repair rather than simply discarding difficult examples.
Benchmarks and scoring GRADUS was evaluated on τ2-Bench (airline, retail, telecom) and CAR-Bench (in-car voice assistant, with base, disambiguation, hallucination splits). Each task is scored by the benchmark’s executable evaluator as pass or fail. Per-split success is reported as pass@1 averaged over five independent runs; pass@1 = (1/5) ∑_{i=1}^5 I[y_i = 1].
Concrete results With off-the-shelf backbones, the GRADUS curriculum raised a 30B model from 59% to 89% on τ2-Bench and from 47% to 83% on CAR-Bench when using an Opus 4.5 proposer. Using an open-weight Qwen3-235B proposer, the same 30B backbone reached 76% on τ2-Bench and 62% on CAR-Bench.
Under matched controls (Opus 4.5 proposer, Qwen3-8B student, fixed data budget), GRADUS exceeded three controlled synthesis baselines on every split. Compared with the strongest baseline, success increased from 59% to 71% on airline, from 71% to 92% on telecom, and from 52% to 74% on CAR hallucination.
Experimental setup Curriculum construction and proposer roles used either Claude Opus 4.5 or Qwen3-235B. Students included Qwen3-8B, Qwen3-14B, and Qwen3-30B-A3B. Shallow lessons (evolution depths 1–3) were routed primarily to supervised fine-tuning (SFT). Deeper lessons (depths 4–5) were routed primarily to reinforcement learning (RL). RL used Group Relative Policy Optimization (GRPO) with the executable acceptance signal from construction as the reward.
Ablations and coverage effects Removing guarded operators, the global coverage ledger, structural evolution, or interaction variation reduced performance. Removing verification caused a substantial drop. Adding parallel solution search, evaluator-guided reflection, and task recovery increased performance and helped preserve difficult examples.
Coverage-guided search discovered roughly 1.4×–2.9× more distinct (policy, outcome) combinations than baseline methods. Structural evolution expanded action-combination coverage by up to about 5× on airline and about 1.5× on retail. Baseline coverage tended to plateau, while GRADUS continued discovering new combinations.
Supported caveats and limitations Proposer strength matters. The two proposers coincided on retail (policy, outcome) coverage but differed by about 1.7× on action combinations. This suggests that composing unseen action sets depends more on proposer capability than on enumerating policy branches.
GRADUS is currently focused on bounded environments with explicit policies and executable outcomes. Extending the approach beyond such environments is identified as important future work.