First-hand information for everyone
A new training framework, Route‑DMD, lets a student model perform generation, editing and processing with only four diffusion steps. The method divides its loss into a distribution‑matching part for generative and editing tasks and a regression part for separation, because the teacher’s distribution no longer matches the student’s noisy states during separation. The student starts from a consistency‑distilled checkpoint and the teacher rollouts use adaptive projected guidance (APG) instead of the traditional classifier‑free guidance (CFG), which can cause artifacts at high guidance scales.
When evaluated on the AuK speech foundation model, the Route‑DMD student matched or slightly outperformed the 32‑step teacher on auditory MOS, intelligibility MOS, and real‑time factor (RTF). On a single NVIDIA H2O GPU, the teacher ran at an RTF of 0.4333, while the Route‑DMD student ran at 0.0958, a 4.5× end‑to‑end speedup, and kept its MOS scores within 0.01 of the teacher. For speech‑to‑speech tasks, the student’s signal‑to‑audio‑model similarity (SIM) and word‑error rate (WER) on the SeedTTS‑Eval benchmark matched the teacher’s 32‑step results (average SIM 0.790, WER 2.85) while halving the RTF.
Route‑DMD’s performance relies on consistency distillation for initialization and on APG for teacher rollouts; without these components, the student’s separation quality degrades or inference slows. Thus, the framework shows that task‑dependent distillation can produce fast, high‑quality speech models without the need for lengthy iterative generation steps.
The results support the claim that a few‑step student can match a teacher on multiple speech tasks while providing a 4.5× speedup, but the method’s effectiveness depends on the chosen guidance strategy and initialization, and it may not directly translate to other architectures or domains.