HiLo-RD: Cross-Budget Self-Distillation for Masked Diffusion Language Models
Aligns a diffusion model’s low-budget hidden states with its own longer-budget trajectories. Distilling states at committed positions reduces early fixation on wrong answer tokens in math and code tasks.
First public: · Added to this list: November 2, 2026
Explore this approach
Alternative generation methods — Change how intermediate steps are generated, revised, or searched—for example, by refining several steps in parallel. Some produce written chains of thought.
Analysis and evaluation — Measure how well these methods work, investigate what their internal computation does, and identify where they fail. This includes experiments, studies of model internals, and mathematical results.