Masked Self-Distillation: Internalizing the Chain-of-Thought in Language Models
Trains a no-think student with on-policy feedback from a copy that sees a prefix of the reasoning trace. Withholding more of the trace from the student teaches it to answer with fewer or no written reasoning steps.
First public: · Added to this list: November 2, 2026
Earlier title: Masked Distillation: Internalizing the Chain-of-Thought in Language Models.
Explore this approach
Internalized reasoning — Learn from worked solutions so that some or all of the reasoning can later happen without writing those steps. Training may use a separate teacher model or progressively remove written steps from the model being trained.