TwT: Thinking without Tokens by Habitual Reasoning Distillation with Multi-Teachers' Guidance
Selects diverse worked solutions from multiple teachers, then trains a student on full solutions, compressed solutions, and finally answers alone. The intermediate compression stage uses the student’s responses to guide how a teacher shortens its explanations.
First public: · Added to this list: November 9, 2026
Explore this approach
Internalized reasoning — Learn from worked solutions so that some or all of the reasoning can later happen without writing those steps. Training may use a separate teacher model or progressively remove written steps from the model being trained.