When Internalization Fails: Finding Better Targets for Reasoning Compression
Tests segment-removal curricula, a segment-based Coconut adaptation, and shorter teacher targets on competition math. Training on the teacher’s post-think explanation gives a better accuracy–token trade-off than the tested internalization curricula.
First public: · Added to this list: November 9, 2026
Author-hosted preprint upload date. Date source.
Cite this paper
Published version
When Internalization Fails: Finding Better Targets for Reasoning Compression
Mourad Heddaya, Manley Roberts, Rohan Wadhawan, Chenhao Tan
Findings of the Association for Computational Linguistics: ACL 2026, 2026, pp. 14935-14946.
Explore this approach
Analysis and evaluation — Measure how well these methods work, investigate what their internal computation does, and identify where they fail. This includes experiments, studies of model internals, and mathematical results.
Internalized reasoning — Learn from worked solutions so that some or all of the reasoning can later happen without writing those steps. Training may use a separate teacher model or progressively remove written steps from the model being trained.