# Truncate or Not? A Scaling Study of Looped Transformers with Truncated Backpropagation

Authors: Xuyuan Xiong, Siyu Chen, Zihan Shao, Yue Su, Weiyang Jin, Ping Luo, Zhuoran Yang, Xihui Liu.

Measures how forward loop counts and truncated backpropagation affect language-model pretraining across model sizes and data budgets. Fits scaling laws to compare full backpropagation through fewer loops with truncated alternatives at matched compute.

First public: 2026-10-19.
ArXiv initial submission: 2026-10-19.

[Original paper](https://arxiv.org/abs/2610.26030) · [PDF](https://arxiv.org/pdf/2610.26030) · [HTML](https://arxiv.org/html/2610.26030v2) · [Reading-list entry](https://latent-implicit-thinking.github.io/papers/2610.26030/)
[Preprint BibTeX](https://latent-implicit-thinking.github.io/papers/2610.26030/citation.bib) · [Preprint CSL-JSON](https://latent-implicit-thinking.github.io/papers/2610.26030/citation.csl.json) · [JSON record](https://latent-implicit-thinking.github.io/papers/2610.26030/record.json)

## Approaches

- [Recurrent reasoning](https://latent-implicit-thinking.github.io/topics/recurrent-reasoning/): Repeat parts of the model or keep intermediate information in memory while working through a problem. Some methods also learn how many passes to run.
- [Analysis and evaluation](https://latent-implicit-thinking.github.io/topics/implicit-reasoning-evaluation/): Measure how well these methods work, investigate what their internal computation does, and identify where they fail. This includes experiments, studies of model internals, and mathematical results.

Reading-list version 0.11.0; literature checked through 2026-10-26.