# Beyond the Training Horizon: Mechanisms and Limits of Length Generalization in Looped Transformers

Authors: Jia Liang, Xi Jin, Liangming Pan.

Compares how two looped-transformer training schemes solve multi-step tasks beyond their training length. Uses hidden-state interventions to identify how intermediate information moves through each model and where that process breaks down.

First public: 2026-09-27.
ArXiv initial submission: 2026-09-27.

[Original paper](https://arxiv.org/abs/2609.33144) · [Reading-list entry](https://latent-implicit-thinking.github.io/papers/2609.33144/)
[Preprint BibTeX](https://latent-implicit-thinking.github.io/papers/2609.33144/citation.bib) · [Preprint CSL-JSON](https://latent-implicit-thinking.github.io/papers/2609.33144/citation.csl.json) · [JSON record](https://latent-implicit-thinking.github.io/papers/2609.33144/record.json)

## Approaches

- [Recurrent reasoning](https://latent-implicit-thinking.github.io/topics/recurrent-reasoning/): Repeat parts of the model or keep intermediate information in memory while working through a problem. Some methods also learn how many passes to run.
- [Analysis and evaluation](https://latent-implicit-thinking.github.io/topics/implicit-reasoning-evaluation/): Measure how well these methods work, investigate what their internal computation does, and identify where they fail. This includes experiments, studies of model internals, and mathematical results.

Summary and categories are editorial. Read the original paper to verify claims and limitations. A citation to a paper does not, by itself, establish that a later method builds on it.

Reading-list version 0.9.5; literature checked through 2026-10-18.