# UpLoop: Training Looped Transformers via Upcycling at No Extra Pretraining Compute

Authors: Cheng Liang, Tengfei Zhang, Ya Zhang, Yanfeng Wang, Zhouhan Lin.

Pretrains a shallow transformer, grows it by copying layers and optimizer states, then adds shared-weight repetitions of its middle block. Uses the compute saved early in training to match a dense baseline in parameters, training tokens, and total training FLOPs.

First public: 2026-10-30.
ArXiv initial submission: 2026-10-30.

[Original paper](https://arxiv.org/abs/2610.41720) · [PDF](https://arxiv.org/pdf/2610.41720) · [HTML](https://arxiv.org/html/2610.41720v1) · [Reading-list entry](https://latent-implicit-thinking.github.io/papers/2610.41720/)
[Preprint BibTeX](https://latent-implicit-thinking.github.io/papers/2610.41720/citation.bib) · [Preprint CSL-JSON](https://latent-implicit-thinking.github.io/papers/2610.41720/citation.csl.json) · [JSON record](https://latent-implicit-thinking.github.io/papers/2610.41720/record.json)

## Approaches

- [Recurrent reasoning](https://latent-implicit-thinking.github.io/topics/recurrent-reasoning/): Repeat parts of the model or keep intermediate information in memory while working through a problem. Some methods also learn how many passes to run.

Reading-list version 0.12.0; literature checked through 2026-11-02.