# Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach

Authors: Jonas Geiping, Sean McLeish, Neel Jain, John Kirchenbauer, Siddharth Singh, Brian R. Bartoldson, Bhavya Kailkhura, Abhinav Bhatele, Tom Goldstein.

Pretrains a depth-recurrent language model whose computation can be extended at inference time, separating test-time compute from generated token length.

First public: 2025-02-07.
ArXiv initial submission: 2025-02-07.

[Original paper](https://arxiv.org/abs/2502.05171) · [Reading-list entry](https://latent-implicit-thinking.github.io/papers/2502.05171/)
[Preprint BibTeX](https://latent-implicit-thinking.github.io/papers/2502.05171/citation.bib) · [Preprint CSL-JSON](https://latent-implicit-thinking.github.io/papers/2502.05171/citation.csl.json) · [JSON record](https://latent-implicit-thinking.github.io/papers/2502.05171/record.json)
[Code](https://github.com/seal-rg/recurrent-pretraining)

## Method comparisons

**Recurrent depth and fixed points**

Recurrent depth and fixed-point models repeat computation in different ways. Geiping and colleagues reuse a transformer block, allowing the number of passes to change at inference. The implicit state-space models studied by Schöne and colleagues instead iterate toward a fixed point, with experiments testing when approximate convergence is sufficient. Both add computation without adding a written reasoning trace, but their stopping rules and training costs differ.

Sources: [Recurrent-depth paper](https://proceedings.neurips.cc/paper_files/paper/2025/hash/3b01972cf31e6fa0fe29e4b8b5c2a0a1-Abstract-Conference.html) · [Implicit state-space paper](https://proceedings.mlr.press/v267/schone25a.html)

Paper details: [Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach](https://latent-implicit-thinking.github.io/papers/2502.05171/) · [Implicit Language Models are RNNs: Balancing Parallelization and Expressivity](https://latent-implicit-thinking.github.io/papers/2502.07827/)


## Published version

Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach. Jonas Geiping, Sean McLeish, Neel Jain, John Kirchenbauer, Siddharth Singh, Brian Bartoldson, Bhavya Kailkhura, Abhinav Bhatele, Tom Goldstein. Advances in Neural Information Processing Systems, 2025.
[Publication](https://proceedings.neurips.cc/paper_files/paper/2025/hash/3b01972cf31e6fa0fe29e4b8b5c2a0a1-Abstract-Conference.html) · [BibTeX](https://latent-implicit-thinking.github.io/papers/2502.05171/publication.bib) · [CSL-JSON](https://latent-implicit-thinking.github.io/papers/2502.05171/publication.csl.json)
Metadata checked 2026-10-18 against [the publication record](https://proceedings.neurips.cc/paper_files/paper/2025/hash/3b01972cf31e6fa0fe29e4b8b5c2a0a1-Abstract-Conference.html).

## Approaches

- [Recurrent reasoning](https://latent-implicit-thinking.github.io/topics/recurrent-reasoning/): Repeat parts of the model or keep intermediate information in memory while working through a problem. Some methods also learn how many passes to run.

Summary and categories are editorial. Read the original paper to verify claims and limitations. A citation to a paper does not, by itself, establish that a later method builds on it.

Reading-list version 0.9.5; literature checked through 2026-10-18.