# Implicit Language Models are RNNs: Balancing Parallelization and Expressivity

Authors: Mark Schöne, Babak Rahmani, Heiner Kremer, Fabian Falck, Hitesh Ballani, Jannes Gladrow.

Connects implicit language models to recurrent computation and analyzes the trade-off between parallel training and expressive sequential depth.

First public: 2025-02-10.
ArXiv initial submission: 2025-02-10.

[Original paper](https://arxiv.org/abs/2502.07827) · [Reading-list entry](https://latent-implicit-thinking.github.io/papers/2502.07827/)
[Preprint BibTeX](https://latent-implicit-thinking.github.io/papers/2502.07827/citation.bib) · [Preprint CSL-JSON](https://latent-implicit-thinking.github.io/papers/2502.07827/citation.csl.json) · [JSON record](https://latent-implicit-thinking.github.io/papers/2502.07827/record.json)

## Method comparisons

**Recurrent depth and fixed points**

Recurrent depth and fixed-point models repeat computation in different ways. Geiping and colleagues reuse a transformer block, allowing the number of passes to change at inference. The implicit state-space models studied by Schöne and colleagues instead iterate toward a fixed point, with experiments testing when approximate convergence is sufficient. Both add computation without adding a written reasoning trace, but their stopping rules and training costs differ.

Sources: [Recurrent-depth paper](https://proceedings.neurips.cc/paper_files/paper/2025/hash/3b01972cf31e6fa0fe29e4b8b5c2a0a1-Abstract-Conference.html) · [Implicit state-space paper](https://proceedings.mlr.press/v267/schone25a.html)

Paper details: [Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach](https://latent-implicit-thinking.github.io/papers/2502.05171/) · [Implicit Language Models are RNNs: Balancing Parallelization and Expressivity](https://latent-implicit-thinking.github.io/papers/2502.07827/)


## Published version

Implicit Language Models are RNNs: Balancing Parallelization and Expressivity. Mark Schöne, Babak Rahmani, Heiner Kremer, Fabian Falck, Hitesh Ballani, Jannes Gladrow. Proceedings of the 42nd International Conference on Machine Learning, 2025.
[Publication](https://proceedings.mlr.press/v267/schone25a.html) · [BibTeX](https://latent-implicit-thinking.github.io/papers/2502.07827/publication.bib) · [CSL-JSON](https://latent-implicit-thinking.github.io/papers/2502.07827/publication.csl.json)
Metadata checked 2026-10-18 against [the publication record](https://proceedings.mlr.press/v267/schone25a.html).

## Approaches

- [Recurrent reasoning](https://latent-implicit-thinking.github.io/topics/recurrent-reasoning/): Repeat parts of the model or keep intermediate information in memory while working through a problem. Some methods also learn how many passes to run.
- [Analysis and evaluation](https://latent-implicit-thinking.github.io/topics/implicit-reasoning-evaluation/): Measure how well these methods work, investigate what their internal computation does, and identify where they fail. This includes experiments, studies of model internals, and mathematical results.

Summary and categories are editorial. Read the original paper to verify claims and limitations. A citation to a paper does not, by itself, establish that a later method builds on it.

Reading-list version 0.9.5; literature checked through 2026-10-18.