Implicit Language Models are RNNs: Balancing Parallelization and Expressivity
Connects implicit language models to recurrent computation and analyzes the trade-off between parallel training and expressive sequential depth.
First public:
Cite this paper
Published version
Implicit Language Models are RNNs: Balancing Parallelization and Expressivity
Mark Schöne, Babak Rahmani, Heiner Kremer, Fabian Falck, Hitesh Ballani, Jannes Gladrow
Proceedings of the 42nd International Conference on Machine Learning, 2025, pp. 53439-53466.
Checked 2026-10-18 against the publication record. The published title, author list or year can differ from the preprint.
Choose the version you used. The arXiv link may serve a revision; check the original paper for the exact result, assumptions and limitations you cite.
Method comparisons
Recurrent depth and fixed points
Recurrent depth and fixed-point models repeat computation in different ways. Geiping and colleagues reuse a transformer block, allowing the number of passes to change at inference. The implicit state-space models studied by Schöne and colleagues instead iterate toward a fixed point, with experiments testing when approximate convergence is sufficient. Both add computation without adding a written reasoning trace, but their stopping rules and training costs differ.
Sources: Recurrent-depth paper · Implicit state-space paper
Paper details: Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach · Implicit Language Models are RNNs: Balancing Parallelization and Expressivity
Explore this approach
Recurrent reasoning — Repeat parts of the model or keep intermediate information in memory while working through a problem. Some methods also learn how many passes to run.
Analysis and evaluation — Measure how well these methods work, investigate what their internal computation does, and identify where they fail. This includes experiments, studies of model internals, and mathematical results.
Read or reuse this entry
Plain-text Markdown · JSON record · Find in the full list
Our summary and categories are a reading aid, not the authors’ abstract. Use the original paper for research claims. Suggest a correction.