# Enhancing Latent Computation in Transformers with Latent Tokens

Authors: Yuchang Sun, Yanxi Chen, Yaliang Li, Bolin Ding.

Adds trainable dummy tokens to a pretrained transformer so attention can use them for extra computation during decoding. The tokens can be inserted in different positions at inference time.

First public: 2025-05-19.
ArXiv initial submission: 2025-05-19.

[Original paper](https://arxiv.org/abs/2505.12629) · [Reading-list entry](https://latent-implicit-thinking.github.io/papers/2505.12629/)
[Preprint BibTeX](https://latent-implicit-thinking.github.io/papers/2505.12629/citation.bib) · [Preprint CSL-JSON](https://latent-implicit-thinking.github.io/papers/2505.12629/citation.csl.json) · [JSON record](https://latent-implicit-thinking.github.io/papers/2505.12629/record.json)

## Approaches

- [Thinking tokens](https://latent-implicit-thinking.github.io/topics/thinking-tokens/): Give a model extra tokens for computation, such as pauses, filler symbols, or learned planning tokens. These tokens need not spell out the solution.

Summary and categories are editorial. Read the original paper to verify claims and limitations. A citation to a paper does not, by itself, establish that a later method builds on it.

Reading-list version 0.9.5; literature checked through 2026-10-18.