# Let's think dot by dot: Hidden computation in transformer language models

Authors: Jacob Pfau, William Merrill, Samuel R. Bowman.

Studies whether meaningless filler-token positions can provide transformers with additional hidden computation without carrying readable reasoning content.

First public: 2024-04-24.
ArXiv initial submission: 2024-04-24.

[Original paper](https://arxiv.org/abs/2404.15758) · [Reading-list entry](https://latent-implicit-thinking.github.io/papers/2404.15758/)
[Preprint BibTeX](https://latent-implicit-thinking.github.io/papers/2404.15758/citation.bib) · [Preprint CSL-JSON](https://latent-implicit-thinking.github.io/papers/2404.15758/citation.csl.json) · [JSON record](https://latent-implicit-thinking.github.io/papers/2404.15758/record.json)
[Code](https://github.com/JacobPfau/fillerTokens)

## Approaches

- [Thinking tokens](https://latent-implicit-thinking.github.io/topics/thinking-tokens/): Give a model extra tokens for computation, such as pauses, filler symbols, or learned planning tokens. These tokens need not spell out the solution.

Summary and categories are editorial. Read the original paper to verify claims and limitations. A citation to a paper does not, by itself, establish that a later method builds on it.

Reading-list version 0.9.5; literature checked through 2026-10-18.