# Reading Between the Dots: Decoding Hidden Computation across Filler Tokens

Authors: Kaley Brauer, Claudio Mayrink Verdun, Samuel Marks.

Studies how models use filler tokens to compute without writing intermediate steps. Reads intermediate values from hidden states and swaps cached states between examples to test their effect on answers, then evaluates an unsupervised decoder on fact retrieval, arithmetic and string tasks.

First public: 2026-07-03.
ArXiv initial submission: 2026-07-03.

[Original paper](https://arxiv.org/abs/2607.03502) · [PDF](https://arxiv.org/pdf/2607.03502) · [HTML](https://arxiv.org/html/2607.03502v2) · [Reading-list entry](https://latent-implicit-thinking.github.io/papers/2607.03502/)
[Preprint BibTeX](https://latent-implicit-thinking.github.io/papers/2607.03502/citation.bib) · [Preprint CSL-JSON](https://latent-implicit-thinking.github.io/papers/2607.03502/citation.csl.json) · [JSON record](https://latent-implicit-thinking.github.io/papers/2607.03502/record.json)
[Code](https://github.com/kaleybrauer/filler-token-reasoning)

## Approaches

- [Thinking tokens](https://latent-implicit-thinking.github.io/topics/thinking-tokens/): Give a model extra tokens for computation, such as pauses, filler symbols, or learned planning tokens. These tokens need not spell out the solution.
- [Analysis and evaluation](https://latent-implicit-thinking.github.io/topics/implicit-reasoning-evaluation/): Measure how well these methods work, investigate what their internal computation does, and identify where they fail. This includes experiments, studies of model internals, and mathematical results.

Reading-list version 0.11.0; literature checked through 2026-10-26.