# Thinking Tokens for Language Modeling

Authors: David Herel, Tomas Mikolov.

Gives a recurrent language model extra hidden-state updates by inserting special tokens after each word. Tests whether these updates improve next-word prediction.

First public: 2023-09-04.
ArXiv initial submission: 2024-05-14.

[Original paper](https://arxiv.org/abs/2405.08644) · [Reading-list entry](https://latent-implicit-thinking.github.io/papers/2405.08644/)
[Preprint BibTeX](https://latent-implicit-thinking.github.io/papers/2405.08644/citation.bib) · [Preprint CSL-JSON](https://latent-implicit-thinking.github.io/papers/2405.08644/citation.csl.json) · [JSON record](https://latent-implicit-thinking.github.io/papers/2405.08644/record.json)

Date note: AITP presentation date; the earlier public release date of the conference abstract is unverified. [Source](https://aitp-conference.org/2023/).

## Approaches

- [Thinking tokens](https://latent-implicit-thinking.github.io/topics/thinking-tokens/): Give a model extra tokens for computation, such as pauses, filler symbols, or learned planning tokens. These tokens need not spell out the solution.

Summary and categories are editorial. Read the original paper to verify claims and limitations. A citation to a paper does not, by itself, establish that a later method builds on it.

Reading-list version 0.9.5; literature checked through 2026-10-18.