Thinking Tokens for Language Modeling
Gives a recurrent language model extra hidden-state updates by inserting special tokens after each word. Tests whether these updates improve next-word prediction.
First public:
AITP presentation date; the earlier public release date of the conference abstract is unverified. Date source. ArXiv initial submission: 2024-05-14.
Cite this paper
ArXiv preprint
arXiv:2405.08644 · Initial submission: 2024-05-14
A published-version citation has not been verified for this entry. This does not mean the paper is unpublished.
Choose the version you used. The arXiv link may serve a revision; check the original paper for the exact result, assumptions and limitations you cite.
Explore this approach
Thinking tokens — Give a model extra tokens for computation, such as pauses, filler symbols, or learned planning tokens. These tokens need not spell out the solution.
Read or reuse this entry
Plain-text Markdown · JSON record · Find in the full list
Our summary and categories are a reading aid, not the authors’ abstract. Use the original paper for research claims. Suggest a correction.