Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking
Trains on ordinary text by generating written rationales before predicting what comes next. Learns from how much those rationales improve subsequent token predictions.
First public:
Cite this paper
ArXiv preprint
arXiv:2403.09629 · Initial submission: 2024-03-14
A published-version citation has not been verified for this entry. This does not mean the paper is unpublished.
Choose the version you used. The arXiv link may serve a revision; check the original paper for the exact result, assumptions and limitations you cite.
Explore this approach
Thinking tokens — Give a model extra tokens for computation, such as pauses, filler symbols, or learned planning tokens. These tokens need not spell out the solution.
Read or reuse this entry
Plain-text Markdown · JSON record · Find in the full list
Our summary and categories are a reading aid, not the authors’ abstract. Use the original paper for research claims. Suggest a correction.