In an explicit chain of thought, a language model writes intermediate steps before its answer. Implicit chain-of-thought methods try to carry out some or all of those steps without expressing them as words. This research is also discussed as latent reasoning or latent chain-of-thought.
Yuntian Deng and colleagues first proposed implicit chain-of-thought reasoning in 2023: training language models to reason through continuous hidden states rather than write out explicit intermediate steps. The paper's distillation method is one implementation of that proposal. Their 2024 work introduced step-by-step internalization.
The aim is not simply to hide a written explanation from the user. Researchers change how a model is trained or how it computes, then test whether it can solve problems without generating the full explanation.
Suppose the question is: “Three tickets cost $8 each, plus a $2 booking fee for the order. What is the total?”
“3 × 8 = 24 for the tickets. Adding the $2 fee gives $26.”
“The total is $26.”
The shorter answer does not prove that the model reasoned internally. It could also have memorized the answer or used a shortcut. The papers below study ways to train and run models, and ways to test what actually happens inside them.
Start with any explanation that interests you. Each link leads to papers in date order.
Learn from worked solutions so that some or all of the reasoning can later happen without writing those steps. Training may use a separate teacher model or progressively remove written steps from the model being trained.
Example and papers →Give a model extra tokens for computation, such as pauses, filler symbols, or learned planning tokens. These tokens need not spell out the solution.
Example and papers →Use vectors—lists of numbers—to carry intermediate reasoning from one step to the next, instead of selecting a text token at every step. Some methods use the model's hidden states; others combine several candidate token representations.
Example and papers →Repeat parts of the model or keep intermediate information in memory while working through a problem. Some methods also learn how many passes to run.
Example and papers →Change how intermediate steps are generated, revised, or searched—for example, by refining several steps in parallel. These are related approaches: some still produce written chains of thought.
Example and papers →Measure how well these methods work, investigate what their internal computation does, and identify where they fail. This includes experiments, studies of model internals, and mathematical results.
Example and papers →A model might learn to replace written steps with vectors, then update those vectors by running the same layers several times. That paper can appear under internalized reasoning, continuous-state reasoning, and recurrent reasoning.
The list also includes related generation methods and earlier background papers. Paper titles retain the authors' terminology.
Browse all 207 papers