Expediting and Elevating Large Language Model Reasoning via Hidden Chain-of-Thought Decoding
Trains one model to compress written reasoning into special-token representations, then trains another to answer using those representations.
Continuous-state reasoning
Use vectors—lists of numbers—to carry intermediate reasoning from one step to the next, instead of selecting a text token at every step. Some methods use the model's hidden states; others combine several candidate token representations.
For example: A model feeds its current hidden-state vector back as the next input. It repeats this several times before returning to text to produce an answer.
How the methods connect and differ
Implicit CoT: proposal and implementation
Deng et al. (2023) proposed implicit chain-of-thought reasoning: language models carry out intermediate reasoning in continuous hidden states rather than generate it as text. Section 2.3 defines this setting; section 3 develops a distillation method to implement it. That method transfers a teacher's reasoning into hidden states across layers. Reasoning through continuous hidden states is the broader proposal; it does not require the particular distillation procedure used in this paper. Its across-layer computation differs from the repeated hidden-state feedback used by later methods such as Coconut.
Sources: Implicit CoT proposal, §2.3 · Distillation implementation, §3 · Coconut, §3
Paper details: Implicit Chain of Thought Reasoning via Knowledge Distillation · Training Large Language Models to Reason in a Continuous Latent Space
From stepwise internalization to Coconut
Coconut explicitly credits Deng et al. (2024) as the inspiration for its multistage training curriculum. Stepwise internalization gradually removes written CoT tokens until the model answers directly. Coconut instead replaces removed steps with continuous hidden-state inputs, retaining a sequence of internal reasoning steps before the answer. It also follows the earlier work in resetting the optimizer between training stages.
Sources: Deng et al. (2024), §3 · Coconut, §3
Paper details: From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step · Training Large Language Models to Reason in a Continuous Latent Space
Coconut and CODI: similar computation, different training
Coconut and CODI both pass continuous states between reasoning steps, but supervise them differently. Coconut gradually replaces written steps with hidden-state inputs and trains on the remaining text. CODI jointly trains explicit and implicit reasoning tasks, matching their hidden states at a selected position before the answer. These models avoid decoding a word at every intermediate step; they still perform successive model passes, so fewer written tokens does not mean no reasoning-time computation.
Sources: Coconut, §3 · CODI, §3
Paper details: Training Large Language Models to Reason in a Continuous Latent Space · CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation
71 papers, oldest first
Dates usually refer to the first arXiv submission. Search or filter these papers →
Latent Space Chain-of-Embedding Enables Output-free LLM Self-Evaluation
Uses the sequence of hidden states produced during inference to estimate whether an answer is correct, without extra training or labeled correctness examples.
Training Large Language Models to Reason in a Continuous Latent Space
Feeds a model's last hidden state back as its next input, enabling recurrent reasoning in continuous space before returning to language.
Compressed Chain of Thought: Efficient Reasoning Through Dense Representations
Compresses verbose reasoning traces into shorter dense representations and evaluates the trade-off between reasoning quality and generated length.
Efficient Reasoning with Hidden Thinking
Compresses visible reasoning into hidden thinking states to reduce generated reasoning length while preserving task performance.
Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning
Interleaves latent and textual tokens so a model can combine compressed internal computation with readable reasoning steps.
LLM Pretraining with Continuous Concepts
Interleaves continuous concept representations with text during pretraining so language models learn to compute over concepts as well as tokens.
LightThinker: Thinking Step-by-Step Compression
Compresses segments of a generated reasoning chain into dense states while retaining a stepwise inference process.
CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation
Uses self-distillation to align continuous internal reasoning states with explicit chain-of-thought behavior while reducing visible reasoning tokens.
Seek in the Dark: Reasoning via Test-Time Instance-Level Policy Gradient in Latent Space
Optimizes latent states for individual problems at test time using policy gradients rather than searching over written reasoning traces.
Text Generation Beyond Discrete Token Sampling
Feeds posterior expectations back into a language model so intermediate generation can remain continuous instead of sampling discrete tokens.
Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chains
Dynamically compresses portions of a reasoning chain into latent states according to the problem and intermediate context.
Hybrid Latent Reasoning via Reinforcement Learning
Uses reinforcement learning to choose and train a mixture of latent and written reasoning steps.
System-1.5 Reasoning: Traversal in Language and Latent Spaces with Dynamic Shortcuts
Learns dynamic shortcuts between language and latent states to skip portions of an otherwise explicit reasoning trajectory.
PonderLM: Pretraining Language Models to Ponder in Continuous Space
Pretrains language models with repeated continuous-space pondering steps before token prediction.
Continuous Chain of Thought Enables Parallel Exploration and Reasoning
Analyzes and demonstrates how continuous intermediate states can explore several candidate reasoning paths in parallel.
DART: Distilling Autoregressive Reasoning to Silent Thought
Distills an autoregressive textual reasoner into a model that carries the intermediate process through silent internal states.
Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens
Uses latent visual tokens as an internal imagery channel for multimodal reasoning without rendering every intermediate state.
Parallel Continuous Chain-of-Thought with Jacobi Iteration
Applies Jacobi-style iteration to update continuous reasoning states in parallel rather than strictly one step at a time.
SynAdapt: Learning Adaptive Reasoning in Large Language Models via Synthetic Continuous Chain-of-Thought
Uses synthetic continuous reasoning trajectories to train language models to allocate internal reasoning adaptively.
LLMs are Single-threaded Reasoners: Demystifying the Working Mechanism of Soft Thinking
Probes soft-token reasoning and finds that tested models often follow the highest-probability component rather than exploring several paths at once. Introduces stochastic soft thinking, including Gumbel-Softmax sampling, to reduce this greedy behavior.
SIM-CoT: Supervised Implicit Chain-of-Thought
Trains continuous intermediate states with explicit supervision so reasoning can proceed without emitting a full textual trace.
LaDiR: Latent Diffusion Enhances LLMs for Text Reasoning
Uses a variational autoencoder to compress textual reasoning steps into blocks of continuous states. A diffusion model repeatedly revises each block before generating the next, then produces a textual answer; diversity guidance encourages alternative reasoning trajectories.
LLM Latent Reasoning as Chain of Superposition
Models latent reasoning as a sequence of superposed token distributions and trains the process with supervised fine-tuning.
SemCoT: Accelerating Chain-of-Thought Reasoning through Semantically-Aligned Implicit Tokens
Compresses textual reasoning into semantically aligned implicit tokens intended to preserve useful intermediate computation with fewer generated steps.
Learning When to Stop: Adaptive Latent Reasoning via Reinforcement Learning
Learns a stopping policy for continuous latent reasoning so the model can allocate a variable number of internal steps.
Do Latent Tokens Think? A Causal and Adversarial Analysis of Chain-of-Continuous-Thought
Tests COCONUT with latent-state interventions and deliberately biased examples on MMLU and HotpotQA. Finds weak sensitivity to steering and dependence on dataset shortcuts in the evaluated models, challenging whether their intermediate states faithfully carry the reasoning.
Latent Chain-of-Thought as Planning: Decoupling Reasoning from Verbalization
Separates an internal planning process from verbalization and studies whether the latent plan supports effective downstream reasoning.
Beyond Imitation: Reinforcement Learning for Active Latent Planning
Uses reinforcement learning to train an active latent-planning process rather than only imitating supervised reasoning traces.
Do Latent-CoT Models Think Step-by-Step? A Mechanistic Study on Sequential Reasoning Tasks
Uses mechanistic interventions on sequential tasks to test whether latent chain-of-thought models actually execute stepwise internal procedures.
Capabilities and Fundamental Limits of Latent Chain-of-Thought
Characterizes tasks that latent chain-of-thought can express and identifies theoretical and empirical limits of the approach.
Latent Reasoning with Supervised Thinking States
Supervises intermediate thinking states that bridge textual reasoning traces and latent computation.
ThinkRouter: Efficient Reasoning via Routing Thinking between Latent and Discrete Spaces
Routes intermediate computation between latent and discrete representations to trade off reasoning quality, readability, and token cost.
Unlocking the Black Box of Latent Reasoning: An Interpretability-Guided Approach to Intervention
Uses probes, ablations, and targeted edits to study how continuous reasoning states affect answers. Builds interventions from learned semantic mappings and geometric constraints, improving the evaluated models by editing their latent states without updating base-model weights.
Observable Patterns Are Not Explanations: A Causal-Geometric Analysis of Latent Reasoning Models
Compares Coconut and CODI with controls lacking their proposed reasoning recurrence or curriculum. Finds that decodable patterns can also appear in controls, while interventions reveal varying causal use of latent states concentrated in low-dimensional directions.
Demystifying Hidden-State Recurrence: Switchable Latent Reasoning with On-Policy Reinforcement Learning
Trains switchable hidden-state recurrence with on-policy reinforcement learning and analyzes how the internal loop contributes to reasoning.
What Makes Effective Supervision in Latent Chain-of-Thought: An Information-Theoretic Analysis
Uses information-theoretic analysis to study which supervision signals make latent chain-of-thought training effective.
DiscoLoop: Looping Discrete Embeddings and Continuous Hidden States for Multi-hop Reasoning
Feeds both token embeddings and hidden states back through repeated transformer layers to connect successive steps of multi-hop reasoning.
Think in Latent, Explain in Language: Self-Explainable Latent Reasoning
Jointly trains latent states for answer prediction and for decoding back into readable reasoning, using one model rather than a separate post-hoc explanation decoder.
Final Checkpoints Are Not Enough: Analyzing Latent Reasoning Faithfulness Along Training Trajectories
Tests how answers respond to changed inputs and altered hidden reasoning states throughout training. These responses can change even while accuracy remains high.
Training Continuous Chain of Thought Models: A Tale of Two Regimes
Introduces direct latent supervision through averaged trace embeddings and finds that both direct and indirect continuous-CoT training degrade sharply on tasks with substantially longer reasoning traces.
GradCuit: Credit-Assigned Gradient Flow Enables Robust and Interpretable Test-Time Latent Reasoning
At test time, adjusts extra hidden states inside the transformer using feedback on generated answers, while keeping the model’s weights fixed.
Does Out-of-Sight Equal Out-of-Mind in CoT Monitorability?
Compares monitoring from explicit traces, latent-state probes, and latent verbalizations, finding that monitorability depends strongly on task structure and level of internal access.
Latent Recurrent Thoughts: Recurrent Refinement of Proposed Latents for Reasoning with Frozen LLMs
Uses a task-specific proposer and small recurrent auxiliary network to iteratively refine continuous thoughts that a frozen language model then decodes.
Structural Process Supervision for Latent Chain-of-Thought Reasoning
Aligns continuous thoughts with written reasoning through shared, learnable prototypes, allowing several text steps to supervise several latent states. A positional alignment constraint is gradually relaxed during training.
Register Tokens for Bounded-State Reasoning in Diffusion Language Models
Trains a diffusion language model to carry a fixed number of continuous register states between generated chunks. The model still writes text within each chunk, but clears earlier text and continues from the prompt and retained registers.
Learn Your Own Thoughts: Abstract Token Curriculum
Trains on progressively harder problems so that a model learns continuous intermediate thoughts without explicit reasoning-step supervision. Analyzes parity learning theoretically and tests graph reachability and arithmetic.
Uni-LaDiR: Latent Diffusion Unifies Multimodal Reasoning
Encodes teacher reasoning steps from text, images, point clouds, and robot states into a shared continuous representation. A jointly trained diffusion reasoner generates these states at inference, evaluated on visual question answering and robotic manipulation.
Schema-Anchored Latent Reasoning for Semantic Parsing-Based Knowledge Base Question Answering
Aligns continuous thoughts with a codebook of knowledge-base relations and classes, using schema traces derived from correct logical forms. Feeds the aligned codes into later reasoning steps before generating an executable query.
CaLR: Causal Latent Revision for Robust Diffusion Reasoning
Uses an expert-derived causal dependency matrix to guide gradient-based revision of diffusion reasoning states. The aim is to preserve logical dependencies while revising several intermediate positions during parallel generation.
When Steering Fails in Latent Reasoning: A Latent-to-Language Transition Gap
Tests whether changing continuous reasoning states changes the model's later text output. Finds weaker steering effects than in explicit chain-of-thought, despite comparable changes to the hidden representations.
ELF-REG: Scaling Continuous Diffusion Language Models to Reasoning Tasks
Adapts continuous diffusion to mathematics and code using intermediate-feature supervision and a global representation from a frozen autoregressive teacher. It denoises continuous response representations and decodes all response tokens only at the end.
Task-Oriented Key-Layer KV Communication for Efficient Latent Multi-Agent Collaboration
Lets agents perform hidden-state reasoning and pass working memory through the key-value cache of one selected transformer layer. Reuses that layer as the entry point for subsequent reasoning updates, avoiding earlier layers and decoding only the final agent's response into text.
Next Thoughts Are Distributions: Generative Autoregressive Reasoning in the Latent Space
Adds a diffusion head that samples the next continuous thought from a distribution rather than predicting one fixed vector. Feeds each sampled thought back into an autoregressive reasoner, supporting variable-length latent traces and multiple reasoning paths.
VaME: Exploring Variational Latent Reasoning for Multimodal Embeddings
Learns a distribution over continuous reasoning trajectories before producing a retrieval embedding. A lightweight decoder provides reconstruction supervision and a semantic reward for reinforcement learning, evaluated on image, video, and visual-document retrieval.
Reasoning with Continuous Latent Diffusion
CEDR learns compact representations from several teacher layers, denoises them at different rates, and trains a smaller prompt encoder to replace the teacher at inference. It generates complete mathematical and coding solutions through continuous refinement before text decoding.
Port-Hamiltonian Latent Deliberation: Mitigating the Deliberation Drift Cliff in Test-Time Compute Scaling
Separates hidden-state updates into attracting and rotating components to reduce errors from running a recurrent reasoner for more steps. Tests a tensor-based implementation on logical and multi-hop reasoning tasks.
LatentHarness: Learning Latent Actions for Memory and Reasoning via Counterfactual Policy Distillation
Learns when to perform another internal reasoning step, recall stored evidence or intermediate states, or emit a token. Trains these choices by branching over alternative actions and measuring their effects on the next emitted token.
Hierarchical Continuous Diffusion Language Models
Keeps a continuous state throughout denoising while reading out tokens at each step and feeding them back into the next state update. Tests whether this coupling preserves dependencies during parallel generation on Sudoku, Countdown, and language modeling.
Large Language Continuous Diffusion Models
Sigma denoises learned continuous token embeddings in blocks, starting from autoregressive pretrained weights. Studies how guidance and temperature affect mathematical reasoning, coding, output diversity, and performance when fewer refinement steps are used.
Enhancing Long-Video VLM Embeddings with Query-Aware Streaming Latent Reasoning
Processes selected video clips sequentially while persistent think tokens accumulate evidence for a fixed-size retrieval embedding. Intermediate retrieval supervision and self-distillation train partial-video readouts; the output is an embedding rather than a generated reasoning trace.
What Matters for Latent Reasoning with Flow Matching
Compresses symbolic reasoning into latent codes and trains a flow model to refine a block of noise into a thought before decoding the answer. Studies training choices, self-training on verified thoughts, and tests of whether the answer actually uses those thoughts.
CALR: Continuous Anchored Latent Reasoning via Render-of-Thought Compression
Compresses images of written derivations into continuous states, training both the answers they support and the reasoning that can be recovered from them. Gradually shifts from generating latent blocks in parallel to generating them in sequence.
Noise Your Prompt: Noising Conditioning Tokens in Continuous Diffusion Language Models
Adds noise to conditioning tokens during training instead of keeping the prompt clean. Tests improvements in Sudoku and N-Queens generalization and solution diversity, while reporting that the gains do not transfer to every language-generation task.
Efficient Reasoning with Flow Language Models
Compares continuous flow and discrete diffusion models at matched sizes and denoising steps. A theoretical account and intermediate-state interventions examine whether retaining information about several candidates helps solve mazes and Sudoku before final discretization.
Think Before You Paint: Recursive Latent Reasoning for Diffusion Models
PaTh uses a small recursive network to refine learned image-state tokens inside each denoising step and steer a frozen diffusion model through adapters. It learns visual puzzles from reconstruction targets without symbolic solutions or an external verifier.
Dynamic K-Head: Difficulty-Supervised Budget Selection for Continuous Latent Reasoning
Predicts from the question how many continuous reasoning updates to perform. Evaluates total cost including latent updates, special tokens, and any written continuation rather than counting latent steps alone.
Backpropagating Verifier Rewards Through Continuous Flow Language Models
Trains a critic to score intermediate continuous token representations, enabling reward gradients through the denoising trajectory before final discrete decoding. Evaluates this post-training approach on four reasoning tasks.
INTACT: Inducing Latent Reasoning on a Frozen Backbone
Trains a small encoder to turn a frozen model's hidden states into successive blocks of continuous thoughts. After this latent prefix, the frozen model writes its remaining reasoning and answer; the encoder learns through distillation and answer-correctness rewards.
Rationale Distribution Distillation via Flow Matching for Distributional Continuous Latent Reasoning
Distills multiple written reasoning paths for each question into a flow model over fixed-length latent sequences, including mixtures of complete rationales. Generates all latent slots together, then uses a frozen language model to decode answers from sampled rationale contexts.
Does Latent Chain-of-Thought Actually Reason? A Single, Unifying Taxonomy for Latent Reasoning
Uses chain deletion, content replacement, probes, and trajectory analysis across 21 model-task configurations. Separates what continuous states encode from whether the model uses their content, finding that dependence on the chain varies with task and training recipe.