Papers added to this reading list since the initial collection. Some are new research; others are older work we missed.
Literature checked through October 18, 2026 · Sort all papers by publication date · Follow by RSS
114 papers · Newest research first within this update
Trains a small drafter to predict several future tokens from early-loop hidden states. Later loops decide which speculative branches to continue, and full-depth verification determines the final output.
Uses chain deletion, content replacement, probes, and trajectory analysis across 21 model-task configurations. Separates what continuous states encode from whether the model uses their content, finding that dependence on the chain varies with task and training recipe.
Distills multiple written reasoning paths for each question into a flow model over fixed-length latent sequences, including mixtures of complete rationales. Generates all latent slots together, then uses a frozen language model to decode answers from sampled rationale contexts.
Separates computing an answer, preserving it through additional loops, and keeping its hidden state usable for later steps. On state-machine and maze tasks, shows that training can improve one ability without securing the others.
Trains a small encoder to turn a frozen model's hidden states into successive blocks of continuous thoughts. After this latent prefix, the frozen model writes its remaining reasoning and answer; the encoder learns through distillation and answer-correctness rewards.
Branches rollouts from uncertain intermediate states and reuses their shared textual prefixes. Local outcome comparisons assign credit to reasoning segments, testing whether more targeted exploration improves diffusion-model reasoning with fewer generated training tokens.
Analyzes the Continuous Thought Machine's repeated updates during image classification. Finds that its hidden states follow moving fixed points and that much of its attention schedule is shared across images.
Trains a critic to score intermediate continuous token representations, enabling reward gradients through the denoising trajectory before final discrete decoding. Evaluates this post-training approach on four reasoning tasks.
Searches over input conditions to change a recursive reasoner's update rule without retraining it. Tests whether these alternative updates can redirect a trajectory away from an incorrect solution, rather than only adding more steps or perturbing its state.
Studies how oscillations can slow convergence in recurrent reasoning models. Proposes an architecture whose rotational state changes are separated from the representation used for reasoning, and tests convergence on puzzle-solving tasks.
Searches for layers to repeat to make a stronger teacher from the current model, then distills its predictions into the original student. Repeats search and distillation as the student improves.
Inspects eight looped language models for state convergence, token decisions, readout quality, attention behavior, and sensitivity to injected noise. Finds important differences between models that receive the prompt once and those that reintroduce it at each loop.
Predicts from the question how many continuous reasoning updates to perform. Evaluates total cost including latent updates, special tokens, and any written continuation rather than counting latent steps alone.
Separately perturbs the answer state and the reasoning state of a recursive puzzle solver to study where extra search is useful. Uses the model's halting prediction to trigger restarts that resample the answer state.
Trains a language model to score labels supplied at inference in one forward pass, without generating a reasoning trace. Distills label distributions from repeated teacher reasoning and studies routing uncertain decisions back to explicit reasoning.
Studies how reducing recurrent depth changes refusal behavior in two looped language-model families. Uses probes and interventions to distinguish recognizing a harmful request from refusing it, and tests safety training across randomly selected depths.
Trains an anchor generator to choose a short text fragment and its position in a frozen diffusion model's output. These visible control tokens guide subsequent denoising, with rewards for correct answers and satisfied constraints.
Evaluates a depth-recurrent model on Sudoku-like worlds with different rule combinations. Compares symbolic rule descriptions with solved examples when the test world contains new puzzles, new combinations, or unfamiliar rule instances.
Analyzes how many extra loops a model trained on graph connectivity can safely run. Derives a finite range where added depth solves harder graphs before further loops create false connections.
Trains a looped language model at a fixed depth first, then introduces varying depths only near the end of training. Compares the resulting training cost and performance across inference depths with varying depth throughout training.
Identifies spatial-coordinate directions in multimodal model activations and tests them through causal interventions. Separating perception from viewpoint transformation reveals cases where a model represents object locations but fails to perform the required spatial computation.
ChemDiffAgent denoises interleaved textual reasoning and tool-use trajectories, letting tool decisions depend on context across the trajectory. A chemistry benchmark and matched-budget comparisons test this alternative to left-to-right agent generation.
Trains one language model to operate at different loop counts by mixing depths during training and conditioning each pass on its index and budget. A learned gate controls each recurrent update.
Adds a learned running summary of previous recurrent states so that each update can retain useful earlier work. Uses content-based weighting and temporal decay with constant aggregation memory, and tests whether puzzle-solving improves over much longer inference runs.
Compresses each completed block of generated reasoning into the key/value cache of learned pause tokens, then discards the block's original cache. Later tokens attend to the compact replacement.
Predicts by averaging over the long-run distribution of recurrent hidden states instead of reading a single final state. This lets the model use changing hidden states without requiring them to settle at one fixed point.
Evaluates harmful-query and jailbreak behavior at different recurrent depths of looped language models. Proposes depth-specific control, state sharing, and joint supervision to align safety behavior across depths.
Trains an intermediate loop using a deeper loop of the same language model as its teacher on student-generated outputs. A dynamic variant refreshes the teacher during training, turning additional recurrent computation into a source of supervision.
PaTh uses a small recursive network to refine learned image-state tokens inside each denoising step and steer a frozen diffusion model through adapters. It learns visual puzzles from reconstruction targets without symbolic solutions or an external verifier.
Compares continuous flow and discrete diffusion models at matched sizes and denoising steps. A theoretical account and intermediate-state interventions examine whether retaining information about several candidates helps solve mazes and Sudoku before final discretization.
Adds noise to conditioning tokens during training instead of keeping the prompt clean. Tests improvements in Sudoku and N-Queens generalization and solution diversity, while reporting that the gains do not transfer to every language-generation task.
Keeps separate short- and long-term matrix memories and lets a transformer update both through bidirectional reads and writes. Tests this recurrent architecture on algorithmic, in-context learning, and reasoning tasks.
Refines an internal state across loops to choose among typed answer options without generating a written reasoning trace. Supervises decisions at multiple depths and evaluates how prediction quality changes with the available computation budget.
Passes each token's final decoder state to the next token, alongside a causal encoder representation. Tests whether this growing computation path supports state tracking on sequences longer than those seen during training.
Compresses images of written derivations into continuous states, training both the answers they support and the reasoning that can be recovered from them. Gradually shifts from generating latent blocks in parallel to generating them in sequence.
Uses near-fixed-point recurrent states to reduce the cost of training, key/value caching, and prefill. Learns how to sample loop depths and changes how the input is reintroduced on each loop.
Compresses symbolic reasoning into latent codes and trains a flow model to refine a block of noise into a thought before decoding the answer. Studies training choices, self-training on verified thoughts, and tests of whether the answer actually uses those thoughts.
Processes selected video clips sequentially while persistent think tokens accumulate evidence for a fixed-size retrieval embedding. Intermediate retrieval supervision and self-distillation train partial-video readouts; the output is an embedding rather than a generated reasoning trace.
Adds a shared recurrent block between an audio encoder and language model. A question-conditioned halting mechanism chooses how many refinement passes to apply, with deeper computation learned for tasks requiring finer acoustic distinctions.
Selects token commitments that most reduce uncertainty over the remaining output. Self-distillation reinforces these pivots from successful diffusion trajectories and discourages selected pivots from failed ones, rather than supervising every token equally.
Tests how recurrent depth affects the ability to monitor a language model through its written chain of thought. Finds task-dependent changes under stress tests, while matched comparisons do not show a general monitorability disadvantage for looped architectures.
Sigma denoises learned continuous token embeddings in blocks, starting from autoregressive pretrained weights. Studies how guidance and temperature affect mathematical reasoning, coding, output diversity, and performance when fewer refinement steps are used.
Shares a long-context KV memory written by the first loop across later loops, which also keep a short local memory. Studies this architecture during pretraining and examines how shared memory changes representations and gradient flow.
Keeps a continuous state throughout denoising while reading out tokens at each step and feeding them back into the next state update. Tests whether this coupling preserves dependencies during parallel generation on Sudoku, Countdown, and language modeling.
Compares a looped model's final prediction with predictions from earlier loops to improve decoding without additional training. Tests contrastive variants using either output logits or hidden states on mathematical reasoning and code generation.
Extends integrated-gradient attribution across layers and denoising steps to examine which inputs influence a diffusion model's developing response. Includes multi-hop graph reasoning; the authors present attribution as a complement to, not a replacement for, causal intervention.
Studies how to train mixture-of-experts transformers with more than two recurrent passes. Combines stabilized residual updates, reintroduced input information, and loop-specific routing to support deeper reuse of the shared model.
Splits the recurrent residual into groups and rotates each group separately, aiming to preserve information across repeated transformer loops. Evaluates state tracking and language modeling.
Gives training recipes for looped language models both from scratch and by converting pretrained models. Uses controlled data and compute comparisons to separate the benefit of recurrence from the training setup.
Changes how outcome rewards weight decisions at different denoising states during diffusion-model reinforcement learning. Derives the effect for binary rewards and uses matched-scale controls to test whether the schedule improves mathematical accuracy and reasoning-trace diversity.
Fits scaling laws that jointly account for recurrent depth, expert sparsity, model size, and training data in looped mixture-of-experts models. Uses them to study how to allocate resources under compute and memory constraints.
Compares recursive reasoning models under a common training and evaluation setup. Isolates the effects of gradient propagation, batch size, state updates, and hierarchical structure on algorithmic tasks and generalization.
Uses graph-walk tasks to study how a looped transformer performs different computations with the same weights. Tests whether changing the state entering a loop can select its computation, using steering and activation-patching experiments.
Learns when to perform another internal reasoning step, recall stored evidence or intermediate states, or emit a token. Trains these choices by branching over alternative actions and measuring their effects on the next emitted token.
Uses a fixed, revisable output and model confidence to search over candidate solutions with a frozen fine-tuned diffusion checkpoint. Evaluates logic and scheduling problems where global constraints can make irrevocable left-to-right commitments costly.
Extends shared-module and whole-model looping to vision-language models, repeatedly updating a joint visual-language state. Trains from scratch through language and multimodal stages and evaluates visual understanding and reasoning.
Uses a shared transformer block to refine a reinforcement-learning policy's internal state before choosing an action. Learns to vary the number of loops with the current state, and studies adaptive computation in puzzles and robotic-control tasks.
Treats each recurrent hidden-state update as an optimization step and derives loop designs from this view. Introduces OperLoop, combining adaptive step sizes and weight decay.
Separates hidden-state updates into attracting and rotating components to reduce errors from running a recurrent reasoner for more steps. Tests a tensor-based implementation on logical and multi-hop reasoning tasks.
Adjusts the size of each hidden-state update based on whether recent updates make consistent progress or fluctuate. Tests the resulting scheduler with and without additional training.
Compares loop layouts and ways of passing information between loops in language models. Examines when recurrence helps reasoning and when it helps less on knowledge-dependent tasks.
Studies when simultaneously committing several predicted tokens breaks their reasoning dependencies. RPD combines stability across final layers with confidence and uncertainty in preceding positions to defer unsupported predictions while retaining parallel text generation.
Checks whether tokens committed earlier in diffusion decoding become less plausible as context fills in. CoDR remasks and regenerates those positions, testing targeted revision against extra-compute controls on reasoning and code tasks.
Builds a looped mixture-of-experts model by pooling experts across layers and revisiting them. Shares experts and routing across passes while giving each pass its own attention parameters.
Trains an iteration controller to give extra transformer loops only to tokens whose predictions benefit from them. Compares reasoning accuracy as both output length and internal computation increase.
CEDR learns compact representations from several teacher layers, denoises them at different rates, and trains a smaller prompt encoder to replace the teacher at inference. It generates complete mathematical and coding solutions through continuous refinement before text decoding.
Compares process-reward guidance with independent sampling and final-answer reranking at matched forward-pass budgets. In the tested diffusion models, early pruning removes promising candidates and intermediate-state scorers can be poor final judges; the generated solutions remain textual.
Uses shallower-loop queries with values from deeper loops to improve draft-token predictions in a looped language model. Studies the different rates at which queries, keys, and values develop, and uses that difference for self-speculative decoding.
Post-trains a looped reinforcement-learning policy to make useful decisions at several recurrent depths. Combines its original objective with distillation between neighboring depths, then uses agreement across depths to decide when to stop.
Distills a teacher's interleaved visual and textual reasoning into a student that no longer generates intermediate visual representations. The student still writes a textual chain of thought; what is internalized is the visual portion of the reasoning.
Regularizes fine-tuning of looped language models by randomly masking adapter updates at selected loops. The shared backbone still runs at every loop; the method targets adapter updates that otherwise become concentrated at particular depths.
Learns a distribution over continuous reasoning trajectories before producing a retrieval embedding. A lightweight decoder provides reconstruction supervision and a semantic reward for reinforcement learning, evaluated on image, video, and visual-document retrieval.
Adds a diffusion head that samples the next continuous thought from a distribution rather than predicting one fixed vector. Feeds each sampled thought back into an autoregressive reasoner, supporting variable-length latent traces and multiple reasoning paths.
Combines a recurrent reasoner with sparse, compositional task embeddings for ARC puzzles. Studies how task-memory size, recurrent depth, and training horizon affect performance.
Compares how two looped-transformer training schemes solve multi-step tasks beyond their training length. Uses hidden-state interventions to identify how intermediate information moves through each model and where that process breaks down.
Analyzes tasks for which a fixed-depth transformer needs an informative chain of thought to exceed a task-specific accuracy limit. Combines circuit-based results with experiments that replace, erase, or shuffle reasoning tokens, including comparisons with filler tokens.
LIRSeg generates a fixed sequence of learned special tokens before a target description and bounding box used for segmentation. Spatial alignment grounds the tokens' hidden states in relevant image regions, followed by training with segmentation rewards.
Lets agents perform hidden-state reasoning and pass working memory through the key-value cache of one selected transformer layer. Reuses that layer as the entry point for subsequent reasoning updates, avoiding earlier layers and decoding only the final agent's response into text.
Adapts continuous diffusion to mathematics and code using intermediate-feature supervision and a global representation from a frozen autoregressive teacher. It denoises continuous response representations and decodes all response tokens only at the end.
Extracts reasoning-guiding token chains and prioritizes them during diffusion-model training to improve denoising speed and accuracy. This is a related text-generating approach, not reasoning without a written trace.
Studies when a recurrent language model settles on which context tokens to attend to. Reuses that selected working set in later loops while still recomputing attention weights and hidden-state updates.
Uses the history of diffusion hidden states to detect answers that become stable but remain wrong. A planner selects targeted repair branches and verifies continuations, testing whether these internal trajectories support better repair decisions than confidence or a single snapshot.
Triggers graph retrieval when new entities become identifiable in a partially denoised draft, then uses graph propagation to provide multi-hop evidence for subsequent updates. This combines retrieval with parallel textual reasoning rather than removing the text trace.
Uses differences between early- and late-loop predictions to adjust token sampling at inference time, without retraining the model.
Uses intermediate loop depths to propose tokens that later loops verify. Arranges speculation across recurrent depths so that more of the draft and verification work can run together.
Tests whether changing continuous reasoning states changes the model's later text output. Finds weaker steering effects than in explicit chain-of-thought, despite comparable changes to the hidden representations.
Inserts a hidden-state refinement step between consecutive text tokens in a recurrent transformer. Compares this extra thinking time with adding physical layers or transformer loops under matched computation budgets.
Uses an expert-derived causal dependency matrix to guide gradient-based revision of diffusion reasoning states. The aim is to preserve logical dependencies while revising several intermediate positions during parallel generation.
Aligns continuous thoughts with a codebook of knowledge-base relations and classes, using schema traces derived from correct logical forms. Feeds the aligned codes into later reasoning steps before generating an executable query.
Encodes teacher reasoning steps from text, images, point clouds, and robot states into a shared continuous representation. A jointly trained diffusion reasoner generates these states at inference, evaluated on visual question answering and robotic manipulation.
Trains on progressively harder problems so that a model learns continuous intermediate thoughts without explicit reasoning-step supervision. Analyzes parity learning theoretically and tests graph reachability and arithmetic.
Adds a small gated memory to a looped transformer and compares it with storing the full history of recurrent states. Uses a cellular-automaton task and language modeling to test whether memory remains useful as computation grows.
Studies how growing a model during training, reusing layers, and modifying the connections between repeated blocks affect language-model scaling. Compares shared and unshared weights and operators that normalize or reintroduce earlier representations.
Drafts future tokens from early loops while later loops verify current tokens, overlapping the work without training an extra model. Final verification preserves the full model's decoding behavior.
Studies updates that initially increase support for a reference answer but overshoot when applied in full. Uses curvature along each update to explain failures and identify smaller useful steps.
Trains a diffusion language model to carry a fixed number of continuous register states between generated chunks. The model still writes text within each chunk, but clears earlier text and continues from the prompt and retained registers.
Learns a router that chooses how many recurrent passes each token receives, and keeps separate key/value caches for each pass.
Starts diffusion-model reinforcement learning with teacher-written reasoning partially filled into the output, then gradually removes that guidance. Tests whether this curriculum improves exploration on mathematics and tool-use tasks; inference still produces textual reasoning.
Aligns continuous thoughts with written reasoning through shared, learnable prototypes, allowing several text steps to supervise several latent states. A positional alignment constraint is gradually relaxed during training.
Trains recurrent reasoners against a distribution of targets near each correct solution, using an adversarial objective instead of a single point target. Evaluates generalization and stability on maze, chess, and Sudoku tasks.
Studies recurrent reasoning trajectories as dynamical systems on puzzles and mathematical logic. Links long solution times in the tested models to transient trapping near unstable states associated with nearly correct solutions.
Places pause tokens at reasoning-step boundaries and masks their training loss. Studies how this affects learning new tasks while retaining earlier capabilities, including during reinforcement learning.
Turns a diffusion denoiser into a recurrent puzzle solver by adding a persistent hidden state and removing timestep conditioning. Separates the role of denoising during training from the role of repeated updates and noise at inference.
Loops the middle half of a mixture-of-experts transformer twice while matching an unlooped baseline on computation, parameter count, and KV-cache size. Studies how the comparison changes with model scale and examines the attention patterns produced by revisiting layers.
Tests when a hierarchical reasoner should revise the internal plan guiding its lower-level computation. Compares learned revision schedules with fixed schedules, separating whether a controller responds to its state from whether that response improves decisions.
Uses state interventions, probes, and feature ablations to study how hierarchical recurrent models solve Sudoku, mazes, and ARC tasks. Compares hierarchical and single-state recurrence, and distinguishes information that can be read from a state from information that causally affects an answer.
Compares behavior under neutral, explicit-CoT, and no-CoT prompts using linguistic and internal-state measurements. Its score measures resemblance to CoT behavior, not direct evidence of a hidden reasoning trace; calibration controls expose important confounds.
Compares Coconut and CODI with controls lacking their proposed reasoning recurrence or curriculum. Finds that decodable patterns can also appear in controls, while interventions reveal varying causal use of latent states concentrated in low-dimensional directions.
Uses probes, ablations, and targeted edits to study how continuous reasoning states affect answers. Builds interventions from learned semantic mappings and geometric constraints, improving the evaluated models by editing their latent states without updating base-model weights.
Trains on synthetic traces that rank document pages and extract relevant evidence, then uses low-strength model merging to internalize the trained reasoning behavior. Compares answering with and without written reasoning on long visual documents.
Analyzes how skipping intermediate steps weakens learning signals in a simplified parity-task model. Introduces NatBool-DAG to test logical dependencies and ALiCoT, which supervises learned token states with intermediate logical labels or explicit-CoT representations.
Tests COCONUT with latent-state interventions and deliberately biased examples on MMLU and HotpotQA. Finds weak sensitivity to steering and dependence on dataset shortcuts in the evaluated models, challenging whether their intermediate states faithfully carry the reasoning.
Uses a variational autoencoder to compress textual reasoning steps into blocks of continuous states. A diffusion model repeatedly revises each block before generating the next, then produces a textual answer; diversity guidance encourages alternative reasoning trajectories.
Probes soft-token reasoning and finds that tested models often follow the highest-probability component rather than exploring several paths at once. Introduces stochastic soft thinking, including Gumbel-Softmax sampling, to reduce this greedy behavior.