# Implicit Chain-of-Thought reading list

Implicit chain-of-thought (implicit CoT) studies intermediate reasoning that is not fully written out as natural language. This research is also discussed as latent reasoning or latent chain-of-thought.

Yuntian Deng and colleagues first proposed implicit chain-of-thought reasoning in [2023](https://arxiv.org/html/2311.01460v1#S2.SS3): training language models to reason through continuous hidden states rather than write out explicit intermediate steps. The paper's [distillation method](https://arxiv.org/html/2311.01460v1#S3) is one implementation of that proposal. Their [2024 work](https://arxiv.org/abs/2405.14838) introduced step-by-step internalization.

Version 0.9.4. Website updated 2026-10-18; literature checked through 2026-10-18. 207 papers and 8 related background papers. The list is not exhaustive.

## Sources and citation

Cite the original papers for scientific claims. The summaries below are written for this list, not copied abstracts. Papers are ordered by their earliest verified public date, usually the first arXiv submission. Earlier conference presentations have a source note.

- [JSON paper list](https://latent-implicit-thinking.github.io/data/papers.json)
- [Recent additions](https://latent-implicit-thinking.github.io/recent/): additions since the initial collection, not necessarily newly published papers
- [CSL-JSON citations](https://latent-implicit-thinking.github.io/data/papers.csl.json) and [BibTeX citations](https://latent-implicit-thinking.github.io/data/papers.bib): arXiv preprints using initial submission dates, not later proceedings; source URLs are not version-pinned
- [Introduction and examples](https://latent-implicit-thinking.github.io/topics/)
- [Downloads and file details](https://latent-implicit-thinking.github.io/data/)
- [Corrections](https://latent-implicit-thinking.github.io/contribute/)

## Topics

Papers can belong to more than one category. Each linked guide explains an approach and lists its papers, oldest first.

### [Internalized reasoning](https://latent-implicit-thinking.github.io/topics/internalized-reasoning/)

Learn from worked solutions so that some or all of the reasoning can later happen without writing those steps. Training may use a separate teacher model or progressively remove written steps from the model being trained.

For example: A teacher model solves a problem in several written steps. A student model learns to produce the answer directly, using information from the teacher's intermediate reasoning during training.

### [Thinking tokens](https://latent-implicit-thinking.github.io/topics/thinking-tokens/)

Give a model extra tokens for computation, such as pauses, filler symbols, or learned planning tokens. These tokens need not spell out the solution.

For example: A model processes several pause tokens before answering a math question. Those extra positions let it perform additional computation, even though they do not display a worked solution.

### [Continuous-state reasoning](https://latent-implicit-thinking.github.io/topics/continuous-states/)

Use vectors—lists of numbers—to carry intermediate reasoning from one step to the next, instead of selecting a text token at every step. Some methods use the model's hidden states; others combine several candidate token representations.

For example: A model feeds its current hidden-state vector back as the next input. It repeats this several times before returning to text to produce an answer.

### [Recurrent reasoning](https://latent-implicit-thinking.github.io/topics/recurrent-reasoning/)

Repeat parts of the model or keep intermediate information in memory while working through a problem. Some methods also learn how many passes to run.

For example: A model runs the same layers several times, updating the information it keeps between passes. If it has learned a stopping rule, it can decide when to produce its answer.

### [Alternative generation methods](https://latent-implicit-thinking.github.io/topics/alternative-reasoning-schedules/)

Change how intermediate steps are generated, revised, or searched—for example, by refining several steps in parallel. These are related approaches: some still produce written chains of thought.

For example: A diffusion model refines representations of a solution over several rounds, rather than committing to each word in left-to-right order. The final reasoning can still be readable text.

### [Analysis and evaluation](https://latent-implicit-thinking.github.io/topics/implicit-reasoning-evaluation/)

Measure how well these methods work, investigate what their internal computation does, and identify where they fail. This includes experiments, studies of model internals, and mathematical results.

For example: A researcher replaces an intermediate state with one taken from a different problem, then checks how the answer changes. This tests what information that state contributes to the solution.

## Papers — oldest first

### 2023-09-04 — Thinking Tokens for Language Modeling

Authors: David Herel, Tomas Mikolov.
Source: [arXiv:2405.08644](https://arxiv.org/abs/2405.08644). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2405.08644/).
Categories: Thinking tokens.
Summary: Gives a recurrent language model extra hidden-state updates by inserting special tokens after each word. Tests whether these updates improve next-word prediction.




Date note: AITP presentation date; the earlier public release date of the conference abstract is unverified. [Date source](https://aitp-conference.org/2023/). Linked arXiv version: 2024-05-14.

### 2023-10-03 — Think before you speak: Training language models with pause tokens

Authors: Sachin Goyal, Ziwei Ji, Ankit Singh Rawat, Aditya Krishna Menon, Sanjiv Kumar, Vaishnavh Nagarajan.
Source: [arXiv:2310.02226](https://arxiv.org/abs/2310.02226). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2310.02226/).
Categories: Thinking tokens.
Summary: Introduces pause tokens that give a language model additional computational positions before it must produce an answer.



**Pausing and planning**

Pause tokens add computation before the answer without choosing a different plan token for each step. Planning tokens instead select a learned token before generating each written reasoning step. The latter still produces a textual chain of thought. In the pause-token experiments, the training setup matters: gains were strongest when pauses were used in both pretraining and fine-tuning, not simply inserted into an unchanged model.

Sources: [Pause-token paper](https://proceedings.iclr.cc/paper_files/paper/2024/hash/76917808731dae9e6d62c2a7a6afb542-Abstract-Conference.html) · [Planning-token paper, §2](https://arxiv.org/html/2310.05707v3#S2)

Paper details: [Think before you speak: Training language models with pause tokens](https://latent-implicit-thinking.github.io/papers/2310.02226/) · [Guiding Language Model Reasoning with Planning Tokens](https://latent-implicit-thinking.github.io/papers/2310.05707/)



### 2023-10-09 — Guiding Language Model Reasoning with Planning Tokens

Authors: Xinyi Wang, Lucas Caccia, Oleksiy Ostapenko, Xingdi Yuan, William Yang Wang, Alessandro Sordoni.
Source: [arXiv:2310.05707](https://arxiv.org/abs/2310.05707). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2310.05707/). [Code](https://github.com/WANGXinyiLinda/planning_tokens).
Categories: Thinking tokens.
Summary: Adds a learned planning token before each written reasoning step, then uses that token to guide the step.



**Pausing and planning**

Pause tokens add computation before the answer without choosing a different plan token for each step. Planning tokens instead select a learned token before generating each written reasoning step. The latter still produces a textual chain of thought. In the pause-token experiments, the training setup matters: gains were strongest when pauses were used in both pretraining and fine-tuning, not simply inserted into an unchanged model.

Sources: [Pause-token paper](https://proceedings.iclr.cc/paper_files/paper/2024/hash/76917808731dae9e6d62c2a7a6afb542-Abstract-Conference.html) · [Planning-token paper, §2](https://arxiv.org/html/2310.05707v3#S2)

Paper details: [Think before you speak: Training language models with pause tokens](https://latent-implicit-thinking.github.io/papers/2310.02226/) · [Guiding Language Model Reasoning with Planning Tokens](https://latent-implicit-thinking.github.io/papers/2310.05707/)



### 2023-10-16 — CoTFormer: A Chain-of-Thought Driven Architecture with Budget-Adaptive Computation Cost at Inference

Authors: Amirkeivan Mohtashami, Matteo Pagliardini, Martin Jaggi.
Source: [arXiv:2310.10845](https://arxiv.org/abs/2310.10845). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2310.10845/).
Categories: Recurrent reasoning.
Summary: Runs shared transformer blocks repeatedly, keeping earlier passes available to attention. A learned router chooses how many passes each token receives.





### 2023-10-23 — Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models

Authors: Yifan Hou, Jiaoda Li, Yu Fei, Alessandro Stolfo, Wangchunshu Zhou, Guangtao Zeng, Antoine Bosselut, Mrinmaya Sachan.
Source: [arXiv:2310.14491](https://arxiv.org/abs/2310.14491). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2310.14491/).
Categories: Analysis and evaluation.
Summary: Uses mechanistic analysis to identify internal components involved in multi-step reasoning and test their causal roles.





### 2023-11-02 — Implicit Chain of Thought Reasoning via Knowledge Distillation

Authors: Yuntian Deng, Kiran Prasad, Roland Fernandez, Paul Smolensky, Vishrav Chaudhary, Stuart Shieber.
Source: [arXiv:2311.01460](https://arxiv.org/abs/2311.01460). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2311.01460/). [Code](https://github.com/da03/implicit_chain_of_thought/).
Categories: Internalized reasoning.
Summary: First proposes implicit chain-of-thought reasoning: language models carry out intermediate reasoning in continuous hidden states rather than generate it as text. Defines this setting separately from its implementation, which distills an explicit-CoT teacher's hidden states into an emulator and a student that produces the answer.
Related terms: implicit CoT; latent reasoning; latent chain-of-thought; latent CoT.


Related-method demo: [Multiplication without written steps](https://huggingface.co/spaces/yuntian-deng/gpt2-multiplication)

Compare GPT-2 models using implicit CoT, explicit CoT, or no CoT on two numbers of up to 20 digits each.

The models can make mistakes, including on valid inputs. The implicit model writes answer digits, not intermediate steps.

[Model details](https://huggingface.co/yuntian-deng/gpt2-implicit-cot-multiplication-20-digits)

Implements: [From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step](https://latent-implicit-thinking.github.io/papers/2405.14838/).

Related method, not implemented by this demo: [Implicit Chain of Thought Reasoning via Knowledge Distillation](https://latent-implicit-thinking.github.io/papers/2311.01460/).

Related-method demo: [Math word problems without written steps](https://huggingface.co/spaces/yuntian-deng/implicit-cot-math)

Try a Mistral-7B model trained with stepwise internalization on grade-school math word problems.

Answers can be wrong. This model is trained for grade-school math word problems, not general conversation. It still generates the answer token by token.

[Model details](https://huggingface.co/yuntian-deng/implicit-cot-math-mistral7b)

Implements: [From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step](https://latent-implicit-thinking.github.io/papers/2405.14838/).

Related method, not implemented by this demo: [Implicit Chain of Thought Reasoning via Knowledge Distillation](https://latent-implicit-thinking.github.io/papers/2311.01460/).


**Distillation and stepwise internalization**

Deng et al. (2023) introduced implicit CoT as reasoning through hidden states rather than written steps; distillation is the paper's implementation of that proposal. It learns to predict a teacher's intermediate hidden states, which help a student produce the answer. Stepwise Internalization instead repeatedly shortens the written solution and fine-tunes the same model on what remains. Both use worked solutions during training; neither requires writing the removed steps at inference. Their reported experiments focus on multiplication and grade-school math, not a guarantee of general reasoning ability.

Sources: [Implicit CoT proposal, §2.3](https://arxiv.org/html/2311.01460v1#S2.SS3) · [Distillation implementation, §3](https://arxiv.org/html/2311.01460v1#S3) · [Stepwise internalization, §3](https://arxiv.org/html/2405.14838v1#S3)

Paper details: [Implicit Chain of Thought Reasoning via Knowledge Distillation](https://latent-implicit-thinking.github.io/papers/2311.01460/) · [From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step](https://latent-implicit-thinking.github.io/papers/2405.14838/)

**Hidden-state distillation: the 2023 method and CODI**

The 2023 implicit CoT paper proposes reasoning through hidden states and implements it with a teacher, a thought emulator and an answer-generating student. CODI also transfers explicit reasoning through hidden-state supervision, but uses one model for both teacher and student tasks. It aligns their states just before the answer, while the student takes successive continuous reasoning steps.

Sources: [Deng et al. (2023), §2.3](https://arxiv.org/html/2311.01460v1#S2.SS3) · [Deng et al. (2023), §3](https://arxiv.org/html/2311.01460v1#S3) · [CODI, §3](https://arxiv.org/html/2502.21074v3#S3)

Paper details: [Implicit Chain of Thought Reasoning via Knowledge Distillation](https://latent-implicit-thinking.github.io/papers/2311.01460/) · [CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation](https://latent-implicit-thinking.github.io/papers/2502.21074/)

**Implicit CoT: proposal and implementation**

Deng et al. (2023) proposed implicit chain-of-thought reasoning: language models carry out intermediate reasoning in continuous hidden states rather than generate it as text. Section 2.3 defines this setting; section 3 develops a distillation method to implement it. That method transfers a teacher's reasoning into hidden states across layers. Reasoning through continuous hidden states is the broader proposal; it does not require the particular distillation procedure used in this paper. Its across-layer computation differs from the repeated hidden-state feedback used by later methods such as Coconut.

Sources: [Implicit CoT proposal, §2.3](https://arxiv.org/html/2311.01460v1#S2.SS3) · [Distillation implementation, §3](https://arxiv.org/html/2311.01460v1#S3) · [Coconut, §3](https://arxiv.org/html/2412.06769v3#S3)

Paper details: [Implicit Chain of Thought Reasoning via Knowledge Distillation](https://latent-implicit-thinking.github.io/papers/2311.01460/) · [Training Large Language Models to Reason in a Continuous Latent Space](https://latent-implicit-thinking.github.io/papers/2412.06769/)



### 2024-02-12 — Diffusion of Thoughts: Chain-of-Thought Reasoning in Diffusion Language Models

Authors: Jiacheng Ye, Shansan Gong, Liheng Chen, Lin Zheng, Jiahui Gao, Han Shi, Chuan Wu, Xin Jiang, Zhenguo Li, Wei Bi, Lingpeng Kong.
Source: [arXiv:2402.07754](https://arxiv.org/abs/2402.07754). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2402.07754/).
Categories: Alternative generation methods.
Summary: Refines intermediate reasoning representations over several rounds of denoising. A multi-pass version can revisit earlier parts of the reasoning as it refines the solution.



**Changing generation order or removing written steps**

Diffusion of Thought changes how a chain of thought is generated: it refines the solution through denoising rather than committing to words strictly from left to right. This changes the generation schedule, not necessarily the visibility of the reasoning. The resulting chain can still be text, unlike methods that only emit the final answer.

Sources: [Diffusion of Thought](https://proceedings.neurips.cc/paper_files/paper/2024/hash/be30024e7fa2c29cac7a6dafcbb8571f-Abstract-Conference.html) · [Deng et al. (2024), §3](https://arxiv.org/html/2405.14838v1#S3)

Paper details: [Diffusion of Thoughts: Chain-of-Thought Reasoning in Diffusion Language Models](https://latent-implicit-thinking.github.io/papers/2402.07754/) · [From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step](https://latent-implicit-thinking.github.io/papers/2405.14838/)



### 2024-02-19 — A Mechanistic Analysis of a Transformer Trained on a Symbolic Multi-Step Reasoning Task

Authors: Jannik Brinkmann, Abhay Sheshadri, Victor Levoso, Paul Swoboda, Christian Bartelt.
Source: [arXiv:2402.11917](https://arxiv.org/abs/2402.11917). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2402.11917/).
Categories: Analysis and evaluation.
Summary: Reverse-engineers a transformer trained on symbolic multi-step reasoning to characterize its learned internal algorithm.





### 2024-02-22 — Understanding and Patching Compositional Reasoning in LLMs

Authors: Zhaoyi Li, Gangwei Jiang, Hong Xie, Linqi Song, Defu Lian, Ying Wei.
Source: [arXiv:2402.14328](https://arxiv.org/abs/2402.14328). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2402.14328/).
Categories: Analysis and evaluation.
Summary: Localizes internal representations involved in compositional reasoning and tests whether targeted interventions can repair failures.





### 2024-02-26 — Do Large Language Models Latently Perform Multi-Hop Reasoning?

Authors: Sohee Yang, Elena Gribovskaya, Nora Kassner, Mor Geva, Sebastian Riedel.
Source: [arXiv:2402.16837](https://arxiv.org/abs/2402.16837). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2402.16837/).
Categories: Analysis and evaluation.
Summary: Probes whether language models internally compose facts across hops before producing an answer, separating internal computation from verbal explanation.



**Recovering information and testing its use**

Recovering an intermediate answer from hidden states is different from showing that the model uses it. The multi-hop study by Yang and colleagues separates recall of a bridge entity from its contribution to the final answer, including interventions on internal representations. Evidence was stronger for the first hop than for the complete two-hop computation and varied by relation type. That distinction matters when interpreting claims that a model reasons internally.

Sources: [Yang et al. (2024)](https://aclanthology.org/2024.acl-long.550/)

Paper details: [Do Large Language Models Latently Perform Multi-Hop Reasoning?](https://latent-implicit-thinking.github.io/papers/2402.16837/)



### 2024-03-14 — Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking

Authors: Eric Zelikman, Georges Harik, Yijia Shao, Varuna Jayasiri, Nick Haber, Noah D. Goodman.
Source: [arXiv:2403.09629](https://arxiv.org/abs/2403.09629). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2403.09629/).
Categories: Thinking tokens.
Summary: Trains on ordinary text by generating written rationales before predicting what comes next. Learns from how much those rationales improve subsequent token predictions.





### 2024-04-24 — Let's think dot by dot: Hidden computation in transformer language models

Authors: Jacob Pfau, William Merrill, Samuel R. Bowman.
Source: [arXiv:2404.15758](https://arxiv.org/abs/2404.15758). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2404.15758/). [Code](https://github.com/JacobPfau/fillerTokens).
Categories: Thinking tokens.
Summary: Studies whether meaningless filler-token positions can provide transformers with additional hidden computation without carrying readable reasoning content.





### 2024-05-23 — From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step

Authors: Yuntian Deng, Yejin Choi, Stuart Shieber.
Source: [arXiv:2405.14838](https://arxiv.org/abs/2405.14838). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2405.14838/). [Code](https://github.com/da03/Internalize_CoT_Step_by_Step).
Categories: Internalized reasoning.
Summary: Uses a curriculum that progressively removes intermediate reasoning steps during fine-tuning, teaching a model to internalize an explicit chain of thought.
Related terms: implicit CoT; latent reasoning; latent chain-of-thought; latent CoT.


Demo: [Multiplication without written steps](https://huggingface.co/spaces/yuntian-deng/gpt2-multiplication)

Compare GPT-2 models using implicit CoT, explicit CoT, or no CoT on two numbers of up to 20 digits each.

The models can make mistakes, including on valid inputs. The implicit model writes answer digits, not intermediate steps.

[Model details](https://huggingface.co/yuntian-deng/gpt2-implicit-cot-multiplication-20-digits)

Implements: [From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step](https://latent-implicit-thinking.github.io/papers/2405.14838/).

Related method, not implemented by this demo: [Implicit Chain of Thought Reasoning via Knowledge Distillation](https://latent-implicit-thinking.github.io/papers/2311.01460/).

Demo: [Math word problems without written steps](https://huggingface.co/spaces/yuntian-deng/implicit-cot-math)

Try a Mistral-7B model trained with stepwise internalization on grade-school math word problems.

Answers can be wrong. This model is trained for grade-school math word problems, not general conversation. It still generates the answer token by token.

[Model details](https://huggingface.co/yuntian-deng/implicit-cot-math-mistral7b)

Implements: [From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step](https://latent-implicit-thinking.github.io/papers/2405.14838/).

Related method, not implemented by this demo: [Implicit Chain of Thought Reasoning via Knowledge Distillation](https://latent-implicit-thinking.github.io/papers/2311.01460/).


**Distillation and stepwise internalization**

Deng et al. (2023) introduced implicit CoT as reasoning through hidden states rather than written steps; distillation is the paper's implementation of that proposal. It learns to predict a teacher's intermediate hidden states, which help a student produce the answer. Stepwise Internalization instead repeatedly shortens the written solution and fine-tunes the same model on what remains. Both use worked solutions during training; neither requires writing the removed steps at inference. Their reported experiments focus on multiplication and grade-school math, not a guarantee of general reasoning ability.

Sources: [Implicit CoT proposal, §2.3](https://arxiv.org/html/2311.01460v1#S2.SS3) · [Distillation implementation, §3](https://arxiv.org/html/2311.01460v1#S3) · [Stepwise internalization, §3](https://arxiv.org/html/2405.14838v1#S3)

Paper details: [Implicit Chain of Thought Reasoning via Knowledge Distillation](https://latent-implicit-thinking.github.io/papers/2311.01460/) · [From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step](https://latent-implicit-thinking.github.io/papers/2405.14838/)

**From stepwise internalization to Coconut**

Coconut explicitly credits Deng et al. (2024) as the inspiration for its multistage training curriculum. Stepwise internalization gradually removes written CoT tokens until the model answers directly. Coconut instead replaces removed steps with continuous hidden-state inputs, retaining a sequence of internal reasoning steps before the answer. It also follows the earlier work in resetting the optimizer between training stages.

Sources: [Deng et al. (2024), §3](https://arxiv.org/html/2405.14838v1#S3) · [Coconut, §3](https://arxiv.org/html/2412.06769v3#S3)

Paper details: [From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step](https://latent-implicit-thinking.github.io/papers/2405.14838/) · [Training Large Language Models to Reason in a Continuous Latent Space](https://latent-implicit-thinking.github.io/papers/2412.06769/)

**Changing generation order or removing written steps**

Diffusion of Thought changes how a chain of thought is generated: it refines the solution through denoising rather than committing to words strictly from left to right. This changes the generation schedule, not necessarily the visibility of the reasoning. The resulting chain can still be text, unlike methods that only emit the final answer.

Sources: [Diffusion of Thought](https://proceedings.neurips.cc/paper_files/paper/2024/hash/be30024e7fa2c29cac7a6dafcbb8571f-Abstract-Conference.html) · [Deng et al. (2024), §3](https://arxiv.org/html/2405.14838v1#S3)

Paper details: [Diffusion of Thoughts: Chain-of-Thought Reasoning in Diffusion Language Models](https://latent-implicit-thinking.github.io/papers/2402.07754/) · [From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step](https://latent-implicit-thinking.github.io/papers/2405.14838/)



### 2024-05-23 — Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization

Authors: Boshi Wang, Xiang Yue, Yu Su, Huan Sun.
Source: [arXiv:2405.15071](https://arxiv.org/abs/2405.15071). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2405.15071/).
Categories: Analysis and evaluation.
Summary: Mechanistically studies transformers that learn implicit reasoning after grokking, focusing on how internal algorithms support out-of-distribution generalization.





### 2024-06-19 — Distributional Reasoning in LLMs: Parallel Reasoning Processes in Multi-Hop Reasoning

Authors: Yuval Shalev, Amir Feder, Ariel Goldstein.
Source: [arXiv:2406.13858](https://arxiv.org/abs/2406.13858). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2406.13858/).
Categories: Analysis and evaluation.
Summary: Finds evidence for parallel internal processes during multi-hop reasoning rather than a single serial written-style chain.





### 2024-07-08 — Distilling System 2 into System 1

Authors: Ping Yu, Jing Xu, Jason Weston, Ilia Kulikov.
Source: [arXiv:2407.06023](https://arxiv.org/abs/2407.06023). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2407.06023/).
Categories: Internalized reasoning.
Summary: Uses outputs from a slower deliberative process as training data for a faster model, studying when System-2-like behavior can be compressed into direct prediction.





### 2024-07-29 — Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Process

Authors: Tian Ye, Zicheng Xu, Yuanzhi Li, Zeyuan Allen-Zhu.
Source: [arXiv:2407.20311](https://arxiv.org/abs/2407.20311). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2407.20311/).
Categories: Analysis and evaluation.
Summary: Studies the internal computation behind grade-school mathematics, including when models learn robust procedures or rely on data shortcuts.





### 2024-09-13 — Expediting and Elevating Large Language Model Reasoning via Hidden Chain-of-Thought Decoding

Authors: Tianqiao Liu, Zui Chen, Zitao Liu, Mi Tian, Weiqi Luo.
Source: [arXiv:2409.08561](https://arxiv.org/abs/2409.08561). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2409.08561/).
Categories: Continuous-state reasoning.
Summary: Trains one model to compress written reasoning into special-token representations, then trains another to answer using those representations.





### 2024-09-21 — Uncovering Latent Chain of Thought Vectors in Language Models

Authors: Jason Zhang, Scott Viteri.
Source: [arXiv:2409.14026](https://arxiv.org/abs/2409.14026). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2409.14026/).
Categories: Analysis and evaluation.
Summary: Searches hidden activations for directions that encode intermediate reasoning progress and tests how those directions affect behavior.





### 2024-10-13 — Dualformer: Controllable Fast and Slow Thinking by Learning with Randomized Reasoning Traces

Authors: DiJia Su, Sainbayar Sukhbaatar, Michael Rabbat, Yuandong Tian, Qinqing Zheng.
Source: [arXiv:2410.09918](https://arxiv.org/abs/2410.09918). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2410.09918/).
Categories: Internalized reasoning.
Summary: Randomizes which reasoning steps remain visible during training so one model can interpolate between direct and explicit reasoning.





### 2024-10-17 — Latent Space Chain-of-Embedding Enables Output-free LLM Self-Evaluation

Authors: Yiming Wang, Pei Zhang, Baosong Yang, Derek F. Wong, Rui Wang.
Source: [arXiv:2410.13640](https://arxiv.org/abs/2410.13640). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2410.13640/).
Categories: Continuous-state reasoning; Analysis and evaluation.
Summary: Uses the sequence of hidden states produced during inference to estimate whether an answer is correct, without extra training or labeled correctness examples.





### 2024-11-24 — Do LLMs Really Think Step-by-step In Implicit Reasoning?

Authors: Yijiong Yu.
Source: [arXiv:2411.15862](https://arxiv.org/abs/2411.15862). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2411.15862/).
Categories: Analysis and evaluation.
Summary: Tests whether successful implicit reasoning reflects an internal step-by-step procedure and documents where that interpretation fails.





### 2024-12-09 — Training Large Language Models to Reason in a Continuous Latent Space

Authors: Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, Yuandong Tian.
Source: [arXiv:2412.06769](https://arxiv.org/abs/2412.06769). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2412.06769/). [Code](https://github.com/facebookresearch/coconut).
Categories: Continuous-state reasoning.
Summary: Feeds a model's last hidden state back as its next input, enabling recurrent reasoning in continuous space before returning to language.



**Implicit CoT: proposal and implementation**

Deng et al. (2023) proposed implicit chain-of-thought reasoning: language models carry out intermediate reasoning in continuous hidden states rather than generate it as text. Section 2.3 defines this setting; section 3 develops a distillation method to implement it. That method transfers a teacher's reasoning into hidden states across layers. Reasoning through continuous hidden states is the broader proposal; it does not require the particular distillation procedure used in this paper. Its across-layer computation differs from the repeated hidden-state feedback used by later methods such as Coconut.

Sources: [Implicit CoT proposal, §2.3](https://arxiv.org/html/2311.01460v1#S2.SS3) · [Distillation implementation, §3](https://arxiv.org/html/2311.01460v1#S3) · [Coconut, §3](https://arxiv.org/html/2412.06769v3#S3)

Paper details: [Implicit Chain of Thought Reasoning via Knowledge Distillation](https://latent-implicit-thinking.github.io/papers/2311.01460/) · [Training Large Language Models to Reason in a Continuous Latent Space](https://latent-implicit-thinking.github.io/papers/2412.06769/)

**From stepwise internalization to Coconut**

Coconut explicitly credits Deng et al. (2024) as the inspiration for its multistage training curriculum. Stepwise internalization gradually removes written CoT tokens until the model answers directly. Coconut instead replaces removed steps with continuous hidden-state inputs, retaining a sequence of internal reasoning steps before the answer. It also follows the earlier work in resetting the optimizer between training stages.

Sources: [Deng et al. (2024), §3](https://arxiv.org/html/2405.14838v1#S3) · [Coconut, §3](https://arxiv.org/html/2412.06769v3#S3)

Paper details: [From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step](https://latent-implicit-thinking.github.io/papers/2405.14838/) · [Training Large Language Models to Reason in a Continuous Latent Space](https://latent-implicit-thinking.github.io/papers/2412.06769/)

**Coconut and CODI: similar computation, different training**

Coconut and CODI both pass continuous states between reasoning steps, but supervise them differently. Coconut gradually replaces written steps with hidden-state inputs and trains on the remaining text. CODI jointly trains explicit and implicit reasoning tasks, matching their hidden states at a selected position before the answer. These models avoid decoding a word at every intermediate step; they still perform successive model passes, so fewer written tokens does not mean no reasoning-time computation.

Sources: [Coconut, §3](https://arxiv.org/html/2412.06769v3#S3) · [CODI, §3](https://arxiv.org/html/2502.21074v3#S3)

Paper details: [Training Large Language Models to Reason in a Continuous Latent Space](https://latent-implicit-thinking.github.io/papers/2412.06769/) · [CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation](https://latent-implicit-thinking.github.io/papers/2502.21074/)



### 2024-12-11 — Multimodal Latent Language Modeling with Next-Token Diffusion

Authors: Yutao Sun, Hangbo Bao, Wenhui Wang, Zhiliang Peng, Li Dong, Shaohan Huang, Jianyong Wang, Furu Wei.
Source: [arXiv:2412.08635](https://arxiv.org/abs/2412.08635). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2412.08635/).
Categories: Alternative generation methods.
Summary: Encodes continuous data such as images as vectors, then generates those vectors with diffusion inside a causal transformer. This is related work on multimodal generation, rather than a method trained directly on reasoning steps.





### 2024-12-17 — Compressed Chain of Thought: Efficient Reasoning Through Dense Representations

Authors: Jeffrey Cheng, Benjamin Van Durme.
Source: [arXiv:2412.13171](https://arxiv.org/abs/2412.13171). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2412.13171/).
Categories: Continuous-state reasoning.
Summary: Compresses verbose reasoning traces into shorter dense representations and evaluates the trade-off between reasoning quality and generated length.





### 2025-01-31 — Efficient Reasoning with Hidden Thinking

Authors: Xuan Shen, Yizhou Wang, Yufa Zhou, Xiangxi Shi, Pu Zhao, Yanzhi Wang, Jiuxiang Gu.
Source: [arXiv:2501.19201](https://arxiv.org/abs/2501.19201). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2501.19201/).
Categories: Internalized reasoning; Continuous-state reasoning.
Summary: Compresses visible reasoning into hidden thinking states to reduce generated reasoning length while preserving task performance.





### 2025-02-05 — Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning

Authors: DiJia Su, Hanlin Zhu, Yingchen Xu, Jiantao Jiao, Yuandong Tian, Qinqing Zheng.
Source: [arXiv:2502.03275](https://arxiv.org/abs/2502.03275). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2502.03275/).
Categories: Thinking tokens; Continuous-state reasoning.
Summary: Interleaves latent and textual tokens so a model can combine compressed internal computation with readable reasoning steps.





### 2025-02-07 — Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach

Authors: Jonas Geiping, Sean McLeish, Neel Jain, John Kirchenbauer, Siddharth Singh, Brian R. Bartoldson, Bhavya Kailkhura, Abhinav Bhatele, Tom Goldstein.
Source: [arXiv:2502.05171](https://arxiv.org/abs/2502.05171). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2502.05171/). [Code](https://github.com/seal-rg/recurrent-pretraining).
Categories: Recurrent reasoning.
Summary: Pretrains a depth-recurrent language model whose computation can be extended at inference time, separating test-time compute from generated token length.



**Recurrent depth and fixed points**

Recurrent depth and fixed-point models repeat computation in different ways. Geiping and colleagues reuse a transformer block, allowing the number of passes to change at inference. The implicit state-space models studied by Schöne and colleagues instead iterate toward a fixed point, with experiments testing when approximate convergence is sufficient. Both add computation without adding a written reasoning trace, but their stopping rules and training costs differ.

Sources: [Recurrent-depth paper](https://proceedings.neurips.cc/paper_files/paper/2025/hash/3b01972cf31e6fa0fe29e4b8b5c2a0a1-Abstract-Conference.html) · [Implicit state-space paper](https://proceedings.mlr.press/v267/schone25a.html)

Paper details: [Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach](https://latent-implicit-thinking.github.io/papers/2502.05171/) · [Implicit Language Models are RNNs: Balancing Parallelization and Expressivity](https://latent-implicit-thinking.github.io/papers/2502.07827/)



### 2025-02-10 — Implicit Language Models are RNNs: Balancing Parallelization and Expressivity

Authors: Mark Schöne, Babak Rahmani, Heiner Kremer, Fabian Falck, Hitesh Ballani, Jannes Gladrow.
Source: [arXiv:2502.07827](https://arxiv.org/abs/2502.07827). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2502.07827/).
Categories: Recurrent reasoning; Analysis and evaluation.
Summary: Connects implicit language models to recurrent computation and analyzes the trade-off between parallel training and expressive sequential depth.



**Recurrent depth and fixed points**

Recurrent depth and fixed-point models repeat computation in different ways. Geiping and colleagues reuse a transformer block, allowing the number of passes to change at inference. The implicit state-space models studied by Schöne and colleagues instead iterate toward a fixed point, with experiments testing when approximate convergence is sufficient. Both add computation without adding a written reasoning trace, but their stopping rules and training costs differ.

Sources: [Recurrent-depth paper](https://proceedings.neurips.cc/paper_files/paper/2025/hash/3b01972cf31e6fa0fe29e4b8b5c2a0a1-Abstract-Conference.html) · [Implicit state-space paper](https://proceedings.mlr.press/v267/schone25a.html)

Paper details: [Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach](https://latent-implicit-thinking.github.io/papers/2502.05171/) · [Implicit Language Models are RNNs: Balancing Parallelization and Expressivity](https://latent-implicit-thinking.github.io/papers/2502.07827/)



### 2025-02-12 — LLM Pretraining with Continuous Concepts

Authors: Jihoon Tack, Jack Lanchantin, Jane Yu, Andrew Cohen, Ilia Kulikov, Janice Lan, Shibo Hao, Yuandong Tian, Jason Weston, Xian Li.
Source: [arXiv:2502.08524](https://arxiv.org/abs/2502.08524). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2502.08524/). [Code](https://github.com/facebookresearch/RAM/tree/main/projects/cocomix).
Categories: Continuous-state reasoning.
Summary: Interleaves continuous concept representations with text during pretraining so language models learn to compute over concepts as well as tokens.





### 2025-02-19 — Inner Thinking Transformer: Leveraging Dynamic Depth Scaling to Foster Adaptive Internal Thinking

Authors: Yilong Chen, Junyuan Shang, Zhenyu Zhang, Yanxi Xie, Jiawei Sheng, Tingwen Liu, Shuohuan Wang, Yu Sun, Hua Wu, Haifeng Wang.
Source: [arXiv:2502.13842](https://arxiv.org/abs/2502.13842). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2502.13842/).
Categories: Recurrent reasoning.
Summary: Routes selected tokens through extra computation while reusing model parameters. Marks successive thinking steps so the model can distinguish one pass from the next.





### 2025-02-21 — LightThinker: Thinking Step-by-Step Compression

Authors: Jintian Zhang, Yuqi Zhu, Mengshu Sun, Yujie Luo, Shuofei Qiao, Lun Du, Da Zheng, Huajun Chen, Ningyu Zhang.
Source: [arXiv:2502.15589](https://arxiv.org/abs/2502.15589). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2502.15589/).
Categories: Internalized reasoning; Continuous-state reasoning.
Summary: Compresses segments of a generated reasoning chain into dense states while retaining a stepwise inference process.





### 2025-02-24 — Reasoning with Latent Thoughts: On the Power of Looped Transformers

Authors: Nikunj Saunshi, Nishanth Dikkala, Zhiyuan Li, Sanjiv Kumar, Sashank J. Reddi.
Source: [arXiv:2502.17416](https://arxiv.org/abs/2502.17416). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2502.17416/).
Categories: Recurrent reasoning.
Summary: Analyzes the expressive power of transformers that repeatedly apply shared layers, treating recurrent depth as a form of internal reasoning.





### 2025-02-28 — CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation

Authors: Zhenyi Shen, Hanqi Yan, Linhai Zhang, Zhanghao Hu, Yali Du, Yulan He.
Source: [arXiv:2502.21074](https://arxiv.org/abs/2502.21074). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2502.21074/).
Categories: Continuous-state reasoning; Internalized reasoning.
Summary: Uses self-distillation to align continuous internal reasoning states with explicit chain-of-thought behavior while reducing visible reasoning tokens.



**Hidden-state distillation: the 2023 method and CODI**

The 2023 implicit CoT paper proposes reasoning through hidden states and implements it with a teacher, a thought emulator and an answer-generating student. CODI also transfers explicit reasoning through hidden-state supervision, but uses one model for both teacher and student tasks. It aligns their states just before the answer, while the student takes successive continuous reasoning steps.

Sources: [Deng et al. (2023), §2.3](https://arxiv.org/html/2311.01460v1#S2.SS3) · [Deng et al. (2023), §3](https://arxiv.org/html/2311.01460v1#S3) · [CODI, §3](https://arxiv.org/html/2502.21074v3#S3)

Paper details: [Implicit Chain of Thought Reasoning via Knowledge Distillation](https://latent-implicit-thinking.github.io/papers/2311.01460/) · [CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation](https://latent-implicit-thinking.github.io/papers/2502.21074/)

**Coconut and CODI: similar computation, different training**

Coconut and CODI both pass continuous states between reasoning steps, but supervise them differently. Coconut gradually replaces written steps with hidden-state inputs and trains on the remaining text. CODI jointly trains explicit and implicit reasoning tasks, matching their hidden states at a selected position before the answer. These models avoid decoding a word at every intermediate step; they still perform successive model passes, so fewer written tokens does not mean no reasoning-time computation.

Sources: [Coconut, §3](https://arxiv.org/html/2412.06769v3#S3) · [CODI, §3](https://arxiv.org/html/2502.21074v3#S3)

Paper details: [Training Large Language Models to Reason in a Continuous Latent Space](https://latent-implicit-thinking.github.io/papers/2412.06769/) · [CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation](https://latent-implicit-thinking.github.io/papers/2502.21074/)



### 2025-03-10 — Implicit Reasoning in Transformers is Reasoning through Shortcuts

Authors: Tianhe Lin, Jian Xie, Siyu Yuan, Deqing Yang.
Source: [arXiv:2503.07604](https://arxiv.org/abs/2503.07604). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2503.07604/).
Categories: Analysis and evaluation.
Summary: Tests whether apparent implicit multi-step reasoning reflects compositional algorithms or learned shortcuts, emphasizing failure modes under distribution shift.





### 2025-04-14 — Beyond Chains of Thought: Benchmarking Latent-Space Reasoning Abilities in Large Language Models

Authors: Thilo Hagendorff, Sarah Fabi.
Source: [arXiv:2504.10615](https://arxiv.org/abs/2504.10615). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2504.10615/).
Categories: Analysis and evaluation.
Summary: Encodes the answer in the language of the model's first response token, forcing a choice before any written explanation. Also tests whether successful answers could come from simple heuristics.





### 2025-04-21 — Efficient Pretraining Length Scaling

Authors: Bohong Wu, Shen Yan, Sijun Zhang, Jianqiao Lu, Yutao Zeng, Ya Wang, Xun Zhou.
Source: [arXiv:2504.14992](https://arxiv.org/abs/2504.14992). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2504.14992/).
Categories: Thinking tokens; Recurrent reasoning.
Summary: Adds hidden decoding tokens during pretraining, then discards their key/value cache entries after use. Keeps the original tokens' entries to preserve longer-range context.





### 2025-05-19 — Enhancing Latent Computation in Transformers with Latent Tokens

Authors: Yuchang Sun, Yanxi Chen, Yaliang Li, Bolin Ding.
Source: [arXiv:2505.12629](https://arxiv.org/abs/2505.12629). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2505.12629/).
Categories: Thinking tokens.
Summary: Adds trainable dummy tokens to a pretrained transformer so attention can use them for extra computation during decoding. The tokens can be inserted in different positions at inference time.





### 2025-05-19 — Seek in the Dark: Reasoning via Test-Time Instance-Level Policy Gradient in Latent Space

Authors: Hengli Li, Chenxi Li, Tong Wu, Xuekai Zhu, Yuxuan Wang, Zhaoxin Yu, Eric Hanchen Jiang, Song-Chun Zhu, Zixia Jia, Ying Nian Wu, Zilong Zheng.
Source: [arXiv:2505.13308](https://arxiv.org/abs/2505.13308). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2505.13308/).
Categories: Continuous-state reasoning.
Summary: Optimizes latent states for individual problems at test time using policy gradients rather than searching over written reasoning traces.





### 2025-05-20 — Do Language Models Use Their Depth Efficiently?

Authors: Róbert Csordás, Christopher D. Manning, Christopher Potts.
Source: [arXiv:2505.13898](https://arxiv.org/abs/2505.13898). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2505.13898/).
Categories: Analysis and evaluation; Recurrent reasoning.
Summary: Skips layers and compares shallow and deep models to examine what extra layers compute. In the models studied, later layers mostly make smaller refinements rather than adding new reasoning steps.





### 2025-05-20 — Internal Chain-of-Thought: Empirical Evidence for Layer-wise Subtask Scheduling in LLMs

Authors: Zhipeng Yang, Junzhuo Li, Siyu Xia, Xuming Hu.
Source: [arXiv:2505.14530](https://arxiv.org/abs/2505.14530). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2505.14530/). [Code](https://github.com/yzp11/Internal-Chain-of-Thought).
Categories: Analysis and evaluation.
Summary: Presents evidence that transformer layers schedule intermediate subtasks internally, offering a layer-wise view of implicit reasoning.





### 2025-05-20 — Text Generation Beyond Discrete Token Sampling

Authors: Yufan Zhuang, Liyuan Liu, Chandan Singh, Jingbo Shang, Jianfeng Gao.
Source: [arXiv:2505.14827](https://arxiv.org/abs/2505.14827). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2505.14827/).
Categories: Continuous-state reasoning; Alternative generation methods.
Summary: Feeds posterior expectations back into a language model so intermediate generation can remain continuous instead of sampling discrete tokens.





### 2025-05-22 — Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chains

Authors: Wenhui Tan, Jiaze Li, Jianzhong Ju, Zhenbo Luo, Ruihua Song, Jian Luan.
Source: [arXiv:2505.16552](https://arxiv.org/abs/2505.16552). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2505.16552/).
Categories: Internalized reasoning; Continuous-state reasoning.
Summary: Dynamically compresses portions of a reasoning chain into latent states according to the problem and intermediate context.





### 2025-05-22 — Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning

Authors: Xinghao Chen, Anhao Zhao, Heming Xia, Xuan Lu, Hanlin Wang, Yanjun Chen, Wei Zhang, Jian Wang, Wenjie Li, Xiaoyu Shen.
Source: [arXiv:2505.16782](https://arxiv.org/abs/2505.16782). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2505.16782/). [Code](https://github.com/EIT-NLP/Awesome-Latent-CoT).
Categories: Analysis and evaluation.
Summary: Surveys latent chain-of-thought methods and organizes a broad, continuously updated bibliography spanning token-wise, structural, representational, and analytical work.





### 2025-05-23 — Language Models Can Learn Implicit Multi-Hop Reasoning, but Only If They Have Lots of Training Data

Authors: Yuekun Yao, Yupei Du, Dawei Zhu, Michael Hahn, Alexander Koller.
Source: [arXiv:2505.17923](https://arxiv.org/abs/2505.17923). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2505.17923/).
Categories: Analysis and evaluation.
Summary: Studies the data requirements for learning implicit multi-hop reasoning and identifies sharp limitations in low-data regimes.





### 2025-05-24 — Hybrid Latent Reasoning via Reinforcement Learning

Authors: Zhenrui Yue, Bowen Jin, Huimin Zeng, Honglei Zhuang, Zhen Qin, Jinsung Yoon, Lanyu Shang, Jiawei Han, Dong Wang.
Source: [arXiv:2505.18454](https://arxiv.org/abs/2505.18454). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2505.18454/).
Categories: Continuous-state reasoning; Internalized reasoning.
Summary: Uses reinforcement learning to choose and train a mixture of latent and written reasoning steps.





### 2025-05-25 — System-1.5 Reasoning: Traversal in Language and Latent Spaces with Dynamic Shortcuts

Authors: Xiaoqiang Wang, Suyuchen Wang, Yun Zhu, Bang Liu.
Source: [arXiv:2505.18962](https://arxiv.org/abs/2505.18962). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2505.18962/).
Categories: Continuous-state reasoning; Internalized reasoning.
Summary: Learns dynamic shortcuts between language and latent states to skip portions of an otherwise explicit reasoning trajectory.





### 2025-05-25 — To CoT or To Loop? A Formal Comparison Between Chain-of-Thought and Looped Transformers

Authors: Kevin Xu, Issei Sato.
Source: [arXiv:2505.19245](https://arxiv.org/abs/2505.19245). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2505.19245/).
Categories: Analysis and evaluation; Recurrent reasoning.
Summary: Formally compares explicit chain-of-thought computation with repeated internal transformations in looped transformer architectures.





### 2025-05-27 — PonderLM: Pretraining Language Models to Ponder in Continuous Space

Authors: Boyi Zeng, Shixiang Song, Siyuan Huang, Yixuan Wang, He Li, Ziwei He, Xinbing Wang, Zhiyu Li, Zhouhan Lin.
Source: [arXiv:2505.20674](https://arxiv.org/abs/2505.20674). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2505.20674/).
Categories: Continuous-state reasoning; Recurrent reasoning.
Summary: Pretrains language models with repeated continuous-space pondering steps before token prediction.





### 2025-05-29 — Continuous Chain of Thought Enables Parallel Exploration and Reasoning

Authors: Halil Alperen Gozeten, M. Emrullah Ildiz, Xuechen Zhang, Hrayr Harutyunyan, Ankit Singh Rawat, Samet Oymak.
Source: [arXiv:2505.23648](https://arxiv.org/abs/2505.23648). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2505.23648/).
Categories: Continuous-state reasoning; Analysis and evaluation.
Summary: Analyzes and demonstrates how continuous intermediate states can explore several candidate reasoning paths in parallel.





### 2025-06-13 — DART: Distilling Autoregressive Reasoning to Silent Thought

Authors: Nan Jiang, Ziming Wu, De-Chuan Zhan, Fuming Lai, Shaobing Lian.
Source: [arXiv:2506.11752](https://arxiv.org/abs/2506.11752). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2506.11752/).
Categories: Internalized reasoning; Continuous-state reasoning.
Summary: Distills an autoregressive textual reasoner into a model that carries the intermediate process through silent internal states.





### 2025-06-20 — Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens

Authors: Zeyuan Yang, Xueyang Yu, Delin Chen, Maohao Shen, Chuang Gan.
Source: [arXiv:2506.17218](https://arxiv.org/abs/2506.17218). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2506.17218/).
Categories: Thinking tokens; Continuous-state reasoning.
Summary: Uses latent visual tokens as an internal imagery channel for multimodal reasoning without rendering every intermediate state.





### 2025-06-23 — Parallel Continuous Chain-of-Thought with Jacobi Iteration

Authors: Haoyi Wu, Zhihao Teng, Kewei Tu.
Source: [arXiv:2506.18582](https://arxiv.org/abs/2506.18582). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2506.18582/).
Categories: Continuous-state reasoning; Alternative generation methods.
Summary: Applies Jacobi-style iteration to update continuous reasoning states in parallel rather than strictly one step at a time.





### 2025-06-26 — Hierarchical Reasoning Model

Authors: Guan Wang, Jin Li, Yuhao Sun, Xing Chen, Changling Liu, Yue Wu, Meng Lu, Sen Song, Yasin Abbasi Yadkori.
Source: [arXiv:2506.21734](https://arxiv.org/abs/2506.21734). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2506.21734/). [Code](https://github.com/sapientinc/HRM).
Categories: Recurrent reasoning.
Summary: Uses coupled recurrent modules operating at different timescales to perform iterative internal computation on difficult reasoning tasks.





### 2025-07-02 — Latent Chain-of-Thought? Decoding the Depth-Recurrent Transformer

Authors: Wenquan Lu, Yuechuan Yang, Kyle Lee, Yanshu Li, Enqi Liu.
Source: [arXiv:2507.02199](https://arxiv.org/abs/2507.02199). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2507.02199/). [Code](https://github.com/wenquanlu/huginn-latent-cot).
Categories: Analysis and evaluation; Recurrent reasoning.
Summary: Probes a depth-recurrent transformer to test what its repeated hidden-state computation represents and whether it behaves like a latent chain of thought.





### 2025-07-08 — A Survey on Latent Reasoning

Authors: Rui-Jie Zhu, Tianhao Peng, Tianhao Cheng, Xingwei Qu, Jinfa Huang, Dawei Zhu, Hao Wang, Kaiwen Xue, Xuanliang Zhang, Yong Shan, Tianle Cai, Taylor Kergan, Assel Kembay, Andrew Smith, Chenghua Lin, Binh Nguyen, Yuqi Pan, Yuhong Chou, Zefan Cai, Zhenhe Wu, Yongchi Zhao, Tianyu Liu, Jian Yang, Wangchunshu Zhou, Chujie Zheng, Chongxuan Li, Yuyin Zhou, Zhoujun Li, Zhaoxiang Zhang, Jiaheng Liu, Ge Zhang, Wenhao Huang, Jason Eshraghian.
Source: [arXiv:2507.06203](https://arxiv.org/abs/2507.06203). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2507.06203/). [Code](https://github.com/multimodal-art-projection/LatentCoT-Horizon).
Categories: Analysis and evaluation.
Summary: Surveys latent-reasoning methods, evaluation, and applications and accompanies a maintained research bibliography.





### 2025-07-10 — Skip a Layer or Loop it? Test-Time Depth Adaptation of Pretrained LLMs

Authors: Ziyue Li, Yang Li, Tianyi Zhou.
Source: [arXiv:2507.07996](https://arxiv.org/abs/2507.07996). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2507.07996/).
Categories: Recurrent reasoning.
Summary: Adapts the depth of pretrained language models at test time by selectively skipping or repeating layers.





### 2025-07-14 — Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation

Authors: Sangmin Bae, Yujin Kim, Reza Bayat, Sungnyun Kim, Jiyoun Ha, Tal Schuster, Adam Fisch, Hrayr Harutyunyan, Ziwei Ji, Aaron Courville, Se-Young Yun.
Source: [arXiv:2507.10524](https://arxiv.org/abs/2507.10524). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2507.10524/). [Code](https://github.com/raymin0223/mixture_of_recursions).
Categories: Recurrent reasoning.
Summary: Learns token-level routing across recursive depths so different positions can receive different amounts of internal computation.





### 2025-08-01 — SynAdapt: Learning Adaptive Reasoning in Large Language Models via Synthetic Continuous Chain-of-Thought

Authors: Jianwei Wang, Ziming Wu, Fuming Lai, Shaobing Lian, Ziqian Zeng.
Source: [arXiv:2508.00574](https://arxiv.org/abs/2508.00574). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2508.00574/).
Categories: Continuous-state reasoning.
Summary: Uses synthetic continuous reasoning trajectories to train language models to allocate internal reasoning adaptively.





### 2025-08-05 — LLMs are Single-threaded Reasoners: Demystifying the Working Mechanism of Soft Thinking

Authors: Junhong Wu, Jinliang Lu, Zixuan Ren, Gangqiang Hu, Zhi Wu, Dai Dai, Hua Wu.
Source: [arXiv:2508.03440](https://arxiv.org/abs/2508.03440). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2508.03440/).
Categories: Analysis and evaluation; Continuous-state reasoning.
Summary: Probes soft-token reasoning and finds that tested models often follow the highest-probability component rather than exploring several paths at once. Introduces stochastic soft thinking, including Gumbel-Softmax sampling, to reduce this greedy behavior.



Added to this list: 2026-10-18.


### 2025-09-02 — Implicit Reasoning in Large Language Models: A Comprehensive Survey

Authors: Jindong Li, Yali Fu, Li Fan, Jiahong Liu, Yao Shu, Chengwei Qin, Menglin Yang, Irwin King, Rex Ying.
Source: [arXiv:2509.02350](https://arxiv.org/abs/2509.02350). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2509.02350/). [Code](https://github.com/digailab/awesome-llm-implicit-reasoning).
Categories: Analysis and evaluation.
Summary: Organizes implicit-reasoning research across model mechanisms, training methods, evaluation, and applications.





### 2025-09-24 — SIM-CoT: Supervised Implicit Chain-of-Thought

Authors: Xilin Wei, Xiaoran Liu, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Jiaqi Wang, Xipeng Qiu, Dahua Lin.
Source: [arXiv:2509.20317](https://arxiv.org/abs/2509.20317). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2509.20317/). [Code](https://github.com/InternLM/SIM-CoT).
Categories: Continuous-state reasoning; Internalized reasoning.
Summary: Trains continuous intermediate states with explicit supervision so reasoning can proceed without emitting a full textual trace.





### 2025-10-06 — LaDiR: Latent Diffusion Enhances LLMs for Text Reasoning

Authors: Haoqiang Kang, Yizhe Zhang, Nikki Lijing Kuang, Nicklas Majamaki, Navdeep Jaitly, Yi-An Ma, Lianhui Qin.
Source: [arXiv:2510.04573](https://arxiv.org/abs/2510.04573). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2510.04573/).
Categories: Continuous-state reasoning; Alternative generation methods.
Summary: Uses a variational autoencoder to compress textual reasoning steps into blocks of continuous states. A diffusion model repeatedly revises each block before generating the next, then produces a textual answer; diversity guidance encourages alternative reasoning trajectories.



Added to this list: 2026-10-18.


### 2025-10-17 — LLM Latent Reasoning as Chain of Superposition

Authors: Jingcheng Deng, Liang Pang, Zihao Wei, Shicheng Xu, Zenghao Duan, Kun Xu, Yang Song, Huawei Shen, Xueqi Cheng.
Source: [arXiv:2510.15522](https://arxiv.org/abs/2510.15522). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2510.15522/). [Code](https://github.com/DJC-GO-SOLO/Latent-SFT).
Categories: Continuous-state reasoning.
Summary: Models latent reasoning as a sequence of superposed token distributions and trains the process with supervised fine-tuning.





### 2025-10-28 — SemCoT: Accelerating Chain-of-Thought Reasoning through Semantically-Aligned Implicit Tokens

Authors: Yinhan He, Wendy Zheng, Yaochen Zhu, Zaiyi Zheng, Lin Su, Sriram Vasudevan, Qi Guo, Liangjie Hong, Jundong Li.
Source: [arXiv:2510.24940](https://arxiv.org/abs/2510.24940). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2510.24940/). [Code](https://github.com/YinhanHe123/SemCoT).
Categories: Internalized reasoning; Continuous-state reasoning.
Summary: Compresses textual reasoning into semantically aligned implicit tokens intended to preserve useful intermediate computation with fewer generated steps.





### 2025-10-29 — Scaling Latent Reasoning via Looped Language Models

Authors: Rui-Jie Zhu, Zixuan Wang, Kai Hua, Tianyu Zhang, Ziniu Li, Haoran Que, Boyi Wei, Zixin Wen, Fan Yin, He Xing, Lu Li, Jiajun Shi, Kaijing Ma, Shanda Li, Taylor Kergan, Andrew Smith, Xingwei Qu, Mude Hui, Bohong Wu, Qiyang Min, Hongzhi Huang, Xun Zhou, Wei Ye, Jiaheng Liu, Jian Yang, Yunfeng Shi, Chenghua Lin, Enduo Zhao, Tianle Cai, Ge Zhang, Wenhao Huang, Yoshua Bengio, Jason Eshraghian.
Source: [arXiv:2510.25741](https://arxiv.org/abs/2510.25741). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2510.25741/). [Code](https://ouro-llm.github.io/).
Categories: Recurrent reasoning.
Summary: Scales looped language models that reuse layers across recurrent depth and studies how performance changes with additional internal computation.





### 2025-11-10 — Teaching Pretrained Language Models to Think Deeper with Retrofitted Recurrence

Authors: Sean McLeish, Ang Li, John Kirchenbauer, Dayal Singh Kalra, Brian R. Bartoldson, Bhavya Kailkhura, Avi Schwarzschild, Jonas Geiping, Tom Goldstein, Micah Goldblum.
Source: [arXiv:2511.07384](https://arxiv.org/abs/2511.07384). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2511.07384/). [Code](https://github.com/mcleish7/retrofitting-recurrence).
Categories: Recurrent reasoning.
Summary: Adds recurrent depth to pretrained language models and trains them to use repeated internal computation without pretraining from scratch.





### 2025-11-11 — Think-at-Hard: Dynamic Looped Transformers for Improved Reasoning

Authors: Tianyu Fu, Yichen You, Zekai Chen, Guohao Dai, Huazhong Yang, Yu Wang.
Source: [arXiv:2511.08577](https://arxiv.org/abs/2511.08577). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2511.08577/). [Code](https://github.com/thu-nics/TaH).
Categories: Recurrent reasoning.
Summary: Uses dynamic looping to vary internal depth with problem difficulty and studies the resulting reasoning and efficiency trade-offs.





### 2025-11-26 — Learning When to Stop: Adaptive Latent Reasoning via Reinforcement Learning

Authors: Alex Ning, Yen-Ling Kuo, Gabe Gomes.
Source: [arXiv:2511.21581](https://arxiv.org/abs/2511.21581). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2511.21581/). [Code](https://github.com/apning/adaptive-latent-reasoning).
Categories: Continuous-state reasoning.
Summary: Learns a stopping policy for continuous latent reasoning so the model can allocate a variable number of internal steps.





### 2025-12-25 — Do Latent Tokens Think? A Causal and Adversarial Analysis of Chain-of-Continuous-Thought

Authors: Yuyi Zhang, Boyu Tang, Tianjie Ju, Sufeng Duan, Gongshen Liu.
Source: [arXiv:2512.21711](https://arxiv.org/abs/2512.21711). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2512.21711/).
Categories: Analysis and evaluation; Continuous-state reasoning.
Summary: Tests COCONUT with latent-state interventions and deliberately biased examples on MMLU and HotpotQA. Finds weak sensitivity to steering and dependence on dataset shortcuts in the evaluated models, challenging whether their intermediate states faithfully carry the reasoning.



Added to this list: 2026-10-18.


### 2026-01-29 — Latent Chain-of-Thought as Planning: Decoupling Reasoning from Verbalization

Authors: Jiecong Wang, Hao Peng, Zhanyi Wang, Chunyang Liu, Guanlin Wu.
Source: [arXiv:2601.21358](https://arxiv.org/abs/2601.21358). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2601.21358/). [Code](https://github.com/yunsaijc/PLaT).
Categories: Continuous-state reasoning.
Summary: Separates an internal planning process from verbalization and studies whether the latent plan supports effective downstream reasoning.





### 2026-01-29 — Chain Of Thought Compression: A Theoretical Analysis

Authors: Juncai Li, Ru Li, Yuxiang Zhou, Boxiang Ma, Jeff Z. Pan.
Source: [arXiv:2601.21576](https://arxiv.org/abs/2601.21576). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2601.21576/).
Categories: Analysis and evaluation; Internalized reasoning; Thinking tokens.
Summary: Analyzes how skipping intermediate steps weakens learning signals in a simplified parity-task model. Introduces NatBool-DAG to test logical dependencies and ALiCoT, which supervises learned token states with intermediate logical labels or explicit-CoT representations.



Added to this list: 2026-10-18.


### 2026-01-29 — Beyond Imitation: Reinforcement Learning for Active Latent Planning

Authors: Zhi Zheng, Wee Sun Lee.
Source: [arXiv:2601.21598](https://arxiv.org/abs/2601.21598). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2601.21598/). [Code](https://github.com/zz1358m/ATP-Latent-master).
Categories: Continuous-state reasoning.
Summary: Uses reinforcement learning to train an active latent-planning process rather than only imitating supervised reasoning traces.





### 2026-01-31 — Do Latent-CoT Models Think Step-by-Step? A Mechanistic Study on Sequential Reasoning Tasks

Authors: Jia Liang, Liangming Pan.
Source: [arXiv:2602.00449](https://arxiv.org/abs/2602.00449). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2602.00449/). [Code](https://github.com/jialiang19/latent-cot-thinking).
Categories: Analysis and evaluation; Continuous-state reasoning.
Summary: Uses mechanistic interventions on sequential tasks to test whether latent chain-of-thought models actually execute stepwise internal procedures.





### 2026-02-01 — Capabilities and Fundamental Limits of Latent Chain-of-Thought

Authors: Jiaxuan Zou, Yaozhong Xiong, Yong Liu.
Source: [arXiv:2602.01148](https://arxiv.org/abs/2602.01148). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2602.01148/).
Categories: Analysis and evaluation; Continuous-state reasoning.
Summary: Characterizes tasks that latent chain-of-thought can express and identifies theoretical and empirical limits of the approach.





### 2026-02-09 — Latent Reasoning with Supervised Thinking States

Authors: Ido Amos, Avi Caciularu, Mor Geva, Amir Globerson, Jonathan Herzig, Lior Shani, Idan Szpektor.
Source: [arXiv:2602.08332](https://arxiv.org/abs/2602.08332). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2602.08332/).
Categories: Thinking tokens; Continuous-state reasoning.
Summary: Supervises intermediate thinking states that bridge textual reasoning traces and latent computation.





### 2026-02-12 — ThinkRouter: Efficient Reasoning via Routing Thinking between Latent and Discrete Spaces

Authors: Xin Xu, Tong Yu, Xiang Chen, Haoliang Wang, Julian McAuley, Saayan Mitra.
Source: [arXiv:2602.11683](https://arxiv.org/abs/2602.11683). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2602.11683/).
Categories: Continuous-state reasoning; Thinking tokens.
Summary: Routes intermediate computation between latent and discrete representations to trade off reasoning quality, readability, and token cost.





### 2026-03-02 — PonderLM-3: Adaptive Token-Wise Pondering with Differentiable Masking

Authors: He Li, Feichen Song, Boyi Zeng, Shixiang Song, Zhiqin John Xu, Ziwei He, Zhouhan Lin.
Source: [arXiv:2603.02023](https://arxiv.org/abs/2603.02023). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2603.02023/).
Categories: Recurrent reasoning.
Summary: Uses differentiable masks to allocate a variable number of recurrent pondering steps to different token positions.





### 2026-03-31 — Internalized Reasoning for Long-Context Visual Document Understanding

Authors: Austin Veselka.
Source: [arXiv:2604.02371](https://arxiv.org/abs/2604.02371). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2604.02371/).
Categories: Internalized reasoning.
Summary: Trains on synthetic traces that rank document pages and extract relevant evidence, then uses low-strength model merging to internalize the trained reasoning behavior. Compares answering with and without written reasoning on long visual documents.



Added to this list: 2026-10-18.


### 2026-04-27 — Latent Agents: A Post-Training Procedure for Internalized Multi-Agent Debate

Authors: John Seon Keun Yi, Aaron Mueller, Dokyun Lee.
Source: [arXiv:2604.24881](https://arxiv.org/abs/2604.24881). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2604.24881/). [Code](https://github.com/johnsk95/latent_agents).
Categories: Internalized reasoning.
Summary: Post-trains a single model to internalize the computation of a multi-agent debate rather than emitting the debate as text.





### 2026-05-27 — Transformers Provably Learn to Internalize Chain-of-Thought

Authors: Yixiao Huang, Hanlin Zhu, Zixuan Wang, Jiantao Jiao, Stuart Russell, Somayeh Sojoudi, Song Mei.
Source: [arXiv:2605.28600](https://arxiv.org/abs/2605.28600). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2605.28600/).
Categories: Internalized reasoning; Analysis and evaluation.
Summary: Provides theoretical and empirical results on when transformers trained with a curriculum learn to internalize explicit reasoning steps.





### 2026-05-31 — Unlocking the Black Box of Latent Reasoning: An Interpretability-Guided Approach to Intervention

Authors: Shuochen Chang, Tong Bai, Xiaofeng Zhang, Qianli Ma, Qingyang Liu, Zhaohe Liao, Yibo Miao, Li Niu.
Source: [arXiv:2606.01243](https://arxiv.org/abs/2606.01243). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2606.01243/).
Categories: Analysis and evaluation; Continuous-state reasoning.
Summary: Uses probes, ablations, and targeted edits to study how continuous reasoning states affect answers. Builds interventions from learned semantic mappings and geometric constraints, improving the evaluated models by editing their latent states without updating base-model weights.



Added to this list: 2026-10-18.


### 2026-06-10 — Observable Patterns Are Not Explanations: A Causal-Geometric Analysis of Latent Reasoning Models

Authors: Darpan Aswal, Thomas Palmeira Ferraz, Yongxin Zhou, Maxime Peyrard.
Source: [arXiv:2606.12689](https://arxiv.org/abs/2606.12689). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2606.12689/).
Categories: Analysis and evaluation; Continuous-state reasoning.
Summary: Compares Coconut and CODI with controls lacking their proposed reasoning recurrence or curriculum. Finds that decodable patterns can also appear in controls, while interventions reveal varying causal use of latent states concentrated in low-dimensional directions.



Added to this list: 2026-10-18.


### 2026-06-11 — Demystifying Hidden-State Recurrence: Switchable Latent Reasoning with On-Policy Reinforcement Learning

Authors: Jiayu Yang, Chao Chen, Shengen Wu, Yinhong Liu, Yuxuan Fan, Lujundong Li, Songning Lai, Chengwei Qin, Zhijiang Guo.
Source: [arXiv:2606.13106](https://arxiv.org/abs/2606.13106). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2606.13106/).
Categories: Continuous-state reasoning; Recurrent reasoning; Analysis and evaluation.
Summary: Trains switchable hidden-state recurrence with on-policy reinforcement learning and analyzes how the internal loop contributes to reasoning.





### 2026-06-16 — Fixed-Point Reasoners: Stable and Adaptive Deep Looped Transformers

Authors: Sajad Movahedi, Vera Milovanović, Shlomo Libo Feigin, Alexander Theus, Thomas Hofmann, Valentina Boeva, T. Konstantin Rusch, Antonio Orvieto.
Source: [arXiv:2606.18206](https://arxiv.org/abs/2606.18206). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2606.18206/). [Code](https://github.com/nilskiKonjIzDunava/fprm).
Categories: Recurrent reasoning.
Summary: Stabilizes repeated transformer layers and stops looping when the hidden state converges, allowing the model to spend different amounts of computation on different problems.





### 2026-06-18 — What Makes Effective Supervision in Latent Chain-of-Thought: An Information-Theoretic Analysis

Authors: Xinghao Chen, Chak Tou Leong, Wenjin Guo, Jian Wang, Wenjie Li, Xiaoyu Shen.
Source: [arXiv:2606.20075](https://arxiv.org/abs/2606.20075). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2606.20075/). [Code](https://github.com/EIT-NLP/Supervision-in-Latent-CoT).
Categories: Analysis and evaluation; Continuous-state reasoning.
Summary: Uses information-theoretic analysis to study which supervision signals make latent chain-of-thought training effective.





### 2026-07-01 — DiscoLoop: Looping Discrete Embeddings and Continuous Hidden States for Multi-hop Reasoning

Authors: Hengyu Fu, Tianyu Guo, Zixuan Wang, Hanlin Zhu, Jason D. Lee, Jiantao Jiao, Stuart Russell, Song Mei.
Source: [arXiv:2607.00341](https://arxiv.org/abs/2607.00341). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2607.00341/).
Categories: Recurrent reasoning; Continuous-state reasoning.
Summary: Feeds both token embeddings and hidden states back through repeated transformer layers to connect successive steps of multi-hop reasoning.





### 2026-07-01 — Think in Latent, Explain in Language: Self-Explainable Latent Reasoning

Authors: Dayuan Zhao, Shengcao Cao, Yu-Xiong Wang, Liang-Yan Gui.
Source: [arXiv:2608.13570](https://arxiv.org/abs/2608.13570). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2608.13570/). [Code](https://jasondayuan.github.io/SELR/).
Categories: Continuous-state reasoning; Analysis and evaluation.
Summary: Jointly trains latent states for answer prediction and for decoding back into readable reasoning, using one model rather than a separate post-hoc explanation decoder.





### 2026-07-07 — Final Checkpoints Are Not Enough: Analyzing Latent Reasoning Faithfulness Along Training Trajectories

Authors: Hengyu Jin, Shu Yang, Di Wang.
Source: [arXiv:2607.06648](https://arxiv.org/abs/2607.06648). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2607.06648/).
Categories: Analysis and evaluation; Continuous-state reasoning.
Summary: Tests how answers respond to changed inputs and altered hidden reasoning states throughout training. These responses can change even while accuracy remains high.





### 2026-07-16 — T²MLR: Transformer with Temporal Middle-Layer Recurrence

Authors: Ziyang Cai, Xingyu Zhu, Yihe Dong, Yinghui He, Sanjeev Arora.
Source: [arXiv:2607.15178](https://arxiv.org/abs/2607.15178). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2607.15178/).
Categories: Recurrent reasoning.
Summary: Carries a cached middle-layer state from one token position into an earlier layer at the next position, providing localized temporal recurrence that can also be retrofitted into pretrained transformers.





### 2026-07-18 — Training Continuous Chain of Thought Models: A Tale of Two Regimes

Authors: Varun Yerram, He He, Eunsol Choi.
Source: [arXiv:2607.16972](https://arxiv.org/abs/2607.16972). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2607.16972/). [Code](https://github.com/Varun221/cmtp_research).
Categories: Continuous-state reasoning; Analysis and evaluation.
Summary: Introduces direct latent supervision through averaged trace embeddings and finds that both direct and indirect continuous-CoT training degrade sharply on tasks with substantially longer reasoning traces.





### 2026-08-03 — GradCuit: Credit-Assigned Gradient Flow Enables Robust and Interpretable Test-Time Latent Reasoning

Authors: Zhaoxin Yu, Qi Shen, Hengli Li, Zhaowei Zhang, Song-Chun Zhu, Chi Zhang, Zilong Zheng.
Source: [arXiv:2608.02585](https://arxiv.org/abs/2608.02585). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2608.02585/).
Categories: Continuous-state reasoning.
Summary: At test time, adjusts extra hidden states inside the transformer using feedback on generated answers, while keeping the model’s weights fixed.





### 2026-08-04 — LoopMTP: A looped transformer guided by latent multi-token prediction

Authors: Behzad Shomali, Markus Frey, David Berghaus, Joachim Koehler, Mehdi Ali.
Source: [arXiv:2608.03624](https://arxiv.org/abs/2608.03624). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2608.03624/).
Categories: Recurrent reasoning.
Summary: Trains each transformer loop to anticipate a different future token, using token embeddings to guide its hidden state.





### 2026-08-05 — Does Out-of-Sight Equal Out-of-Mind in CoT Monitorability?

Authors: Pedro Ferreira, Wilker Aziz, Ivan Titov.
Source: [arXiv:2608.04928](https://arxiv.org/abs/2608.04928). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2608.04928/).
Categories: Analysis and evaluation; Continuous-state reasoning.
Summary: Compares monitoring from explicit traces, latent-state probes, and latent verbalizations, finding that monitorability depends strongly on task structure and level of internal access.





### 2026-08-18 — Think Shallow, Solve Deep: Controlling Recurrent Dynamics for Reliable Test-Time Depth

Authors: Ivan Viakhirev, Kirill Borodin, Amirah Almutairi, Serguei Barannikov, Maxim Abramov, Grach Mkrtchian.
Source: [arXiv:2608.18222](https://arxiv.org/abs/2608.18222). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2608.18222/).
Categories: Recurrent reasoning; Analysis and evaluation.
Summary: Studies when extra transformer loops improve an answer or make it worse. Gives a condition under which further loops cannot change the answer, and trains models toward this stable behavior.





### 2026-08-18 — Allocating Recurrent Compute in Looped Language Models

Authors: Ruhai Lin, Yiyang Guo, Rui-Jie Zhu, Hao Ye, Jason K. Eshraghian.
Source: [arXiv:2608.18230](https://arxiv.org/abs/2608.18230). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2608.18230/).
Categories: Recurrent reasoning; Analysis and evaluation.
Summary: Studies which transformer components should recur and introduces MixerLoop, which repeats the sequence mixer while applying the dense feed-forward component once.





### 2026-08-30 — Detecting Hidden Chain-of-Thought in Large Language Models with Linguistic, Behavioral, and Mechanistic Indicators

Authors: Armaan Singh, Ryan Trinh Le, Jasmine Kaur, Abdullah Sultan, Edward Lue Chee Lip, Kiran Nijjer, Adnan Ahmed, Vasu Sharma.
Source: [arXiv:2608.29956](https://arxiv.org/abs/2608.29956). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2608.29956/).
Categories: Analysis and evaluation.
Summary: Compares behavior under neutral, explicit-CoT, and no-CoT prompts using linguistic and internal-state measurements. Its score measures resemblance to CoT behavior, not direct evidence of a hidden reasoning trace; calibration controls expose important confounds.



Added to this list: 2026-10-18.


### 2026-08-30 — Dissecting Hierarchical Reasoning Models: A Mechanistic Study

Authors: Leo Raphael Rodrigues, Jian Kang.
Source: [arXiv:2609.22197](https://arxiv.org/abs/2609.22197). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.22197/).
Categories: Analysis and evaluation; Recurrent reasoning.
Summary: Uses state interventions, probes, and feature ablations to study how hierarchical recurrent models solve Sudoku, mazes, and ARC tasks. Compares hierarchical and single-state recurrence, and distinguishes information that can be read from a state from information that causally affects an answer.



Added to this list: 2026-10-18.


### 2026-09-01 — A Score Is Not a Policy: Measuring the Value of Adaptive Revision

Authors: Ayushi Chadha.
Source: [arXiv:2609.00874](https://arxiv.org/abs/2609.00874). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.00874/).
Categories: Analysis and evaluation; Recurrent reasoning.
Summary: Tests when a hierarchical reasoner should revise the internal plan guiding its lower-level computation. Compares learned revision schedules with fixed schedules, separating whether a controller responds to its state from whether that response improves decisions.



Added to this list: 2026-10-18.


### 2026-09-01 — Latent Recurrent Thoughts: Recurrent Refinement of Proposed Latents for Reasoning with Frozen LLMs

Authors: Zhaoliang Chen, Jie Fu.
Source: [arXiv:2609.01117](https://arxiv.org/abs/2609.01117). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.01117/).
Categories: Continuous-state reasoning; Recurrent reasoning.
Summary: Uses a task-specific proposer and small recurrent auxiliary network to iteratively refine continuous thoughts that a frozen language model then decodes.





### 2026-09-01 — SMELT: Scaling Laws for Compute-Matched MoE Looped Transformers

Authors: Shaowen Wang, Ge Zhang, Kairong Luo, Yuhao Wu, Shaofan Liu, Jiaheng Liu, Wenhao Huang, Shen Yan, Jian Li.
Source: [arXiv:2609.01343](https://arxiv.org/abs/2609.01343). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.01343/).
Categories: Recurrent reasoning; Analysis and evaluation.
Summary: Loops the middle half of a mixture-of-experts transformer twice while matching an unlooped baseline on computation, parameter count, and KV-cache size. Studies how the comparison changes with model scale and examines the attention patterns produced by revisiting layers.



Added to this list: 2026-10-18.


### 2026-09-01 — Diffusion as a Training Curriculum for Timestep-Free Iterative Reasoning

Authors: Mariia Drozdova, Aidan Sirbu, Pietro Miotti, Robert Obryk, Mayalen Etcheverry, Eyvind Niklasson, Blake Richards.
Source: [arXiv:2609.01449](https://arxiv.org/abs/2609.01449). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.01449/).
Categories: Recurrent reasoning; Alternative generation methods.
Summary: Turns a diffusion denoiser into a recurrent puzzle solver by adding a persistent hidden state and removing timestep conditioning. Separates the role of denoising during training from the role of repeated updates and noise at inference.



Added to this list: 2026-10-18.


### 2026-09-01 — Looped Transformers under the Jacobian Lens: Does the Global Workspace Survive Recurrence?

Authors: Wenlong Wang, Fergal Reid.
Source: [arXiv:2609.01924](https://arxiv.org/abs/2609.01924). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.01924/).
Categories: Analysis and evaluation; Recurrent reasoning.
Summary: Tracks information that can be decoded from, and altered within, repeated transformer layers. Finds that Ouro reconstructs this information each loop, while Huginn carries it forward.





### 2026-09-03 — RecurTrace: Adaptive Latent Reasoning with Loop-Time Memory

Authors: Yuxiang Wang, Kunyu Feng, Yingda Shen, Haoning Xu, Junyu Wang, Zhizheng Wu.
Source: [arXiv:2609.03379](https://arxiv.org/abs/2609.03379). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.03379/).
Categories: Recurrent reasoning.
Summary: Lets looped layers attend to their own earlier iteration states and learns a halting head that allocates recurrent depth per input.





### 2026-09-03 — Almost Free State Prediction Separation

Authors: John Langford, Nathan Godey, Giovanni Monea, Yoav Artzi, Harry Dong, Ying Fan, Gustavo de Rosa, Zheng Zhan.
Source: [arXiv:2609.03807](https://arxiv.org/abs/2609.03807). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.03807/).
Categories: Thinking tokens; Alternative generation methods.
Summary: Moves pause-token computation into a parallel, weight-shared prediction stream, providing additional internal compute without increasing generated sequence length or key/value-cache length.





### 2026-09-03 — Towards Understanding Pause Token Fine-Tuning Dynamics: A Mode Retention Perspective

Authors: Jaehyeon Kim, Suhwan Kim, Nakyung Lee, Yeongoon Kim, Jimin Seo, Giho Lee, Jungwoo Lee.
Source: [arXiv:2609.04489](https://arxiv.org/abs/2609.04489). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.04489/).
Categories: Thinking tokens; Analysis and evaluation.
Summary: Places pause tokens at reasoning-step boundaries and masks their training loss. Studies how this affects learning new tasks while retaining earlier capabilities, including during reinforcement learning.



Added to this list: 2026-10-18.


### 2026-09-04 — Fractal basins trap latent reasoning

Authors: Jeffrey Lai, Anthony Bao, John Quinn, William Gilpin.
Source: [arXiv:2609.04963](https://arxiv.org/abs/2609.04963). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.04963/).
Categories: Analysis and evaluation; Recurrent reasoning.
Summary: Studies recurrent reasoning trajectories as dynamical systems on puzzles and mathematical logic. Links long solution times in the tested models to transient trapping near unstable states associated with nearly correct solutions.



Added to this list: 2026-10-18.


### 2026-09-05 — DART: Distributional Adversarial Recurrent Training for Algorithm Learning

Authors: Hieu Tran Bao, Phung Thanh Dang, Pham Quang Nhat Minh, Hoang Thanh Tung.
Source: [arXiv:2609.05988](https://arxiv.org/abs/2609.05988). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.05988/).
Categories: Recurrent reasoning.
Summary: Trains recurrent reasoners against a distribution of targets near each correct solution, using an adversarial objective instead of a single point target. Evaluates generalization and stability on maze, chess, and Sudoku tasks.



Added to this list: 2026-10-18.


### 2026-09-09 — Structural Process Supervision for Latent Chain-of-Thought Reasoning

Authors: Yiqi Li, Xu Chen, Chen Ju, Jiangchao Yao, Zhaoyang Li, Jinsong Lan, Xiaoyong Zhu, Yu Wang.
Source: [arXiv:2609.09928](https://arxiv.org/abs/2609.09928). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.09928/).
Categories: Continuous-state reasoning.
Summary: Aligns continuous thoughts with written reasoning through shared, learnable prototypes, allowing several text steps to supervise several latent states. A positional alignment constraint is gradually relaxed during training.



Added to this list: 2026-10-18.


### 2026-09-11 — CanvasAnneal: Curriculum Reinforcement Learning for Diffusion Language Models

Authors: Blake Olson, Yuhang Song, Emmett McQuinn, Yuan Shangguan.
Source: [arXiv:2609.13060](https://arxiv.org/abs/2609.13060). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.13060/).
Categories: Alternative generation methods.
Summary: Starts diffusion-model reinforcement learning with teacher-written reasoning partially filled into the output, then gradually removes that guidance. Tests whether this curriculum improves exploration on mathematics and tool-use tasks; inference still produces textual reasoning.



Added to this list: 2026-10-18.


### 2026-09-14 — T-LoopFormer: Token-Level Elastic-Depth Looped Transformers with Dynamic Routing

Authors: Mingqian Yu, Wenpeng Zhang, Shaobo Cui, Peilin Zhao.
Source: [arXiv:2609.15160](https://arxiv.org/abs/2609.15160). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.15160/).
Categories: Recurrent reasoning.
Summary: Learns a router that chooses how many recurrent passes each token receives, and keeps separate key/value caches for each pass.



Added to this list: 2026-10-18.


### 2026-09-14 — Register Tokens for Bounded-State Reasoning in Diffusion Language Models

Authors: Albert Ge, Chandan Singh, Yufan Zhuang, Xiaodong Liu, Jianfeng Gao, Frederic Sala.
Source: [arXiv:2609.16372](https://arxiv.org/abs/2609.16372). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.16372/).
Categories: Continuous-state reasoning; Alternative generation methods.
Summary: Trains a diffusion language model to carry a fixed number of continuous register states between generated chunks. The model still writes text within each chunk, but clears earlier text and continues from the prompt and retained registers.



Added to this list: 2026-10-18.


### 2026-09-15 — Right Direction, Wrong Step: Geometric Analysis of Finite-Step Failure in Looped Transformers

Authors: Zhihao Guo, Zonghan Wu, Haizhou Du, Huan Huo, Yilei Shao, Athanasios V. Vasilakos, Qingsong Wen.
Source: [arXiv:2609.16665](https://arxiv.org/abs/2609.16665). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.16665/).
Categories: Analysis and evaluation; Recurrent reasoning.
Summary: Studies updates that initially increase support for a reference answer but overshoot when applied in full. Uses curvature along each update to explain failures and identify smaller useful steps.



Added to this list: 2026-10-18.


### 2026-09-15 — LoopSpec: Pipelined Self-Speculative Decoding for Looped Transformers

Authors: SangLyul Cho, Langqing Cui, Sehoon Kim, Dongsu Han, Insu Han.
Source: [arXiv:2609.17184](https://arxiv.org/abs/2609.17184). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.17184/).
Categories: Recurrent reasoning; Alternative generation methods.
Summary: Drafts future tokens from early loops while later loops verify current tokens, overlapping the work without training an extra model. Final verification preserves the full model's decoding behavior.



Added to this list: 2026-10-18.


### 2026-09-16 — How Model Growth, Recursion, and Boundary Operators Influence Scaling Exponents

Authors: Zixi Chen, Akshay Vegesna, Samip Dahal, Andrew Gordon Wilson.
Source: [arXiv:2609.19107](https://arxiv.org/abs/2609.19107). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.19107/).
Categories: Recurrent reasoning; Analysis and evaluation.
Summary: Studies how growing a model during training, reusing layers, and modifying the connections between repeated blocks affect language-model scaling. Compares shared and unshared weights and operators that normalize or reintroduce earlier representations.



Added to this list: 2026-10-18.


### 2026-09-17 — Storing Is Not Remembering: LSTM-UT and Bounded Gated Memory for Looped Transformers

Authors: Aras Kavuncu, Muhammad Burhan Hafez.
Source: [arXiv:2609.19521](https://arxiv.org/abs/2609.19521). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.19521/).
Categories: Recurrent reasoning; Analysis and evaluation.
Summary: Adds a small gated memory to a looped transformer and compares it with storing the full history of recurrent states. Uses a cellular-automaton task and language modeling to test whether memory remains useful as computation grows.



Added to this list: 2026-10-18.


### 2026-09-17 — Learn Your Own Thoughts: Abstract Token Curriculum

Authors: Khashayar Gatmiry, Avrajit Ghosh, Parsa Mirtaheri, Jason D. Lee, Nika Haghtalab, Emmanuel Abbe, Peter Bartlett.
Source: [arXiv:2609.19717](https://arxiv.org/abs/2609.19717). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.19717/).
Categories: Continuous-state reasoning.
Summary: Trains on progressively harder problems so that a model learns continuous intermediate thoughts without explicit reasoning-step supervision. Analyzes parity learning theoretically and tests graph reachability and arithmetic.



Added to this list: 2026-10-18.


### 2026-09-17 — Uni-LaDiR: Latent Diffusion Unifies Multimodal Reasoning

Authors: Haoqiang Kang, Yizhe Zhang, Nikki Lijing Kuang, Jiatao Gu, Yian Ma, Lianhui Qin.
Source: [arXiv:2609.19878](https://arxiv.org/abs/2609.19878). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.19878/).
Categories: Continuous-state reasoning; Alternative generation methods.
Summary: Encodes teacher reasoning steps from text, images, point clouds, and robot states into a shared continuous representation. A jointly trained diffusion reasoner generates these states at inference, evaluated on visual question answering and robotic manipulation.



Added to this list: 2026-10-18.


### 2026-09-17 — Schema-Anchored Latent Reasoning for Semantic Parsing-Based Knowledge Base Question Answering

Authors: Guangze Gao, Zixuan Li, Sikui Zhang, Chunfeng Yuan, Wenjuan Li, Bing Li, Xiaolong Jin, Weiming Hu.
Source: [arXiv:2609.20398](https://arxiv.org/abs/2609.20398). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.20398/).
Categories: Continuous-state reasoning.
Summary: Aligns continuous thoughts with a codebook of knowledge-base relations and classes, using schema traces derived from correct logical forms. Feeds the aligned codes into later reasoning steps before generating an executable query.



Added to this list: 2026-10-18.


### 2026-09-17 — CaLR: Causal Latent Revision for Robust Diffusion Reasoning

Authors: Wei Cai, Jian Zhao, Yuchen Yuan, Xuelong Li.
Source: [arXiv:2609.20981](https://arxiv.org/abs/2609.20981). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.20981/).
Categories: Continuous-state reasoning; Alternative generation methods.
Summary: Uses an expert-derived causal dependency matrix to guide gradient-based revision of diffusion reasoning states. The aim is to preserve logical dependencies while revising several intermediate positions during parallel generation.



Added to this list: 2026-10-18.


### 2026-09-18 — Trading Depth for Time in Recurrent Transformers

Authors: Zeyi Huang, Xuehai He, Yelong Shen, Yong Jae Lee.
Source: [arXiv:2609.21605](https://arxiv.org/abs/2609.21605). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.21605/).
Categories: Recurrent reasoning; Thinking tokens.
Summary: Inserts a hidden-state refinement step between consecutive text tokens in a recurrent transformer. Compares this extra thinking time with adding physical layers or transformer loops under matched computation budgets.



Added to this list: 2026-10-18.


### 2026-09-18 — When Steering Fails in Latent Reasoning: A Latent-to-Language Transition Gap

Authors: Gaoxiang Huang, Lei Qi.
Source: [arXiv:2609.21662](https://arxiv.org/abs/2609.21662). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.21662/).
Categories: Analysis and evaluation; Continuous-state reasoning.
Summary: Tests whether changing continuous reasoning states changes the model's later text output. Finds weaker steering effects than in explicit chain-of-thought, despite comparable changes to the hidden representations.



Added to this list: 2026-10-18.


### 2026-09-19 — WaveFront Decoding: Parallelized Self-Speculative Decoding for Looped Language Models

Authors: Hyeongju Ha, Jae-Joon Kim.
Source: [arXiv:2609.23033](https://arxiv.org/abs/2609.23033). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.23033/).
Categories: Recurrent reasoning; Alternative generation methods.
Summary: Uses intermediate loop depths to propose tokens that later loops verify. Arranges speculation across recurrent depths so that more of the draft and verification work can run together.



Added to this list: 2026-10-18.


### 2026-09-21 — LoopCD: Loop-wise Contrastive Decoding for Improving Reasoning in Looped Language Models

Authors: Byeongho Yu, Junhyuk So, Eunhyeok Park.
Source: [arXiv:2609.24196](https://arxiv.org/abs/2609.24196). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.24196/).
Categories: Recurrent reasoning.
Summary: Uses differences between early- and late-loop predictions to adjust token sampling at inference time, without retraining the model.



Added to this list: 2026-10-18.


### 2026-09-21 — LADDER: Graph-Guided Diffusion Language Models for Efficient Multi-Hop Reasoning

Authors: Senlei Zhang, Linhao Luo, Qian-Wen Zhang, Siyu An, Junnan Dong, Shuhao Zhang, Xing Sun.
Source: [arXiv:2609.24346](https://arxiv.org/abs/2609.24346). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.24346/).
Categories: Alternative generation methods.
Summary: Triggers graph retrieval when new entities become identifiable in a partially denoised draft, then uses graph propagation to provide multi-hop evidence for subsequent updates. This combines retrieval with parallel textual reasoning rather than removing the text trace.



Added to this list: 2026-10-18.


### 2026-09-23 — LOCKR: A Hidden-State Trajectory-Guided Planner for Detecting and Repairing Stable-but-Wrong Lock-In in Diffusion Language Models

Authors: Guoshenghui Zhao, Tan Yu, Weijie Zhao.
Source: [arXiv:2609.27220](https://arxiv.org/abs/2609.27220). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.27220/).
Categories: Analysis and evaluation; Alternative generation methods.
Summary: Uses the history of diffusion hidden states to detect answers that become stable but remain wrong. A planner selects targeted repair branches and verifies continuations, testing whether these internal trajectories support better repair decisions than confidence or a single snapshot.



Added to this list: 2026-10-18.


### 2026-09-23 — Attention Routing Stabilizes Early: Working-Set Inference for Recurrent Language Models

Authors: Ke Wan, Chen Chen.
Source: [arXiv:2609.27373](https://arxiv.org/abs/2609.27373). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.27373/).
Categories: Recurrent reasoning; Analysis and evaluation.
Summary: Studies when a recurrent language model settles on which context tokens to attend to. Reuses that selected working set in later loops while still recomputing attention weights and hidden-state updates.



Added to this list: 2026-10-18.


### 2026-09-23 — Towards Efficient Reasoning: Learning Causal Shortcuts for Diffusion Language Models

Authors: Dian Jin, Kairong Han, Baohong Li, Xinpeng Dong, Zijing Hu, Nuanqiao Shan, Fei Wu, Kun Kuang.
Source: [arXiv:2609.28272](https://arxiv.org/abs/2609.28272). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.28272/).
Categories: Alternative generation methods.
Summary: Extracts reasoning-guiding token chains and prioritizes them during diffusion-model training to improve denoising speed and accuracy. This is a related text-generating approach, not reasoning without a written trace.



Added to this list: 2026-10-18.


### 2026-09-24 — ELF-REG: Scaling Continuous Diffusion Language Models to Reasoning Tasks

Authors: Zeyu Michael Li, William Xingxu Chen, Bingshuo Qian, Jiayin Liu, Xiang Cheng.
Source: [arXiv:2609.29102](https://arxiv.org/abs/2609.29102). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.29102/).
Categories: Continuous-state reasoning; Alternative generation methods.
Summary: Adapts continuous diffusion to mathematics and code using intermediate-feature supervision and a global representation from a frozen autoregressive teacher. It denoises continuous response representations and decodes all response tokens only at the end.



Added to this list: 2026-10-18.


### 2026-09-24 — Task-Oriented Key-Layer KV Communication for Efficient Latent Multi-Agent Collaboration

Authors: Dongsen Zhang, Peipei Li, Zekun Li, Wenjun Xu.
Source: [arXiv:2610.08820](https://arxiv.org/abs/2610.08820). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.08820/).
Categories: Continuous-state reasoning.
Summary: Lets agents perform hidden-state reasoning and pass working memory through the key-value cache of one selected transformer layer. Reuses that layer as the entry point for subsequent reasoning updates, avoiding earlier layers and decoding only the final agent's response into text.



Added to this list: 2026-10-18.


### 2026-09-25 — Skip the Talk, Re-Focus on Vision: Latent Reasoning for Reasoning Segmentation in Multimodal Large Language Models

Authors: Tianhang Guo, Yulin He, Wei Chen, Wenjuan Zhou, Yuhang Li, Xinbiao Gan.
Source: [arXiv:2609.30783](https://arxiv.org/abs/2609.30783). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.30783/).
Categories: Thinking tokens.
Summary: LIRSeg generates a fixed sequence of learned special tokens before a target description and bounding box used for segmentation. Spatial alignment grounds the tokens' hidden states in relevant image regions, followed by training with segmentation rewards.



Added to this list: 2026-10-18.


### 2026-09-27 — Ceiling of a Task: When Can a Transformer Succeed Without Its Chain of Thought?

Authors: Jiashu He, Jinxuan Fan, Xiao Xiao, Radu Marculescu, Alejandro Ribeiro.
Source: [arXiv:2609.33134](https://arxiv.org/abs/2609.33134). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.33134/).
Categories: Analysis and evaluation; Thinking tokens.
Summary: Analyzes tasks for which a fixed-depth transformer needs an informative chain of thought to exceed a task-specific accuracy limit. Combines circuit-based results with experiments that replace, erase, or shuffle reasoning tokens, including comparisons with filler tokens.



Added to this list: 2026-10-18.


### 2026-09-27 — Beyond the Training Horizon: Mechanisms and Limits of Length Generalization in Looped Transformers

Authors: Jia Liang, Xi Jin, Liangming Pan.
Source: [arXiv:2609.33144](https://arxiv.org/abs/2609.33144). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.33144/).
Categories: Analysis and evaluation; Recurrent reasoning.
Summary: Compares how two looped-transformer training schemes solve multi-step tasks beyond their training length. Uses hidden-state interventions to identify how intermediate information moves through each model and where that process breaks down.



Added to this list: 2026-10-18.


### 2026-09-27 — Structured Sparse Memory for Recurrent Reasoning

Authors: Zixuan Zhao, Samuel Wheeler, Neil Getty, Xiaotian Duan, Rick Stevens, Fangfang Xia.
Source: [arXiv:2609.33270](https://arxiv.org/abs/2609.33270). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.33270/).
Categories: Recurrent reasoning.
Summary: Combines a recurrent reasoner with sparse, compositional task embeddings for ARC puzzles. Studies how task-memory size, recurrent depth, and training horizon affect performance.



Added to this list: 2026-10-18.


### 2026-09-27 — Next Thoughts Are Distributions: Generative Autoregressive Reasoning in the Latent Space

Authors: Yang Li, Yi Wang, Shiyuan Huang, Yang Liu, Hao Wang, Chengzhi Mao.
Source: [arXiv:2609.33271](https://arxiv.org/abs/2609.33271). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.33271/).
Categories: Continuous-state reasoning; Alternative generation methods.
Summary: Adds a diffusion head that samples the next continuous thought from a distribution rather than predicting one fixed vector. Feeds each sampled thought back into an autoregressive reasoner, supporting variable-length latent traces and multiple reasoning paths.



Added to this list: 2026-10-18.


### 2026-09-27 — VaME: Exploring Variational Latent Reasoning for Multimodal Embeddings

Authors: Peixi Wu, Mingzhou Jiang, Feipeng Ma, Biao Yang, Yunhao Zhou, Wei Yuan, Bosong Chai, Huizu Lin, Jie Chen, Zhangchi Hu, Fan Yang, Wenwu Ou, Hebei Li, Xiaoyan Sun.
Source: [arXiv:2609.33402](https://arxiv.org/abs/2609.33402). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.33402/).
Categories: Continuous-state reasoning.
Summary: Learns a distribution over continuous reasoning trajectories before producing a retrieval embedding. A lightweight decoder provides reconstruction supervision and a semantic reward for reinforcement learning, evaluated on image, video, and visual-document retrieval.



Added to this list: 2026-10-18.


### 2026-09-28 — Loop Dropout: Regularizing Shared Updates in Looped Language Models

Authors: Zirui Zhu, Hailun Xu, Xuanlei Zhao, Yong Liu, Yingxuan Ren, Kanchan Sarkar, Kun Xu, Yang You.
Source: [arXiv:2609.34218](https://arxiv.org/abs/2609.34218). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.34218/).
Categories: Recurrent reasoning.
Summary: Regularizes fine-tuning of looped language models by randomly masking adapter updates at selected loops. The shared backbone still runs at every loop; the method targets adapter updates that otherwise become concentrated at particular depths.



Added to this list: 2026-10-18.


### 2026-09-28 — Distilling Visual Reasoning into Text Space

Authors: Wenhan Yang, Nilay Naharas, Ali Payani, Baharan Mirzasoleiman.
Source: [arXiv:2609.34408](https://arxiv.org/abs/2609.34408). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.34408/).
Categories: Internalized reasoning.
Summary: Distills a teacher's interleaved visual and textual reasoning into a student that no longer generates intermediate visual representations. The student still writes a textual chain of thought; what is internalized is the visual portion of the reasoning.



Added to this list: 2026-10-18.


### 2026-09-28 — FlexLoop: Depth-Elastic Looped Policies for Adaptive Test-Time Computation in Deep RL

Authors: Xun Wang, Ruishuo Chen, Yu Chen, Zhuoran Li, Longbo Huang.
Source: [arXiv:2609.34488](https://arxiv.org/abs/2609.34488). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.34488/).
Categories: Recurrent reasoning.
Summary: Post-trains a looped reinforcement-learning policy to make useful decisions at several recurrent depths. Combines its original objective with distillation between neighboring depths, then uses agreement across depths to decide when to stop.



Added to this list: 2026-10-18.


### 2026-09-28 — Shallow Queries, Mature Values: Depth-Asynchronous Self-Speculation for Looped Transformers

Authors: Guanghao Li, Zihan Su, Hao Yu, Jinyang Jiang, Tao Ren, Zehao Li, Feng Lu, Ming Tang, Chun Yuan.
Source: [arXiv:2609.34538](https://arxiv.org/abs/2609.34538). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.34538/).
Categories: Recurrent reasoning; Alternative generation methods.
Summary: Uses shallower-loop queries with values from deeper loops to improve draft-token predictions in a looped language model. Studies the different rates at which queries, keys, and values develop, and uses that difference for self-speculative decoding.



Added to this list: 2026-10-18.


### 2026-09-28 — Why Deterministic PRM Guidance Underperforms in Discrete Diffusion Reasoning

Authors: Yan Zhan, Shaobo Liu, Zhijun Gao.
Source: [arXiv:2609.35472](https://arxiv.org/abs/2609.35472). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.35472/).
Categories: Analysis and evaluation; Alternative generation methods.
Summary: Compares process-reward guidance with independent sampling and final-answer reranking at matched forward-pass budgets. In the tested diffusion models, early pruning removes promising candidates and intermediate-state scorers can be poor final judges; the generated solutions remain textual.



Added to this list: 2026-10-18.


### 2026-09-28 — Reasoning with Continuous Latent Diffusion

Authors: Xiang Cheng.
Source: [arXiv:2609.35694](https://arxiv.org/abs/2609.35694). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.35694/).
Categories: Continuous-state reasoning; Alternative generation methods.
Summary: CEDR learns compact representations from several teacher layers, denoises them at different rates, and trains a smaller prompt encoder to replace the teacher at inference. It generates complete mathematical and coding solutions through continuous refinement before text decoding.



Added to this list: 2026-10-18.


### 2026-09-28 — Improving Test-Time Scaling with Adaptive Looped Transformers

Authors: Yichen You, Tianyu Fu, Aosong Feng, Xingtai Lv, Xuefei Ning, Ning Ding, Yu Wang.
Source: [arXiv:2609.35748](https://arxiv.org/abs/2609.35748). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.35748/).
Categories: Recurrent reasoning.
Summary: Trains an iteration controller to give extra transformer loops only to tokens whose predictions benefit from them. Compares reasoning accuracy as both output length and internal computation increase.



Added to this list: 2026-10-18.


### 2026-09-28 — How to Loop MoE: Flatten the Experts, Untie the Attention

Authors: Shouren Wang, Chuang Ma, Mohsen Hariri, Debargha Ganguly, Wang Yang, Xiaoqing Tong, Qianying Liu, Xiaotian Han, Vipin Chaudhary.
Source: [arXiv:2609.35751](https://arxiv.org/abs/2609.35751). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.35751/).
Categories: Recurrent reasoning.
Summary: Builds a looped mixture-of-experts model by pooling experts across layers and revisiting them. Shares experts and routing across passes while giving each pass its own attention parameters.



Added to this list: 2026-10-18.


### 2026-09-28 — CoDR: Training-Free Confidence-Drift Remasking for Diffusion Language Models

Authors: Yue Wu, Qinghe Zhang, Yu Zhang, Jian Huang.
Source: [arXiv:2610.08833](https://arxiv.org/abs/2610.08833). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.08833/).
Categories: Alternative generation methods.
Summary: Checks whether tokens committed earlier in diffusion decoding become less plausible as context fills in. CoDR remasks and regenerates those positions, testing targeted revision against extra-compute controls on reasoning and code tasks.



Added to this list: 2026-10-18.


### 2026-09-29 — Reliable Parallel Decoding in Masked Diffusion Language Models

Authors: Zhenghao He, Bohan Liu, Guangzhi Xiong, Aidong Zhang.
Source: [arXiv:2609.36452](https://arxiv.org/abs/2609.36452). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.36452/).
Categories: Analysis and evaluation; Alternative generation methods.
Summary: Studies when simultaneously committing several predicted tokens breaks their reasoning dependencies. RPD combines stability across final layers with confidence and uncertainty in preceding positions to defer unsupported predictions while retaining parallel text generation.



Added to this list: 2026-10-18.


### 2026-09-29 — What Makes Recurrence Effective in Looped Language Models?

Authors: Xinlin Zhuang, Siyuan Wang, Imran Razzak, Weiyang Liu.
Source: [arXiv:2609.36636](https://arxiv.org/abs/2609.36636). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.36636/).
Categories: Recurrent reasoning; Analysis and evaluation.
Summary: Compares loop layouts and ways of passing information between loops in language models. Examines when recurrence helps reasoning and when it helps less on knowledge-dependent tasks.



Added to this list: 2026-10-18.


### 2026-09-29 — Scheduling Recursive Reasoning in Looped Transformers

Authors: Boyuan Wang, Chengyao Yu, Jiaxi Ren, Hongxin Wei, Bingyi Jing, Yuxin Tao.
Source: [arXiv:2609.36653](https://arxiv.org/abs/2609.36653). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.36653/).
Categories: Recurrent reasoning; Analysis and evaluation.
Summary: Adjusts the size of each hidden-state update based on whether recent updates make consistent progress or fluctuate. Tests the resulting scheduler with and without additional training.



Added to this list: 2026-10-18.


### 2026-09-29 — Port-Hamiltonian Latent Deliberation: Mitigating the Deliberation Drift Cliff in Test-Time Compute Scaling

Authors: Zeyu Jia.
Source: [arXiv:2609.37351](https://arxiv.org/abs/2609.37351). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.37351/).
Categories: Recurrent reasoning; Continuous-state reasoning.
Summary: Separates hidden-state updates into attracting and rotating components to reduce errors from running a recurrent reasoner for more steps. Tests a tensor-based implementation on logical and multi-hop reasoning tasks.



Added to this list: 2026-10-18.


### 2026-09-29 — Looped Transformers as Optimizers

Authors: Yulong Huang, Chen Jiang, Zhanpeng Zhou, Hongtao Zhang, Tianyu Li, Tianyu He, Xiangyu Zhang, Bojun Cheng.
Source: [arXiv:2609.37379](https://arxiv.org/abs/2609.37379). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.37379/).
Categories: Recurrent reasoning; Analysis and evaluation.
Summary: Treats each recurrent hidden-state update as an optimization step and derives loop designs from this view. Introduces OperLoop, combining adaptive step sizes and weight decay.



Added to this list: 2026-10-18.


### 2026-09-29 — Looped Actor: Depth-Recurrent Reasoning Models for Reinforcement Learning

Authors: T. Konstantin Rusch, Tim Seyde, Jared Boyer, Zach J. Patterson, Daniela Rus.
Source: [arXiv:2609.37432](https://arxiv.org/abs/2609.37432). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.37432/).
Categories: Recurrent reasoning.
Summary: Uses a shared transformer block to refine a reinforcement-learning policy's internal state before choosing an action. Learns to vary the number of loops with the current state, and studies adaptive computation in puzzles and robotic-control tasks.



Added to this list: 2026-10-18.


### 2026-09-29 — LoopVL: Recurrent Visual Intelligence

Authors: Zhe Qian, Ziyang Gong, Zhongxing Xu, Hehan Li, Zhonghua Wang, Hanmeng Liu, Liwei Qian, Xin Pei, Peizhi Xu, Fei Luo, Mingxuan Wang, Xue Yang, Shiwei Liu, Yanbiao Ma, Junchi Yan, Jungong Han.
Source: [arXiv:2609.38426](https://arxiv.org/abs/2609.38426). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.38426/).
Categories: Recurrent reasoning.
Summary: Extends shared-module and whole-model looping to vision-language models, repeatedly updating a joint visual-language state. Trains from scratch through language and multimodal stages and evaluates visual understanding and reasoning.



Added to this list: 2026-10-18.


### 2026-09-30 — Blackboard Intelligence Can Surpass Autoregressive on Globally Constrained Problems

Authors: Woosang Jeon, Jaeyeon Kim, Sham Kakade, Yilun Du, Amrit Singh Bedi, Arun Kumar Chithanar, Chul Lee, Taehyeong Kim, Sitan Chen.
Source: [arXiv:2609.38806](https://arxiv.org/abs/2609.38806). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.38806/).
Categories: Alternative generation methods.
Summary: Uses a fixed, revisable output and model confidence to search over candidate solutions with a frozen fine-tuned diffusion checkpoint. Evaluates logic and scheduling problems where global constraints can make irrevocable left-to-right commitments costly.



Added to this list: 2026-10-18.


### 2026-09-30 — LatentHarness: Learning Latent Actions for Memory and Reasoning via Counterfactual Policy Distillation

Authors: Xiaoqiang Wang, Suyuchen Wang, Bang Liu.
Source: [arXiv:2609.39740](https://arxiv.org/abs/2609.39740). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.39740/).
Categories: Continuous-state reasoning; Recurrent reasoning.
Summary: Learns when to perform another internal reasoning step, recall stored evidence or intermediate states, or emit a token. Trains these choices by branching over alternative actions and measuring their effects on the next emitted token.



Added to this list: 2026-10-18.


### 2026-09-30 — Shared Weights, Selected Computations: How Looped Transformers Route What Each Loop Does

Authors: Jiaju Wu, Yi Hu, Muhan Zhang.
Source: [arXiv:2609.39892](https://arxiv.org/abs/2609.39892). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.39892/).
Categories: Analysis and evaluation; Recurrent reasoning.
Summary: Uses graph-walk tasks to study how a looped transformer performs different computations with the same weights. Tests whether changing the state entering a loop can select its computation, using steering and activation-patching experiments.



Added to this list: 2026-10-18.


### 2026-09-30 — What Limits Recursive Reasoning Models: Optimization, Architecture and Test-Time Scaling

Authors: Yuliana Shakhvalieva, Dmitrii Kharchev, Viacheslav Bezrukov, Inessa Fedorova, Dmitry Bocharov, Ivan Oseledets, Valerii Ternovskii.
Source: [arXiv:2609.39967](https://arxiv.org/abs/2609.39967). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.39967/).
Categories: Recurrent reasoning; Analysis and evaluation.
Summary: Compares recursive reasoning models under a common training and evaluation setup. Isolates the effects of gradient propagation, batch size, state updates, and hierarchical structure on algorithmic tasks and generalization.



Added to this list: 2026-10-18.


### 2026-09-30 — Scaling Laws for Looped Mixture of Experts

Authors: Yanbei Chen, Anirudh Goyal, Raghuraman Krishnamoorthi.
Source: [arXiv:2609.40316](https://arxiv.org/abs/2609.40316). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2609.40316/).
Categories: Recurrent reasoning; Analysis and evaluation.
Summary: Fits scaling laws that jointly account for recurrent depth, expert sparsity, model size, and training data in looped mixture-of-experts models. Uses them to study how to allocate resources under compute and memory constraints.



Added to this list: 2026-10-18.


### 2026-09-30 — Exploring More, Reasoning Better: Stepwise Risk-Sensitive GRPO for Diffusion Language Models

Authors: Yue Yu, Bowen Zuo, David Crandall, Yinglun Zhu, Dongruo Zhou.
Source: [arXiv:2610.00661](https://arxiv.org/abs/2610.00661). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.00661/).
Categories: Alternative generation methods.
Summary: Changes how outcome rewards weight decisions at different denoising states during diffusion-model reinforcement learning. Derives the effect for binary rewards and uses matched-scale controls to test whether the schedule improves mathematical accuracy and reasoning-trace diversity.



Added to this list: 2026-10-18.


### 2026-09-30 — Closing the Loop: Practical Training Recipes for Looped Language Models

Authors: Andrei Marchenko, Viacheslav Bezrukov, Oleg Kashurin, Inessa Fedorova, Dmitry Bocharov, Yuliana Shakhvalieva, Maria Tikhonova, Valerii Ternovskii.
Source: [arXiv:2610.00673](https://arxiv.org/abs/2610.00673). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.00673/).
Categories: Recurrent reasoning.
Summary: Gives training recipes for looped language models both from scratch and by converting pretrained models. Uses controlled data and compute comparisons to separate the benefit of recurrence from the training setup.



Added to this list: 2026-10-18.


### 2026-09-30 — Rotate the Loop: Blockwise Conformality for Looped Transformers

Authors: Alexey Zemtsov, Albina Klepach, Vadim Lomshakov, Andrei Polubarov, Nikita Lyubaykin, Alexander Derevyagin, Vladislav Kurenkov, Alexander Nikulin.
Source: [arXiv:2610.14030](https://arxiv.org/abs/2610.14030). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.14030/).
Categories: Recurrent reasoning.
Summary: Splits the recurrent residual into groups and rotates each group separately, aiming to preserve information across repeated transformer loops. Evaluates state tracking and language modeling.



Added to this list: 2026-10-18.


### 2026-10-01 — Looping Beyond Twice: A Scalable Recipe for Looped Mixture-of-Experts

Authors: Di He, Pengxiang Li, Da Chang, Qingyan Meng, Lu Yin, Shiwei Liu.
Source: [arXiv:2610.01153](https://arxiv.org/abs/2610.01153). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.01153/).
Categories: Recurrent reasoning.
Summary: Studies how to train mixture-of-experts transformers with more than two recurrent passes. Combines stabilized residual updates, reintroduced input information, and loop-specific routing to support deeper reuse of the shared model.



Added to this list: 2026-10-18.


### 2026-10-01 — Temporally-Resolved Token Attribution Reveals the Generation Dynamics of Diffusion Language Models

Authors: Darpan Aswal, Céline Hudelot.
Source: [arXiv:2610.01177](https://arxiv.org/abs/2610.01177). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.01177/).
Categories: Analysis and evaluation; Alternative generation methods.
Summary: Extends integrated-gradient attribution across layers and denoising steps to examine which inputs influence a diffusion model's developing response. Includes multi-hop graph reasoning; the authors present attribution as a complement to, not a replacement for, causal intervention.



Added to this list: 2026-10-18.


### 2026-10-01 — Decoding Looped Transformers Better for (Almost) Free

Authors: Weihao Liu, Huangjie Zheng, Tianrong Chen, Rohit Dilip, Richard He Bai, Yizhu Jiao, Yuyang Wang, Ruixiang Zhang.
Source: [arXiv:2610.02185](https://arxiv.org/abs/2610.02185). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.02185/).
Categories: Recurrent reasoning.
Summary: Compares a looped model's final prediction with predictions from earlier loops to improve decoding without additional training. Tests contrastive variants using either output logits or hidden states on mathematical reasoning and code generation.



Added to this list: 2026-10-18.


### 2026-10-01 — Hierarchical Continuous Diffusion Language Models

Authors: Hui Ren, Zihan Li, Chang Liu, Huidong Liu, Alexander Schwing.
Source: [arXiv:2610.02193](https://arxiv.org/abs/2610.02193). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.02193/).
Categories: Continuous-state reasoning; Alternative generation methods.
Summary: Keeps a continuous state throughout denoising while reading out tokens at each step and feeding them back into the next state update. Tests whether this coupling preserves dependencies during parallel generation on Sudoku, Countdown, and language modeling.



Added to this list: 2026-10-18.


### 2026-10-01 — The Surprising Effectiveness of Shared Memory in Looped Transformers

Authors: Giovanni Monea, Keshav Ramji, Yousef El-Kurdi, Luis A. Lastras, Yoav Artzi, Nathan Godey, Ramón Fernandez Astudillo.
Source: [arXiv:2610.02383](https://arxiv.org/abs/2610.02383). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.02383/).
Categories: Recurrent reasoning.
Summary: Shares a long-context KV memory written by the first loop across later loops, which also keep a short local memory. Studies this architecture during pretraining and examines how shared memory changes representations and gradient flow.



Added to this list: 2026-10-18.


### 2026-10-02 — Large Language Continuous Diffusion Models

Authors: Zhihan Yang, Wei Guo, Jean-Marie Lemercier, Simon Welker, Yonggan Fu, Mohammad Mahdi Kamani, Sajad Norouzi, Julius Berner, Tomas Geffner, Karsten Kreis, Yongxin Chen, Molei Tao, John Thickstun, Pavlo Molchanov, Ante Jukić, Arash Vahdat, Morteza Mardani.
Source: [arXiv:2610.02665](https://arxiv.org/abs/2610.02665). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.02665/).
Categories: Continuous-state reasoning; Alternative generation methods.
Summary: Sigma denoises learned continuous token embeddings in blocks, starting from autoregressive pretrained weights. Studies how guidance and temperature affect mathematical reasoning, coding, output diversity, and performance when fewer refinement steps are used.



Added to this list: 2026-10-18.


### 2026-10-02 — On the Chain-of-Thought Monitorability of Looped Language Models

Authors: Han Wang, Ishwar B Balappanawar, Huan Zhang.
Source: [arXiv:2610.02741](https://arxiv.org/abs/2610.02741). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.02741/).
Categories: Analysis and evaluation; Recurrent reasoning.
Summary: Tests how recurrent depth affects the ability to monitor a language model through its written chain of thought. Finds task-dependent changes under stress tests, while matched comparisons do not show a general monitorability disadvantage for looped architectures.



Added to this list: 2026-10-18.


### 2026-10-02 — Pivot-SD: Efficient Self-Distillation for Masked Diffusion Language Models

Authors: Seo Hyun Kim, Sunwoo Hong, Younwoo Choi, Chen-Hao Chao, Se-Young Yun, Rahul G. Krishnan.
Source: [arXiv:2610.03665](https://arxiv.org/abs/2610.03665). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.03665/).
Categories: Alternative generation methods.
Summary: Selects token commitments that most reduce uncertainty over the remaining output. Self-distillation reinforces these pivots from successful diffusion trajectories and discourages selected pivots from failed ones, rather than supervising every token equally.



Added to this list: 2026-10-18.


### 2026-10-03 — AdaLoop: Adaptive-Depth Latent Reasoning for Audio Language Models

Authors: Lee Seung-woo, Bowen Qi.
Source: [arXiv:2610.06949](https://arxiv.org/abs/2610.06949). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.06949/).
Categories: Recurrent reasoning.
Summary: Adds a shared recurrent block between an audio encoder and language model. A question-conditioned halting mechanism chooses how many refinement passes to apply, with deeper computation learned for tasks requiring finer acoustic distinctions.



Added to this list: 2026-10-18.


### 2026-10-04 — Enhancing Long-Video VLM Embeddings with Query-Aware Streaming Latent Reasoning

Authors: Haozhe Chi, Song Jin, Yang Jin, Yadong Mu.
Source: [arXiv:2610.04864](https://arxiv.org/abs/2610.04864). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.04864/).
Categories: Continuous-state reasoning; Recurrent reasoning.
Summary: Processes selected video clips sequentially while persistent think tokens accumulate evidence for a fixed-size retrieval embedding. Intermediate retrieval supervision and self-distillation train partial-video readouts; the output is an embedding rather than a generated reasoning trace.



Added to this list: 2026-10-18.


### 2026-10-05 — What Matters for Latent Reasoning with Flow Matching

Authors: Yassine Ouali, Adrian Bulat, Georgios Tzimiropoulos.
Source: [arXiv:2610.06666](https://arxiv.org/abs/2610.06666). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.06666/).
Categories: Continuous-state reasoning; Alternative generation methods.
Summary: Compresses symbolic reasoning into latent codes and trains a flow model to refine a block of noise into a thought before decoding the answer. Studies training choices, self-training on verified thoughts, and tests of whether the answer actually uses those thoughts.



Added to this list: 2026-10-18.


### 2026-10-05 — Towards Looped Models Done Right, Part II: Rethinking at Fixed Points

Authors: Benhao Huang, Chufan Shi, Junlin Chen, Shicheng Wen, Zhengzhong Liu, Eric Xing, Xuezhe Ma.
Source: [arXiv:2610.06833](https://arxiv.org/abs/2610.06833). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.06833/).
Categories: Recurrent reasoning.
Summary: Uses near-fixed-point recurrent states to reduce the cost of training, key/value caching, and prefill. Learns how to sample loop depths and changes how the input is reintroduced on each loop.



Added to this list: 2026-10-18.


### 2026-10-05 — CALR: Continuous Anchored Latent Reasoning via Render-of-Thought Compression

Authors: Zhaoyang Wei, Bowen Jiang, Yanchao Hao, Wenchao Ding, Zheng Wei, Shaocheng Wu, Zhenjun Han, Jianbin Jiao.
Source: [arXiv:2610.07175](https://arxiv.org/abs/2610.07175). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.07175/).
Categories: Continuous-state reasoning.
Summary: Compresses images of written derivations into continuous states, training both the answers they support and the reasoning that can be recovered from them. Gradually shifts from generating latent blocks in parallel to generating them in sequence.



Added to this list: 2026-10-18.


### 2026-10-06 — Recurrent Looped Transformer

Authors: Yifan Zhang, Jichen Feng, Shihan Qin, Sun-Yuan Kung, Mengdi Wang.
Source: [arXiv:2610.07591](https://arxiv.org/abs/2610.07591). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.07591/).
Categories: Recurrent reasoning.
Summary: Passes each token's final decoder state to the next token, alongside a causal encoder representation. Tests whether this growing computation path supports state tracking on sequences longer than those seen during training.



Added to this list: 2026-10-18.


### 2026-10-06 — SanSi: A Looped Typed Decision Model for System 1.5 Thinking

Authors: Shuyu Gan, Young-Jun Lee, Dongyeop Kang.
Source: [arXiv:2610.07730](https://arxiv.org/abs/2610.07730). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.07730/).
Categories: Recurrent reasoning.
Summary: Refines an internal state across loops to choose among typed answer options without generating a written reasoning trace. Supervises decisions at multiple depths and evaluates how prediction quality changes with the available computation budget.



Added to this list: 2026-10-18.


### 2026-10-06 — Continuous Memory Machines

Authors: Ciaran Regan, Kai Arulkumaran, Luke Darlow, Stefania Druga, Sebastian Risi, Llion Jones.
Source: [arXiv:2610.07907](https://arxiv.org/abs/2610.07907). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.07907/).
Categories: Recurrent reasoning.
Summary: Keeps separate short- and long-term matrix memories and lets a transformer update both through bidirectional reads and writes. Tests this recurrent architecture on algorithmic, in-context learning, and reasoning tasks.



Added to this list: 2026-10-18.


### 2026-10-06 — Noise Your Prompt: Noising Conditioning Tokens in Continuous Diffusion Language Models

Authors: Justin Jung.
Source: [arXiv:2610.09145](https://arxiv.org/abs/2610.09145). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.09145/).
Categories: Continuous-state reasoning; Alternative generation methods.
Summary: Adds noise to conditioning tokens during training instead of keeping the prompt clean. Tests improvements in Sudoku and N-Queens generalization and solution diversity, while reporting that the gains do not transfer to every language-generation task.



Added to this list: 2026-10-18.


### 2026-10-07 — Efficient Reasoning with Flow Language Models

Authors: Hanru Bai, Faissal Izermine, Oscar Davis, T. Konstantin Rusch.
Source: [arXiv:2610.09416](https://arxiv.org/abs/2610.09416). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.09416/).
Categories: Analysis and evaluation; Continuous-state reasoning; Alternative generation methods.
Summary: Compares continuous flow and discrete diffusion models at matched sizes and denoising steps. A theoretical account and intermediate-state interventions examine whether retaining information about several candidates helps solve mazes and Sudoku before final discretization.



Added to this list: 2026-10-18.


### 2026-10-07 — Think Before You Paint: Recursive Latent Reasoning for Diffusion Models

Authors: Paweł Skierś, Małgorzata Grzanka, Wojciech Masarczyk, Jan-Willem van de Meent, Kamil Deja.
Source: [arXiv:2610.09876](https://arxiv.org/abs/2610.09876). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.09876/).
Categories: Recurrent reasoning; Continuous-state reasoning; Alternative generation methods.
Summary: PaTh uses a small recursive network to refine learned image-state tokens inside each denoising step and steer a frozen diffusion model through adapters. It learns visual puzzles from reconstruction targets without symbolic solutions or an external verifier.



Added to this list: 2026-10-18.


### 2026-10-07 — Recurrent Self-Improvement: Dynamic Cross-Loop On-Policy Distillation for Looped Language Models

Authors: Yi Wang, Rui Qian, Yu Li, Haoyang Yao, Wenjie Wang.
Source: [arXiv:2610.10623](https://arxiv.org/abs/2610.10623). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.10623/).
Categories: Recurrent reasoning.
Summary: Trains an intermediate loop using a deeper loop of the same language model as its teacher on student-generated outputs. A dynamic variant refreshes the teacher during training, turning additional recurrent computation into a source of supervision.



Added to this list: 2026-10-18.


### 2026-10-07 — Safe at One Loop, Risky at Another: Aligning Safety Across Recurrent Depths in Looped Language Models

Authors: Yi Wang, Xiuyuan Qi, Dongqi Han, Dongsheng Li, Wenjie Wang.
Source: [arXiv:2610.10625](https://arxiv.org/abs/2610.10625). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.10625/).
Categories: Recurrent reasoning; Analysis and evaluation.
Summary: Evaluates harmful-query and jailbreak behavior at different recurrent depths of looped language models. Proposes depth-specific control, state sharing, and joint supervision to align safety behavior across depths.



Added to this list: 2026-10-18.


### 2026-10-07 — Invariant-Measure Reasoners: Stable Representations for Latent Reasoning

Authors: Yuto Inui, Takuya Konishi, Yoshinobu Kawahara.
Source: [arXiv:2610.10996](https://arxiv.org/abs/2610.10996). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.10996/).
Categories: Recurrent reasoning.
Summary: Predicts by averaging over the long-run distribution of recurrent hidden states instead of reading a single final state. This lets the model use changing hidden states without requiring them to settle at one fixed point.



Added to this list: 2026-10-18.


### 2026-10-07 — AbridgeKV: Compressing Long Reasoning into the KV Cache of Learned Pause Tokens

Authors: Jack Cai, Lijie Yang, Vasilis Kontonis, Dimitris Papailiopoulos, Shivam Garg.
Source: [arXiv:2610.20641](https://arxiv.org/abs/2610.20641). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.20641/).
Categories: Thinking tokens.
Summary: Compresses each completed block of generated reasoning into the key/value cache of learned pause tokens, then discards the block's original cache. Later tokens attend to the compact replacement.



Added to this list: 2026-10-18.


### 2026-10-08 — Scaling to Tens of Thousands of Test-Time Iterations with Loop-Native Attention Residuals

Authors: Pengxiang Li, Dilxat Muhtar, Di He, Guinan Su, Lu Yin, Shiwei Liu.
Source: [arXiv:2610.11570](https://arxiv.org/abs/2610.11570). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.11570/).
Categories: Recurrent reasoning.
Summary: Adds a learned running summary of previous recurrent states so that each update can retain useful earlier work. Uses content-based weighting and temporal decay with constant aggregation memory, and tests whether puzzle-solving improves over much longer inference runs.



Added to this list: 2026-10-18.


### 2026-10-08 — One Model to Loop Them All: Elastic Latent Recurrence for Language Models

Authors: Faisal Ladhak, Shantanu Acharya, Santiago Akle Serano, Simeng Sun, Boris Ginsburg, Krishna C Puvvada.
Source: [arXiv:2610.12582](https://arxiv.org/abs/2610.12582). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.12582/).
Categories: Recurrent reasoning.
Summary: Trains one language model to operate at different loop counts by mixing depths during training and conditioning each pass on its index and budget. A learned gate controls each recurrent update.



Added to this list: 2026-10-18.


### 2026-10-09 — Masked Diffusion Language Agents for Tool-Integrated Chemical Reasoning

Authors: Mengdi Liu, Chenghao Jia, Hong Chang, Shiguang Shan.
Source: [arXiv:2610.13330](https://arxiv.org/abs/2610.13330). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.13330/).
Categories: Alternative generation methods.
Summary: ChemDiffAgent denoises interleaved textual reasoning and tool-use trajectories, letting tool decisions depend on context across the trajectory. A chemistry benchmark and matched-budget comparisons test this alternative to left-to-right agent generation.



Added to this list: 2026-10-18.


### 2026-10-09 — S-Space: Exploring Spatial Workspace in Multimodal Language Models

Authors: Zhiyuan Shi, Xiang Li, Xingzhuo Guo, Shangchen Miao, Weiliang Chen, Qiyu Dai, Jinshan Ren, Jiangran Lyu, Fangfu Liu, Mingsheng Long, Jialong Wu.
Source: [arXiv:2610.13527](https://arxiv.org/abs/2610.13527). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.13527/).
Categories: Analysis and evaluation.
Summary: Identifies spatial-coordinate directions in multimodal model activations and tests them through causal interventions. Separating perception from viewpoint transformation reveals cases where a model represents object locations but fails to perform the required spatial computation.



Added to this list: 2026-10-18.


### 2026-10-09 — Adaptive-Depth Is a Finishing Stage

Authors: Andrei Piterbarg, Sean McLeish, Dhruv Pai, Benjamin Keigwin.
Source: [arXiv:2610.14252](https://arxiv.org/abs/2610.14252). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.14252/).
Categories: Recurrent reasoning.
Summary: Trains a looped language model at a fixed depth first, then introduces varying depths only near the end of training. Compares the resulting training cost and performance across inference depths with varying depth throughout training.



Added to this list: 2026-10-18.


### 2026-10-10 — Learning to Extrapolate in Depth with Looped Transformers

Authors: Huitao Yang, Jianhao Huang, Zixuan Wang, Baharan Mirzasoleiman.
Source: [arXiv:2610.14456](https://arxiv.org/abs/2610.14456). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.14456/).
Categories: Analysis and evaluation; Recurrent reasoning.
Summary: Analyzes how many extra loops a model trained on graph connectivity can safely run. Derives a finite range where added depth solves harder graphs before further loops create false connections.



Added to this list: 2026-10-18.


### 2026-10-10 — Knowledge transfer between simple discrete worlds

Authors: Silvère Gangloff, Jan Hula.
Source: [arXiv:2610.15081](https://arxiv.org/abs/2610.15081). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.15081/).
Categories: Recurrent reasoning; Analysis and evaluation.
Summary: Evaluates a depth-recurrent model on Sudoku-like worlds with different rule combinations. Compares symbolic rule descriptions with solved examples when the test world contains new puzzles, new combinations, or unfamiliar rule instances.



Added to this list: 2026-10-18.


### 2026-10-10 — Anchors Enhance Controllability and Reasoning of Diffusion Language Models

Authors: Feiyu Qu, Zheng Huang, Andrea Zanette.
Source: [arXiv:2610.15097](https://arxiv.org/abs/2610.15097). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.15097/).
Categories: Alternative generation methods.
Summary: Trains an anchor generator to choose a short text fragment and its position in a frozen diffusion model's output. These visible control tokens guide subsequent denoising, with rewards for correct answers and satisfied constraints.



Added to this list: 2026-10-18.


### 2026-10-10 — Running Fewer Loops Removes Refusal Behaviour in Recurrent-Depth Language Models

Authors: Seyedarmin Azizi, Erfan Baghaei Potraghloo, Arya Fayyazi, Saeid Shokoufa, Massoud Pedram.
Source: [arXiv:2610.15218](https://arxiv.org/abs/2610.15218). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.15218/).
Categories: Analysis and evaluation; Recurrent reasoning.
Summary: Studies how reducing recurrent depth changes refusal behavior in two looped language-model families. Uses probes and interventions to distinguish recognizing a harmful request from refusing it, and tests safety training across randomly selected depths.



Added to this list: 2026-10-18.


### 2026-10-11 — Large Decision Models: Repurposing Large Language Models as Single-Pass Decision Makers

Authors: Ziyi Chen, Yan Zhang, Jianhui Wei, Jianing Yin, Daoan Zhang.
Source: [arXiv:2610.15501](https://arxiv.org/abs/2610.15501). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.15501/).
Categories: Internalized reasoning.
Summary: Trains a language model to score labels supplied at inference in one forward pass, without generating a reasoning trace. Distills label distributions from repeated teacher reasoning and studies routing uncertain decisions back to explicit reasoning.



Added to this list: 2026-10-18.


### 2026-10-12 — Resample the Answer: Where to Spend Test-Time Compute in Recursive Reasoning

Authors: Maxime Illouz.
Source: [arXiv:2610.16427](https://arxiv.org/abs/2610.16427). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.16427/).
Categories: Recurrent reasoning; Analysis and evaluation.
Summary: Separately perturbs the answer state and the reasoning state of a recursive puzzle solver to study where extra search is useful. Uses the model's halting prediction to trigger restarts that resample the answer state.



Added to this list: 2026-10-18.


### 2026-10-13 — Dynamic K-Head: Difficulty-Supervised Budget Selection for Continuous Latent Reasoning

Authors: Ying Li, Yuqi Chen, Sixuan Li, Yunfeng Cai.
Source: [arXiv:2610.17679](https://arxiv.org/abs/2610.17679). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.17679/).
Categories: Continuous-state reasoning.
Summary: Predicts from the question how many continuous reasoning updates to perform. Evaluates total cost including latent updates, special tokens, and any written continuation rather than counting latent steps alone.



Added to this list: 2026-10-18.


### 2026-10-13 — Monitoring latent reasoning in looped language models

Authors: Pawel Olszowiec, Paulina Tomaszewska, Maciej Żelaszczyk, Michal Byra, Neo Christopher Chung.
Source: [arXiv:2610.18127](https://arxiv.org/abs/2610.18127). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.18127/).
Categories: Analysis and evaluation; Recurrent reasoning.
Summary: Inspects eight looped language models for state convergence, token decisions, readout quality, attention behavior, and sensitivity to injected noise. Finds important differences between models that receive the prompt once and those that reintroduce it at each loop.



Added to this list: 2026-10-18.


### 2026-10-13 — LoOPSD: On-Policy Self-Distillation with Looped Teachers

Authors: Wei Fan, Qingyue Yang, Chao-Hong Tan, Qian Chen, Wen Wang, Xiangang Li, Kejiang Chen, Weiming Zhang, Nenghai Yu.
Source: [arXiv:2610.18162](https://arxiv.org/abs/2610.18162). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.18162/).
Categories: Recurrent reasoning.
Summary: Searches for layers to repeat to make a stronger teacher from the current model, then distills its predictions into the original student. Repeats search and distillation as the student improves.



Added to this list: 2026-10-18.


### 2026-10-13 — Looped Reasoning Finishes Earlier Than You Think

Authors: Sumin Park, T. Konstantin Rusch.
Source: [arXiv:2610.18174](https://arxiv.org/abs/2610.18174). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.18174/).
Categories: Recurrent reasoning; Analysis and evaluation.
Summary: Studies how oscillations can slow convergence in recurrent reasoning models. Proposes an architecture whose rotational state changes are separated from the representation used for reasoning, and tests convergence on puzzle-solving tasks.



Added to this list: 2026-10-18.


### 2026-10-13 — Input-Conditioned Search for Efficient Test-Time Scaling of Recursive Reasoners

Authors: Haochen Shi, Tianwen Fu, Qianggang Ding, Yinhan Lu, Yizhan Li, Huan Zhang, Yajie Zhao, Xingdi Yuan, Hengyang Lu, Bang Liu.
Source: [arXiv:2610.18547](https://arxiv.org/abs/2610.18547). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.18547/).
Categories: Recurrent reasoning.
Summary: Searches over input conditions to change a recursive reasoner's update rule without retraining it. Tests whether these alternative updates can redirect a trajectory away from an incorrect solution, rather than only adding more steps or perturbing its state.



Added to this list: 2026-10-18.


### 2026-10-13 — Backpropagating Verifier Rewards Through Continuous Flow Language Models

Authors: Xu Han, Jiajing Hu, Li-Ping Liu.
Source: [arXiv:2610.19080](https://arxiv.org/abs/2610.19080). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.19080/).
Categories: Continuous-state reasoning; Alternative generation methods.
Summary: Trains a critic to score intermediate continuous token representations, enabling reward gradients through the denoising trajectory before final discrete decoding. Evaluates this post-training approach on four reasoning tasks.



Added to this list: 2026-10-18.


### 2026-10-13 — Riding the Fixed-Point Curve: Image Classification in the Continuous Thought Machine

Authors: Thomas Hannagan.
Source: [arXiv:2610.19220](https://arxiv.org/abs/2610.19220). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.19220/).
Categories: Analysis and evaluation; Recurrent reasoning.
Summary: Analyzes the Continuous Thought Machine's repeated updates during image classification. Finds that its hidden states follow moving fixed points and that much of its attention schedule is shared across images.



Added to this list: 2026-10-18.


### 2026-10-14 — ForkGRPO: Efficient Exploration and Fine-Grained Credit Assignment for Diffusion Language Models

Authors: Zhaoyan Jia, Chengwei Zhou, Gourav Datta, Massoud Pedram.
Source: [arXiv:2610.19408](https://arxiv.org/abs/2610.19408). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.19408/).
Categories: Alternative generation methods.
Summary: Branches rollouts from uncertain intermediate states and reuses their shared textual prefixes. Local outcome comparisons assign credit to reasoning segments, testing whether more targeted exploration improves diffusion-model reasoning with fewer generated training tokens.



Added to this list: 2026-10-18.


### 2026-10-14 — INTACT: Inducing Latent Reasoning on a Frozen Backbone

Authors: Kutay Tire, Atula Tejaswi, Haris Vikalo, Sujay Sanghavi.
Source: [arXiv:2610.19568](https://arxiv.org/abs/2610.19568). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.19568/).
Categories: Continuous-state reasoning.
Summary: Trains a small encoder to turn a frozen model's hidden states into successive blocks of continuous thoughts. After this latent prefix, the frozen model writes its remaining reasoning and answer; the encoder learns through distillation and answer-correctness rewards.



Added to this list: 2026-10-18.


### 2026-10-14 — Learning to Compute Is Not Learning to Retain or Reuse in Looped Transformers

Authors: Peilong Zhou, Hongtao Zhang, Yulong Huang, Cangyuan Li, Mingyu Chen, Ying Wang.
Source: [arXiv:2610.19768](https://arxiv.org/abs/2610.19768). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.19768/).
Categories: Analysis and evaluation; Recurrent reasoning.
Summary: Separates computing an answer, preserving it through additional loops, and keeping its hidden state usable for later steps. On state-machine and maze tasks, shows that training can improve one ability without securing the others.



Added to this list: 2026-10-18.


### 2026-10-14 — Rationale Distribution Distillation via Flow Matching for Distributional Continuous Latent Reasoning

Authors: Mingyeong Kim, Gyusang Cho, Juho Lee, Giung Nam.
Source: [arXiv:2610.19871](https://arxiv.org/abs/2610.19871). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.19871/).
Categories: Continuous-state reasoning; Alternative generation methods.
Summary: Distills multiple written reasoning paths for each question into a flow model over fixed-length latent sequences, including mixtures of complete rationales. Generates all latent slots together, then uses a frozen language model to decode answers from sampled rationale contexts.



Added to this list: 2026-10-18.


### 2026-10-15 — Does Latent Chain-of-Thought Actually Reason? A Single, Unifying Taxonomy for Latent Reasoning

Authors: Jingjie Qiu, Sheldon Yu, Xunyi Jiang, Jiajin Liu, Zihan Huang, Zhendong Chu, Tong Yu, Julian McAuley, Junda Wu.
Source: [arXiv:2610.21182](https://arxiv.org/abs/2610.21182). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.21182/).
Categories: Analysis and evaluation; Continuous-state reasoning.
Summary: Uses chain deletion, content replacement, probes, and trajectory analysis across 21 model-task configurations. Separates what continuous states encode from whether the model uses their content, finding that dependence on the chain varies with task and training recipe.



Added to this list: 2026-10-18.


### 2026-10-15 — LoopAhead: Speculating Ahead for Faster Decoding in Looped Language Models

Authors: Zixuan Liu, Nan Xue, Rui Bao, Yuhang Xu, Xiaofeng Cao, Jiangchao Yao, Zhiyong Chen.
Source: [arXiv:2610.21594](https://arxiv.org/abs/2610.21594). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2610.21594/).
Categories: Recurrent reasoning; Alternative generation methods.
Summary: Trains a small drafter to predict several future tokens from early-loop hidden states. Later loops decide which speculative branches to continue, and full-depth verification determines the final output.



Added to this list: 2026-10-18.


## Related background papers

### 2016-03-29 — Adaptive Computation Time for Recurrent Neural Networks

Authors: Alex Graves.
Source: [Original paper](https://arxiv.org/abs/1603.08983). [Entry and citations](https://latent-implicit-thinking.github.io/papers/1603.08983/).
Why it is relevant: Introduces learned, input-dependent computation time for recurrent networks.


### 2018-07-10 — Universal Transformers

Authors: Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, Łukasz Kaiser.
Source: [Original paper](https://arxiv.org/abs/1807.03819). [Entry and citations](https://latent-implicit-thinking.github.io/papers/1807.03819/).
Why it is relevant: Applies recurrence across transformer depth and supports dynamic halting.


### 2019-09-03 — Deep Equilibrium Models

Authors: Shaojie Bai, J. Zico Kolter, Vladlen Koltun.
Source: [Original paper](https://arxiv.org/abs/1909.01377). [Entry and citations](https://latent-implicit-thinking.github.io/papers/1909.01377/).
Why it is relevant: Represents effectively infinite depth through a learned fixed point.


### 2021-06-08 — Can You Learn an Algorithm? Generalizing from Easy to Hard Problems with Recurrent Networks

Authors: Avi Schwarzschild, Eitan Borgnia, Arjun Gupta, Furong Huang, Uzi Vishkin, Micah Goldblum, Tom Goldstein.
Source: [Original paper](https://arxiv.org/abs/2106.04537). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2106.04537/).
Why it is relevant: Studies recurrent computation as a route to algorithmic length generalization.


### 2021-07-12 — PonderNet: Learning to Ponder

Authors: Andrea Banino, Jan Balaguer, Charles Blundell.
Source: [Original paper](https://arxiv.org/abs/2107.05407). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2107.05407/).
Why it is relevant: Learns a probabilistic halting policy for adaptive internal computation.


### 2023-05-24 — Can Transformers Learn to Solve Problems Recursively?

Authors: Shizhuo Dylan Zhang, Curt Tigges, Stella Biderman, Maxim Raginsky, Talia Ringer.
Source: [Original paper](https://arxiv.org/abs/2305.14699). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2305.14699/).
Why it is relevant: Tests whether transformers learn recursive procedures and generalize beyond training depth.


### 2023-11-21 — Looped Transformers are Better at Learning Learning Algorithms

Authors: Liu Yang, Kangwook Lee, Robert Nowak, Dimitris Papailiopoulos.
Source: [Original paper](https://arxiv.org/abs/2311.12424). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2311.12424/).
Why it is relevant: Shows how looped transformer depth can support learning iterative algorithms.


### 2024-02-21 — AlgoFormer: An Efficient Transformer Framework with Algorithmic Structures

Authors: Yihang Gao, Chuanyang Zheng, Enze Xie, Han Shi, Tianyang Hu, Yu Li, Michael K. Ng, Zhenguo Li, Zhaoqiang Liu.
Source: [Original paper](https://arxiv.org/abs/2402.13572). [Entry and citations](https://latent-implicit-thinking.github.io/papers/2402.13572/).
Why it is relevant: Builds iterative algorithmic structure into transformer computation.


## Acknowledgment and reuse

Thanks to the [Awesome-Latent-CoT bibliography](https://github.com/EIT-NLP/Awesome-Latent-CoT) and its associated survey, which helped us find many of these papers.

Original curation metadata, taxonomy, and editorial material: [CC BY 4.0](https://latent-implicit-thinking.github.io/license/). Underlying papers and code retain their own terms. [Citation guidance](https://latent-implicit-thinking.github.io/data/#citation).