# Implicit Reasoning in Transformers: Bridging the Compositionality Gap in GPT-2-like Models

Authors: Davide Venuto, Leonardo Venuto.

Investigates two-hop factual reasoning in a GPT-2-like model trained from scratch. Aligns the positions of atomic facts with the second hop of composite queries, then tests whether the model can answer unseen combinations without written intermediate steps.

First public: 2026-10-28.
ArXiv initial submission: 2026-10-28.

[Original paper](https://arxiv.org/abs/2610.39016) · [PDF](https://arxiv.org/pdf/2610.39016) · [HTML](https://arxiv.org/html/2610.39016v1) · [Reading-list entry](https://latent-implicit-thinking.github.io/papers/2610.39016/)
[Preprint BibTeX](https://latent-implicit-thinking.github.io/papers/2610.39016/citation.bib) · [Preprint CSL-JSON](https://latent-implicit-thinking.github.io/papers/2610.39016/citation.csl.json) · [JSON record](https://latent-implicit-thinking.github.io/papers/2610.39016/record.json)

## Approaches

- [Analysis and evaluation](https://latent-implicit-thinking.github.io/topics/implicit-reasoning-evaluation/): Measure how well these methods work, investigate what their internal computation does, and identify where they fail. This includes experiments, studies of model internals, and mathematical results.

Reading-list version 0.12.0; literature checked through 2026-11-02.