Implicit Reasoning in Transformers: Bridging the Compositionality Gap in GPT-2-like Models
Investigates two-hop factual reasoning in a GPT-2-like model trained from scratch. Aligns the positions of atomic facts with the second hop of composite queries, then tests whether the model can answer unseen combinations without written intermediate steps.
First public: · Added to this list: November 2, 2026
Explore this approach
Analysis and evaluation — Measure how well these methods work, investigate what their internal computation does, and identify where they fail. This includes experiments, studies of model internals, and mathematical results.