What Makes Looped Transformers Perform Better Than Non-Recursive Ones
Studies how repeated attention changes optimization through a model of loss-landscape geometry. Proposes SHIFT, which starts with one attention pass and adds shared-weight repetitions after learning stalls, and tests the schedule on synthetic sequence tasks.
First public: · Added to this list: November 3, 2026
Earlier title: What Makes Looped Transformers Perform Better Than Non-Recursive Ones (Provably).
Explore this approach
Recurrent reasoning — Repeat parts of the model or keep intermediate information in memory while working through a problem. Some methods also learn how many passes to run.
Analysis and evaluation — Measure how well these methods work, investigate what their internal computation does, and identify where they fail. This includes experiments, studies of model internals, and mathematical results.