Best-of-Depth: Learning Exploratory Dynamics in Looped Transformers

Shota Takashiro, Takeshi Kojima, Masanori Koyama, Kohei Hayashi, Yusuke Iwasawa

Trains a looped transformer using the lowest-loss prediction along its recurrent trajectory. Different depths can produce different valid solutions, which are evaluated on puzzles and code generation.

First public: November 3, 2026 · Added to this list: November 9, 2026