ScaleRLT: Exact Log-Probabilities and Fast Deferred Rollouts for Reinforcement Learning on Looped Transformers

Dezhen Lu, Zixiang Ni, Runguo Li

Batches later transformer loops across generated positions to accelerate reinforcement-learning rollouts. Replays the rollout schedule to recover its exact token log probabilities while using a dense pass for gradients, and evaluates the resulting training system on math problems.

First public: October 25, 2026 · Added to this list: November 2, 2026