Not All Loops Cost the Same: Phase-Dependent Depth for Serving Looped Transformers

Zhixu Du, Yuhan Cheng, Hai "Helen" Li, Yiran Chen

Uses fewer recurrent steps for prompt tokens while retaining more computation for prompt endpoints and generated tokens. Removes finished positions from later loop calls and adjusts prompt depth as the serving queue grows.

First public: November 4, 2026 · Added to this list: November 9, 2026