Loop Dropout: Regularizing Shared Updates in Looped Language Models

Zirui Zhu, Hailun Xu, Xuanlei Zhao, Yong Liu, Yingxuan Ren, Kanchan Sarkar, Kun Xu, Yang You

Regularizes fine-tuning of looped language models by randomly masking adapter updates at selected loops. The shared backbone still runs at every loop; the method targets adapter updates that otherwise become concentrated at particular depths.

First public: September 28, 2026 · Added to this list: October 18, 2026