How to Loop MoE: Flatten the Experts, Untie the Attention

Shouren Wang, Chuang Ma, Mohsen Hariri, Debargha Ganguly, Wang Yang, Xiaoqing Tong, Qianying Liu, Xiaotian Han, Vipin Chaudhary

Builds a looped mixture-of-experts model by pooling experts across layers and revisiting them. Shares experts and routing across passes while giving each pass its own attention parameters.

First public: September 28, 2026 · Added to this list: October 18, 2026