Asymmetric Looped Transformers: Fixed Global Representations for an Iterative Decoder

Cheng Liang, Tengfei Zhang, Wenjie DU, Jianyu Wu, Changlian Ma, Shuchen Pu, Ya Zhang, Yanfeng Wang, Zhouhan Lin

Runs a causal encoder once, then repeatedly applies a shared decoder that attends to fixed encoder context and its evolving hidden states. Evaluates this division of work against fully looped and standard language models during pretraining.

First public: October 20, 2026 · Added to this list: October 22, 2026