AlterNet: Adaptive-Depth Mixture-of-Experts with Learned Halting for Elastic Inference
Routes each sequence through shallow experts repeatedly, using the current hidden state and depth to choose experts. Trains expert-specific stopping signals alongside the language model so computation can end at different depths.
First public: · Added to this list: November 9, 2026
Explore this approach
Recurrent reasoning — Repeat parts of the model or keep intermediate information in memory while working through a problem. Some methods also learn how many passes to run.