LSTM-Style Looped Transformers with Shared KV Caches
Adds an LSTM-style gated memory across transformer loops and shares one KV cache across the recurrent core. Aligns cache entries from two training passes with a consistency loss to reduce the mismatch during autoregressive decoding.
First public: · Added to this list: November 2, 2026
Explore this approach
Recurrent reasoning — Repeat parts of the model or keep intermediate information in memory while working through a problem. Some methods also learn how many passes to run.