Predictive Latents for Autoregressive Language Models: Possibility, Gauge, and Obstruction
Studies when predicting the next hidden state preserves information needed for next-token prediction. Gives conditions and counterexamples showing why good one-step predictions need not yield reliable multi-step hidden-state rollouts.
First public: · Added to this list: October 22, 2026
Explore this approach
Analysis and evaluation — Measure how well these methods work, investigate what their internal computation does, and identify where they fail. This includes experiments, studies of model internals, and mathematical results.