Reading Between the Dots: Decoding Hidden Computation across Filler Tokens
Studies how models use filler tokens to compute without writing intermediate steps. Reads intermediate values from hidden states and swaps cached states between examples to test their effect on answers, then evaluates an unsupervised decoder on fact retrieval, arithmetic and string tasks.
First public: · Added to this list: October 23, 2026
Explore this approach
Thinking tokens — Give a model extra tokens for computation, such as pauses, filler symbols, or learned planning tokens. These tokens need not spell out the solution.
Analysis and evaluation — Measure how well these methods work, investigate what their internal computation does, and identify where they fail. This includes experiments, studies of model internals, and mathematical results.