Do Transformers Use their Depth Adaptively? Evidence from a Relational Reasoning Task
Studies when transformer layers combine family relationships to answer multi-hop questions. Uses logit-lens probes and causal activation patching to compare depth use before and after task fine-tuning.
First public: · Added to this list: November 9, 2026
Explore this approach
Analysis and evaluation — Measure how well these methods work, investigate what their internal computation does, and identify where they fail. This includes experiments, studies of model internals, and mathematical results.