What earlier OPD work established before the one-shot result
On-policy distillation has become a core post-training technique because the student samples its own rollouts while the teacher supplies dense token-level supervision at every visited prefix, avoiding the exposure bias of off-policy distillation on fixed teacher-generated sequences [3]. Industry pipelines including Qwen3, MiMo, and GLM-5 adopt OPD alongside supervised fine-tuning and outcome-reward reinforcement learning, and Thinking Machines Lab replicated the Qwen3 OPD recipe at a fraction of the RL compute cost [3]. The precursor paper by Li et al. identified two conditions governing OPD success: the student and teacher must share compatible thinking patterns, and the teacher must offer genuinely new capabilities beyond what the student has already seen [3]. It also showed that successful OPD is characterized by progressive alignment on high-probability overlap tokens at student-visited states, a small shared token set that concentrates 97%-99% of the probability mass [3]. Critically, benchmark performance did not predict OPD outcome: R1-Distill-7B outscored JustRL-1.5B yet produced the same regressed distillation result, because same-family 1.5B and 7B teachers were distributionally indistinguishable from the student's perspective [3].
One query recovers most of full-data OPD's gain
The anchor paper reduces the data supply to its minimum by training on a single query, and the learning curve is strikingly similar to one-shot RLVR: the student keeps improving for hundreds of steps [1]. In mathematics, one-shot OPD reaches 68.5 against 69.8 for full-data OPD at step 300, recovering 69% of the teacher-student gap and 87% of full-data OPD's gain; beyond step 300 both curves stay within about 3 points, and the recovered fraction ranges from 62% to 89% through step 1000 [1]. The effect is robust across model families: averaged math scores rise from 77.1, 28.2, and 70.8 for R1-Distill-1.5B, Llama-3B-It, and OLMo-7B-It-DPO baselines to 85.5, 40.2, and 82.4 after one-shot OPD [1]. It also extends beyond mathematics, with one-shot OPD recovering 73%, 66%, and 64% of the teacher-student gap on code generation, instruction following, and agentic tool use respectively [1]. The gain is insensitive to query difficulty, response-length budget, and sampling temperature, and a query the student never solves is as effective as one it always solves [1].
State coverage, not query count, explains the data efficiency
The anchor paper's central measurement is state coverage: states are represented by the teacher's final-layer hidden vector at each prefix, clustered into 200 groups via PCA and K-means over full-data OPD rollouts, and coverage is the fraction of clusters a setting reaches [1]. One-shot OPD reaches 71.5% state coverage by step 300, with 65.9% already reached by step 100 and only 5.6 percentage points added over the next 200 steps [1]. Adding semantically distinct queries raises coverage and validation accuracy together: 4 queries reach 79.8% coverage and 0.699 accuracy, 16 queries reach 98.9% coverage and 0.709 accuracy, matching full-data OPD's 0.708 [1]. The causal ablation holds the query fixed while varying distinct trajectories from 1 to 64, and accuracy rises from 0.606 to 0.666, confirming that state diversity rather than query identity drives the gain [1]. This connects to the precursor's token-overlap mechanism: OPD succeeds by progressively aligning on high-probability tokens at student-visited states, so what matters is which states the rollouts reach, not how many problems are collected [3].
The algorithm, not the data, sets how long a run takes
The anchor paper's second finding is that alignment slows at a similar pace whether OPD trains on one query or the whole dataset, and even a fixed set of states takes hundreds of steps to absorb [1]. The top-16 overlap ratio climbs to the full-data level, the overlap-token advantage approaches zero, and the absolute entropy gap nearly closes over hundreds of steps under both settings [1]. Removing the supply of fresh states therefore does not shorten the run, which suggests a fixed set of states is enough on its own to keep an OPD run going for hundreds of steps [1]. This is where the anchor paper diverges from a purely data-centric reading of the precursor: Li et al. showed that OPD learns thinking patterns and that prompt sets closer to the teacher's post-training data can sharpen alignment, but they did not isolate whether data quantity or absorption rate limits the run [3]. The anchor paper's answer is that the optimizer, not the training set, sets the run length, which is why it calls OPD data-overfed but algorithm-starved [1].
Multi-teacher OPD and content-light inputs test the boundary
The state-coverage result extends to multi-teacher OPD, where one student trains on several domains in a single run and each query is routed to its domain teacher [1]. Full-data MOPD raises average validation accuracy from 43.5 to 52.8, recovering 79% of the teacher-student gap and coming within 1.0 point of three separate full-data OPD runs at 53.8 [1]. Increasing the query count from 1 to 16 per domain raises average accuracy from 50.1 to 52.9, matching full-data MOPD and recovering 101% of its gain, with per-domain proportions of 93% for mathematics, 136% for code generation, and 109% for instruction following [1]. As a further stress test, content-light templates and off-domain WildChat queries also approach the real-query baseline, reaching that level on a third to a half of its rollout tokens, though a scaffold that closes the thinking block immediately collapses into short meta-level replies [1]. The authors caution that this does not imply task content is generally dispensable; it shows that content is not the only source of useful signal, and explicit domain content changes final performance by about one point in these experiments [1].
Competing evidence and where the conclusion stops
A competing line of work reaches a different conclusion about what makes distillation data useful. SRPO uses reflection-conditioned teacher scores on student on-policy rollouts as dynamic targets, and its authors report that semantic content is essential: matching format, style, and token count of reflections does not transfer benefits, and only task-relevant semantic content produces improvements [4]. They argue this rules out explanations based on teacher forcing or exposure to future information, because mismatched reflections should provide similar benefits if those mechanisms were responsible [4]. The anchor paper's content-light templates appear to conflict with this, but the settings differ: SRPO distills from a self-teacher conditioned on a reflection patch, while the anchor paper distills from a separate domain teacher on student rollouts, and the anchor paper explicitly notes that a scaffold closing the thinking block immediately collapses [1][4]. The anchor paper's own limitations are also explicit: state coverage is a semantic-level proxy measured against a reference space built from full-data rollouts, so it reports how much of that space a query set reaches rather than what it covers on its own, and it weights every cluster equally regardless of visit frequency or remaining teacher signal [1]. What sets the absorption rate is still open, and the MOPD runs use only three domains with one teacher each, leaving open how far the diversity result extends as teachers are added [1].
Why this matters for post-training practice
The anchor paper shifts data design from collecting problems to choosing teachers and states: an input is useful when the student's rollouts reach regions where the teacher can provide useful supervision, and stating a problem is only one way to reach them [1]. This connects to a broader data-centric theme in distillation research, where dataset distillation synthesizes small informative datasets to compress training data, and surveys note that distilled data can discard semantically meaningful information and produce overconfident predictions [5]. The anchor paper's state-coverage measure offers a selection criterion once it can be estimated from queries themselves without a full-data run to measure against, and the authors propose making training more step-efficient by reusing a batch for several epochs under a trust region on the per-token gap or weighting tokens by how much teacher signal they still carry [1]. The comparison with one-shot RLVR sharpens the distinction: RLVR updates from trajectory outcomes and its signal weakens as the model learns to solve the training query, while OPD can continue learning from local teacher-student gaps after the query is solved, though those gaps gradually shrink [1]. Over 1000 steps, OPD closes 72% of its gap to the teacher and its gain in validation accuracy is more than twice that of RLVR on the same query, and the advantage remains when matched by rollout tokens rather than training steps [1].
About These Sources
This research page is built on 5 studies (4 peer-reviewed, 1 preprint) — published from 2023 to 2026, 4 from 2024 or later, collectively cited 420 times — selected as the most relevant from 11 studies that passed quality screening, drawn from 67 papers retrieved from a database of over 500 million.
Sources used in this answer
Rethinking On-Policy Distillation of Large Language Models II: One Training Example
The anchor paper shows that one-shot on-policy distillation recovers most of full-data OPD's gain across four task domains and three model families, explains this via state coverage where one query reaches 71.5% of full-data states and 16 diverse queries reach 98.9%, and identifies the absorption rate rather than data supply as the limiting factor.
Knowledge Distillation in Object Detection: A Survey from CNN to Transformer.
This survey establishes knowledge distillation as a foundational technique for compressing large object detection models into efficient students while maintaining accuracy, covering CNN-to-Transformer architectures and applications from 2D detection to 3D reconstruction and document analysis.
Rethinking on-policy distillation of large language models: Phenomenology, mechanism, and recipe
The precursor paper identifies thinking-pattern compatibility and genuinely new teacher knowledge as the two conditions governing OPD success, shows that successful OPD aligns on high-probability overlap tokens at student-visited states, and demonstrates that benchmark performance does not predict OPD outcome.
SRPO: Self-Reflective Policy Optimization for Long-Horizon Reasoning
SRPO presents competing evidence that semantic content in reflections is essential for self-reflective policy optimization, with mismatched reflections failing to transfer benefits, and reports that self-distillation achieves competitive performance without larger teachers while better preserving capabilities in continual learning.
A comprehensive survey of dataset distillation
This dataset distillation survey taxonomizes methods into meta-learning and data matching frameworks, notes limitations in distilling high-resolution data and complex label spaces, and observes that models trained on distilled data tend to output overconfident predictions due to discarding semantically meaningful information.