From single-source RAG to multi-agent survey workflows
The prior frontier treated survey generation as a two-stage pipeline: retrieve literature, then generate text. SurveyGen formalized this as a quality-aware framework using semantic retrieval and citation expansion, and argued that recall-only citation evaluation overstates model performance, motivating precision as a complementary metric [2]. Citegeist pushed retrieval further with dynamic RAG over the arXiv corpus, embedding-based similarity, multi-stage filtering, and user-facing breadth, depth, and diversity parameters, but its evaluation relied on LLM-as-a-judge because human annotators were unavailable [3]. SurveyAgent-HKA inherits this retrieval-centric view but rejects the one-pass generation assumption: it retrieves from multiple databases (Semantic Scholar, arXiv, PubMed), applies co-citation expansion to recover foundational papers that semantic similarity misses, and re-ranks candidates using citation counts, recent popularity, author and venue influence, and content diversity [1].
The practical consequence is that retrieval quality and generation quality are decoupled and separately optimized. In the paper's own analysis, purely similarity-based retrieval performs worst and barely improves as Top-K grows, whereas citation-aware methods improve until roughly K=50 before saturating; the authors' combined strategy continues to improve because it also captures recent, not-yet-highly-cited work [1]. This is a methodological claim about what evidence a survey agent should see, not merely an engineering detail.
Human outlines and peer-review comments as revision signals
The anchor paper's distinctive move is to treat published surveys and their peer reviews as reusable expert knowledge. The [RETRIEVER] agent collects human-authored survey outlines and topic-relevant peer-review comments alongside papers; the [OUTLINER] builds an initial outline from clustered literature and then refines it using those human outlines; the [REVIEWER] simulates a peer reviewer by assessing the draft against issues experts actually raised in published survey reviews, and the [REFINER] polishes the result [1]. This is a different use of human knowledge than SurveyGen's quality-aware retrieval [2] or Citegeist's retrieval parameterization [3]: here human input shapes structure and revision rather than only candidate selection.
The few-shot outline analysis supports the mechanism but also bounds it. Structural consistency for first-level and second-level headings improves as human-written outline examples increase, peaks at 10 examples, and then declines because additional examples introduce irrelevant or redundant structural cues [1]. So the benefit is real but non-monotonic, and the paper does not establish how to select the optimal example set automatically.
What the comparisons actually show, and what they do not
Against baselines including Naive-RAG, AutoSurvey, SurveyX, SurveyForge, SurveyGen, and DeepResearch, SurveyAgent-HKA reports the best citation quality in both domains: 14.95% precision, 19.53% recall, and 16.94% F1 in computer science, and 16.78%, 22.95%, and 19.39% in medicine [1]. It also reports the highest structural match with human-written surveys and the best LLM-judged structural relevance (3.5/5 computer science, 3.9/5 medicine), plus key-point recall of 53.85% and 57.07% [1]. These numbers matter because they are measured against human-written surveys as gold standards, not only against other systems.
Two comparisons temper the headline. First, the paper itself notes that its content overlap ranks second to the commercial DeepResearch model, so the framework does not dominate on every content dimension [1]. Second, the distribution analysis shows that all models, including this one, still diverge significantly from human reference selection across citation count, author H-index, journal H-index, and time gap; human surveys may cite papers up to 30 years old that never entered mainstream databases [1]. The authors state plainly that human selection behavior is complex and difficult to replicate.
Competing architectures and the evaluation-validity question
Not every multi-agent design converges on the same conclusion. CREW uses collaborative multi-agent reinforcement learning with a learned decentralized policy for related work generation, reporting a 0.06-point overall quality gain over a single-agent baseline and a 23.54% improvement in citation verification while reducing token usage by 12.90% [4]. Its ablation shows that removing the feedback action causes the largest degradation, dropping overall score to 2.58, which independently supports the anchor paper's emphasis on critique-and-revise loops [4]. However, CREW operates over a fixed candidate pool rather than live scholarly databases and acknowledges that its automated rewards may not capture field-specific scholarly conventions [4].
Evaluation validity is the sharper unresolved issue. GREP shows that specialized state-of-the-art LLMs match expert judgments only about 53% of the time on citation faithfulness, while its multi-turn framework reaches 78% and 66% agreement, and that the best model could coherently cite prior work only 20% of the time [6]. A separate attribution study finds that retrieval augmentation, not the citation paradigm, is the primary driver of citation accuracy, with post-hoc citation favored for high-stakes coverage and generation-time citation for precision-critical verification [7]. Since SurveyAgent-HKA relies on LLM-based structural and content scoring alongside human-survey gold standards [1], these findings suggest its reported structural gains should be read as agreement with one evaluation protocol, not as a validated proxy for expert preference.
Boundaries: two domains, no visual content, no substitute for experts
The evidence boundary is explicit in the paper. Experiments cover only computer science and medicine, and the authors state that applicability to other disciplines, venues, survey lengths, and emerging areas remains unexamined [1]. Generated surveys contain no figures or tables, and the evaluation set is small because of long-form API costs [1]. The authors also flag potential data contamination: GPT-4o's October 2023 and Claude-3.5-Sonnet's April 2024 knowledge cutoffs mean some human-written reference surveys may have appeared in pretraining data, which could inflate both generation and LLM-based evaluation, even though they excluded each target survey during experiments [1].
The authors' own conclusion is the correct frame for readers: generated surveys still lag expert-written ones in identifying key references, maintaining balanced outlines, and providing in-depth cross-paper comparisons, and should be treated as supportive material for literature exploration rather than substitutes [1]. A separate multi-agent citation-graph framework is noted as reporting that human surveys remain superior on coverage [5], which is consistent with this boundary. The open questions are therefore not whether multi-agent survey generation works at all, but how to select human revision signals without overfitting, how to evaluate against expert preference rather than LLM judges, and whether the approach transfers beyond the two tested domains.
About These Sources
This research page is built on 7 peer-reviewed studies — published from 2025 to 2026, 7 from 2024 or later — selected as the most relevant from 8 studies that passed quality screening, drawn from 80 papers retrieved from a database of over 500 million.
Sources used in this answer
SurveyAgent-HKA: A multi-agent framework for scientific survey generation with LLMs and human knowledge augmentation
SurveyAgent-HKA introduces a multi-agent framework that augments LLM survey generation with human-written outlines and peer-review comments, reporting the best citation quality and structural consistency among tested baselines in computer science and medicine while acknowledging that outputs remain supportive rather than expert-level.
Surveygen: Quality-aware scientific survey generation with large language models
SurveyGen established a quality-aware survey generation framework using semantic retrieval and citation expansion, and argued that citation recall alone overstates performance, motivating the addition of citation precision.
Citegeist: Automated Generation of Related Work Analysis on the arXiv Corpus
Citegeist demonstrated dynamic retrieval-augmented generation over the arXiv corpus for citation-backed related work, exposing breadth, depth, and diversity as user parameters but relying on LLM-as-a-judge evaluation without human annotators.
Assembling the CREW: A Collaborative Multi-agent Reinforcement Learning Framework for Automated Related Work Generation
CREW applies collaborative multi-agent reinforcement learning with a learned decentralized policy to related work generation, improving citation verification by 23.54% over a single-agent baseline while operating over a fixed candidate pool and acknowledging limits in capturing scholarly conventions.
A Multi-Agent LLM Framework with Hierarchical Citation Graph for Automated Survey Generation
A multi-agent LLM framework with a hierarchical citation graph is reported to show superior robustness and generalization across survey generation tasks, but notes that human surveys remain superior on coverage.
Expert Preference-based Evaluation of Automated Related Work Generation
GREP shows that specialized LLMs match expert judgments only about 53% of the time on citation faithfulness, while its multi-turn evaluation framework reaches 78% and 66% agreement, and that the best model coherently cited prior work only 20% of the time.
Generation-Time vs. Post-hoc Citation: A Holistic Evaluation of LLM Attribution
A holistic evaluation of generation-time versus post-hoc citation finds that retrieval augmentation is the primary driver of citation accuracy, with post-hoc citation favored for high-coverage high-stakes use and generation-time citation for precision-critical verification.