OpenNovelty: The LLM-Powered Agentic System Making Scholarly Novelty Verifiable

In the era of "publication explosion," the burden on academic reviewers has reached a breaking point. Top-tier conferences like NeuriPS and ICLR receive tens of thousands of submissions, making it nearly impossible for a single human reviewer to maintain a comprehensive view of the rapidly evolving frontier. This leads to subjective, inconsistently verified novelty judgments.

Enter OpenNovelty, a sophisticated agentic system designed by researchers from Fudan University to provide transparent, evidence-based novelty analysis. Unlike naive LLM approaches that rely on "parametric memory" (and thus hallucinate), OpenNovelty is built on a "Retrieval and Full-Full-Text Verification" philosophy.

The Architecture of Truth: How OpenNovelty Works

OpenNovelty operates through a four-phase pipeline designed to move from a single PDF to a structured, auditable report.

1. Information Extraction & Query Expansion

Using Claude-Sonnet-4.5, the system extracts the "Core Task" (the problem addressed) and specific "Claimed Contributions." To solve the ambiguity of academic terminology, the system generates primary queries and multiple semantic variants (e.g., "RL" vs "Reinforcement Learning") to ensure maximum coverage during retrieval.

2. Precision Retrieval and Filtering

The system uses the Wispaper search engine to recall hundreds of papers. It then applies a multi-layer filtering strategy:

• Quality Filtering: Retaining only papers with high semantic relevance.
• Temporal Filtering: Excluding papers published after the target paper.
• Deduplication: Merging results across different contribution queries.

3. Taxonomy Construction & Full-Text Comparison

This is the "analytical heart" of the system. Instead of simple clustering, OpenNovelty builds a Hierarchical Taxonomy where every retrieved paper is assigned to a leaf node following MECE (Mutually Exclusive, Collectively Exhaustive) principles.

Crucially, the system performs Contribution-level Comparison. It doesn't just look at abstracts; it compares the full text of the target paper against the full text of top candidates.

4. Verifiable Evidence (No More Hallucinations)

A "can_refute" judgment is only allowed in the final report if the system can provide verbatim quotes from both the target and the candidate paper. These quotes are verified via a token-level anchor alignment algorithm. If the quote cannot be found exactly in the source PDF, the judgment is automatically downgraded.

Core Breakthroughs

• MECE Taxonomy: Provides a systematic map of the research field, allowing reviewers to see if a paper sits in a "crowded" or "sparse" sub-topic.
• Plagiarism Detection: Includes a textual similarity module to identify extensive unattributed reuse or duplicate submissions.
• Scalability: Already deployed on over 500 ICLR 2026 submissions with public reports available at opennovelty.org.

Limitations and Future Outlook

While powerful, the authors acknowledge that OpenNovelty still struggles with:

1. Mathematical Formulas: PDF extraction of complex equations remains unstable.
2. Visual Content: It cannot currently analyze figures, tables, or architecture diagrams.
3. Indexing Boundaries: It is limited by the search engine's database (Wispaper).

Conclusion

OpenNovelty is not meant to replace human reviewers but to empower them. By providing a "pre-check" that surfaces relevant prior work and highlights potential novelty overlaps with verified evidence, it brings a new level of rigor to the scientific peer review process.