Merton101 · Case 17 · Science and publishing

Major AI conferences desk-reject papers over LLM-hallucinated citations

Reported 5 Sep 2026 · entered the ledger 25 Sep 2026 · last checked 25 Sep 2026 · Side effect

Academic machine learning conferences introduced automated reference verification and desk rejections after discovering widespread LLM-hallucinated citations in submitted and accepted papers, leading to hundreds of desk rejections, high false-positive rates, and appeals.

The institution Academic computer science conferences (NeurIPS, ICLR, ICML, ACL) institution

The mechanism Friction removed externality scale 3 of 5, fast

Adaptation verify every submission is checked before it counts

The case as the ledger holds it

The assumption that broke: Citations in submitted scientific papers correspond to real literature verified by human authors and scrutinised by peer reviewers.
The first-order effect: Four major machine learning and NLP conferences (NeurIPS, ICLR, ICML, ACL) instituted automated cross-checks of citations against bibliographic databases and desk-rejected submissions with non-existent references, causing over 100 desk rejections at ACL alone, elevated desk-rejection rates at ICLR, and emergency review processes for false positives.
Who pays: Program chairs, senior area chairs, and reviewers spending hours adjudicating flagged references, plus authors navigating erroneous desk-rejections caused by false positives · group: Volunteers and reviewers
Scale and speed: 3 of 5 · fast

Evidence 1 page read

StrictCite ↗ 5 Sep 2026
Four major venues this cycle — NeurIPS, ICLR, ICML, and ACL — have all treated a hallucinated citation as grounds for desk rejection, and three of the four have published enough about their process to check.

The second reader every claim, checked against the page

Claim	Verdict	Note
Four major venues this cycle — NeurIPS, ICLR, ICML, and ACL — have all treated a hallucinated citation as grounds for desk rejection	supported	Page states verbatim: "Four major venues this cycle — NeurIPS, ICLR, ICML, and ACL — have all treated a hallucinated citation as grounds for desk rejection".
ACL's program chairs published a statement confirming that more than 100 accepted papers were desk-rejected over citations to works that do not exist	supported	Page: "ACL's program chairs published a statement confirming that more than 100 accepted papers were desk-rejected over citations to works that do not exist".
48,095 accepted papers and 2.6 million extracted references across ICLR, ICML, NeurIPS and USENIX Security from 2021 through 2026 were audited against Semantic Scholar, OpenAlex, Crossref, DBLP and the ACL Anthology	supported	Page: "48,095 accepted papers and 2.6 million extracted references across ICLR, ICML, NeurIPS and USENIX Security from 2021 through 2026, checked against Semantic Scholar, OpenAlex, Crossref, DBLP and the ACL Anthology".
In 2025 accepted papers, at least one hallucinated reference appeared in 18.7% of ICLR, 23.3% of ICML, 26.2% of NeurIPS, and 34.9% of USENIX Security papers	supported	Table "Venue (2025) \| ≥1 hallucinated ref": ICLR 18.7%, ICML 23.3%, NeurIPS 26.2%, USENIX Security 34.9%.
In 2025 accepted papers, two or more hallucinated references appeared in 1.9% of ICLR, 3.4% of ICML, 5.1% of NeurIPS, and 4.8% of USENIX Security papers	supported	Table "Venue (2025) \| ≥2 hallucinated refs": ICLR 1.9%, ICML 3.4%, NeurIPS 5.1%, USENIX Security 4.8%.
NeurIPS 2026 area chair Danish Pruthi posted that in his batch, '5 out of 8 submissions have 2+ hallucinated references and will likely be desk rejected'	supported	Page: "a NeurIPS 2026 area chair did, in public, under their own name. Danish Pruthi posted that in his own area chair batch, '5 out of 8 submissions have 2+ hallucinated references and will likely be desk rejected'." Quote and figures match.
Comparing reviewer scores on affected versus clean papers, the gap was +0.02 at ICLR, +0.005 at ICML, +0.04 at NeurIPS, showing peer review failed to catch fabricated citations	supported	Page: "the gap was +0.02 at ICLR, +0.005 at ICML, +0.04 at NeurIPS — noise, not signal... a fabricated citation moves the outcome not at all." Numbers match and the page's own framing is that peer review does not catch this.