Browse
Reference
Directory
Company
Help & Legal
Follow
Subscribe
Quick Take
Anthropic’s Claude Fable 5.2 correctly named the writer of 56% of 398 reviews and meta-reviews in a small test of unsigned peer reviews. Six volunteers wrote the reports. The model saw one report at a time, without web access, tools, a list of possible authors or other writing samples.
In a Nov. 19 preprint, Atrey Desai and colleagues analyzed reports written between 2015 and 2026 for 26 double-blind conferences and workshops in machine learning and natural language processing. The researchers knew who wrote each report, but the reviewers’ names had never been publicly linked to them. In 19 cases, the text itself was already publicly available without a name.
Claude named the writer on 68.4% of 266 standard reviews and 31.1% of 132 meta-reviews, which summarize the reviews of a submission. The meta-reviews were shorter and came from just two of the six contributors. The study did not separate the effects of report length, type and authorship.
The 56% result counts each report equally, so contributors who supplied more reports had more influence on the overall figure. When the researchers gave each reviewer equal weight, Claude’s average accuracy was 49.1%. The test showed that an unsigned report can give its writer away. Estimating how often that happens across confidential peer review would take a larger, more varied sample.
Can one unsigned peer review reveal its author?
In the study, Claude Fable 5.2 saw one report at a time, without web access, tools, candidate names or extra writing samples.
Only two of the six volunteers wrote the meta-reviews.
The meta-reviews were shorter. The study cannot separate the effects of report length, type and writer.
Sources
Supporting facts
Method
Claude Fable 5.2 received one text per attempt, without web access, tools, candidate names or additional writing samples. Both bars use a common 0% to 100% scale. The rates are reported in Table 32.
Scope
Public reviews yielded a lower overall score
The researchers also tested 1,559 public reviews from 47 open-review journals. Claude correctly identified the reviewer in 21.4% of the 1,530 cases it answered. The paper reports 39.3% accuracy for a subset containing 789 reviews that Pangram 4.1, an AI-text detector, had labeled human or mixed. The paper’s abstract cites that filtered rate, rounded to 39%.
Accuracy was higher on public reviews whose writers cited their own work. Claude recorded 80.2% accuracy in the group of 106 reviews with self-citations and 17.1% in the group of 1,453 without them.
Claude made mistakes even in the group where it performed best. An editor or author presented with a name from Claude would still need independent evidence to establish who wrote an unsigned review. CryptoSlate has also reported on how frontier AI models handled fabricated research data. The team says it plans to add results from a substantially larger set of anonymous peer reviews in an update to the preprint.
Anthropic's Claude completes a 16-hour human research job in under 3 hours
Mentioned in this article
AI agents built their own scientific journal and found that reviewing research was harder than expected
Cookie Consent
We use cookies to improve your experience on our site. By using our site, you consent to cookies.
Cookie Preferences
Manage your cookie preferences below:
Essential cookies enable basic functions and are necessary for the proper function of the website.
CloudFlare provides web performance and security solutions, enhancing site speed and protecting against threats.
Service URL: developers.cloudflare.com (opens in a new window)
Google Analytics is a powerful tool that tracks and analyzes website traffic for informed marketing decisions.
Service URL: policies.google.com (opens in a new window)
Statistics cookies collect information anonymously. This information helps us understand how visitors use our website.
Clarity is a web analytics service that tracks and reports website traffic.
Service URL: clarity.microsoft.com (opens in a new window)
You can find more information in our Privacy Policy and .
Loading image…
This image could not be loaded.
The Catalyst




