This paper calls on the research community not only to investigate how human biases are inherited by large language models (LLMs) but also to explore how these biases in LLMs can be leveraged to make society’s “unwritten code” — such as implicit stereotypes and heuristics — visible and accessible for critique. We introduce a conceptual framework through a case study in science: uncovering hidden rules in peer review - the factors that reviewers care about but rarely state explicitly due to normative scientific expectations.

Recent advancements in large language models have sparked interest in utilizing them to assist the peer review process of scientific publication. Instead of having AI models generate reviews in the same way as human reviewers, we propose adopting them as manuscript quality checkers. We introduce several baseline approaches and an extendable automatic evaluation framework using top LLMs as judges to tackle the difficulty of recruiting domain experts for manual evaluation. Utilizing papers withdrawn from arXiv, we validated our proposed methods with several leading reasoning LLMs from different providers and assessed their performance and API costs for identifying critical errors and unsoundness problems. The OpenAI o3 model performed the best, while o4-mini was the most cost-effective one in our evaluation.

The peer-review process, which serves as the quality-control mechanism of scientific knowledge production, has been criticized for its bias, unreliability, and inefficiency. Academic conferences and journals typically rely on a centralized mechanism for reviewer assignment and paper assessment. We argue that this centralization is a major factor contributing to the unreliability of the review process, leading to deficiencies in the current knowledge-assessment systems. To address this, we propose a novel decentralized model that democratizes peer review by shifting decision-making rights from centralized authorities to all scholars participating in a scholarly community. Our model includes a dual-rewarding incentive mechanism that motivates scholars to actively participate in peer review by recognizing both their effort and scientific contributions. This model transforms peer review from passive judgment to active collaboration.

A very extreme form of equilibrium behavior is an explicitly adversarial attack on the LLM reviewer. The possibilities are near-endless. On the low-effort end of the spectrum, some authors will no doubt attempt to smuggle new instructions into hidden text in the paper, instructing the LLM to leave a more positive review. This may be explicitly forbidden by conferences, with automated checks implemented to detect such instructions. However, more sophisticated attacks abound. Authors could attempt steganographic attacks, using an LLM of their own to attempt millions of paraphrases of the original paper until discovering a version that is semantically similar but garners a much more positive review. Or, since sophisticated LLM reviewers will employ web search, authors could attempt to poison the search results, introducing papers into arxiv or less-selective venues which exist only to influence the LLM reviewer once retrieved into context.

I have written before, as have many other people, about the sheer and unending silliness of dealing with regular research integrity processes. The people involved on ‘the other side’, wherever that may be, are slow. Or, slower than that. They lack accountability. They have guidelines for pursuing research integrity questions, like COPE, which are neither followed nor enforced. Complaints are often bounced between disinterested parties.

And if we find active threats to human health in this literature, I think it is immoral to get tied up in this silly process that you can’t trust or rely on.

Let's say we found an unambiguously contaminated treatment guideline. Should we enter into some nonsense exchange of emails that takes two years to issue an expression of concern while people are hurt that whole time?

No - we’re going straight to press.

No filter, no university intermediary, no silliness. It’ll look more like this:
1. Problem identified.
2. Report written about problem.
3. If serious enough, problem peer reviewed internally (AND I HAVE A LINE ITEM IN THE BUDGET TO PAY THEM)
4. Problem immediately reported in press.