• Scalene
  • Posts
  • Scalene 17: Scepticism, ICLR, Watchmen

Scalene 17: Scepticism, ICLR, Watchmen

Author

Chris Leonard
October 20, 2024

Humans | AI | Peer review. The triangle is changing.

I’ve spent the last two weeks lamenting a spelling mistake in the last newsletter and attending 3 major events. I’ll gloss over the Knitting and Stitching Show at Alexandra Palace where I was helping my wife, and instead focus on the STM Frankfurt meeting and the Frankfurt Book Fair. It was my first time at both and I found them to be excellent ways to meet and stay in touch with people, learn from others, and to gauge the ‘feel’ of the industry toward certain initiatives. There was a general sense of more optimism around the use of AI in peer review than I was expecting, but some of that was against a background of integrity scandals which broke whilst we were there - so perhaps a pragmatic acceptance that some things needs to change, or at least experimented with?

20th October 2024

// 1
AI-Assisted Academia: Navigating the Nuances of Peer Review With ChatGPT 4
J Pediatr Pharmacy Ther. - 13 Aug 2024 - 8 min read

The meetings in Frankfurt last week highlighted to me the need for publishers and authors/reviewers to find a gentle on-ramp to utilising AI in peer review. A few of those are described in this editorial, ranging from interpreting reviewer comments to make them more actionable, to authors drafting rebuttal letters and suggesting revisions.

The style of this article felt familiar. Then I noticed at the end “The author acknowledges that this article was partially generated by ChatGPT (powered by OpenAI’s language model, GPT-3; http://openai.com). The editing was performed by the human author.”

CL: I did an interview this week where I was asked about various instances in which it is acceptable to use AI in scholarly publishing. The interviewer was surprised (given some of my other answers) that I thought it should not be used for drafting articles that a human then edits. Two reasons: 1) they all start sounding the same and consist of listicles within paragraphs that become tiresome after reading a few, but more importantly 2) writing is thinking. It’s important for authors to write the first draft of their article to think through their arguments and make sure the correct emphasis is on what they want to communicate.
Particularly with articles with your name on, it’s important that they reflect your thinking and are in your writing style. If we rely on ChatGPT or others to draft content, we need to attack it with a heavier editing mindset. But better, IMHO, to do the SFD yourself.

// 2
Overcoming Skepticism Through Experimentation: The Role of AI in Transforming Peer Review
TSK - 09 Oct 2024 - 11 min read

By conducting pilot programs, publishers and academic institutions can gather valuable insights into how AI can be integrated into the peer review process without compromising quality or security. Proposed scenarios include AI adoption at the publisher level, allowing data access for models, and creating AI playgrounds to get feedback from researchers and reviewers. Familiarity breeds trust, and it is through these low-stakes environments that AI can begin to integrate more seamlessly into peer review.

CL: It’s great to see others are also thinking about parallel track experiments to see how AI-assistance can help in peer review. I love the phrase ‘familiarity breeds trust’ - much of the scepticism around AI seems to to stem from a lack of familiarity with what current systems can do. In another conversation I had last week (and quoting myself seems wildly narcissistic, but here we are) I said ‘what seems problematic now, will not be in 6 months time - what seems impossible now, will not be in 18 months time’. Don’t quote me on the exact month, but the essence of the statement was that things are changing quickly, and if your opinions of where LLMs are now is based on your first interactions with ChatGPT, it’s time to see how far things have come.

// 3
Assisting ICLR 2025 reviewers with feedback
ICLR - October 2024 - 3 min read

Computer science conferences are at the forefront of AI assistance in peer review, and this year ICLR plan to implement a reviewer feedback agent to "help make reviews more constructive and actionable for authors." There are few technical details, but it’s heartening to see experiments to improve the peer review process at scale.

// 4
The artificial intelligence revolution...in unethical publishing: Will AI worsen our dysfunctional publishing system?
J Gen Physiol - 4 Nov 2024 -29 min read

In case you were getting optimistic about things, here’s a startling down-to-earth thud for you:

Although these models can be used in positive ways, the metrics and pressures of academia, along with our dysfunctional publishing system, stimulate their indiscriminate and uncritical use to speed up research outputs. Thus, LLMs are likely to amplify the worst incentives of academia, greatly increasing the volume of scientific literature while diluting its quality. At present, no effective solutions are evident to overcome this grim scenario, and nothing short of a cultural revolution within academia will be needed to realign the practice of science with its traditional ideal of a rigorous search for truth.

// 5
‘Quis custodiet ipsos custodes?’ Who will watch the watchmen? On Detecting AI-generated Peer Reviews
arXiv - 13 October 2024 - 34 min read

Our focus here is to solve a real-world problem by assisting the editor or chair in determining whether a review is written by ChatGPT or not. To address this, we introduce the Term Frequency (TF) model, which posits that AI often repeats tokens, and the Review Regeneration (RR) model, which is based on the idea that ChatGPT generates similar outputs upon re-prompting. We stress test these detectors against token attack and paraphrasing. Finally, we propose an effective defensive strategy to reduce the effect of paraphrasing on our models.

And finally…

Lots of actions for me to follow up from last week, but I’m always happy to hear from you if you’re considering how best to dip your toes into the world of AI-assisted peer reviewing.
And on that point, I’m trying to think of a new name for AI-assisted peer review. It’s not a ‘peer’ in the traditional sense. I also wanted it to reflect the sense that the validity and quality evaluation is a larger process involving ethical writing, research integrity checks, and traditional review. ‘Quality control’ feels too generic - any suggestions?

Let's do coffee!
I’m in London on November 6th for anyone who wants to meet up.

Curated by Chris Leonard.
If you want to get in touch with me, please simply reply to this email.