Frequently Asked Questions
Why does isitcredible.com exist?
Our website provides simulated peer review reports on academic texts.
Why "Reviewer 2"?
According to the meme, Reviewer 2 is the archetype of the bad peer reviewer: critical and pedantic in an unconstructive way, primarily interested in promoting their own work and blocking anything that challenges it. At isitcredible.com, we aim to invert that meme: our Reviewer 2 is trained to be critical but also fair, and it has no interest of its own to promote.
When you use the private service, are the reports private?
We may read the report produced by our system for quality control purposes. However, the report will not be made public. Nor do we read your uploaded files, which are automatically deleted from our server as soon as the report is emailed to you. We keep the generated review on our server for two weeks in case you find there were any technical issues. After two weeks, the report itself is also permanently deleted.
Can I nominate a text for the Public Archive?
Yes. You can nominate a text here. We require a minimum of 100 words of context explaining why we should review the text and add it to the archive; we may not be able to review every text due to limited resources.
Who provides the large language models (LLMs)?
Our system uses the LLMs of the standard major providers, such as Anthropic and Google. We strictly monitor their privacy policies to ensure your work is not used to train their LLMs and is deleted from their systems after processing.
Don't LLMs do this sort of thing for free?
What you get out of an LLM depends entirely on how you prompt it. Standard chat interfaces are often sycophantic—within a few prompts, it is easy to get Claude to tell you that your paper is brilliant. The value added from isitcredible.com comes from the system of prompts that we use to produce our reports.
Does Reviewer 2 have access to the Internet?
No. We do not currently trust LLMs to reliably extract information from the live web. Our agents are restricted to their training data and the text contained within your PDF. This makes our system particularly effective at finding issues related to internal consistency, research design, execution, and interpretation.
Do human researchers contribute to the reviews in the Public Archive?
The first version of a report is fully automated. However, in revised versions of the report, human researcher assistants working for isitcredible.com can add potential issues for Reviewer 2 to consider. We always try to consult with the authors of the reviewed text before adding any potential issues.
Furthermore, whenever a report is added to the Public Archive, we try to ensure that the authors of the reviewed text are notified; we encourage them to leave a private comment if there are any errors in the report and we will correct them in a revised version.
Who is Joseph Francis?
Joseph Francis is an economic historian who lives on a hill in Wales. He is also the founder and director of The Catalogue of Errors Ltd, the company that owns isitcredible.com, where he also works as a research assistant. His own research focuses on the political economy of Argentina and the United States since the eighteenth century. You can follow him on X or read his blog, The Poor Rich World.