OFFLINE PREVIEW — for review only. PAPERS / LEGAL / ACCOUNT in the menu link to the live site; GitHub and external paper links are reachable if you are online.

Case studies

Below you will find two audits that demonstrate the work The Catalogue of Errors Ltd does. The first is on an article from The Financial Times that claims smartphones have caused the global fertility decline. The second is on a widely-cited academic article on the medieval origins of antisemitism in Nazi Germany.

1. The Financial Times on smartphones and fertility

In "Why Birth Rates Are Falling Everywhere All at Once" (May 16, 2026), John Burn-Murdoch presents a graph that appears to show how eight countries' "smartphone take-off" year was followed by plunging total fertility rates. To replicate it below, we extracted the underlying data from the infographics from ft.com:

Replication of the FT chart: eight country deviation series extracted from the saved Flourish slide, plotted against years before and after each country's chart-implied smartphone take-off year.

The first question we asked was how the FT determined the take-off years. The original graph's footnote states "Smartphone take-off based on surveys of smartphone adoption and Google searches for app stores," and cites, among other things, Google Trends and Pew Research Center, among other sources. So we checked.

We were not able to find anything to support the FT's choice.

When searching Google Trends for “App Store,” “Play Store,” and “Android Market” (the name Google Play went by before March 2012) in the local languages, we found this:

Google Trends interest for 'App Store', 'Play Store' and 'Android Market' in local languages across the eight FT countries, with the FT's take-off year marked.

The FT appears to have chosen arbitrary years for each country, without any relationship to the actual Google Trends data they cite.

The Pew Research Center's data on smartphone adoption also fails to support their take-off years:

Country FT chart take-off year Smartphone ownership in that year
United States 2007~0% (iPhone launched mid-2007; Pew's first US observation is 35% in 2011)
Australia 2008<10% (ACMA/Deloitte's first observation is 65% in 2013; iPhone 3G launched mid-2008)
United Kingdom 2009~15% (Pew's first UK observation is 55% in 2013; back-extrapolation gives mid-teens for 2009)
Indonesia 2012~10% (Pew 2013: 14%; linear back-extrapolation gives ~10% for 2012)
Mexico 201321% (Pew 2013 Global Attitudes round, direct observation)
Senegal 2013~14% (Pew 2014: 16% and Pew 2015: 19%; back-extrapolation gives ~14%)
Egypt 2014~22% (interpolation between Pew 2013: 17% and Pew 2015: 28%)
Iran 2015~36% (ISPA estimate; consistent with Pew 2014: 25% and Pew 2017: 58%)

It seems, then, that the FT may have simply cherry-picked years to produce an apparent treatment effect. The chart's published "% deviation" is the gap between each country's actual TFR and the straight line you get by fitting the eleven years up to take-off and extending it forward:

Eight small-multiples showing each country's actual TFR alongside the OLS pre-trend line extrapolated forward past the chart-implied take-off year.

Furthermore, the total fertility rate data for one of the eight countries appears to be of unknown provenance. For seven countries, the data come from the UN World Population Prospects 2024, but for Mexico, they do not match. The FT also cites "national statistical agencies." For Mexico, they seem to have used the older, superseded 2017 edition of the UN World Population Prospects for 2004, 2007 and 2012 as benchmark years before the supposed "smartphone take-off" in 2013. They then interpolated between those benchmarks to get the pre-treatment period. Two of their post-2013 anchor values (1.75 for 2020 and 1.517 for 2023) do not match any published source we have been able to locate, and the resulting series shows a far greater treatment effect compared to the actual series from the UN World Population Prospects 2024, which is identical to the series from Mexico's national statistical agency:

Mexico TFR deviation under the FT's recipe: the FT's Mexico series falls to about -21 per cent by year +10, while the same recipe applied to UN WPP 2024 produces only about -2 per cent.

All of our replication scripts and data are available on our GitHub.

2. Voigtländer and Voth on the origins of Nazism

Nico Voigtländer and Hans-Joachim Voth's "Persecution Perpetuated: The Medieval Origins of Anti-Semitic Violence in Nazi Germany" (Quarterly Journal of Economics, 2012) is a widely cited article that is used to teach historical economics in universities.

The article claims to have found a correlation between antisemitic pogroms during the Black Death and support for the Nazis and antisemitic violence in the 1920s and 1930s.

To test this claim, we went back to Voigtländer and Voth's German-language source material:

Page from Avneri's Germania Judaica on Aachen.
Avneri, Germania Judaica (1968), on Aachen.
Entry from Alicke's Lexikon der jüdischen Gemeinden im deutschen Sprachraum on Aachen.
Alicke, Lexikon der jüdischen Gemeinden im deutschen Sprachraum (2008), on Aachen.

We used artificial intelligence to perform an initial check for discrepancies between their coding of their dataset and what the published source material said. We then checked every apparent discrepancy by hand. We identified 169 apparent discrepancies between Voigtländer and Voth's coding and their published sources, and have made all the source materials that document those discrepancies available online for anyone to check. We also found unexplained discrepancies in Voigtländer and Voth's dataset calculated from the Gedenkbuch, the official government record of the Jewish victims of the Holocaust in Germany.

In addition, in an appendix to our paper, we demonstrate fragilities in Voigtländer and Voth's econometrics, even when their original dataset is used.

Our paper and all its replication material are available on this website.

You can write to us at contact@isitcredible.com if you would like us to do similar work for you.