In this last week even the IMF has told George Osborne to back off on austerity. Osborne is of course not interested (never mind the risk of an unprecedented triple-dip recession or that he’s quite happy to waste £10 million for Margaret Thatcher’s funeral). And the intellectual justification for austerity has never been shakier. Osborne is fond of citing. Or to quote him at the 2010 Mais Lecture “As Rogoff and Reinhart demonstrate convincingly, all financial crises ultimately have their origins in one thing – rapid and unsustainable increases in debt.”
But this isn’t about Osborne. It’s about his favourite economists, Rogoff and Reinhart – and about the Circles of Scientific Hell.
Let’s assume we are a couple of moderately obscure economists and want to make a name for ourselves. we want to be the most cited authors in public and policy debates about the currently hot topic of public debt. we want to be cited in books by US Senators. we want to be cited by the Vice-President of the European Commission and a US Vice-Presidential candidate. we want to be name-dropped by the British Chancellor and one of his predecessors. we want to be cited by reputable papers more than 70 times in the year.
We produce a paper that says what the rich and powerful want to hear. And then based on the paper a book. But how do we do this assuming we start off in limbo? One way would be to look at the circles of scientific hell and start digging.
We want a very oversold paper – level 2 of scientific hell. And if it’s a paper on economic history which is a field we already know, we’re already well into level 3 of scientific hell. we know what results we want and are trying to fit the data to the patterns. But this isn’t enough.
Our next step is to go down yet another level, past ordinary malpractice and into p-value fishing. But we aren’t fishing for p-values; we aren’t doing anything remotely that rigorous. Instead with no reason given we break debt down into arbitrary baskets (0-30% of GDP, 30-60% of GDP, 60-90% of GDP, >90% GDP). Never mind that these baskets are not equivalent and that Japan’s public debt is over 200% of GDP. Or that public debt is often high because of a recession rather than the other way – after all the Keynesian recommendation is to borrow in a recession.
But drilling down to level 4 isn’t going to get we what we want. So we pick up our spades and keep digging. Level 5 is creative outliers, and was where the investigation to uncover what happened started to be needed. we use really odd weighting in this case to give the outliers disproportionate influence. we have a result from New Zealand that has a growth rate of -7.6% for one year with over 90% of debt-to-GDP (out of five), and a result from Britain that has 19 years of over 90% debt to GDP and a growth rate of 2.4%. And then we average them by country – never mind that there are almost four times as many data points for Britain as New Zealand – Britain’s 19 data points are given equal weight to New Zealand’s four data points. I’d say we’ve hit level five very comfortably here – that outlier is about four times as influential as any of the positive growth data points in Britain.
But that still isn’t enough. Drill, baby, drill! We find our next level of scientific hell is entirely empty – it’s plagiarism, and the punishment for plagiarism is to be given punishments in all the other levels of scientific hell. Admittedly this is one place where we are innocent.
So we drill a little bit more, and find ourselves in the non-publication level. It’s quite simple. At the time we don’t publish the underlying data – and no one will be able to reproduce our findings. But that isn’t going to get the results we want on its own.
By now we’ve given up on the jackhammer and need an industrial strength borer to go down low enough. Partial publication of data. You remember that -7.6% figure for one of New Zealand’s five years? What if we simply discarded the other four? After all they were the previous years. We can also start Canada and Australia a few years late as well. They came out of World War 2 with debts greater than 90% of GDP and very solid growth (2.5% for several years for New Zealand and 3% for Canada). And yes this means starting New Zealand a year before Canada or Australia.
Our numbers are already screaming at us and begging for mercy. But if we were really determined we could dynamite our way into the ninth level. Or possibly just blow ourselves up with our own incompetence. Either way we could screw up an excel formula and miss a datapoint off our sum, pushing the growth below the psychologically important 0% when debt is over 90%. You think I’m joking?
And whatever we do we must not let people like Thomas Herndon, Michael Ash, and Robert Pollin get ahold of our data. They might publish a paper of their own. And show a very different relationship to the one we did.
Still, even after torturing our data into submission, we need to find a way of publishing it in a prestigious journal, and slipping past peer review. The American Economic Review is a superb journal, and every May it has an issue entitled Papers and Proceedings designed for exploration and discussion rather than as formal peer-reviewed papers.
From the foreword to the 2010 Papers and Proceedings of the AER
Papers are published only if the data used in the analysis are clearly and precisely documented and are readily available to any researcher for purposes of replication. Otherwise, the guidelines under which papers are published in the Papers and Proceedings differ considerably from those governing regular issues of the Review. First, the length of papers is strictly controlled. Second, papers are not subjected to a formal refereeing process. However, a paper can be rejected if, after reading it, we conclude that it is without merit. Third, the content and range of subject matter reflect the wishes of the President-elect to investigate and expose the current state of economic research and thinking. In most cases, therefore, the papers are exploratory and discursive, rather than formal presentations of original research.”
I trust that no readers will take the advice given here seriously. And hope either that I am misunderstanding matters or that the entire paper following this approach is me fitting a hypothesis to the data (and ending up in ring 3 myself).