Published daily by the Lowy Institute

How not to Excel in economics

How not to Excel in economics
Published 18 Apr 2013 

Hugh Jorgensen is a Research Associate in the Lowy Institute's G20 Studies Centre.

Drama, tension and controversy are not usually phenomena associated with high-level econometric analysis, but arguments over a paper written three years ago by two Harvard Professors, Carmen Reinhart and Kenneth Rogoff (R&R), titled Growth in a Time of Debt, are presently setting the economic wonkosphere alight

R&R's paper looked for a relationship between GDP growth and government debt by assessing the post-war experience of 20 selected advanced economies. It was released at a time when government debt/GDP ratios were ballooning after the bank bailouts and massive stimulus programs of 2008/2009. R&R's key observation was that:

The relationship between government debt and real GDP growth is weak for debt/GDP ratios below a threshold of 90 percent of GDP. Above 90 percent, median growth rates fall by one percent, and average growth falls considerably more.

R&R's actual finding for average growth is that it declines by 0.1% when the debt/GDP ratio climbs above the 90% threshold. While the data itself simply points to a correlation between periods of high government debt and low economic growth, R&R's discussion of their result strongly implies that the 90% debt/GDP ratio actually represents something of a critical breakpoint. Exceed this level and there is a strong chance your economy could be consigned to a long period of very low growth.

This 'cutoff statistic' quickly entered the rhetorical arsenal of pro-austerity policy-makers as a justification for cutting deficits hard and fast, having since been cited by the likes of Paul Ryan (US House budget committee chairman and former vice-presidential candidate, and author of the budget plan passed earlier this year by the Republican controlled House), UK Chancellor George Osborne, US Senator Kent Conrad (chairman of the Senate Budget Committee) and Olli Rehn, Vice-President of the European Commission. [fold]

The 90% cutoff has also been raised in the G20 'Framework for Growth' working group, and has even been submitted for discussion at the G20 finance ministers meeting being held this week in Washington (although it is unlikely it will get very far).

Yet the underlying methodology of the paper has long been criticised. Other economists have tried and failed to replicate R&R's results using public data, and they have been critical of R&R's initial refusal to release their data. There is also the matter of reverse causation: the data could also be interpreted as evidence that sustained lower growth is itself the cause of a higher debt/GDP ratio (think Japan).

Enter PhD student Thomas Herndon from the University of Massachusetts. Herndon, tasked to investigate R&R's work for a homework assignment, finally obtained the data from R&R and alongside Professors Michael Ash and Robert Pollin, discovered the 90% cutoff finding was actually based on a 'fat-fingered' coding mistake in an Excel spreadsheet.

While R&R's data supposedly drew on the experiences of 20 advanced economies, five were accidentally excluded. When these five countries are included (Australia, Austria, Belgium, Canada and Denmark), Herndon-Ash-Pollin (HAP) find that 'the average real GDP growth rate for countries carrying a public debt-to-GDP ratio of over 90 percent is actually 2.2 percent, not -0.1 percent'.

R&R have penned a response to HAP that accepts the error but argues that their government debt/GDP slow-growth claim still broadly holds up. Paul Krugman notes the problem here — substituting other more tangential results when the initial evidence has been dismissed is not a strong defence.

Yet it is worth noting that there has been a lot of embellishment in coverage of this debate: austerity-minded policies do not exist because of Reinhart and Rogoff, and the discovery of an Excel coding error is not going to lead to earth-shattering changes in fiscal policy in Europe, the US or anywhere else. Furthermore, any errors in R&R's initial work does not undermine the basic message: the higher public debt gets, the more the chance that markets will require a risk premium.* That R&R's work was so widely praised by austerity-minded thinkers likely has more to do with 'motivated reasoning' – individuals often add more credence to work that matches their ideological preference.

Put another way, the stimulus-austerity debate is as much about political ideology as it is economics, and focusing on the legitimacy of a 'magic' 90% debt/GDP cutoff ratio underplays the incredibly complex and contingent circumstances that each country finds itself in, post-crisis. We are no clearer about how to put the global economy back together than we were two days ago, and it is quite possible that the only objectively good thing from all of this is that Herndon received a deserved A for his paper.

For more on this debate, check out this and this. And for info on how the actual coding error occurred, see this. The New Yorker also has a piece by John Cassidy that links the story all the way back to the austerity of Margaret Thatcher.

Photo by Flickr user wiccked

* This sentence added at 15.07.



You may also be interested in