The IMF's Independent Evaluation Office (IEO) was set up in 2001; since then it has produced 26 thoughtful and substantial reports on a wide range of IMF activities. Now the head of the IEO has co-authored a book with the key recommendation that the IMF should use the IEO evaluations to enhance its capacity as a 'learning organisation', and be more prepared to change its mindset and operations. Why is it so hard for institutions such as the IMF to accept criticism?
The main catalyst for the IEO's formation in 2001 was probably the 1997-98 Asian crisis. There was a widespread view that the IMF had not performed well, misunderstanding the nature of the problems in Thailand and Indonesia and providing policy advice that was inappropriate and unhelpful. This unhappiness was manifest in the Chiang Mai Initiative, a (largely failed) attempt by Asian countries to create a regional alternative to the IMF to handle crises like this.
Hence one of the first IEO evaluations was of the late-1990s crises in Indonesia, Korea and Brazil. The title of this report was, in itself, an indictment of the IMF's approach to the 1998 Asian crisis: The IMF and Recent Capital Account Crises: Indonesia, Korea, Brazil. This title identified the central issue as international capital flows – excessive before the crisis and devastatingly harmful when the flows reversed. Not only had the IMF focused on other, less crisis-critical issues (crony capitalism, budget deficits, weak prudential supervision, monetary policy inadequacies), but its policy response did little to address capital outflows or provide sufficient funding to offset them. The fiscal and monetary policy levers were pushed in the wrong direction, and the fall in GDP dramatically exceeded the IMF's early estimates.
Having got the title right, did this evaluation contain the hard-hitting indictment that might have provoked some serious soul-searching among the IMF's staff? Not at all. By the time the IEO's initial draft had been rewritten to take account of the voluminous rebuttals and self-serving assertions offered by IMF staff, the published version was so bland, and so free of criticism that there was little that the IMF might learn about its own performance (disclosure: I was a member of the IEO evaluation team).
The IMF staff reaction was largely the Bart Simpson defence: 'I didn't do it'. It was all the fault of the national governments in the crisis countries. This was like the fire brigade assessing its operational performance solely in terms of how the fire had started, without addressing its own specific remit to contain the blaze.
That was then. Now, well over a decade later, the IEO is feeling much bolder (hence the current self-initiated book). The chief criticism is that the IMF is not a 'learning organisation'. The particular episode that seems to have provoked this book was the Greek crisis that began in 2010, evaluated in a 2016 IEO report. In the course of writing this evaluation, the IEO was excluded from what it sees as important material on the IMF's decision process:
Conducting this project proved difficult for the IEO and strained relations between the IEO and the rest of the organization. The project tested the extent of IEO's independence, and while the IEO was able to conclude this evaluation and maintain its independence, the exercise offers interesting insights on the challenges and frictions that characterize independent evaluation.
Many of the problems of the IMF's 1998 Indonesian experience were still apparent in its 2010 response to Greece. Structural weaknesses had not been highlighted in pre-crisis surveillance. When the crisis occurred, financial support was insufficient for the task; necessary debt rescheduling was 'kicked down the road'; coordination with other supporting parties (the European Central Bank and the European Commission) was inadequate; and excessive reliance on budget tightening ensured that the recovery forecast would prove to be hopelessly optimistic.
Bureaucracies everywhere have trouble in admitting error ('some parts of management and senior staff have shown defensive and antagonistic attitudes'), conscious that any admissions will be used against them. The IMF's doctrine has, in fact, evolved quite a bit since 1998, especially on capital flows. But none of this evolution was incorporated in the Greek program.
Perhaps the IMF's staffing provides an additional explanation. The IMF staff is, to a large degree, like a priesthood, where academically bright, just-graduated economists are inculcated with the long-standing core beliefs of the institution. These beliefs are absorbed by osmosis from their longer-serving seniors, through taking part in annual 'missions' (the term is evocative) to evaluate economic performance in each of the 189 member states; drafting the key semi-annual publications on the state of the global economy; and researching the workings of the global economy at an esoteric level. When things go wrong in a member country, a support program always includes detailed advice on what should be done. The most-faithful of the novices move up through the hierarchy; those with doubts move elsewhere. Many of the IMF's staff spend their entire career there, and few of them have had much exposure to the real world of policy compromise and second-best policy solutions. Often they have little detailed knowledge of the particularities of the country they seek to advise.
The IMF also has is an intractable governance structure that any private company would regard as unworkable. It has a 24-member executive board residing in Washington, made up of some who represent a single large country (such as the US), others who represent a constituency of like-minded countries, and some (such as Australia) who represent a disparate collection of countries with little in common. It is inevitable that this board will be dominated by the single-country constituencies (especially the US, which has veto voting power), or tightly-knit groups like Europe, who use their preeminence to pursue self-interested, sometimes deeply political policies. But other board members can kick up a lot of dust. Each member brings a heavy load of intellectual baggage to the policy discussion. Meeting more-or-less continuously, the board provides substantial input that the managing director has somehow to accommodate.
Of course this is not all that different from the problems faced by national bureaucrats, constrained by the political process in which first-best policy-making has to compromise with vested interests and often idiosyncratic but firmly-held opinion. This is, then, the context within which the IMF's perceived failures should be judged. Perhaps it explains (and maybe even justifies) the IMF's less-than-welcoming attitude to the IEO. The IEO book has this frank comment from IMF Managing Director Christine Lagarde: 'The Independent Evaluation Office is an entity that not many organizations would tolerate. It goes under the skin of the institution, and under the skin of projects, reports, and ways of operating.'
This ongoing tension is reflected in the book's conclusion:
The Fund has not yet established the proper atmosphere or culture to truly absorb and apply IEO evaluation findings and lessons. We conclude that there is no incentive to systematically learn from independent evaluation because the Fund identifies the IEO as essentially an accountability device rather than as both an accountability and learning instrument.
