‘Evidence-based aid must not become the latest in a long string of development fads’

Angus Deaton

8 While skepticism about foreign aid used to be the preserve of the political right, it has now spread to some who clearly recognize the moral imperative of the world’s rich to help the world’s poor.

Abhijit Banerjee is skeptical of aid as we know it, but he has both a diagnosis and a plan. The diagnosis is that donors are shooting in the dark because they refuse to collect solid evidence on what works. His plan is to collect this evidence using randomized controlled trials and to confine aid to projects that the evidence supports. Aid would then do a great deal of good. And although Banerjee does not say so, there would be much less of it, because only a fraction of projects that currently receive aid could be subject to randomized controlled trials.

I agree with Banerjee on a good deal of this. I too am skeptical of current practice and I too believe in the value of empirical evidence. But I am also skeptical about the general usefulness of randomized controlled trials in this context. Because Banerjee is far from a voice in the wilderness—the arguments of the “randomistas” are having considerable success—it is important that we get this right, and that evidence-based aid does not become the latest in a long string of development fads.

The historical record tells us that it is possible to grow and eliminate poverty without foreign aid; all of the now-rich countries did so. We also know that some of the most successful poor countries, such as India and China, grew with very little aid relative to their size, or with aid that was dictated by their own priorities rather than donors’. The least successful countries, many of them in sub-Saharan Africa, have been given large amounts of aid relative to their size and have neither grown nor reduced poverty. Isolating the role of aid in these outcomes is clearly very difficult, and a convincing statistical demonstration may not be possible. Yet empirical work has improved considerably, and some of us who had previously discounted the econometric literature are beginning to think that, indeed, there may be no effect to be found. Aid as we have known it has not helped countries to grow.

There are many explanations for why this might be the case. Recently some commentators have drawn a parallel between the effects of foreign aid and the effects of commodity price booms in economies that primarily export commodities such as copper or cotton. The prices of primary commodities are notoriously volatile, and price booms generate bonanzas of discretionary government revenues that often leave trails of destruction. One of many historical examples is the cotton price boom in Egypt during the “cotton famine” induced by the American Civil War that eventually led to the collapse of the government and to British occupation. In many African countries, foreign aid and taxes on commodity exports provide almost all of governments’ discretionary spending—money often wasted on unsustainable but politically desirable projects, or stolen outright. The rent-seeking generated by the resulting economic environment does nothing for development.

Why then not do what Banerjee suggests: take donor money away from governments and use it to build roads, power lines, schools, and clinics? No doubt we can do something of the sort. But as soon as these projects become large enough to do much good, they also become large enough to attract the rent seekers. In cases where there is no real government commitment to the poor, the money from such projects will be diverted into the projects the government would prefer or into the pockets of corrupt politicians or officials. To a government that would have built some roads and clinics in any case, the new funds are readily and legally fungible. According to the European Community, the total value of stolen assets in individual foreign accounts is equivalent to half of Africa’s outstanding debt. Of course, there are some governments that do have a real commitment to poverty reduction. But they have a good chance of doing it on their own, and the provision of large sources of discretionary funds may make it harder for them to control rent-seeking and corruption. When project aid is fungible or can be stolen, there may not be much difference between aid for projects and direct government aid. The view that aid only works when the country is already committed to improvement may or may not be “resolutely puritanical,” but it does have the virtue of recognizing the reality.

So where does evidence come in? Understanding how to improve the world is a global public good, and institutions such as the World Bank can gather evidence, store it, and help countries interpret it. Such a “knowledge bank” would be invaluable to governments that are genuinely looking for poverty reduction and want to learn from others’ mistakes and successes. I have no doubt that randomized controlled trials would play a useful part in constructing this storehouse of knowledge.

But there are limits. Take Banerjee’s example of flip charts. The effectiveness of flip charts clearly depends on many things, of which the skill of the teacher and the age, background, and previous training of the children are only the most obvious. So a trial from a group of Kenyan schools gives us the average effectiveness of flip charts in the experimental schools relative to the control schools for an area in western Kenya, at a specific time, for specific teachers, and for specific pupils. It is far from clear that this evidence is useful outside of that situation. This qualification also holds for the much more serious case of worms, where the rate of reinfection depends on whether children wear shoes and whether they have access to toilets. The results of one experiment in Kenya (in which there was in fact no randomization, only selection based on alphabetical order) hardly prove that deworming is always the cheapest way to get kids into school, as Banerjee suggests.

The comparison with the FDA is very much to the point, but only because exactly the same problems come up. For a specific doctor facing a specific patient, the average outcome of a randomized controlled trial will often be unhelpful. The physician usually has some theory of how the drug works and also an understanding of her patient, who might, for example, be elderly, frail, overweight, and an ex-smoker, with a history of responding to some medications and not others. Therefore the physician will often not prescribe a drug that passed its randomized controlled trial with flying colors and instead prescribe one that did less well but that is a better fit for the patient. Much of medicine is not “evidence-based,” for good reason.

There is no simple way to use randomized controlled trials to eliminate global poverty. They are expensive and technically and politically difficult to do well. We must be careful to apply them only where there is a good chance that the results will be applicable elsewhere. Otherwise, we will be gathering evidence, not knowledge. <

Angus Deaton is the the Dwight D. Eisenhower Professor of Economics and International Affairs at Princeton University.

Click here to return to the New Democracy Forum “Making Aid Work.”

Originally published in the July/August 2006 issue of Boston Review