Effective Altruism, Continued: On Measuring Impact

Photograph: International Organization for Migration.
 

The debate on effective altruism continues. Here Hauke Hillebrandt, Director of Research at Giving What We Can, responds to Emily Clough's essay on the political impact of NGOs. 

For more debate on the forum, read Rwanda's Minister of Health Agnes Binagwaho's response to Angus Deaton. Read Deaton's reply here.

---

We at Giving What We Can would like to thank Emily Clough for her thoughtful response. We constantly strive to update our beliefs on the basis of new evidence, and so we welcome criticism of both the ideas behind effective altruism as well as the practical matter of which charities we recommend. It is particularly valuable to hear the thoughts of scholars of development on these matters. The main idea of effective altruism is doing more to help others with your life and trying to help as much as possible, using evidence and reason. We think what Emily Clough proposes is very much in line with this. This is why we largely agree with many of the points made in the article.

However, we also wanted to clarify a few misunderstandings. We’re sorry for any failure on the part of Giving What We Can (and the effective altruism community) to communicate more clearly why we recommend the charities we do.

We could not agree more that identifying a highly effective organization is far from simple. Evaluating charities in terms of their effectiveness is an immensely complex undertaking, which is precisely why independent charity evaluators such as our colleagues at GiveWell and ourselves are needed. One gap we try to fill is translating the research in development and global health to guide a rapidly increasing number of non-experts to make an informed decision on where best to donate their money. As mentioned by Peter Singer in his rebuttal, our colleagues at GiveWell as well as ourselves try our best to keep abreast of the vast literature in development and global health. For instance, we did consider the systematic Cochrane review, along with many other peer reviewed articles and expert opinions, in our recent review of the Deworm the World Initiative. We are also aware of the ongoing debate within macroeconomics concerning what the main drivers of economic growth are, and what relation economic growth has to poverty alleviation. We have less developed opinions as to whether it is only strong, transparent institutions that benefit the very poor, but we do think that certain investments in human capital can have exceptionally high returns on investment.

We agree with Clough, who argues that some overseas development assistance (ODA) has had some negative unintended consequences, which have hurt the poor. However, it is our opinion that the positive benefits of ODA vastly outweigh the cases in which it had negative consequences (compare disease eradication and the effects of social and economic assistance).

As correctly pointed out in Clough’s article, some NGOs are very ineffective and opaque­—an extreme example would be Homeopaths without Borders—but even when considering less extreme examples, there exist vast differences between NGOs in terms of their effectiveness and transparency.

This is precisely why we need to engage in careful analysis when deciding where our limited funds should go. We only recommend a very small fraction of the charities that our colleagues at GiveWell and we review. The charities we do recommend carry out interventions (deworming, bed net distributions, food fortification) that are also endorsed by major organizations such as the WHO. Bed net distribution for instance, is considered an integral part of any malaria eradication program.

Clough argues that there is an overreliance on RCTs among effective altruists. It is true that our charity recommendations are heavily influenced by empirical methods, such as randomized controlled trials (RCT) as well as natural experiments, surveys, computational modeling data and even correlational studies (though we treat those with caution). However, we do not exclusively rely on RCTs, but supplement our analysis with qualitative analysis, impact evaluations, cost-benefit analyses, scientific analyses, expert opinion, and common sense. The RCTs are usually used to see whether a particular health or poverty intervention has an effect in principle and to determine its effective size. This can then be used as a steppingstone for further investigation.

It is also true that for many of the charities that we currently recommend we do have data from RCTs, which makes it very likely that they have high impact. This is because traditionally, small donors had to face a high degree of uncertainty about whether their donations to developing world charities are having an impact at all. We try to meet the demand of providing recommendations that allow donors to be relatively sure that their donation has an impact. As such, we do not fund any projects, and in that sense do not have a “funding philosophy,” but merely recommend certain charities to our members. We also highlight charities such as Deworm the World Initiative and Project Healthy Children, as “promising” charities, because we are more uncertain about their impact, but think that it might be potentially very high, these charities might be more suitable for donors who are less risk averse. Some effective altruists are even less risk averse and do donate to very speculative causes, such as advocacy.

We believe that the main misunderstanding in the article is that we support charities that do direct work in India or other low- to middle-income countries with relatively capable governments that deliver services to the poor. We believe that the criticism would be valid if that were the case; however, it is not so. The only charity we recommend that works in low- to middle-income countries, is the Deworm the World Initiative, which merely provides technical support for deworming programs to the Indian government. India is a special case: because of its population size and resulting scalability, helping India carry out better deworming programs is potentially very cost-effective. In general, we do not recommend charities that do not closely work together with local governments and think this is of utmost importance for reasons stated by Clough. Moreover, we primarily recommend charities that provide direct support, working in low-income countries–often even the least developed countries (LDCs). These countries have very weak welfare and health systems, such as the Democratic Republic of Congo whose nominal GDP per capita is $411.914 and Malawi  whose nominal GDP per capita is $253. In these countries political and economic hurdles prevent the government from mobilizing the resources needed to provide even the most basic health services, such as malaria nets, for all its citizens, let alone finance a malaria eradication program. As such, we're not “pumping money into a parallel set of institutions,” as was the case in the example of funding private schools in India.

Even though our charities always work closely with local Ministries of Health, one might argue that we risk moral hazard and unintended consequences because there is a possibility that governments will neglect health spending and let NGOs fund their health care systems. However, this is not in line with the empirical literature on this topic. A recent cross-national systematic analysis of public financing of health in developing countries showed that development assistance for health by nonprofits significantly increased domestic government health spending. This is a great example of why we disagree with the statement that many unintended side effects are principally outside the realm of empirical analysis.

We agree that supporting advocacy groups that would lobby the governments to spend more on health is an interesting approach to supporting development in low- to middle-income countries. However, it is simply unrealistic to think that some low-income countries could fund an adequate level of health for their own populations at this point in time. With low- to middle-income countries, we agree that the issue is entirely different. For example, in India, very little is spent on health as a proportion of GDP. Advocacy for increasing health spending might be a good option, because the political and economic situation in India makes domestic health financing reforms realistic.

The charities we recommend that work in low-income countries primarily serve the poorest people and not the “middle poor” as was claimed. Indeed, one of our main criteria for evaluation is whether a space is really neglected and where people would likely not receive any assistance from the government. As such we do all we can to limit unintended negative consequences.

We believe that scalable health interventions such as school-based and community-based deworming, bed net distribution to remote villages, and micronutrient fortification of staple foods reach some of the poorest and most marginalized people, and are very scalable. We are aware that there might be situations in which these treatments do not reach some remote people, who do not send their children to school to receive deworming treatment, who do not purchase staple foods from official markets, or who live in small, remote villages where malaria nets are not distributed. But reaching these places is very expensive and providing services to people in these places therefore comes at the cost of providing services to many more people who are also destitute.

Clough correctly acknowledges that there may be some specific types of services such as eye surgeries that NGOs would especially efficient at delivering. However, she describes these services as low-hanging fruit and there soon may come the point at which “bed nets and eye surgeries are no longer the most pressing need.” However, the point at which there won’t be any very effective health interventions to fund will not arrive in the near future unless there is a stark increase in financing in global health: a recent report estimates that it would cost an additional $38 billion per year in 2016–25 and $53 billion per year in 2026–35, in order to get low-income countries to a similar level of health as middle income countries today by 2035. Until then, one would still avert a death for only $8000 on average—this can still be considered a low-hanging fruit, because it is only about twice as much as it costs currently to avert a death.