Response to Hauke Hillebrandt

I was glad to see Hauke Hillebrandt’s excellent response to my essay on the politics of effective altruism (EA), which clarified Giving What We Can’s research strategy. I’d like to respond to three points: research methods for EA, the political context of NGOs, and the problem of identifying effective advocacy organizations.

It is laudable that Giving What We Can integrates insights from different kinds of research into their assessment of charities. That said, to my knowledge RCTs remain the most systematically used tool for understanding the impact of NGO interventions in the philanthropic sector, even among those who identify with the EA movement. I agree that RCTs can be designed to incorporate institutional, social, or political variables into their survey instruments, in which case they are capable of capturing unintended consequences of an intervention (Give Directly has incorporated measures like alcohol consumption and domestic violence into its RCTs to find out whether cash transfers were having these unintended side effects—they weren’t). The consistent incorporation of institutional and political variables into RCTs that are used in the EA community would help us identify and avoid the kind of harm I cautioned against in my essay. In addition to the continued use of RCTs, EA as a broader community could benefit from systematic integration of rigorous qualitative, inductive research into existing vetting practices with the aim of doing two specific things: to detect unanticipated harm and to learn about scope conditions for effective interventions. Since EA is a growing field with potential to leverage substantial resources, in the next decade we are likely to see a number of attempts to bring EA initiatives to scale. The hazards of implementing interventions under untested conditions, highlighted in a 2013 study on the difficulty of scaling good interventions, are to some extent unavoidable. But while RCTs typically aren’t designed to tell us about external validity, qualitative research can help us identify the logic of a causal effect, which can yield ideas about how well an intervention will travel to other settings and help EAs target their efforts to scale up.

On the issue of whether NGOs undermine the state, Hillebrandt suggests that targeting support for service provision NGOs toward countries with very low state capacity mitigates concerns about eroding existing institutions. Unfortunately, the cross-national study he refers to does not provide the evidence we would need to broadly conclude that funding NGOs does not displace state resources for social services. The study demonstrates a correlation between aid expenditure on NGOs and government expenditure on health, but it does not offer any evidence to defend against critiques of reverse causation (that aid is responding strategically to exogenous changes in government spending) or spurious correlation (that increases in both NGO aid and government funding are being driven by a third factor, like perceived need or tractability). Because we cannot observe the counterfactual, we cannot be sure that governments would not have spent more than they did had NGOs received less funding. Cross-national observational data is not a reliable way to make the generalized claim that NGOs do not crowd out state provision. More detailed research is beginning to be done on the political conditions under which NGOs displace the state. Meanwhile, the potential for harm from state substitution only becomes truly significant in the medium or long run. In the short term, I agree that from an EA perspective, in weak-state settings the marginal likelihood of harm to state institutions from NGO service providers is clearly swamped by welfare gains. But because these dynamics can be path dependent, the endgame also demands our attention now.

In his response essay, Hillebrandt proposes that in places like India where state systems are robust, effective altruists should recommend charities that work closely with local government. Cooperative approaches hold promise where state systems are already extensive, and many NGOs and donors are shifting in favor of this strategy. However, it is worth noting some scope conditions for the effectiveness of this approach. In cases where a lack of resources is the key constraint, a partnership model that offers the government new policy ideas, trainings, or inputs can indeed build the state’s capacity substantially. But some systems that deliver services to the poor are constrained not by resources but by corrupt or politically captured state officials. Unlike advocacy groups, NGOs that partner with states are often not in the position to put serious pressure on these agents, and such a system is unlikely to be broadly responsive to an influx of resources from a partnership program. These insights suggest that advocacy, partnership, and direct service provision may each be called for under different political circumstances.

The challenge Hillebrandt poses to identify specific advocacy groups is on point, and scholars of advocacy should work with effective altruists to try to meet this challenge, given the potential benefits of directing resources toward effective advocacy. Given the nature of advocacy—it cannot be done experimentally at any significant scale, and its effects are generally understood as being long-term and uneven—it presents a methodological challenge to identify effective advocacy organizations ex ante (or even ex post) using the usual approaches to impact measurement. As such, identifying worthwhile expenditures in this area may require a different approach to assessment. My hope is that my research and that being done by other social scientists on the politics of state and non-state welfare provision can be helpful in continuing to develop best practices for making these decisions in the EA community.