Category: RCTs

When economists shut off your water

jethron akallahLeave a comment
Researcher Irene Nduta in Kayole-Soweto.

By Adrian Wilson, Faith Kasina, Irene Nduta and Jethron Ayumbah Akallah

In August 2020, people all over the development world started talking about water in Nairobi. There was a lot of anger, and some calls for sending people to the guillotine. The reason: the publication of results from a development randomized controlled trial (RCT), run by two American development economists, working together with the World Bank. In order to compel property owners in Kayole-Soweto—a relatively poor neighborhood in eastern Nairobi—to pay their water bills, this experiment disconnected the water supply at randomly selected low-income rental properties.

There’s no doubt that water is a problem in Nairobi. As Elizabeth Wamuchiru tells us, the water system in the city has a built-in spatial inequality inherited from the British colonial era. Visitors to the city can readily see the differences between the cool, leafy, green neighborhoods of Kilimani and Lavington—segregated white neighborhoods under colonialism, now home to rich Kenyans, foreigners, and NGOs—and the gray and dusty tin-roof neighborhoods of Mathare, Kibera, Mukuru, and Kayole, home to the lower-income Kenyans excluded from Nairobi’s prosperity.

Today’s water system reflects this history of inequality. Nairobi’s water is harnessed from a combination of surface and groundwater sources; however, the city’s groundwater is naturally salty and very high in fluoride. Piped water systems, provided to upper- and middle-income housing estates, do not exist in the vast bulk of the city’s poorer neighborhoods, where people must instead buy water from vendors—often salty water pumped from boreholes, or siphoned off from city pipes through rickety connections that are frequently contaminated with sewage. In the richer neighborhoods, Nairobi Water Company, a public utility, sells relatively clean piped surface water for a fraction of the price paid by poorer Nairobians—a disparity that research has shown to often be the case in other cities in the global South. As the Mathare Social Justice Centre puts it, in poorer neighborhoods such as Kayole-Soweto, “water provision costs more, is less safe, and is less consistent than in other richer parts of the city.”

Read More »

Neoclassical Economics and Urban Planning: A Contentious Theoretical and Policy Making Relationship

ngarzacsudheduLeave a comment

Neoclassical economics – and contemporary extensions of it – has an outsized presence in academic and policy making circuits. This position of privilege builds upon more than a century of theoretical development, comprising the contemporary “mainstream” of economic science. The characteristics and rise of this mainstream, determined in many cases by means beyond pure intellectual merit, has been regularly documented in the existing scholarship.

Economic imperialism has been one of the results of mainstream dominance, and its academic impact on other social sciences has been widely documented, including their corresponding areas of policy making. In this regard, I present here an approach to the problematic relationship between Neoclassical Urban Economics and Urban Planning. These are two related social science disciplines, which however have very different epistemologies and approaches to policy advice.

The main difference between academic mainstream Economics and Urban Planning is methodological, in terms of what is considered a valid approach to scientific knowledge. Economics builds upon logical positivism; it first performs deductive theory construction that “describes” reality, and then subsequently tests its theoretical predictions, which in some cases (not all the cases) lead to policy prescriptions. In contrast, Urban Planning is an action-oriented and problem-solving scientific discipline. It inductively produces normative theory, which explicitly shows the analyst’s point of view regarding the topic and how to intervene on it (public policy advice).

Mainstream Economics is in essence defined by the method and theoretical approach, not by the topic (the economy). This allows it to engage with a wide variety of topics, one of them being the spatial analysis of the built environment, which is also the topic of academic Urban Planning.

Read More »

On the perils of embedded experiments

jaandaraz1 Comment

There is growing interest in ‘embedded experiments’, conducted by researchers and policymakers as a team. Aside from their potential scale, the main attraction of these experiments is that they seem to facilitate speedy translation of research into policy. Discussing a case study from Bihar, Jean Drèze argues that this approach carries a danger of distorting both policy and research. 

Evidence-based policy is the rage, to the extent that even village folk in Jharkhand (where I live) sometimes hold forth about the importance of ‘ebhidens’, as they call it. No one, of course, would deny the value of bringing evidence to bear on public policy, as long as evidence is understood in a broad sense and does not become the sole arbiter of decision-making. However, sometimes evidence-based policy gets reduced to an odd method that consists of using randomised controlled trials (RCTs) to find out ‘what works’, and then ‘scale up’ whatever works. That makes short shrift of the long bridge that separates evidence from policy. Sound policy requires not only evidence – broadly understood – but also a good understanding of the issues, considered value judgements, and inclusive deliberation (Drèze 2018a, 2020a).

Enormous energy has been spent on the quest for rigorous evidence, much less on the integrity of the process that leads from evidence to policy. As illustrated in an earlier contribution to Ideas for India (Drèze et al. 2020), it is not uncommon for the scientific findings of an RCT to be embellished in the process. This follow-up post presents another case study that may help to convey the problem. It also illustrates a related danger – casual jumps from evidence to policy advice. The risk of a short-circuit is particularly serious in ‘embedded experiments’, where the research team works ‘from within’ a partner government in direct collaboration with policymakers.

The case study pertains to an experiment conducted in Bihar in 2012-2013 and reported in Banerjee, Duflo, Imbert, Mathew and Pande (2020)1. This is a large-scale, influential experiment by some of the leading lights of the RCT movement – indeed, a formidable quartet of first-rate economists reinforced by one of India’s brightest civil servants, Santhosh Mathew. The high technical standards of the study are not in doubt, and nor is the integrity of the authors. And yet, I would argue that something is amiss in their accounts of the findings and policy implications of this study.

Read More »

Abolition Will Not Be Randomized

caseybuchholz1 Comment

28476745294_cf7a695ec4_o

By Anastasia Wilson and Casey Buchholz

In the wake of the current uprising in support of Black Lives Matter, there has been increasing interest in the use of mainstream empirical methods in economics — like randomized control trials (RCTs) and administrative data evaluation — to address issues of racism and violence in the institution of policing. These interests are well intentioned, but similar to prior debates, we are reminded that “there is reason for concern” about the relevance of these approaches amidst a mass movement calling for deep structural and institutional change. In just two weeks, mass protests have sprung up across the U.S. and the world calling for the defunding, disbanding, and abolition of police as well as the dismantling of white supremacy. This moment has the potential to bring about an institutional and structural shift in our politics, society, and economy. Given this, we will echo many of the concerns shared by economists about the limits of some empirical methods, the biases embedded in administrative data, and the relevancy of these approaches to the current moment calling for immediate change. Read More »

Misunderstanding the average impact of microcredit?

milfordbateman2 Comments

6886186408_5de7def831_o.jpg
Photo: Peter Haden. Microfinance center leaders tally the week’s loan payments in India.

By Milford Bateman and Maren Duvendack

A recent article on the “average impact of microcredit” by Dr. Rachel Meager (LSE) has received much praise over the past few weeks. Meager deploys Bayesian hierarchical modelling to provide a new take on the argument in favour of a reformed system of microcredit. Her work builds on the data provided by six randomized control trials (RCTs) conducted by Abhijit Banerjee and colleagues (see Banerjee, Karlan and Zinman, 2015). Meager makes an attempt to exculpate the microcredit model from the awkward fact that its impact on the poor has been very much less than originally envisaged. She also claims to show that the critics have overstated the negative impact of microcredit. Microcredit should therefore continue to be a policy intervention, she goes on to say, but there need to be changes in the operating methodology for a more meaningful development impact to be possible in the future.

While seemingly a well-meaning attempt to explore the impact of microcredit, we were struck by the way that her overall argument appears to seriously misunderstand, and it definitely misrepresents, the existing research on microcredit as a development instrument. Read More »

Why positive thinking won’t get you out of poverty

Farwa Sial11 Comments

attitude-be-positive-draw-262532.jpgBy Farwa Sial and Carolina Alves

In a recent article in the New York Times, the development economist Seema Jayachandran discusses three studies that used Randomised Controlled Trials (or RCTs) to understand the benefits of enhancing the self-worth of poor people. Despite wide differences in context, all the cases explore the viability of ‘modest interventions’ to ‘instill hope’ in marginalised communities, concluding that ‘remarkable improvements’ in the quest for poverty reduction are possible.

One of the studies from Uganda, for example, argues that “a role model can have significant effects on students’ educational attainment,” so the suggestion for policy-makers might be “to place more emphasis on motivation and inspiration through example.” Another case study of sex workers in Kolkata Brothels argues that “psychological barriers impede such disadvantaged groups from breaking the vicious circle and achieving better outcomes in life,” so small but effective changes that address these psychological constraints can alleviate the effects of poverty and social exclusion.

The underlying theme of these studies is that individuals can surmount the structural challenges of poverty through their own efforts using tools like ‘effective role models,’ the generation of ‘more hope,’ and the ‘improvement of their mental health.’ Positive psychology of this kind and an emphasis on behavior change to meet the goals of individuals have been around at least since the 1950s, first in the popular literature of self-help books and now in academia, where they form part of an increasingly fashionable trend to ‘do poverty reduction differently.’Read More »

Poor Behavior, Good Behavioral Policies? Double Standards for the North and South

Svenja Flechtner2 Comments

behavioral.jpg

Behavioral approaches to development economics and policy have gained momentum in recent years. A growing number of papers studying behavior of people in poor countries have been published in top journals, accompanied by the rise of randomized controlled trials (RCTs). In 2015, the World Development Report was dedicated to behavioral and cognitive research and policy. Papers studying how to nudge farmers to use fertilizers or increase savings have become classics in the field. Lots of hope has been placed into social experiments and behavioral policies to fight global poverty.

Behavioral policies are of course not reserved for policy-making in poor countries. In fact, nudges became famous with a US-American savings plan. Many behavioral instruments have been discussed and tested in and for rich countries. But there has been an important difference as compared to the debates in development economics: when debating behavioral policies in rich countries, scholars have also devoted lots of time to consider normative and ethical concerns. For example, following Thaler and Sunstein’s exposition of Libertarian Paternalism (see also here), a debate unfolded on whether nudges could be anti-libertarian (here, here, here, or here). Implications of the use of nudges as a new form of government policy have been analyzed, for example, from a Foucauldian perspective, or with a focus on institutional change. Books have been written about ethical concerns. The debate has reached a great level of differentiation, e.g. when authors argue that so-called social nudges (these are nudges that seek to stimulate voluntary cooperation in social dilemma situations) may be justified for different reasons than those targeting individual welfare. Overall, the debate has become really sophisticated, and the autonomy, welfare, and dignity of citizens in rich countries as well as consequences of the use of behavioral policies for these countries’ modes of government have received lots of careful scrutiny (recently again here).Read More »