What are Randomized Controlled Trials?

By Ayesha Imran Malik

Few fields are as exciting as Experimental Economics: the use of experimental methods to study
economic questions.

There’s a lot of talk about Randomized Controlled Trails (RCTs), but what are they exactly?

Think of a medical drug trial. There are two groups; a treatment group, which receives the real drug
and a control group, which receives a placebo. Before the drug is given, the treatment and control
group are selected in such a way that they are exactly identical on average, in terms of the variables
of interest. Once the drug is administered, the researchers can compare both groups and gauge the
true effect of the drug.

In development economics, the same method is gaining popularity in terms of evaluating social
programs. These are called impact evaluations.

Consider a literacy program to improve reading skills of primary school children in rural areas. The
primary outcome variable we are looking at are reading scores. Suppose that after the program was
implemented, reading scores improved. Do we declare the intervention to be successful? Not if we’re
scientists.

Simply comparing before and after is not enough. We need to be able to attribute the improvement
to the program and nothing else. What we need is an insight into an alternate reality where the
intervention never took place and then see how reading scores fare. We could then subtract the
differences in reading scores in these two realities and obtain an estimate of the real impact of the
program. This alternate reality is called a counterfactual.

Unfortunately, the counterfactual does not exist. It is purely hypothetical.

We need something to mimic the counterfactual. This is where the control group comes in. So we
collect data on two groups, one that receives the program and the other that doesn’t. Then, we
compare their reading scores.

The beauty of RCTs is that the treatment and control group are identical on average at the baseline,
the only difference being that one receives the treatment and the other does not.

How do we make sure that they are identical? By randomizing who receives the treatment and who
does not. This means that everyone in the relevant population has an equal probability of being
selected for receiving the program.

But it is not so straightforward. The social programs are carried out in the real world, where there are
many influences and environmental factors, not to mention the externalities that the treatment
generates. There are administrative issues as well. Furthermore, RCTs of social programs are costly.
Arianna Legovini, evaluation-division head at the World Bank, stated that the average price of an
impact evaluation is $500,000.

Nonetheless, the zeitgeist of the development world revolves around evidence these days and
practitioners of RCTs, known as “randomistas” are leading the movement with fervour and gusto.