In 2009 the idea was simple. We would create a space inside schools where girls could meet regularly, with an adult facilitator, to talk about the things that affected their lives: their education, their health, and the marriages their families were planning for them.
The theory was that girls speaking as a group would be heard differently from one girl speaking alone. A family that could dismiss a daughter’s objection might hesitate when twenty girls from her school arrived at the door.
It worked, but by a stranger and more interesting route than the theory predicted.
The collective voice
The collectives were set up across several Indian states and reached thousands of schools. In some districts they measurably delayed the age of marriage for girls in the surrounding community, including girls who were not members.
They did this by attaching a social cost to early marriage. A family thinking of marrying a daughter early knew the girls would talk about it, to teachers, to the panchayat and to other families. Child marriage depends on privacy, and the existence of the collective took that privacy away.
What the data missed
The standard programme evaluation measured the expected things: school attendance, age at marriage, and how often the collective met. Funders ask for these indicators and logframes are built around them.
The more interesting change was harder to measure. Girls who had been in a collective for a year behaved differently. They spoke up in class. They negotiated with their parents about what time to come home. They asked visiting health workers questions the health workers had never been asked before.
This is agency, and it does not fit neatly into an indicator or a number. It is what the collective produced, and it is what made the delay in marriage possible. The delay followed from the agency, and the agency was the main outcome.
The measurement problem
We did not have a good instrument for measuring agency in 2009, and we barely have one now. The Child and Youth Resilience Measure (CYRM) captures part of it and self-efficacy scales capture a little more. But a moment like a fourteen-year-old girl standing up at a village meeting to say publicly that her classmate should not be married yet cannot be reduced to a Likert scale.
This is the gap between what programmes produce and what evaluations can detect. The collective produced agency, while the evaluation could detect attendance and age at marriage. Much of the important work in development measurement still lies in the space between the two.
What we took from this
Three lessons have stayed with us in every programme we have worked on since.
The first is that the unit of change was the collective and not the individual girl. One girl speaking alone was dismissed, and twenty girls speaking together changed how the community weighed its decisions. Programme design that targets individuals misses this.
The second is that the most important outcome was the hardest to measure. Agency showed in how the girls carried themselves, in their confidence, and in the questions they asked. It did not show in the logframe.
The third is that the programme worked because it was built into an existing institution, the school. The collective met inside the school, during school hours, with a teacher present, and did not run as a parallel structure. That is what made it last and what made the community take it seriously.
We are still working on how to build measurement systems that can see these changes without reducing them to numbers that miss what happened.