Somebody asked last week why we are writing a book about measurement instead of something more cheerful. The plain answer is that twenty years in development research have meant watching the same failure produce the same report over and over.
The failure goes like this. A programme is designed, and an indicator is built with it. Over a couple of years the programme adapts to the indicator, because the metric becomes the target, an old observation that many people have made. The indicator stops tracking what the programme was meant to do. The evaluation happens anyway, the report is written, and the report says the programme worked.
We have written that report. We have read many versions of it, and we have lost colleagues over it.
The Measurement Trap tries to make the trap easier to see. It springs at the moment the indicator becomes the thing being managed and the original question drops out of the discussion. The book’s real subject is politics: who decides what to count, and what is left off the list. Statistics are the means by which this happens. The argument is about care, about doubt, and about keeping a public list of what is being left out.
More soon.