Innovation Advisory · Framework 07
5 questions · VALIDThe Indicator Test
Five questions to apply to every indicator before it goes into the logframe. Use it when somebody has proposed a metric and you have to decide whether it earns its place.
Does the indicator measure the construct named in the ToC, or only a convenient proxy?
Write the construct in plain language (‘do mothers feel supported during the perinatal period’) and put the indicator next to it (‘count of calls received’). If a stranger reading the two would not believe the indicator captures the construct, you have a proxy problem. The proxy may still be useful, as long as you call it a proxy.
Does a result on this indicator tell the implementer what to do next?
If the number moves, what changes in the programme on Monday? If the answer is “we report it” and there is no “we do X”, the indicator serves the dashboard and does nothing for the work. Either find an indicator that drives a decision, or admit this one is for accountability only and stop claiming it guides delivery.
Does the indicator correspond to a specific node in the causal pathway?
Point at the box on the diagram. If the indicator floats above the whole programme (‘lives improved’, ‘outcomes achieved’), it is measuring vibes. Anchor it to one step in the chain, so that when it moves you know which arrow it is testing.
How hard is the indicator to manipulate without producing the underlying change?
This is Goodhart's law in operating clothes. If a frontline worker under pressure can move the number without doing the work, through repeat calls to the same beneficiary, easy enrolment of households already served, or training counts that double-count attendance, the indicator will be gamed long before the programme is evaluated. Test the metric against the incentives it creates.
Can the indicator be broken down by the equity dimensions that matter for this programme?
An average improvement is where inequities hide. Decide in advance which breakdowns the indicator must support (caste, class, gender, geography, age, disability) and check that the data system can produce them. If it cannot, the indicator will report that the programme worked while the people you most wanted to reach were left behind.
What the five questions are doing
The five letters divide indicator review into five separate jobs. Valid asks whether the metric measures the construct. Actionable asks whether a change in the metric drives a decision. Linked ties the metric to a specific node in the causal pathway instead of the programme as a whole. Independent of gaming tests the metric against the incentives it creates, which is Goodhart's law in operating clothes. Disaggregable checks that the data system can produce the equity breakdowns that matter before an average hides them.
Use it on every indicator in a logframe, results framework or M&E plan, whether it is your own or one you have been asked to review. Pair it with the Causal Pathway Review when testing a full Theory of Change.