A funder wants to know the impact of an art programme on children in government schools.
The evaluator reaches for the standard toolkit: a pre-post test, a control group, attendance records, and perhaps a socio-emotional learning scale such as the CASEL framework or the SDQ (Strengths and Difficulties Questionnaire). These are validated instruments. They measure anxiety, prosocial behaviour, peer problems and emotional symptoms.
None of them measures joy.
The problem with measuring art
Art does not work in a straight line. A child who spends an hour painting does not produce a measurable output in the way a child who spends an hour on maths drills does. The drill yields a score. The painting yields an experience, and that experience might change how the child relates to school, to self-expression and to taking risks, and whether she feels allowed to make something that has no correct answer.
These are real outcomes and they matter for a child’s development. They are also very hard to capture with instruments designed for cognitive outcomes that move in a straight line.
The usual response is to measure what can be measured and ignore the rest. The report gives attendance, the number of art sessions delivered and perhaps a teacher satisfaction survey. The funder gets a number and the evaluator gets paid, and the children’s joy goes unrecorded.
We decided to try something different.
What we built
The evaluation of an art programme in Mumbai schools used mixed methods, aiming to capture the whole of what changed and to go beyond the part that is easy to measure.
Classroom observation was the first strand. Trained observers sat in art sessions and coded specific behaviours: spontaneous laughter, voluntary participation such as raising a hand without being asked, helping a peer, trying something new after a failed first attempt, and staying engaged after the session formally ended. These behaviours are proxies for joy and engagement that a pre-post test cannot capture.
Child self-report was the second. Children were asked simple questions, one to one, about how the session made them feel. We used open-ended prompts in place of Likert scales, because the children were not all comfortable with formal assessment. The prompts included “What was the best part?”, “Did you want to keep going?” and “Would you come back tomorrow if you could?”
Teacher perception was the third. Teachers were asked about changes they had seen in particular children, as named observations and without aggregated scores. One example: “Riya used to sit in the back. Now she draws first and talks about it.” These small accounts cannot be generalised, but they capture things a scale misses.
The three streams were then cross-referenced. When a child appeared in the observation data as spontaneously engaged, in the self-report as wanting to continue, and in a teacher’s account as changed, we treated that agreement as evidence of impact, even though no single data point would satisfy a journal referee on its own.
What we found
The programme changed the atmosphere of the classrooms it worked in. Children in art sessions were more willing to take risks, both artistic and social. They spoke up more, helped each other more and laughed more. Observers saw these changes, and teachers reported them independently.
The method showed that arts-based outcomes can be evaluated fairly if the evaluation uses methods suited to the shape of the intervention. A randomised controlled trial would have needed a control group of children denied art. Combining observation, self-report and teacher accounts was messier, but it was ethical and it captured what mattered.
The measurement lesson
Joy is a legitimate outcome. The lack of standardised instruments for it is a failure of measurement science, and it says nothing against the outcome. When a funder asks about impact and the evaluator says joy cannot be measured, the honest answer is to build a better instrument.
This argument sits at the centre of the measurement work we keep returning to. The indicators we have shape what we see. When the indicator cannot see joy, the programme is evaluated on attendance, and the thing that changed the children’s lives drops out of the record.
The Measurement Checklist asks whether an indicator is easy to count or worth knowing. Joy is worth knowing and hard to count, and most of the interesting work in evaluation lies between those two facts.