Political parties in India are federated organisations, and they have a lasting problem with knowledge. The central office wants to know how each district unit is doing: whether the local committee functions, whether it holds meetings, whether it has found credible candidates, and whether the campaign infrastructure will be ready for the next cycle. That knowledge sits with the observer the party sends to look, and in the reports the observer files.
The reports are odd documents. They are usually PDFs uploaded through a portal. They contain candidate names and meeting attendance, sometimes photographs, and sometimes only a signature and a date. They pile up in a database that nobody reads.
Our data project turned that pile into a monthly grade, one score per observer in one of four bands, across more than seven hundred districts in twenty-three states. This note covers what the exercise measured, what it chose not to measure, and what happened when the numbers were shown to the people they described.
The seven parameters
The framework scores each observer on seven parameters. Each has a defined weight and each is drawn from information already in the database.
Report quality. This checks whether the report is complete, whether the fields have real content or the same paragraph copied across three districts, whether the photographs are legible, and whether the dates are consistent with each other. The measure is deliberately plain. Ordinary reports pass and unusual ones get flagged.
Daily activity. This checks whether the observer’s logged activity shows any signal across the month. It catches the observer who files everything on the last day at 11:47pm, with the same timestamp on twelve district reports.
Documentation. This checks whether the supporting attachments (meeting minutes, photographs, candidate resumes) are present and whether they match the report. If the report says a meeting happened on the tenth and the attached photograph carries a metadata timestamp from the seventeenth, the score drops. Neither fact disqualifies the report, but the mismatch is worth reading.
Proposed candidates. This checks whether the observer has put forward names, whether those names carry the required biographical detail, and whether the demographic mix of the proposed pool matches the district’s population. This parameter does the real political work, and I come back to it below.
Timeliness. This checks whether the report arrived before the reporting window closed. A late report drops a full band and a missing report drops two.
Cross-verification. This checks the observer’s claims about attendance, resources and candidate viability against the two nearest data sources, which are the district committee’s own reports and the state secretariat’s activity logs. Any divergence is flagged for review.
Signal-to-noise. This is a rolling metric that looks for anomalies in each observer’s own history: a sudden change in reporting style, a lull followed by a burst, or a run of grade-A reports from a district known to be dormant. It is calibrated to the individual observer and not to the population, so it catches changes within one person that a cross-sectional analysis would miss.
The bias diagnostic
The seven parameters assess the observer. A separate diagnostic assesses the observer’s choices, specifically the demographic composition of the candidates each observer ranks first across the districts they cover.
It turned up two patterns the party would not have found by reading reports one at a time, and a third finding related to both.
In districts where the observer’s own caste was one of the two dominant local castes, more than ninety percent of first-preference candidates came from those two castes. That does not by itself disqualify anyone, since dominant-caste candidates may have the strongest local base. But the pattern was uniform across observers from that background, and it disappeared when the analysis was repeated for observers whose caste was not locally dominant. The party can decide what to do about this. After the analysis, it can no longer claim not to know.
In several majority-Adivasi districts, the first-preference candidates included no Scheduled Tribe names at all. This came to light only because the diagnostic compared the caste mix of the proposed pool with the census demographics of the district. Without that automatic comparison, the pattern would have stayed in the observer’s inbox indefinitely.
The third finding was that proposer phone numbers sometimes matched candidate phone numbers. The proposer is required to be an independent local leader, and the candidate is the person being proposed. When both numbers belong to the same person, the process has failed. The diagnostic flagged eleven such cases in the first quarter. Each one needs a human decision, and that decision can now be made with data in hand.
Why the score has to be visible
On its own, the score would have been an academic exercise. The project became useful because each observer received their grade at the end of every monthly cycle, with a breakdown by parameter. It was presented as a mirror and not as an accusation.
Behaviour changed quickly, though unevenly. Reports got longer. Attachments were attached. Timestamps spread across the month instead of clustering on the last day. Some observers took their grade to the state office to dispute it, which is what should happen, because an argument about a grade is an argument about what the grade measures, and that is a useful conversation to have.
The caste-representation diagnostic moved the slowest. Changing it requires a political decision about who is in the candidate pool, and better paperwork cannot supply that. No scoring framework can.
What the framework is for
The framework is an operating diagnostic and should not be mistaken for an audit. Audits are carried out under different constraints, by different people, with powers to sanction. This is a monthly reading of what the reports say collectively, so that the party can act on a pattern before it becomes a headline.
The distinction matters in practice. When the exercise is described as an audit, people respond defensively. When it is described as a mirror the observer chose to hold up, they respond differently. In this kind of work, what you call the tool affects how it is received.
If you run a similar exercise in an institution organised differently and it does something we did not model here, please write in. The seven parameters are deliberately generic, and any of them can be replaced with one that fits your data.