Innovation Advisory · Framework 16
6 questions · before the listThe Administrative List Test
Six questions to ask before you read, publish or argue from a list the state keeps, such as an electoral roll, a beneficiary register or a list of deletions. The canvas comes from building Off the Roll, which reads the Special Intensive Revision of the electoral rolls from the Chief Electoral Officers' own tables.
Do the rows add up to the totals the same office published?
Sum every column and compare it with the printed district and state totals. Off the Roll will not build until each state’s table matches the Chief Electoral Officer’s published figures, because a misread digit breaks at least one of those sums. Watch for a total row left inside the data: summing a column that contains its own total doubles it.
Who wrote down each reason, and has anyone checked it?
A reason on a list is what an official recorded, often after a single visit. Treat entries such as dead, shifted or absent as claims that still have to be tested, and say on the page who made them. In the SIR lists, absent is the reason that rests on the least evidence, and it is where checking should start.
Which record kept by someone else can you set the list against?
Off the Roll set Bihar’s names entered as dead against the Sample Registration System’s death rates. The count came to about 3.4 years of adult deaths and the ages fitted, but 52 per cent of the names were women against 43 per cent of adult deaths. Find one independent record, such as vital registration, census migration tables or a survey, and note where the list departs from it.
Are the places you are comparing measuring the same thing?
Some states publish the names left off the roll. Others publish only the roll on two dates, and the fall between them includes ordinary deaths, moves and new voters. Name what each figure counts before you set them side by side, and rank places only when they count the same thing.
What can the list not show, and is that written where a reader will meet it?
A list of removals shows how many and where. It cannot show why, and it cannot show intent. Write the limit in one plain sentence at the top of every table you publish, so the reader meets it before the numbers.
Could anything you publish identify a person?
Publish counts and keep the rows. Off the Roll publishes aggregates only: the row-level lists with names and EPIC numbers stay out of its repository, and a check fails the build if any published file carries a field that looks like an EPIC number.
What the six questions are doing
An administrative list records what an office did, and every number drawn from it carries that office's habits. The first question checks that the list agrees with itself. The second and third separate what an official wrote down from what happened, and test the entry against a record somebody else keeps. The fourth stops two different measures being set side by side as if they were one. The fifth and sixth decide what a reader is told and what a reader is never shown.
Use it before the first chart is drawn. Pair it with Source Before You Say It when the numbers are going into a report or a public page.