Innovation Advisory · Framework 12
6 questionsThe Baseline Design Test
Six questions to check whether your baseline survey can carry the endline it is meant to feed. This canvas is written for programme designers, MEL teams and funders about to sign a baseline TOR.
What comparison will the endline make against this baseline?
A baseline exists so that the endline can be read against it. Write down the specific comparison you plan to make (before and after, treatment and control, or one cohort against another) on the first page of the design. If the comparison is vague, the baseline will measure things you will not need and miss things you will.
Is the sample large enough to detect the effect you are hoping to see?
If the baseline sample is too small, adding respondents at endline cannot rescue it, because the link between the two rounds is already broken. Do the power calculation for the smallest effect that would matter in practice, and resist using the largest effect that would look impressive. Report the calculation with the design.
Will the sample frame you draw today still describe the population at endline?
In-migration, out-migration, deaths, new construction, seasonal households and urban expansion all change the frame. If the endline is two years away and the population moves, you need either a rolling refresh of the frame or a design that can tolerate frame drift. Decide which now.
Will you use the same instrument at endline, item for item?
If the endline instrument changes, with new items added, old items dropped or wording revised because the pilot exposed problems, the comparison between rounds weakens in ways the report will struggle to defend. Freeze the core items at baseline, add exploratory items separately, and record the freeze.
What does the baseline already know that you have not asked?
Every population knows things about itself that a survey cannot produce. Before spending money on a household questionnaire, run a small, qualitative and deliberately unstructured listening exercise. What you hear will change the survey design, and it often changes the programme design as well.
How many separate comparisons will this baseline eventually feed, and are you correcting for them?
A baseline with many variables invites fishing for results through multiple comparisons at endline. Decide in advance how many bivariate tests you plan to run and pre-register the correction. Bonferroni is the minimum correction to use. If the baseline design cannot support the correction, either reduce the analysis or increase the sample.
What the six questions are doing
Together, the six questions cover the six common ways a baseline fails to last until endline. The comparison question makes the design name the specific evaluative question. The power question checks that the sample is large enough to answer it. The frame drift question catches the population change that separates the baseline from the endline. The instrument continuity question protects the match between the two rounds. The already-knowns question stops the design from paying for information the community could have given for free. The corrections question deals with the inflation from multiple comparisons that turns a baseline into a fishing ground.
Run this before the survey firm is procured. The design decisions you leave to the firm will shape the findings you are able to defend.