Indicator tracking table: recalculation, double counting and reference sheet

Q3: 1.96 × √(0.381 × 0.619 / 310) = 1.96 × 0.0276 = ± 5.4 points, i.e. an interval of 32.7% to 43.5%.

Baseline: 1.96 × √(0.22 × 0.78 / 300) = ± 4.7 points, i.e. 17.3% to 26.7%.

The intervals do not overlap: the improvement is statistically defensible. However, it cannot be entirely attributed to the project (no comparison group; other actors distribute soap in the area).

The upper bound (43.5%) remains below the 50% target: three months before the end, the target is unlikely to be reached if nothing changes. The barriers must be analysed (cost of soap, access to water, lack of a facility) and the response adjusted (distribution of facilities, soap refills).

71% / 38% gap: a 33-point difference between what people say and what is observed. This is a classic social desirability bias: households give the answer they think is expected of them. This justifies keeping observation as the method.

• Definition: proportion of targeted households where the enumerator observes a handwashing facility (fixed or mobile) with water and soap (or ash if the project accepts it, to be specified) at the time of the visit, and whose respondent gives a correct demonstration (wet, lather, rub for at least 20 seconds, rinse, air-dry or dry with a clean cloth).
• Numerator: households meeting both conditions. Denominator: surveyed households that agreed to the observation.
• Method: household survey using a simple random sample from the list of beneficiaries; direct observation, not a closed question; digital form with a photo of the facility only if the household consents, with no faces.
• Data collectors: trained enumerators, independent of the community health workers who run the sessions (to avoid courtesy bias).
• Frequency and timing: baseline, midline (Q2) and end of project, in the same season to limit the effect of water availability.
• Disaggregation: sex of the head of household, village or site, household with a person with a disability, status (displaced, returnee, host).
• Baseline and target: 22% (n = 300, month 1); target 50%.
• Responsible: MEAL officer for collection and analysis; project manager for the use of the results.
• Quality control: back-check of at least 5% of households; margin of error reported with the result.
• Known limitations: a one-off observation does not prove practice at critical times; presence of soap varies depending on the day of the visit.

Sample answer: "I understand the concern, but we cannot keep 6,940: this figure counts the same people several times and the indicator is about unique people. An audit or a final evaluation would easily detect it, and correcting it then would cost us far more in credibility. The Q2 figure (4,630) had the same error. I suggest we report 4,870 unique people in Q3, with a footnote explaining the change of method and the revision of the Q2 figure. We can also show the 6,940 attendances as additional data: they show that 1,450 people came back, which is a positive point. I will prepare the note for validation tomorrow."

Measures: (1) mandatory individual identifier in the attendance form and automatic deduplication before each consolidation; no more paper forms without a number, or data entry with matching against the list of names; (2) validated reference sheets for all indicators and training of the teams on the difference between counting people and counting acts, and on how to calculate a cumulative percentage; (3) quarterly data quality review by MEAL before anything is sent to the donor, with consistency checks (no percentage indicator may exceed 100% achievement without verification, comparison with primary sources).