Verification of reported data and quality control of a survey

The overall factor (-4.5%) is below the 5% threshold, but it hides two sites with anomalies: this is why verification is done site by site.

Figure reported: 1,619 households verified, with a note stating that 305 households in Nova Sofala will be reported once the lists have been found and verified (maximum possible: 1,924). The figure of 2,000 is never reported.

Follow-up: Guara-Guara: trace the 22 households (visit or call); if they confirm they received the kit, have the list completed with a witness and a signed note; if not, find out where the kits are. Grudja: 290 kits released, 236 households served and no returns, hence 54 unaccounted-for kits: immediate physical check of the stock on site, interview with the distribution team and the committees; if diversion is possible, report it under the anti-fraud policy, without conducting a personal investigation. Nova Sofala: search for the lists in Beira; failing that, verification with a sample of households (calls, visits). For all: systematic reconciliation of warehouse, lists and report before each monthly report.

Back-check rate: 39 / 380 = 10.3% of interviews, which is a good level.

Discrepancy rate: E1 0%; E2 2 / 6 = 33%; E3 1 / 7 = 14%; E4 5 / 9 = 56%; E5 0%; E6 1 / 5 = 20%.

E4: a set of converging signs of invented data or skipped modules: median duration of 12 min for a 30-min questionnaire; 61 / 88 = 69% of interviews under 15 min; no "don't know" answers (implausible); 88 interviews against an average of 58 for the others; discrepancies in more than half of the back-checks. This is not proof of fraud, but the data are not reliable.

Decision: suspend E4 from data collection pending verification; exclude his 88 interviews from the analysis; have his households re-interviewed by other enumerators. Without E4, 380 - 88 = 292 interviews remain, fewer than the 324 required: at least 32 are missing. But adding 32 households taken elsewhere is not enough: E4's 9 villages (30% of the villages) would be under-represented. The households drawn in those villages must be redone.

E2: discrepancies on household size, a key question for per-person indicators: review the definition of a household with him, back-check 20% of his next interviews. E6 and E3: normal follow-up, to be monitored.

Going forward: daily rather than weekly control table, alert thresholds (duration less than half the pre-test median, no "don't know" answers, discrepancy of more than 20% in back-checks).

"Thank you for your message. I understand the time constraint, but I cannot keep E4's data as they are: 69% of his interviews last less than 15 minutes for a 30-minute questionnaire, he has no "don't know" answers and 5 back-checks out of 9 show discrepancies. If these data go into the analysis, the whole report loses its value, and the donor or an evaluation could see it. I propose to re-interview his households with two enumerators over three days and to inform the project manager today that the report will go out three days late, or that a preliminary note without the 9 villages will be sent on Friday, with its limitations clearly stated."

Fairness towards E4: no accusation; the findings are presented to him factually and he can explain himself (misunderstanding of instructions, pressure on the number of interviews, tablet problem). What follows is a matter for the HR procedure and, if fabrication is proven, for the anti-fraud policy. The findings remain confidential. We also check whether the system encouraged speed (payment per interview, daily targets too high).