Survey analysis, logical framework and donor report

Weaknesses: (1) the result 1 indicator ("number of vouchers distributed") is an output indicator, not an outcome indicator; (2) the target of 7,200 vouchers does not correspond to anything measurable at household level and says nothing about use; (3) result 2 ("satisfied") is not a change expected from the project but an accountability measure; (4) no disaggregation (sex of head of household, disability, site); (5) no second objective-level indicator even though the rCSI is collected.

Proposal: Specific objective: "Targeted displaced households improve their food consumption and reduce their use of negative coping strategies". Indicators: % of households with acceptable FCS (baseline 38%, target 60%, household survey) and mean rCSI (baseline 24, target ≤ 16, same source), disaggregated by sex of head of household and by site. Result 1: "Targeted households have regular access to diversified food on local markets": % of households that received and used 6 out of 6 vouchers (target 95%, distribution lists and trader redemption reports); mean number of food groups consumed (household dietary diversity score). Result 2: "Households take part in the assistance and can assert their rights": % of complaints handled and closed in less than 14 days (target 90%, complaints mechanism register); % of households aware of at least one complaints channel (target 85%, post-distribution monitoring).

Margin of error = 1.96 × √(p(1 − p) / n) = 1.96 × √(0.61 × 0.39 / 182) = 1.96 × 0.0362 ≈ ± 7.1 points. The confidence interval is therefore approximately 54% to 68%.

The point estimate (61%) exceeds the target (60%), but the lower bound (54%) is below it: we cannot state with certainty that the target has been reached. We can, however, state a significant improvement compared with the baseline (38%, margin ± 5.6 points on n = 290: interval 32% to 44%), because the intervals do not overlap.

Non-coverage bias must be added: Lushagala is missing, so the endline sample is not representative of both sites. If the survey used cluster sampling, the design effect (often 1.5 to 2) widens the margin further.

Female-headed households: an increase of +15 points (29% to 44%) against +23 points overall; the gap has widened. Beware of the small sample size (n = 61, margin of about ± 12 points): this is a signal to be confirmed, not proof.

Complaints: 500 complaints; the main category (exclusion from targeting, 212, i.e. 42%) is handled on time in only 29% of cases (61 out of 212), which points to a targeting problem and an overloaded response mechanism. 148 complaints about prices suggest a price increase at approved traders, which reduces the real value of the voucher. The 4 complaints about staff behaviour may relate to sexual exploitation and abuse: they must follow the confidential PSEA procedure, not the normal channel, and only one was handled on time.

Recommendations: (1) review targeting with a mixed committee and an appeals process, and strengthen the team handling exclusion complaints; (2) monitor prices weekly at approved traders, add more traders and index the voucher value if the increase exceeds 10%; (3) analyse the barriers faced by female heads of household (care burden, distance, safety) through focus group discussions, and adapt (timetables, home delivery for the most vulnerable); (4) refer the 4 sensitive cases immediately to the PSEA focal point and check that the people concerned receive support.

Sample answer: "At the end of the project, 61% of surveyed households had an acceptable food consumption score, compared with 38% at the start (margin of error ± 7 points), and the reduced coping strategies index fell from 24.1 to 15.3. The improvement is clear; however, achievement of the 60% target cannot be statistically confirmed. Female-headed households made less progress (29% to 44%), a gap we are analysing. Three limitations should be noted: the Lushagala site could not be surveyed at the end of the project for access reasons, which may overestimate the results; the absence of a comparison group prevents us from attributing all the change to the project (harvest and other assistance); finally, 42% of complaints concerned exclusion from targeting, and only 29% of these were handled on time. Corrective measures are under way (review of targeting, price monitoring, strengthening of the complaints mechanism)."

I diplomatically refuse to remove this information: it is necessary for data integrity and for accountability to the donor and to affected populations; donors check methods, and an omission discovered later would do more damage to the organisation's credibility. Instead, I propose presenting it constructively (what we learned, measures taken) and adding a follow-up survey in Lushagala as soon as access allows. If the pressure continues, I refer the matter to the regional MEAL manager or to headquarters in line with internal policy.