Survey designs, statistical analysis, data quality and…

International NGO · Written recruitment test

Technical test (MCQ) · Officer level

Monitoring and Evaluation (MEAL) Officer

Duty station: Tillabéri region (Niger)
Time allowed: 30 min
Marks: 25 questions
Pass mark: 70 %
Profile: Officer, 2 to 5 years of experience

Instructions

The test has 25 questions and lasts 30 minutes. The timer starts when the test is opened and the test closes automatically when the time is up.
Each question has a single correct answer out of the four proposed. Some questions require a calculation or a judgement: read each question to the end.
A wrong or missing answer scores zero, with no negative marking. A calculator is allowed.
Any outside help (another person, shared document, artificial intelligence) is forbidden. The pass mark is 70% (18 correct answers out of 25).

Original practice test, built on the format of written tests used by United Nations agencies and international NGOs. The context and figures are fictitious.

Question 11 marks

You are preparing a cluster outcome survey. Expected prevalence of the indicator: 30%. Desired precision: ±7 percentage points. Confidence level: 95% (z = 1.96). Design effect: 1.5. What minimum sample size should you use, before adjusting for non-response?

A. 165 households B. 247 households C. 294 households D. 329 households

Question 21 marks

For a selection of 30 clusters with probability proportional to size, the cumulative population of the villages in the area is 84,000 people. The random starting number is 1,150. Which cumulative population figure identifies the third cluster?

A. 3,950 B. 8,400 C. 6,750 D. 9,550

Question 31 marks

A stratified survey interviews 200 households in each of two communes. Commune A: 12,000 households, 40% have access to a latrine. Commune B: 4,000 households, 70% have access. Which estimate should be used for the area as a whole?

A. 55.0% B. 47.5% C. 52.5% D. 62.5%

Question 41 marks

A cluster survey of 400 households gives a result of 50% for an indicator. The observed design effect is 2. What is the approximate margin of error at 95%?

A. ±4.9 percentage points B. ±9.8 percentage points C. ±3.5 percentage points D. ±6.9 percentage points

Question 51 marks

Project group: 32% of households apply the practice at baseline, 51% at endline. Comparison group: 30% at baseline, 38% at endline. What effect do you estimate using the difference-in-differences method?

A. +19 percentage points B. +13 percentage points C. +11 percentage points D. +21 percentage points

Question 61 marks

Annual report. Q1: 1,200 people assisted, all new. Q2: 1,500 people, of whom 400 were already assisted in Q1. Q3: 900 people, of whom 600 were already assisted previously. The annual target is 4,000 unique people. What cumulative total and achievement rate should be reported?

A. 3,600 people, i.e. 90% of the target B. 1,500 people, i.e. 37.5% of the target C. 2,600 people, i.e. 65% of the target D. 2,200 people, i.e. 55% of the target

Question 71 marks

Out of 100 randomly selected households, average monthly spending on water is USD 4.20, with a standard deviation of USD 2.00. What is the 95% confidence interval of the mean?

A. From USD 2.20 to 6.20 B. From USD 3.81 to 4.59 C. From USD 0.28 to 8.12 D. From USD 4.00 to 4.40

Question 81 marks

In the database, average household size is 6.1 people, with a standard deviation of 2.4. One questionnaire records a household of 18 people. What is its z-score and what should you do?

A. About 2.9: a normal value to keep as it is B. About 5.0: a value to delete automatically from the database C. About 7.5: a value to replace with the mean D. About 5.0: an outlier to check with the enumerator or the household, without deleting it automatically

Question 91 marks

A quality check re-interviews 50 households on 12 key variables, i.e. 600 values compared. 54 discrepancies are found. The protocol requires at least 95% agreement. What is the conclusion?

A. 91% agreement: threshold not met, analyse discrepancies by enumerator and by variable, then retrain or re-interview B. 91% agreement: threshold met C. 9% agreement: the whole survey must be redone D. 89% agreement: threshold not met

Question 101 marks

In LQAS sampling, 19 households are selected per supervision area. For a coverage target of 80%, the decision rule is 13. One area obtains 12 positive answers. How should this be interpreted?

A. The area is classified as not reaching the 80% target and should be prioritised for supervision B. Coverage in the area is estimated precisely at 63% (12 / 19) C. The area reaches the target, because 12 is more than half the sample D. The sample is too small to draw any conclusion

Question 111 marks

The school dropout rate falls from 8% to 6%. How should this be presented correctly?

A. A 2% decrease B. A decrease of 25 percentage points C. A decrease of 2 percentage points, i.e. a relative decrease of 25% D. A relative decrease of 33%

Question 121 marks

Two communes have handwashing rates of 45% and 52%, each measured on a simple random sample of 300 households. Can you state that the difference is statistically significant at the 95% level?

A. No: the confidence interval of the difference (about ±8 points) includes zero B. Yes: a difference of 7 points is always significant C. Yes: the relative difference exceeds 15% D. No: two communes can never be compared with each other

Question 131 marks

Which statement best describes the difference between a theory of change and a logical framework?

A. The logical framework is reserved for public donors and the theory of change for private foundations B. The theory of change sets out the causal links and assumptions that lead to change; the logical framework presents a summary of it with indicators and means of verification C. The theory of change replaces indicators with life stories D. They are two names for the same tool; only the format changes

Question 141 marks

Random assignment is impossible, but baseline data exist for comparable villages not covered by the project. Which design best estimates the project's contribution?

A. Compare the change in the project group with that of the comparison group between baseline and endline (difference-in-differences) B. Compare the endline value of the project group with the logframe target C. Compare before and after in the project villages only D. Gather project staff's views on perceived changes

Question 151 marks

In a qualitative evaluation, when should individual interviews with a given group of respondents stop?

A. When saturation is reached: new interviews no longer provide new information B. After exactly 30 interviews, the threshold for statistical representativeness C. As soon as the answers confirm the project's assumptions D. When the planned budget is used up, whatever the content collected

Question 161 marks

Which document specifies, for each indicator, its definition, calculation method, disaggregation, source, data collection frequency and the person responsible?

A. The logical framework B. The annual work plan C. The Performance Indicator Reference Sheet (PIRS) D. The interim narrative report

Question 171 marks

For a survey of adolescents aged 15 to 17 on their living conditions, which consent practice is appropriate?

A. The village chief's agreement is enough for all minors in the village B. The adolescent's consent alone is enough in all cases from age 15 C. No consent is required since the survey is anonymous D. Obtain the consent of the parent or guardian and the assent of the adolescent, leaving them free to refuse at any time

Question 181 marks

In an XLSForm (KoboToolbox, ODK), which column displays the question "Age of the child" only if the household has reported at least one child?

A. constraint B. relevant C. calculation D. choice_filter

Question 191 marks

In an XLSForm, which expression should be placed in the constraint column of the "age" question to accept only values from 0 to 110 years inclusive?

A. . >= 0 and . <= 110 B. . >= 0 or . <= 110 C. . > 0 and . < 110 D. 0 <= age <= 110

Question 201 marks

An evaluation question asks whether resources were converted into results economically and in a timely way. Which OECD-DAC criterion is concerned?

A. Effectiveness B. Relevance C. Sustainability D. Efficiency

Question 211 marks

What is the main purpose of a real-time evaluation conducted in the first weeks of an emergency response?

A. To provide findings and recommendations quickly in order to adjust the ongoing response B. To measure the long-term impact of the response C. To prepare the end-of-project financial audit D. To replace the regular monitoring system

Question 221 marks

Of 210 pieces of feedback received in one month through the accountability mechanism, 101 concern perceived exclusions from targeting, concentrated in two villages. What is the most relevant response?

A. Close the feedback cases, since the lists were validated by the local committee B. Automatically add all complainants to the beneficiary lists C. Analyse these cases with the programme team, check how the criteria were applied in these villages and inform the communities of the action taken D. Wait for the quarterly report to discuss it

Question 231 marks

Enumerators wearing the NGO's vest measure satisfaction at the exit of a distribution. 97% of people say they are satisfied. Which main bias should be feared?

A. Recall bias B. Social desirability or courtesy bias C. Bias due to the small sample size D. Non-response bias from households absent from the area

Question 241 marks

Post-distribution monitoring only interviews households present on the day of the next distribution. What main risk does this create?

A. A design effect that is too high B. A risk of double counting beneficiaries C. No risk, as all these households are beneficiaries D. A selection bias: households that no longer have access to distributions, often those experiencing problems, are excluded

Question 251 marks

An outcome indicator is measured at 62% (95% confidence interval: 55% to 69%) against a target of 60%. What should be reported?

A. The target is widely exceeded B. The target is not reached, because the lower bound is 55% C. The result is invalid, because the interval is too wide D. The target is reached as a point estimate, but the confidence interval does not allow you to state that it has been exceeded

Other tests for this profession

Panel interview question for this profession

Tell us about a time when you discovered, during data collection, that some data were not reliable. How did you detect it and what did you do?

Competency assessed : Data quality and supervision of data collection

What the panel expects : A concrete control system (durations, back-checks by phone, consistency), detection based on several signs, a decision on the data concerned, fair treatment of the enumerator and an improvement to the system.

Possible follow-up questions

Which indicators did you monitor every day during data collection?
How did you avoid accusing the enumerator before you had the facts?
What was the effect on the sample and the timeline, and how did you announce it?

Strong answer

In Niger, I was supervising a post-distribution monitoring survey of 420 households with eight enumerators. My daily control table showed that one enumerator had a median duration of 11 minutes for a 28-minute questionnaire, and no "don't know" answers. I had 12 of her households called back: 7 did not remember any visit. I presented these facts to the enumerator in private, with the supervisor; she admitted to filling in questionnaires from the list, under pressure from a target of 15 interviews a day. We discarded her 64 interviews and had her households re-interviewed in three days. The report went out four days late, which I announced to the project manager on the first day. Afterwards, I reduced the target to 8 interviews a day, increased back-checks to 10% and added automatic alerts on duration.

Weak answer

"I always check the data. Once, an enumerator had strange results, so I dismissed him and we kept the other data." This answer does not say how the problem was detected, does not check the facts before sanctioning, and ignores the effect on the sample and on the conclusions.

← All tests

Survey designs, statistical analysis, data quality and accountability

Instructions

Result

Common mistakes that cost marks