📊 Quantitative Impact Evaluation Design: Three Key Questions

 

How do we design a quantitative impact evaluation that generates credible evidence about whether a program has made a difference?

As part of my ongoing learning in Monitoring, Evaluation, Accountability and Learning (MEAL), I am exploring three fundamental questions that guide the design of an impact evaluation.

These questions follow a logical sequence: What will be measured? → Who will be measured? → How will the impact be measured?

🔹 1. What Will Be Measured?

The first step is to define precisely what the evaluation will assess.

• What is the treatment or intervention, and how is it expected to work?
• What changes are expected?
• Which evaluation questions need to be answered?
• What indicators will measure the expected changes?

A clear program theory helps establish the relationship between the intervention, its expected outcomes, and the changes to be measured.

🔹 2. Who Will You Take Measurements About?

Once we know what to measure, we need to identify the relevant population and units.

• Hierarchy: Are the units organized within relevant structures, such as individuals, households, communities, or districts?
• Units of observation: From whom or what will data be collected?
• Units of analysis: At what level will differences be examined and conclusions drawn?
• Stratification: Do different groups need to be represented in the evaluation?

These decisions help define the study population and guide the sampling and analysis approach.

🔹 3. How Will You Measure the Impact?

The final question concerns the methods used to collect, compare, and analyze evidence.

• Data sources: Will information come from primary data collection, secondary data, or administrative records?
• Measurement timing: How many observations are required, such as baseline, midline, and endline?
• Counterfactual: How will we establish a credible understanding of what would have happened without the intervention?
• Causal attribution: How, and to what extent, can observed changes be attributed to the intervention?

The evaluation design must be appropriate for the questions being asked and capable of producing credible evidence.

💡 My key learning as a MEAL professional:

A strong quantitative impact evaluation begins by defining what change to measure, identifying whose experiences or outcomes to examine, and selecting appropriate methods to estimate the intervention's effect.

Define the change → Identify the population → Design the measurement → Assess the impact.

These decisions are interconnected and form the foundation of a rigorous, evidence-informed impact evaluation.

I welcome recommendations for courses and practical resources that can further strengthen my knowledge of quantitative impact evaluation and MEAL.

#MEAL #ImpactEvaluation #QuantitativeResearch #MonitoringAndEvaluation #EvaluationDesign #DataAnalysis #EvidenceBasedPractice #ProfessionalDevelopment #ContinuousLearning