Figures enable us to track developments, compare results and steer a project. Testimonials, on the other hand, give meaning to the data and shed light on the mechanisms of change. When measuring social impact, the challenge is therefore not to choose between qualitative and quantitative approaches, but to know when to use each method and how to combine them effectively.
What is the difference between qualitative and quantitative data?
Before choosing your indicators or data collection method, it is essential to distinguish between the two main types of data used in impact measurement. The quantitative method relies on closed-ended questions. Presented as key figures, the responses are used to measure an activity or an effect. They are easier to analyse than qualitative data.
The qualitative method, on the other hand, is based on open-ended questions that help to describe a situation, a need or an issue faced by your beneficiaries. The responses therefore take the form of text or observations.
Objective and subjective impact indicators reflect quantitative data.
- Objective indicators take the form of numerical data, amounts or percentages, and measure an effect.
- Subjective indicators measure a stakeholder’s perception of a situation and are based on observations or statements. To this end, linear-scale indicators, single-choice and multiple-choice questions, etc., will be used.
Once the data to be collected has been identified, the next step is to choose the method best suited to your target audience and your resources.
What methods can be used to collect impact data?
Not all data collection methods have the same cost or the same level of accuracy. The choice depends mainly on your audience and the relationship you have with them.
- Data already available: Inexpensive and quick to analyse, but it mainly provides factual information (profile, participation). It does not allow you to assess people’s feelings or the changes they have experienced.
- Online survey: Effective when you have the contact details and the beneficiaries are comfortable using digital tools. This allows you to collect quantitative data from many people, but the qualitative accuracy remains limited.
- Remote interviews (telephone/video call): More engaging than an online questionnaire. They allow for more nuanced responses but require more time and organisation.
- Face-to-face interviews: The most insightful method for understanding the changes experienced, but also the most time-consuming. Very useful when human interaction is key, or if people have difficulty using digital tools.
- Focus group: A collective qualitative method that helps to bring shared perceptions to the fore. It requires skilled facilitation and dedicated time.
There is no ‘right’ method as such: one should choose the method that is realistic, suited to the target audience, and consistent with the available resources.
Why combine qualitative and quantitative data in an impact assessment?
Qualitative data can be used at various stages of the impact assessment or the project lifecycle. Firstly, when defining the theory of change or the project itself. Conducting qualitative interviews with your stakeholders – particularly those for whom you wish to measure the project’s effects – helps you to better identify the issues they face, their unmet needs and their expectations of the project. This is what community centres do as part of their ‘local area assessment’ before drafting the social project proposal.
⚠️A tip: limit the number of interviews so as not to be overwhelmed by the volume of data to analyse (five interviews are sufficient).
In the context of impact assessment, an in-depth qualitative study is frequently combined with impact indicators. Impact assessment is generally a one-off exercise, most often carried out at the end of a programme by a third party.
It is also useful to include open-ended questions in your online surveys. This allows you to collect verbatim responses and to open the discussion. Beneficiaries can thus explain which aspects of the project they liked best and make suggestions for improvement.
⚠️ A word of advice: once again, keep open-ended questions to a minimum. To maximize answers, surveys should not take too long for your beneficiaries to complete (5 to 8 minutes maximum). And leave a sufficient space for impact indicators.
When managing your organisation’s impact, you should prioritise quantitative methods that allow you to track indicators over time and monitor how they change, helping you to improve and showcase your activities. Collecting verbatim comments, meanwhile, provides you with useful testimonials for your communications. Combined with the figures, they enable you to tell a story – that of your project – and to humanise the data and your beneficiaries’ journeys.
Conclusion
Ultimately, the question is not whether qualitative data is more relevant than quantitative data, or vice versa. Both serve different purposes and are complementary.
Quantitative data enables us to measure, compare and track changes in outcomes over time. Qualitative data helps us to understand why these outcomes occur, to identify emerging needs and to provide depth to the results.
For most organisations, an effective approach to measuring social impact is based on a simple balance: relying primarily on quantitative indicators to steer activities, whilst incorporating some qualitative data collection at key stages of the project. This combination makes it possible to improve initiatives, demonstrate their value and share more meaningful results with partners, funders and beneficiaries.
Ultimately, the best method is not the most sophisticated one, but the one that your organisation will be able to implement consistently and sustainably.
To go further
👉 1st step : build your theory of change with our illustrated guide
👉 You are looking fo support training to build you impact assessment? Chat with an Impact Track expert!