Analysing quantitative data for evaluation

Use this page to learn about how to analyse quantitative data.

Depending on what type of data collection method you’ve used in your evaluation work, you’ll then need to analyse it using either quantitative or qualitative data.

Quantitative data uses numbers to count or measure. For example, we can count the responses to multiple choice or rating scale questions in a questionnaire.

Analysing quantitative data will help you generate findings on how much change has occurred as a result of your work and who has experienced change. Quantitative data is objective.

“Data from the OBR shows food and non-alcoholic beverage prices rose 14.5% in the year to September 2022 – the highest rate in more than 40 years.”

The Road Ahead 2023: The ongoing impact of cost of living, NCVO

Start by making sure your data is in a format you can analyse. If you have paper forms or questionnaires, you will need to type these into a spreadsheet or database.

Microsoft Excel (MS Excel) or a similar software is good enough for most things you will need. Some survey tools such as SurveyMonkey and Google Forms export easily into MS Excel or CSV (Comma-separated values) formats.

Next, clean your data. You need to remove any:

• completely blank responses
• duplicates
• obvious errors – for example, someone ticking two boxes when they were asked to tick one.

You should also make sure that each of these variables is in the right number format, especially if you are using Excel or statistical software. Make sure that dates are formatted as dates, numbers as numbers, amounts of money as currency and so on.

You now have your data set.

Statistics help to organise and understand numerical data so you can present it clearly.

If you have developed an evaluation framework, it will help you decide which statistics to use.

For example, you may want to report on the proportion of people who have experienced an outcome (percentage) or the type of people who have benefitted most and least from your work (cross-tabulation).

There are two categories of statistics in data analysis:

• Descriptive statistics help to illustrate and summarise data - they describe the data
• Inferential statistics help to understand connections between variables, decide whether something could have happened by chance, or make reasonable guesses about a larger population based on the sample data we have.

We concentrate on descriptive data here.

Frequencies and percentages

Frequency is how often something happens. You might use a frequency table to demonstrate how often your services have been accessed or how many campaigning activities you have delivered.

You can often present frequency clearly by expressing it as a percentage of a total.

Here is a simple frequency table showing attendance at training courses, with numbers and percentages.

Which of the following courses did you attend?

Tips for using percentages

• Don’t use percentages when presenting data from small samples as it is easy to be misleading. As a rule of thumb, avoid percentages for samples of fewer than 50. You can use them for samples of 50-100 but don’t draw firm conclusions based on small differences in your data.
• Make sure you refer to the correct number of respondents when calculating percentages. For example, if you’re analysing data from a survey, use the number of people who have responded to a specific question to calculate percentages in the data for that question and not the number of people responding to the whole survey.

Measure how much change there has been

If you have asked the same questions before and after your intervention, you can compare responses to find out how much change individuals or organisations have experienced. To do this:

1. For each respondent, subtract their ‘before’ score from their ‘after’ score.
2. Next, put data from all your respondents into a frequency table.
3. You can then work out the average change for your whole group or for sub-groups, or what percentage of respondents experienced positive or negative change.

Compare who has experienced change

Cross-tabulation is a way of comparing results for different types of people or organisations you have worked with. For example, if you want to know if your intervention is more effective for people who are unemployed or in employment, you could use cross-tabulation to compare their experiences.

You could also use it to compare how people rated different interventions or different aspects of an intervention. You can do this using pivot tables in MS Excel .

The following example of cross-tabulation compares how people in different roles rated a course.

How would you rate the course?

From this, you can see that frontline workers rated the course more positively than managers.

Averages

Averages are used to give a summary of a whole data set in a single number. This number represents the middle of the distribution (the spread of the data).

They can be used to report on the average experience of the individuals or organisations you have worked with.

There are three main types of average: mean, mode and median.

Mean

The mean is what we normally think of when we say ‘average’.

This is the most useful average to use if you have used a rating scale such as 1-5 or 1-10. The mean can be used to understand levels of wellbeing or confidence, for example.

To find the mean, add up all the values and divide by the number of responses.

If we have a string of numbers:

1, 1, 1, 1, 1, 2, 2, 3, 4, 4, 4, 5, 5, 5, 5

we add all the numbers and get 44. There are 15 values so we divide 44 by 15, which is 2.9. So the mean is 2.9.

Mean = the sum of values divided by the count of values

The mean is less helpful if your data is skewed (if the top or bottom values have a higher frequency - occur more often - than the middle value).

It’s particularly unhelpful if your data has outliers (values far above or below the bulk of values in the data set). Imagine what would happen to the mean if our list of numbers was 1, 1, 1, 1, 1, 2, 2, 3, 4, 4, 4, 5, 5, 5, 670.

Median

The median is the value in the middle of a data set arranged from smallest to largest. It may be more helpful than the mean if your data is skewed.

The median is commonly used when reporting income or wealth as the data tends to be highly skewed, with a few very high salaries at the top.

If we use the string of numbers above, the median is 3.

1, 1, 1, 1, 1, 2, 2, 3, 4, 4, 4, 5, 5, 5, 5

Even if the top values are massive outliers, the median is still 3. If you have an even number of values, then you take the two middle values and divide them by two.

If your data is skewed, you may also want to report quartiles or percentiles. For the bottom quartile, you use the value that is one-quarter of the way through the data set, arranged from smallest to largest.

For the top quartile, you use the value three- quarters of the way through. If you are using MS Excel, you can use the quartile function to calculate quartiles.

Mode

The mode is the value in a data set that occurs most frequently - it’s useful to report on what most individuals or organisations you worked with have experienced.

So, in the series of numbers we have been using, the mode would be 1, as this value occurs five times.

1, 1, 1, 1, 1, 2, 2, 3, 4, 4, 4, 5, 5, 5, 5

5 2 1 3 4

The main disadvantage of using the mode is that there might be two modes in the same data set.

Measures of dispersion

These are single numbers that tell you how much variation there is in your data set. They are usually used alongside an average to give a summary of the data set.

Range

The range is simply the difference between the smallest and largest value in your data set. You subtract the smallest number from the largest to get the range. If we use our data series above, the range is 4:

5 - 1 = 4

You can also use an interquartile range to tell you about the distribution of the middle 50% of values in your data set. This is where you subtract the bottom quartile from the top quartile.

Standard deviation

The standard deviation tells you the average distance between each value in the data set and the mean value. It shows how well the mean represents a data set. The higher the standard deviation, the more dispersed the data set is.

In our example above, the standard deviation is 1.7.

Excel and statistical software packages will calculate the standard deviation for you.

Present your quantitative data clearly to make it easier to understand.

Try to combine categories

For some data sets, you can combine categories for simplicity. For example, if you have used an agree/disagree rating scale, you may want to combine ‘strongly agree’ and ‘agree’ into a single category unless this would lose important detail.

Consider tables and charts

Presenting your data in a table or chart emphasises its importance, so use tables or charts for the data that’s most important for people using your evaluation to understand.

Tables and charts also help readers understand complex findings.

It’s important to choose the right chart for your data. Here are some initial suggestions.

Ann K Emmery has a very useful chart choosing tool to help you select the right tool to display your data.

Be clear and transparent

Always report your sample base. This is the number of respondents that answered a particular question or the number of people in your sample (sometimes called n).

Whenever you report a percentage, it’s good practice to report the sample base. This helps the reader understand how many people you're talking about.

For example, ‘80% of participants who completed the survey (n=250) said that they were more confident after the training course’.

Report any limitations. If your sample is small or biased in any way, or if you weren’t able to reach particular target groups, it’s important to report this. It's a strength of your analysis, rather than a weakness.

Once you have decided what statistics to use and have done the calculations, look again at your data to draw out key findings - don’t assume the data speaks for itself! Here are some things to consider:

• Is 80% (for example) good or bad? How do you know? You may be able to decide on this by comparing your data to the previous year’s data, or to other similar interventions.
• Are there any other patterns, themes or trends? For example, does one group consistently achieve more, or less, than other groups? Is the number of people accessing events increasing?
• Can you explain some of the less common responses? You may need some qualitative analysis to help you here.
• Is there anything in the data that has surprised you?
• Do you know anything about why some of the results are as they are? For example, can you link your percentages to qualitative data that explains why some people achieved an outcome while others did not?

Now you’re ready to bring together your data analysis into a report or other presentation format. Read our guidance on writing an evaluation report.