Costs of the Evaluation

Evaluation is a valuable resource because it helps to ensure that road safety education; training and promotional projects provide value for money and are effective in achieving their aims and objectives. However, evaluation will always incur some financial cost.

It is considered good practice in the UK to allocate up to 10% of the total programme budget for evaluation. In the US, this figure has been known to rise to 20%. Therefore, a budget for evaluation should be set when planning interventions.

Organisations recommending a total evaluation budget of around 10% include:

• The Department for Transport ¹
• The Drug Education and Prevention Information Service (DEPIS) ²
• The World Health Organisation ³
• The Big Lottery Fund ⁴

The greatest costs are likely to be staff time, and/or external evaluation consultant fees. Other costs may include travel, printing and photocopying, computer software/technical equipment, participant compensation payments, and postage.

The size of this budget will depend upon a number of factors:

1. The intervention being evaluated

The scale and type of intervention that is to be evaluated will determine how costly the evaluation will be. The more people that the intervention involves then the larger the number of people that will need to be included in the evaluation.

Additionally if those taking part in the intervention are a difficult group to access then this can make it more time consuming and therefore more costly to evaluate. Publicity campaigns are a good example of this. When handing out a flyer or distributing a video clip online you will not be sure who has seen it, and therefore, will find it difficult to access that audience to take part in the evaluation.

2. The evaluation design

Some types of evaluation are more complex to implement and so tend to take more time and expertise to deliver. Experimental designs (people are randomly allocated to a group who receive the intervention or those who don't) are the strongest and most robust types of evaluation. As a result, however, they are likely to be the most expensive to run, especially as it is almost certain that external expertise will be required. Those which are the least robust (such as non-experimental designs, which only analyse those who have received the intervention) are likely to cost the least.

Evaluation is always a balancing act – try to conduct the strongest evaluation you can with the resources available. Quasi-experiments (allocation to a group who receive the intervention or not, but not randomly allocated) can be a good compromise but at the very least try to collect some data on your audience before the intervention takes place so you can compare this with their answers afterwards.

3. Methods chosen for data collection

How you collect your data, whether it be questionnaires, focus groups, observations and so forth, will also come with cost implications. Here are some tips that could help to keep some of those costs down:

• Telephone interviews can be a cheaper alternative to focus groups – there are no travel or room-hire costs, which can become quite expensive.
• Online surveys can be cheaper than postal surveys – there are no printing or postage costs. If you do not have an email list of who you would like to complete the survey, however, then it can become expensive to pay a company to send the survey to their list of contacts.
• If you think you will need an incentive then prize draws can be a cheaper alternative to paying each respondent who takes part.

4. Internal staff costs

Although internal staff costs may seem 'free', they are not. Without having an estimate of the staff costs involved then you could be seriously underestimating the cost of the evaluation. Try to estimate a day rate for the staff conducting the evaluation. You may already have a day rate that you use. If not you can create an estimate by calculating the following for each member of staff:

5. External evaluation consultants

You may decide to get an external evaluation consultant to conduct all or some of the evaluation for you. External evaluators will have the expertise to ensure the evaluation is as strong and robust as the budget will allow. Adding an external cost to the evaluation may seem off-putting, however, it is likely that a consultant will be able to conduct the evaluation much more efficiently than staff in-house and so may actually offer good value for money.

References

1. Guidelines for Evaluating Road Safety Education Interventions (2004:Pg.7).
2. Evaluation Guidance for Practitioners: A practical guide to evaluating drug education and prevention services for young people (2005: Pg. 21)
3. Checklist for developing a monitoring and evaluation plan for malaria control (2008: Pg.10)
4. Research and Learning Strategy: 2009-2015 (Pg.7)