of the Intervention
It is highly likely that an intervention will have some outcomes that were not anticipated at the planning stage. These unintended consequences can be positive or negative and an evaluation should always try to find them out.
Unexpected benefits, once known about, can be designed into future interventions and included in future evaluation plans. They may also be of interest to other agencies.
Example A: Unintended Benefit
A Local Authority initiative offered returning University students the opportunity to have free refresher driving lessons with Approved Driving Instructor's (ADI’s). They targeted parents of students with the offer, and students did take part. The unexpected benefit was that as a result of this scheme, some of the parents took the initiative to organise refresher lessons for themselves. Thus, the benefit extended beyond the group intended to be reached.
This finding informed how the intervention was targeted the next time it was repeated.
Negative unintended consequences resulting from an intervention could, potentially, outweigh the intended benefits. Even if they do not outweigh the benefits, unintended consequences need to be identified and reported, so that they can be minimised, if not avoided.
Example B: Potential Negative Consequence
A Local Authority initiative provided resources to ADI's, learner drivers, and supervising drivers. The resources were intended to increase and improve private driving practice for learners; in combination with paid lessons. Some ADI’s thought that the provision of the resources to parents resulted in learners booking fewer professional lessons, because they thought they could replace, rather than supplement, the professional lessons with private practice. Although no hard evidence of this was found in the evaluation, knowing that this was a concern helped the Authority to improve future publicity of the intervention. It also meant that this potential consequence could now be specifically looked for in future evaluations.
Other examples include where the intervention has increased the level of risk on the roads, rather than reducing it:
- Practical and educational pre-driver education courses can lead to an increase in risky attitudes (1) and crash risk (2)
- The use of shock tactics and fear appeals are equally likely, if not even more likely, to create the opposite of their intended effect (3).
Unintended outcomes can occur in the short-term or long-term so if the immediate evaluation findings do not reveal anything unexpected, they should still be looked for in any follow-up studies conducted.
A good time to think about possible positive and negative consequences is during the construction of the logic model (See the Logic Model help page). These consequences could be included in a separate box or section of text. This will remind the person doing the evaluation to look for them in the evaluation study.
References:
- Glendon, A I, et al.(2014). Evaluating a Novice Driver and Pre-driver Road Safety Intervention, Accident Analysis and Prevention, pp. 100-110.
- Roberts, I, Kwan, I and Reviewers, Cochrane Injuries Group Driver Education.(2001) School-based driver education for the prevention of traffic crashes, Vol.3.
- Elliot, B J.(2003) The Psychology of Fear Appeals Re-visited.