Best practice for an evaluation

Looking in more detail at the most effective way to carry out your evaluation.

Developing an effective evaluation

Any evaluation needs to relate to your Intervention aims and objectives, which should be clearly set out from the start. 

It’s also important to be aware of your Intervention outcomes so that you know what needs to be measured.

Choosing the right type of evaluation

There are two main types of evaluation you can use:

Impact evaluation

This is where you quantify the impact of an intervention, or policy change, on the outcomes of interest. This is the most common type of evaluation.

Process evaluation

This aims to understand the ‘how’ and ‘why’ of the intervention’s impact – ultimately evaluating how the content has been received.

For example, whether students have taken away the intended messages, or not. This is often carried out along with an impact evaluation.

Getting the evaluation design right

There are various designs for running evaluations. Generally, the best approach will depend on three key elements:

  • The intervention design
  • Delivery of the message
  • Participant sample

What’s really important is that you think about the evaluation when the intervention is being designed, not after it’s been delivered.

There are two important factors you need to bear in mind from the start of the evaluation process:

Establishing a baseline

You need to know the attitudes or behaviours before the intervention’s delivered. If you don’t do this, you won’t know whether anything’s changed.
This means you need to collect data that can be used as a baseline. Then you can work out if your intervention has been successful.

Using a control or comparison group

It’s a good idea to collect data from a group who’ve received the intervention, and a similar group who haven’t. This lets you see what would’ve happened if the intervention hadn’t been carried out. And, it means you can properly understand the effect of the intervention.

The ‘gold standard’ is thought to be Randomised Control Trials (RCTs), which are often used in public health and increasingly in Road Safety when appropriate.

Randomised Control Trials

  • An RCT can help to identify the causal impact of an intervention on a measured outcome
  • In these trials, participants are randomly allocated to either:
    • The treatment group (intervention), or
    • Control group (no intervention)
  • It’s random to make sure that the two groups are the same, apart from the intervention
    • This also helps to balance natural variance in the sample (if it’s large enough)
  • If small samples are being tested, it’s better to match them on key demographics instead
    • This is known as a quasi-experimental design, which is common in road safety
Back to top