My first experience was delivering monitoring and evaluation training for voluntary sector organisations, who were delivering crime reduction initiatives in London. This was not communications specific and we taught the Weavers Triangle technique which focused on the basics of aims, outcomes and outputs.
During my diploma (which had a private sector marketing emphasis), I learnt about the importance of measuring return on investment (ROI). Something new (and initially challenging!) to me as a public sector comms person.
I have attended training courses on monitoring and evaluation and developed systems to use with my team at Lambeth Council. This ensured consistency of approach across the team with clear targets that were linked to organisational priorities.
The main challenges were measuring outcomes where data, e.g. employment, health or crime statistics and benchmarks were not always immediately available. The long-term benefits of community safety initiatives tackling fear of crime or gangs, employability or apprenticeships schemes, were all examples of service outcomes that were longer-term and more challenging to measure.
Measuring recycling rates, voter registration and online signups were easier to work with, as the data was available within a short timeframe. It was important to continually improve and refine the approach and stay up to date with new approaches.
Which framework should I use?
Monitoring and evaluation of communications delivery is a well documented area of work. There is lots of advice and a variety of models available.
While the Barcelona Principles were intended to provide a foundation for PR programs, the updated Principles recognise that they can also be applied to the larger communication function of any organisation, government, company or brand globally. In fact, measurement, evaluation and goal-setting should be holistic across media and paid, earned, owned and shared channels.
- Chartered Institute of Public Relations (CIPR) principles
This is a key issue for the CIPR as thought leaders for the industry. This overview document provides non-CIPR members with a short overview of the research, planning and measurement process and includes information on measuring social media, with a glossary of social media terms.
The following statement of best practice in measurement and evaluation is useful when thinking about monitoring and evaluation, with particular reference to award entries. There is a strong move away from Advertising Value Equivalency (AVE) measurement, something which has been debated over a number of years, as summarised in PR Week’s article: ‘The AVE debate: Measuring the value of PR’.
- Government Communication Service (GCS) Evaluation Framework
The GCS Evaluation Framework is designed to help standardise the set of evaluation measures we collect and report on for each type of communications activity and for the range of communication objectives we set out to achieve. The use of consistent data and reporting means that we can draw more meaningful conclusions about our work. It can also assist with benchmarking and target setting.
The emphasis is on measuring, understanding and responding to the following:
Inputs: What you do before and during the activity
Outputs: What is delivered/target audience reached
Outtakes: What the target audience think, feel or do to make a decision
Outcomes: The result of your activity on the target audience
Organisational Impact: The quantifiable impact on the organisation goals/ Key Performance Indicators (KPI’s)
You can find a number of related resources which I have put together.
How should I approach this?
With so many approaches on offer it can be daunting, but my advice is to first understand your organisations’ goals and how they translate into communications targets. Then review how previous monitoring and evaluation work has been undertaken, best practice and benchmarking in your sector before choosing the model which is the best fit for your needs. Good luck!