Learning Through Evaluation
If you are considering undertaking any kind of communication activity it’s important that you put in place some kind of mechanism that will tell you whether your communication was a success or not. “Evaluation” is the process of collecting and analysing the kind of information that tells you about the effectiveness and impact of your communications.
All too often, people don’t consider evaluation when undertaking communication. Which makes it difficult to know how successful your activity was or if you met your set objectives. Evaluation of your communications activity can be valuable for a number of reasons:
- Evaluating of the activity will allow you to assess how well it did in relation to the objectives that you set at the outset of your campaign.
- It also allows you to set benchmarks for any future campaigns that you undertake. e.g. The last time we did this activity we achieved X responses, so we have an idea of how many responses we may get this time.
- Evaluation of a previous campaign should be used to inform future campaigns. If you ask the right questions then you know what worked well and any improvements that could be made so that your next campaign is more effective.
- Evaluation can also be a very important tool when it comes to securing funding for future activity. If you do it well, you will be able to demonstrate the effect that your campaign had on the target audience. By being able to demonstrate the value of your activity, you will have a certain amount of leverage the next time you want to undertake any similar activity.
Planning Evaluation
As with most things – planning how you intend to evaluate your communications in advance will mean that you get the most out of it. Here are some things that you need to think about:
- Aims and objectives should be set and qualified before you undertake any activity. If they are measurable then evaluation of success is possible
- If similar campaigns have been done in the past, find out how they went so that you have some results to compare them against
- Make sure you have some form of baseline data before you start your activity – you need to know where you are starting from so that you can assess how much you’ve achieved
- Make sure you have a clear idea of the type and level of information that is needed to evaluate your campaign in the most effective way
- Be clear how you intend to gather the data
- Set aside an amount of your budget for evaluation
- Know how you are going to report the information that you gather from your evaluation
Pre-communication evaluation
In developing a piece of printed communication such as a leaflet or a poster to meet a specific aim, you are investing time and money. If at all possible, pre-evaluation is a useful stage to build into the project. This means showing the work to and assessing the reactions of a selected cross-section of your intended audience. The communication can then be modified in accordance with reactions to ensure it achieves it’s objectives – or has a better chance of doing so. This needn’t be an expensive or necessarily wide-reaching exercise. You could maybe tap into a local group or use relevant partners to assess the work in question.
Quantitative or Qualitative
The type of evaluation that you do is also dependent on the type of activity that you have undertaken.
Although quantitative evaluation is generally easier to manage, it can only be used when there are tangible outputs from a piece of communications. For instance, if the call to action asks the respondent to access a website or call a telephone line, the hits to the website can be measured, or contacts with the phone line can be monitored, but the motivation behind doing these things is less easily assessed.
For communications with no perceivable call to action, ones that encompass encouragement, attitude change or information, the impact is less easily monitored as there is no response mechanism. This may require qualitative research to really assess in depth the success or other wise of a campaign.
Monitoring
This is a quantitative method that involves routinely collecting information to check on actual progress. It can be applied to situations such as phone-lines, drop in centres, or attendance at events or groups for example. You can use the ongoing results to modify or change your activity as you go.
Qualitative Methods
Qualitative research tends to take more time and money to organise, but it can prove extremely useful in assessing some of the less tangible impacts of your communications. Using qualitative methods allows you to learn more about how people feel and respond to your communications, not just to measure tangible responses.
Informal evaluation using focus groups from certain audience areas may be the best way to achieve this, or more formal research through a research company will produce measureable, if more costly results. The sort of questions that can be asked and issues addressed at these focus groups/interviews are as follows:
- Did you see X communication?
- Where did you see it?
- What was your initial response?
- What did you think about seeing it?
- What did it make you think?
- Did the communication make you change your opinions or attitudes?
- Did the communication make you consider your own behaviours?
- Did you take any action as a result of this communication?
- What were the positives about this piece?
If you were to commission a professional research agency to do this, they would be responsible for recruitment of participants, development of research questions and topic guides and provision of stimulus material. You would drive all this, but they would undertake the legwork. In reality, such projects cost thousands and a more cost effective alternative would be to access informal research groups.