“Everything we do, we should evaluate. Evaluation is everything.”
The fifth and final meeting of Evaluation Exchange was held on 21st March at Project Ability’s project space at 103 Trongate, hoping for inspiration from the artistic space!
We began by reviewing everyone’s progress over the life of the project. Nearly everyone had moved forward. Most had at least devised a plan and some were already conducting focus groups, arranging interviews and designing questionnaires. Even for those who had not seen much concrete progress, there was a real perception that involvement in Evaluation Exchange had increased their own and their organisation’s focus on the importance of evaluation.
“There’s no doubt about it, if I had not come to this we wouldn’t have got anywhere. People are talking about it. People are aware of how important it is.”
There were also plans to further emphasise the importance of evaluation internally with one group member planning to showcase their evaluation work at an upcoming conference and another seeing the techniques and knowledge gained at Evaluation Exchange used across their organisation to achieve consistent application of outcomes.
After this general review, we split into small groups to discuss specific concerns and receive feedback about our individual evaluations.
The second half of the session was designed to reflect on the success of Evaluation Exchange and what lessons we had learned and could share about the process.
Supporting peer support
Based on discussions throughout the project, Emma had written some lessons for supporting peer support. We had seven ‘headlines’, which included:
- The knowledge really is in the room
- Dis-own it
- Keep it concrete
- Build trust and confidence
- Don’t be the middleman
- Stay regular and clear on commitment
- Chill out
After discussion about whether this was an accurate and complete reflection of what had helped the group to work, we added:
- Set it up
- Use processes that work
When the group was happy with the headlines we moved on to prioritise which themes were most important. We used a diamond ranking exercise, where categories are rated in order of importance using a diamond template, resulting in one single most important category, two categories in second place, three in third, two in fourth and one in fifth (the least important) – see picture below.
Diamond ranking is also a useful way of ensuring that all views in the room are included and the final ordering is reached by consensus. This works by splitting into pairs (although with more time or where there is likely to be significant disagreement you can start with individuals) and each pair prioritising the categories. The pairs then merge into two groups of four and negotiatea new order based on their original rankings, then the groups of four come together to negotiate the (for our group) final order.
The final part of this session was for the group to rate how well Evaluation Exchange had performed on each of the elements we had identified as being important to a successful peer support group. We did this by using a traffic light system, where green meant Great, amber meant OK and red meant Not So Good. As you can see from the picture, the project did pretty well!
Self-evaluating… ourselves!
We also wanted to look at how well Evaluation Exchange had worked in areas other than peer support. Specifically, we wondered what participants had learned from the experience, whether it had met their expectations and whether it had had any wider impact on their organisation. To look at this explicitly, we asked the members of the group to interview each other using three questions. These were:
- What has been the key learning point for you from being part of Evaluation Exchange?
- Has Evaluation Exchange met your initial expectations?
- Have you made an impact with your organisation because of your participation in Evaluation Exchange?
The answers overall were very positive, showing that Evaluation Exchange had improved participants’ perceptions of evaluation, had increased their knowledge, and helped them to progress with their evaluations. It had also, in some cases, been a catalyst for improved knowledge and prioritisation of evaluation at an organisational level. These quotes give a flavour about what the group thought of being involved.
“[Evaluation Exchange] has led to focus and a clear plan. It’s been beneficial meeting members of other organisations. Maybe surpassed expectations.”
“I learnt a lot from this group that I wouldn’t necessarily have got from a training session.”
“Creativity and fun. I’m now sure that is my approach. […] I used to think of evaluation as a black hole, but now think it can be fun.”
“Feel that there is recognition by the Council and Capability Scotland because of the association with IRISS and ESS.”
What now?
So you can see from this brief overview that the project has been, by all accounts, an all round success! We are thrilled about how well it went and excited about looking in more detail at what went well, what the group has learned and what we can share.
Watch this space for further reflections of what we’ve learnt from the process, our own evaluation of Evaluation Exchange, and a tool to help people wanting to set up, run or participate in a peer support network.