Evaluation and Learning
The Life Changes Trust is committed to evaluation being integrated into the everyday work of organisations that it funds. We believe that evaluative activity, when well planned at the beginning of projects, can contribute to a positive cycle of learning and improvement. This can be both within organisations and between organisations where learning is shared (see the section describing the Trust's approach to Learning from Each Other).
This integrated approach to evaluation benefits those we work with by allowing learning to be acted upon quickly. The Trust therefore encourages its fundees to adopt a ‘formative’ self-evaluation approach. This is described further in the Evaluation Design section, under Types of Evaluation.
We feel that those we fund are usually best placed to do this evaluative work. However, we recognise that in some circumstances it is appropriate to commission an external evaluator. The use of an external evaluator must still complement an internal evaluative culture that learns and makes changes as good information becomes available.
The Evaluation Pathway
Evaluation is best thought of a routine cycle of activity that forms a fundamental part of how your project is designed delivered and continuously improved. Evaluation Support Scotland’s Evaluation Pathway is consistent with this view.
Evaluation supports gathering evidence to measure the value and quality of your project, activity or services. It’s a way of collecting evidence and analysing it so that you understand and can change in response to new information. It is also used to demonstrate to others whether your project met or exceeded expectations.
Learning on the Go
In evaluating your work, you will make judgements about the quality of the process and outcomes of that work. Asking questions about quality as part of a continuous self-evaluation process is valuable and helps to promote learning and improvement.
You should create chances to stop, take time and space to recognise progress, ask questions, raise doubts, and retrace steps in order to learn to know what to do differently to reproduce good practice and produce ‘better practice’.
It is important that you understand how your project has worked, as well as whether or not it has achieved intended or other outcomes. You can ask a structured set of questions (Western Michigan University have some useful checklists) to help to develop an understanding of what approaches, processes and factors have helped or hindered your work.
The table below provides a framework of key questions to challenge and stimulate this reflective process through a ‘Learning on the Go’ process of continuous improvement, which can be adapted to suit different organisational purposes and beneficiary needs.
Learning on the Go Framework
- Have you gained any new insights into yourself or your organisation? What are these insights? What evidence led you to them?
- Are you gaining better understanding of the views, opinions and actions of others? What are some new understandings you have gained? What evidence led you to them?
- Have you identified any areas in need of new actions or approaches? What new actions and behaviours do we need in order to produce change?
- What relevant stakeholders need to be included in conversations regarding these changes? What kind of evidence will you gather to find out if these new changes are working?
- Does your data show that you are moving towards your original aims? If so, what is working well? If not, what are the barriers to achieving those aims?
- Are there outcomes that you did not anticipate? If so, what are they?
- Are there any gaps in evidence or voices missing? Are the voices of all relevant people being honoured in a fair and balanced way?
- Have differences of opinion arisen regarding activities, data collection, ethics, or interpretation? If so, how have those differences of opinion been dealt with?
- What would enhance the quality of your evaluation at this stage?
The Life Changes Trust has also created a template, available for download here.
You can use these questions to:
- Check in and review progress from time to time, for example monthly or quarterly. This might be at team meetings or dedicated reflection sessions.
- Record evidence and learning as it emerges so it is not lost and can influence the next stages of the process.
- Notice gaps in evidence and where you may need to focus your evidence gathering efforts.
- Record and share insights and ‘ah-ha’ moments.
- Share thoughts about what is working and what needs to change to make it better.
- Check-out what others are thinking.
- Talk about your approach and progress in more depth – What helps? What gets in the way? What seems to be important or distinctive about your project?
- Talk about your desired outcomes – are you on the right path? How are you doing?
- Decide whether you need to collect different kinds of evidence, better quality evidence or include new stakeholders in the process.