Overview

Contribution analysis is a structured approach to understanding to what extent observed outcomes are a consequence of a particular activity, as opposed to other factors.  It is not intended to prove direct causality.  As we discuss elsewhere, it would be unlikely in the complex social world we work within for any intervention to be the only influence on particular outcomes.  

Contribution analysis focuses on increasing understanding of why the observed results have occurred, the roles played by the activity, and other internal and external factors. By gathering and providing evidence around a line of reasoning (atheory of change), a reasonable conclusion can be drawn.

It is a useful approach for assessing the impact of particular activities when many variables could have an impact on outcomes.  It is also helpful for initiatives that have multiple components, involve multiple partners and stakeholders, has an emergent/evolving focus, and/or is being implemented within a dynamic changing setting.

Selected methods and techniques

There is a clear 6-point step-by-step guide to carrying out contribution analysis:

  1. Set out the attribution problem to be assessed: Has the activity influenced the observed result? Why has the result occurred? What conditions are needed to make this type of activity succeed?
  2. Develop a theory of change / logic model: The theory of change should lead to a plausible association between the activities and the outcomes sought. The theory of change must include the assumptions made, as well as the risks and external influences at play.
  3. Fill out the model with existing data and evidence: The model sets out the intended results (outputs, intermediate and end outcomes), and what evidence is currently available related to the outcomes and programme. The links in the theory of change also need to be assessed within the model.
  4. Assemble and assess the evidence and challenges to it: Does the pattern of results observed match the logic model or theory of change? Where are the main weaknesses in the story?
  5. Seek out additional evidence: Additional evidence is gathered to supplement the initial data. This evidence can include the collection of new data from surveys, field visits, administrative data, focus groups, national statistical data, etc., as well as thesynthesisof evidence from other research and evaluations.
  6. Revise the conclusions and strengthen with additional evidence: Here additional evidence could make it stronger and more plausible or, on the contrary, challenge the original conclusions. This is an excellent step for stakeholders to critically assess the evidence and story.

A reasonable contributional claim can then be made if:

  • There is a reasoned theory of change for the intervention.  That is, the key assumptions behind why the intervention is expected to work make sense, are plausible, and are supported by evidence and/or existing research.
  • The activities of the intervention were implemented as set out in the theory of change.
  • Other influencing factors have been assessed and either shown not to have made a significant contribution or their relative role in contributing to the desired result has been recognised.

Benefits

  • Contribution analysis can help sharpen the planning and implementing phases of an initiative by increasing the need for clearly specified intended theory of change and the outcomes and activities need to meet them.
  • Provides a clear step by step guide to help those involved to assess contribution to outcomes.
  • Useful in situations where the programme has been funded on the basis of a relatively clearly articulated theory of change and where there is little or no scope for varying how the program is implemented.
  • Enables some inference of causality in real-life programme evaluations, or at least reduces uncertainty about the contribution the intervention is making.

Challenges

  • Definitive claims of attribution are difficult to make when an intervention is understood to occur within complex social systems.
  • Different terminology in the literature can lead to some confusion about the approach.
  • As a fairly new approach there are few practical examples of how to approach this type of work and the challenges in it.
butterfly.png

Resources

The Better Evaluation Rainbow Toolkit has a section on contribution analysis and also links to other useful resources.

The Scottish Government has produced a short guide to contribution analysis as a tool to understanding policy impact. 

Catherine-Rose Stocks-Rankin from IRISS has published a review of Contribution analysis and its use in evaluating the PROP (Practitioner Research: Older People) Project.