{"id":42,"date":"2013-08-05T16:56:59","date_gmt":"2013-08-05T15:56:59","guid":{"rendered":"http:\/\/blogs.iriss.org.uk\/evidenceandpractice\/?p=42"},"modified":"2013-08-05T17:07:12","modified_gmt":"2013-08-05T16:07:12","slug":"contribution-analysis","status":"publish","type":"post","link":"https:\/\/blogs.iriss.org.uk\/evidenceandpractice\/2013\/08\/05\/contribution-analysis\/","title":{"rendered":"Contribution Analysis"},"content":{"rendered":"<p><strong>CONTRIBUTION ANALYSIS <\/strong><\/p>\n<p>I recently attended a session on contribution analysis. I was on a fact-finding mission. At the end, I came away thinking it wasn\u2019t anything new and what was involved didn\u2019t include anything I hadn\u2019t done before as a researcher or evaluator. It just wasn\u2019t called contribution analysis!<\/p>\n<p>On further reflection, however, I saw that it might offer certain benefits \u2013 but before going any further\u2026.<\/p>\n<p><strong>What is contribution analysis?<\/strong><\/p>\n<ul>\n<li>It was \u2018invented\u2019 by John Mayne, at the end of the 1990s when he was working for the Office of the Auditor General in Canada. He\u2019s since become an independent advisor on public sector performance.<\/li>\n<\/ul>\n<ul>\n<li>It works towards or back from evidencing the impact of a project\/ programme or policy\u2013 and fits nicely with the shift to focusing on impact (or \u2018outcomes\u2019) as we move away from an evaluation culture where we list all the things we have done or produced! If you are smart\u00a0 &#8211; and depending on what your project outcomes are &#8211; you should be able to link your outcomes to higher strategic ones eg those in Scotland\u2019s national performance framework. \u00a0Policymakers will love you?!<\/li>\n<\/ul>\n<ul>\n<li>It acknowledges right up front that your project\/programme or policy is only part of a more complicated landscape\u2013 with multiple partners, different initiatives targeting the same groups, and spheres of direct and indirect influence all playing a part. You can also use it to explain why your project has, for example, limited impact due to other factors out with your control and postulate alternative explanations of change. In short, it doesn\u2019t ignore the rest of the world.<\/li>\n<\/ul>\n<ul>\n<li>Importantly, Mayne recognised the limitations we work under- that you can\u2019t always <em>prove <\/em>causality by testing cause in effect in controlled conditions in the positivist tradition\u2013 however, YOU CAN provide evidence and reduce uncertainty about the difference you are making.<\/li>\n<\/ul>\n<ul>\n<li>Where projects\/programmes or policies <em>don\u2019t <\/em>have clear outcomes or objectives, targets or timelines, it can help re-dress this \u2013 by making them explicit and thereby possible to evaluate.<\/li>\n<\/ul>\n<p><strong>So how do you do it?<\/strong><\/p>\n<p><strong><\/strong>Mayne identified a number of steps &#8211; which seem to have been slightly amended or re interpreted by a range of people \u2013 but essentially these are:<\/p>\n<p>1. Set out the attribution problem to be addressed \u00a0&#8211; this could either be: identify the outcome you hope to achieve OR ask if you project\/ programme\/policy has achieved your stated outcome?<\/p>\n<p>2. Develop a theory of change or logic model \u00a0&#8211; this involves developing a results chain to make explicit the theory of change upon which the project, programme or policy is based eg. if I invest time, money and human resources in delivering information, mentoring schemes, role models and taster events targeted at pupils from schools where few go onto higher education (HE), I can inform pupils and their influencers about the benefits of higher education, raise their aspirations, motivate them to achieve better exam results at school and successfully apply to university \u2013 thereby widening access to higher education (the ultimate outcome)!<\/p>\n<p>Results chains can get complicated \u2013 with multiple chains or \u2018nested logic models\u2019 -but that is for another day.<\/p>\n<p>This should also involve identifying the risks and assumptions you have made eg that your funding will continue and policymakers believe widening access to HE is a good thing; that you can keep and retain good staff; that your activities are well-received and pupils (and parents) react as you expect; that your investment is significant enough to sustain positive change in behaviour; that changes outside your sphere of influence such as charging or raising fees for courses or a reduction in university places doesn\u2019t happen. Agreeing risks and assumptions will be an important part in assessing whether implementation has occurred as expected \u2013 with lessons to be learnt if assumptions are incorrect. You might also consider ways of reducing risks.<\/p>\n<p>It\u2019s been said that contribution analysis is good for confirming or reviewing a recognised theory of change \u2013 but not so good at creating or uncovering one! If so, does this mean it\u2019s not suitable for trial or experimental projects?<\/p>\n<p>3. Gather evidence to substantiate your theory of change or logic model \u2013 using data you have been gathering whether statistical or qualitative. You can also draw on others\u2019 evidence eg to back up your assumptions around how people react to activities similar to your own \u2013 perhaps in the same or slightly different environments?<\/p>\n<p>4. Assemble a contribution or performance story based on the evidence \u2013 this should assess where the evidence is strong\/credible\/weak or contested.<\/p>\n<p>5. Seek out additional evidence<\/p>\n<p>6. Revise and strengthen your contribution story using the additional evidence \u2013 your final version should be short as well as plausible (and perhaps with a visual representation of your theory of change). Policymakers seem to like this, preferring stories to reading vast tomes of research.<\/p>\n<p><strong>Different uses<\/strong><\/p>\n<p>The session also revealed that people have used contribution analysis in different ways. Some have used it to simply evaluate projects (often retrospectively) while others have used it as a planning tool to identify with partners the outcomes they want to achieve, agreeing a theory of change and results chain to determine their project before it starts.<\/p>\n<p>In the second case, they also identify key measures for evaluating the different points in the chain, building in a system for \u00a0\u2018quality assurance,\u2019 \u2018performance management\u2019 or \u2018continuous improvement\u2019 to track progress over time and inform future design. \u00a0How \u2018informative\u2019 it is will depend on the type of data collected of course\u2026.\u00a0 Nevertheless, this approach offers the benefit of integrating your planning, monitoring and evaluation data into one set. You can also build your evidence year on year.<\/p>\n<p>Of course, this assumes that you have the luxury of long-term funding as it may take time for you to build your evidence. Having said that, contribution analysis can be useful in that you have identified interim steps \u2013 or short, medium and longer-term goals \u2013 which you can report against rather than trying to demonstrate impact prematurely.<\/p>\n<p><strong>The challenges- of which there are many<\/strong><\/p>\n<ul>\n<li>Contribution analysis doesn\u2019t remove the difficulties in identifying what it is you want to achieve, your theory of change and what indicators you use to measure it. If you\u2019re working with others to plan, it doesn\u2019t make the challenges of partnership working nor the time and commitment involved any less.<\/li>\n<\/ul>\n<ul>\n<li>Sometimes your measures will be constrained and compromised by the data that is available- which may be a weak proxy or ill-fitting indicator!<\/li>\n<\/ul>\n<ul>\n<li>Contribution analysis may lead to you discovering evidence that makes you want to revise your key outcome \u2013 eg you might find out that it\u2019s better for more women to leave their husbands to protect them from domestic abuse, even though your project is funded to keep families together\u2026. This may not sit well you\u2019re your funders!<\/li>\n<\/ul>\n<ul>\n<li>As mentioned, it can take a very long time to generate, gather and build credible evidence to attribute positive change\u2013 which can be frustrating and problematic, especially if the pressure is on to cut projects or activities within these to make savings. Those at the session who had used contribution analysis said they were nowhere near this stage \u2013 and didn\u2019t know of anyone else who was either!!! Linking your impact to financial data to provide an indicator on value for money is yet another challenge!<\/li>\n<\/ul>\n<p><strong>To sum up<\/strong><\/p>\n<p>Contribution analysis is emergent practice, but I like that it provides a theoretical framework and set of methods &#8211; the end results of which are more likely to be accepted as credible by policy makers and funders and which focus on impact. I like that it builds in others\u2019 spheres of influence to your contribution story\u00a0 &#8211; with Steve Montague to be name checked for developing thinking in this area. \u00a0Having said all of that, it doesn\u2019t make the doing any easier!<\/p>\n<p><strong>REFERENCES<\/strong><\/p>\n<p>Mayne J (1999) Addressing attribution through contribution analysis: using performance measures sensibly<\/p>\n<p>Montague S\u00a0 (2000) Three spheres of performance governance spanning the boundaries from single-organisation focus towards a partnership: <a href=\"http:\/\/evi.sagepub.com\/content\/13\/4\/399\">http:\/\/evi.sagepub.com\/content\/13\/4\/399<\/a><\/p>\n<p>Scottish Evaluation Network event, 18 June 2013, Contribution analysis: a half day workshop on the practicalities of using contribution analysis. Contributions from Lisa Cohen (NHS Health Scotland) and Sarah Morton (University of Edinburgh)<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>CONTRIBUTION ANALYSIS I recently attended a session on contribution analysis. I was on a fact-finding mission. At the end, I came away thinking it wasn\u2019t anything new and what was involved didn\u2019t include anything I hadn\u2019t done before as a researcher or evaluator. It just wasn\u2019t called contribution analysis! On further reflection, however, I saw &hellip; <a href=\"https:\/\/blogs.iriss.org.uk\/evidenceandpractice\/2013\/08\/05\/contribution-analysis\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Contribution Analysis&#8221;<\/span><\/a><\/p>\n","protected":false},"author":61,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_coblocks_attr":"","_coblocks_dimensions":"","_coblocks_responsive_height":"","_coblocks_accordion_ie_support":""},"categories":[1],"tags":[7884],"_links":{"self":[{"href":"https:\/\/blogs.iriss.org.uk\/evidenceandpractice\/wp-json\/wp\/v2\/posts\/42"}],"collection":[{"href":"https:\/\/blogs.iriss.org.uk\/evidenceandpractice\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.iriss.org.uk\/evidenceandpractice\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.iriss.org.uk\/evidenceandpractice\/wp-json\/wp\/v2\/users\/61"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.iriss.org.uk\/evidenceandpractice\/wp-json\/wp\/v2\/comments?post=42"}],"version-history":[{"count":0,"href":"https:\/\/blogs.iriss.org.uk\/evidenceandpractice\/wp-json\/wp\/v2\/posts\/42\/revisions"}],"wp:attachment":[{"href":"https:\/\/blogs.iriss.org.uk\/evidenceandpractice\/wp-json\/wp\/v2\/media?parent=42"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.iriss.org.uk\/evidenceandpractice\/wp-json\/wp\/v2\/categories?post=42"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.iriss.org.uk\/evidenceandpractice\/wp-json\/wp\/v2\/tags?post=42"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}