What’s wrong with the Theory of Change?

Evaluation Methodology, Research & Evaluation, Theories of ChangeComments are off for this post.

Image Credit: Sidney Harris

What’s wrong with the Theory of Change?

Geoff Mulgan (CEO of Nesta) recently wrote an interesting blog in which he examines the concept known as the ‘theory of change’ (or theories of change). The full article can be found here. Although the article highlights some positive support towards the concept, in terms of what it means and how it is used, several contentious issues were raised. Below Julie Rasmussen from the Infinitas team provides her response to each of the less favourable points raised in Geoff’s blog.

Firstly, we thought it would be helpful to summarise what the Theory of Change aims to achieve.  Listen to this short video below, before we discuss the problems with the concept.

So what’s the problem with the Theory of Change?

[Geoff]: The problems come with the phrase ‘theory of change’, in which, to put it bluntly, each of the words is somewhat misplaced and doesn’t stand up to much scrutiny. Take the use of the word ‘theory’. A theory is generally taken to mean an idea, principle or law that is separate from, and more general than, the thing being explained. So a theory of change should be general – like the theory claiming that higher income in a country tends to lead to more democracy, or that higher levels of education for girls will tend to lead to a lower birth rate, or that sending criminals into classrooms will dissuade teenagers from becoming criminals themselves. The advocates of theories of change use the word ‘theory’ in an almost opposite sense to describe a specific explanation of a specific example (albeit one that should then have predictive power), but never explain why they do so.

[Julie]: There seems to be some confusion about the use of the word ‘theory’. This confusion may be due to the distinct ways in which various types of theory have different roles and uses. Social (and behavioural) science theories are explanatory and attempt to provide generalisable and verifiable knowledge about the principles that shape social behavior. These are broad concepts about the nature of social problems. In comparison, programme theory (as it is used in evaluation science with the theory of change) focuses on the nature of the program being evaluated. Programme theory is more modest than social science theory. It is expected to guide the evaluation questions, design and deal with the assumptions that guide the way specific programmes or interventions are implemented and expected to bring about change (Donaldson & Lipsey, 2006). A recent paper by Leeuw & Donaldson (2015) presents two typologies of these theories and encourages readers to join with them in a their debate of their pros and cons [Further information about the confusion surrounding the theory in ‘theory of change’ see articles written by Leeuw & Donaldson, 2015 and Donaldson & Lipsey, 2006].

[Geoff]: The next problem is the use of the word ‘theory’ in the singular form, not plural. Any serious explanation of anything in the social world should be suspect if it only uses one theory – e.g. a theory of financial incentives, or peer influences. All successful models are assemblies of multiple elements and theories. I’ve no idea why the advocates chose to do this.

[Julie]: In the literature, the term theory of change is used interchangeably, described as theories of change or theory of change. The evaluation literature on programme theory emphasises that there may be different theories actively influencing a programme at any one time. These may include ‘espoused theory’ (theory that is claimed to be followed but is not in practice), ‘theories-in-use’ (theory that can be inferred from the actions of the actors), someone’s own preferred theory or theories, preferred theory or theories from the research literature or a combination of these (Funnell and Rogers, 2011; Argyris and Schon, 1974). These theories may be articulated in different ways, using 1) a deductive approach 2) an inductive approach 3) mental models or 4) a collaborative approach [See Laing and Todd, 2015 for case studies on how different theories may be combined].

About the concept

[Geoff]: There are problems with the concept. The typical theory of change, as set out in many accounts, risks being misleading in two important ways.

The first criticism is that they tend to be too linear, assuming that inputs lead to outputs, and that outputs lead to outcomes. This sometimes happens. But anyone familiar with systems thinking will be dubious of linear explanations, especially where complex social phenomena like homelessness, poverty or isolation are concerned.

[Julie]: A recent report by the ODI argues that the term Theory of Change has become an extension of the assumptions column of the logframe and there exists confusion around what these different terms mean and how they relate to each other (Valters, 2015). Often theories of change is a term used by organisations to imply that change revolves around them and their programme, rather than around a range of interrelated contextual factors, of which their programme plays a part. We agree with the view put forward by Valters (2015), that there is no reason why these different tools or approaches cannot be used together, with logframes reflecting a blueprint or ‘control-oriented’ approach to project planning and theories of change representing a more process-orientated approach. If developing a theory of change or theories of change is to become useful, it must form part of a critical and evidence-based attempt to unpack the black box of causality between inputs, outputs and outcomes [See review by Valters, 2015, p6].

[Geoff]: The second criticism is that theories of change risk squeezing out space for learning.

[Julie]: Several authors have mentioned that critical reflection, which is vital to double-loop learning, can be dampened by the over-prescription of new approaches, such as the use of theories of change (Allana, 2014; O’Keefe et al., 2014). Difficulties may arise due to the tension between being accountable to donors on the one hand (often seen as a priority in terms of the results-based agenda) and learning from our work. Valters (2015) suggests four principles that are necessary to encourage creativity and learning when using a theory or theories of change approach. The principles are summarised below. It is my opinion that we should add a Principle 5: Embed a Learning Culture into the Organisation. This is vital if we are to encourage people to speak out about their learning experiences (negative as well as positive). Read our previous blog post about how organisational silence is not conducive to a learning culture.

Principle 1: Focus on Process

Conventional programme management tools, such as logic models, tend to ignore ‘process elements’, treating projects as ‘closed, controllable and unchanging systems (Mosse, 1998, p5). Such tools are often seen as formal records to be completed at the start of a project which then sit gathering dust on a shelf. It is the actual process of bringing to the surface and critically appraising the assumptions that is helpful to learning. This process needs to be iterative. As further information emerges that confirms or contradicts the underlying assumptions these need to be reappraised. The theory of change then becomes a living breathing document of ‘evidence’of learning and adjustment, with nothing being erased. The process is similar to action-research, where for example, diaries could be used to help stakeholders reflect and capture internal changes to learning during the life of a programme.

Principle 2: Prioritise Learning

Tension between learning on one hand and accountability for results on the other can act as a barrier to reflection and adaptive learning. According to Pritchett et al. (2013) Theories of Change can be operationalised as a means of ‘structured experiential learning’, by seeking to build ‘learning objectives into the cycle of project design, implementation, completion, and evaluation’. Three questions are posed to ask 1) learning for what purpose 2) learning for whom and 3) what type of learning – single or double loop learning?

Principle 3: Be locally led

Being locally led means that development of the theory of change takes place in a genuine participatory and inclusive manner. The idea is to ground causal assumptions in local realities, giving priority to the community to seek out solutions to contextually identified problems. Being locally led is more likely to result in ownership of a programme, meaning that learning is more likely to take place.

Principle 4: Think compass not map

Some advocates of the theory of change approach focus on developing a ‘roadmap to get you from here to there’ (Center for Theory of Change, 2015). Use of a ‘roadmap’ can give rise to the misleading notion of linearity, as in the use of log frames. Far more useful is the idea of a compass to guide our way through the fog of a complex dynamic system, where emergent risks and unforeseen issues arise as we move along the pathway (Green, 2015). This process is described as being adaptive, reflective, and iterative. It recognising that plans often reflect best guesses about the future (and the past) and will likely shift over time (Hummelbrunner and Jones, 2013).

[Geoff]: I wrote this note because I couldn’t find much debate about what is good or bad about the theory of change approach – another example of how civil society adopts ideas uncritically. Too often ideas just become fashionable and spread thanks to neat branding. It is much better to interrogate, question and then adapt by keeping the good bits and rejecting the bad ones. But where are the places, magazines, journals or websites supporting serious debate on issues of this kind? At least in the English-speaking world there’s a bit of a gap.

[Julie]: There is quite a lot of debate in the international development and the evaluation science literature critically assessing the theory of change approach. Further reading about some of those debates including the articles used to put together our response to Geoff’s blog can be found in the reference list.

Enjoyed reading our blog? Sign up below to receive future posts directly to your inbox!

References

Allana, A. (2014). Navigating Complexity: Adaptive Management in the Northern Karamoja Growth, Health & Governance Program. Ottawa: Engineers Without Borders Canada and Mercy Corps.

Argyris, C., & Schon, D.  (1974) Theory in practice: Increasing professional effectiveness. San Francisco: Jossey Bass.

Center for Theory of Change (2015) ‘What is Theory of Change?’ http://www.theoryofchange.org/ what-is-theory-of-change/

Donaldson, S.I., & Lipsey, M.W. (2006). Roles for theory in evaluation practice. In I. Shaw, J. Greene, & M. Mark (Eds.), Handbook of Evaluation. Thousand Oaks: Sage.

Funnell, S.C. & Rogers, P.J. (2011). Purposeful Program Theory: effective use of Theories of Change and Logic Models. Jossey-Bass, CA.

Green, D. (2015). Where Have We Got to on Theories of Change? Passing Fad or Paradigm Shift? From Poverty to Power Blog, 16 April, http://oxfamblogs.org/fp2p/ where-have-we-got-to-on-theories-of-change-passing- fad-or-paradigm-shift/

Hummelbrunner, R. and Jones, H. (2013). A Guide for Planning and Strategy Development in the Face of Complexity. Background Note. London: ODI.

Laing, K & Todd, L. (2015). Theory-based Methodology: Using theories of change in educational development, research and evaluation. Research Centre for Learning and Teaching, Newcastle University.

Leeuw, F.L. & Donaldson, S.I. (2015). Theory in evaluation: Reducing confusion and encouraging debate. Evaluation, 21(4): 467-480.

Mosse, D. (1998). Process-Orientated Approaches to Development Practice and Social Research, in D. Mosse, J. Farrington and A. Rew (1998) Development as Process, ODI Development Policy Studies. New York: Routledge.

O’Keefe, M., Sidel, J., Marquette, H., Roche, C., Hudson, D. and Dasandi, N. (2014). Using Action Research and Learning for Politically Informed Programming. Research Paper 29. Birmingham: DLP.

Pritchett, L., Samji, S. and Hammer, J. (2012). It’s All About MeE. Working Paper 2012/104. Helsinki: UNU-WIDER.

Valters, C. (2015). Theories of Change. Time for a radical approach to learning in development. ODI, London.

Facebooktwittergoogle_plusredditpinterestlinkedinmail

One thought on “What’s wrong with the Theory of Change?

Comments are closed.

Top