The Use of Theories of Change in Media and Governance Programmes

"[D]ifferent Theories of Change for different project activities would make projects conceptually better, easier to plan and to monitor..., and ultimately much more suitable to learn from via evaluation studies."

Inspired through a collaboration between the Swiss Development Cooperation (SDC) unit on democratisation, decentralisation, and local governance (DDLG) and the Institute of Development Studies (IDS) at the University of Sussex, this paper presents an analysis of the theories of change used by selected organisations working in media support in post-conflict countries: Fondation Hirondelle (FH), Deutsche Welle Akademie (DWA), Free Press Unlimited (FPU), and BBC Media Action. In addition, it places these theories of change in the context of insights gleaned from academic empirical research on media effects in the field of governance. The purpose of the review, which was carried out by researchers from the Institute of Applied Media Studies (IAM) of Zurich University of Applied Sciences (ZHAW), is to support donors in their decision-making on strategies for use in future programmes on governance with media components. The results could also help to clarify how media support projects work, how they aim to achieve their objectives on various levels, and how to develop meaningful indicators and conduct insightful evaluations in the future.

A theory of change is a means to evaluate complex interventions by making stakeholders more aware of their (hidden) assumptions and making it easier to monitor the intermediate steps that lead to achieving long-term goals. SDC differentiates 3 types of theory of change:

• Narrative model: an impact hypothesis that describes in one or two sentences how the specific intervention being planned is intended to work;
• Overview model: a visual product, often a causal pathways diagram, depicting the proposed impact pathway or chain of influences; and
• Causal model: a detailed description of the causal chain to show how change happens.

The 3 models can be interpreted as complementary, with the narrative model as the overarching one, followed by the overview model, with some more details on achievements at different levels or with partners, and then the causal model, which elaborates on full details in order to specify all interlocking elements.

The review refers to the SDC's guidelines on media support, which are based on a model that describes various layers of the media sector that all have an influence on the final product of media content that is delivered to and used (or not used) by the audience. (Here, the audience is understood to include ordinary citizens, the government and other state institutions, political parties, economic actors, and an organised civil society. Audiences are understood to be active, not passive, consumers.) The layers include: journalists; media organisations; media institutions; media economics; media law, media politics, regulations, and reality; and societal beliefs and values. The media sector model also integrates new developments in digitisation; for example, online newspapers are by definition media organisations, even if they do not have a print version.

The review is based on project documents (evaluation studies, project proposals, and descriptions) from development cooperation donors and implementers from 2011 to 2018, on the topic of media/journalism and governance. Documents about programmes working with the "communication for social change" approach were excluded on purpose, because they usually do not work with news journalism.

In brief, within the collection of SDC programmes in the realm of media and governance, there were 3 types of programmes:

1. Media as priority: programmes in which media support is the dominating part (e.g., programmes to transform the former state broadcaster in Tunisia and Burkina Faso, a programme in the Grands Lacs region, in Somalia, or programmes in Tanzania).
2. Comprehensive governance projects with strong media component: programmes that work on various factors that have a potential to influence governance (e.g., a programme in Bhutan that combines better access to services, the fight against corruption, youth engagement, and supporting independent media in one programme).
3. Minor role for media: programmes in which media are mainly used for dissemination of selected results (e.g., a civil society support programme in Macedonia, where the Council of Media Ethics and an association of journalists were supported).

The programmes of specialised media supporting organisations who participated in this study - FH, DWA, FPU, and BBC Media Action - fall into the category of "media as priority". They involve: projects with purely media-oriented objectives (e.g., projects working on safety of journalists) or objectives beyond the media sector (e.g., projects aiming to contribute to democratisation, civic participation, social cohesion, lowering of tensions between groups in society, etc.).

The organisations participating in this study show different profiles with regard to the layers in media support, which are outlined in the document. In brief:

• FH works mainly in setting up media organisations and media outlets, building the capacity of journalists, and directly producing content.
• DWA sees its efforts explicitly as a human rights-based approach and works in 4 key areas of action: political and legal frameworks; qualification; professionalism and economic viability; and participation in society.
• FPU has a strong focus on the layer of legal environment and legal reality, with another focus on training for journalists in various topics.
• In the area of governance, BBC is engaged in capacity building of journalists (via producing a joint programme in local radio stations), content production (various outputs like dialogue programmes), active audience engagement (in dialogue programmes), and some work on economic sustainability of local radio stations.

In all the documents studied, the review found mostly the narrative and overview models of theories of change that mainly summarise the already-existing logframe in a text or a graph. In some programmes, the organisations have started to formulate more elaborated theories of change with more intermediate elements, and towards describing concrete and complete result chains.

Another observation is that there are no special theories of change for different aspects of governance, such as one for decentralisation and another one for civic participation. Making the theories of change more specific would require a more detailed description of the various steps from activities of projects to expected outcomes and, finally, impacts.

At the outcome level, the review found that there are some differences in theories of change with regard to the layers of the media sector, which are the entry points of the media initiatives. However, they are not very well accentuated when the narrative or the overview model of theory of change is used. For example, expected outcomes are formulated on a general level (e.g., "social cohesion is increased"), but the details are vague (e.g., it remains unclear which groups in society are expected to come closer together and in what concrete aspects of social life the cohesion will be improved). In contrast, the causal model tries to formulate a system of interlocking parts that transmits causal forces between a cause and an outcome.

The study also looked at existing studies based on empirical research on the topic of media effects on governance and other political outcomes. First, the researchers present findings on general insights and limitations; second, they examine specific findings according to the areas of interest (knowledge, interest, engagement, participation); and third, they look at specific theories of media effects in order to gain insights beyond the limited scope of strictly governance outcomes. Based on the overview they provide on the empirical academic evidence on media effects on governance issues, they state the following with regard to theories of change used in media development:

• The agenda setting effect of media exposure - meaning: media establish what people see as relevant and what topics people discuss - has so far been neglected by donors and implementing organisations with regard to both the public agenda and the political agenda. Agenda setting might be a pre-condition for increasing public pressure on governments.
• Framing - meaning: how the media frame stories in their reporting - can also play a significant role in governance issues, but is so far not clearly acknowledged by donors and implementing organisations.
• The literature demonstrates that media exposure to quality media increases political interest, at least in Western countries.
• Empirical studies on media in developing countries are lacking, especially from Africa.
• The connection between media exposure and political participation is complex, as it depends on many intervening variables.
• The effects of digital and social media look to be rather limited and are yet to be more thoroughly investigated.
• Changing attitudes is a long-term endeavour and depends on many factors.

The review found many reports about community media, but hardly any that are based on an empirical, methodological design. Although these reports discuss media effects, they often do not really measure effects. For instance, some studies offer some anecdotal evidence of effects of community radio stations on political participation, peace development, and education, but don't provide a clear picture.

Based on the conceptual considerations of the causal model of theories of change, the researchers propose formulation of (various) result chains according to the layers within the media sector. This way, every layer that is part of a media programme has its own theory of change, linking activities with outputs and outcomes in much more detail. They present a series of graphs using different colours: activities in blue, first outputs in yellow, the next level of outputs or achieved outcomes in green, the final step in outcomes in orange, and external factors or enabling assumptions in a red box below.

In conclusion, the main recommendation for donors and implementing organisation is to work on causal models of theories of change as a standard tool in project planning, as it can give clear context-related insights into what can be expected from projects and what is feasible in the specific environment. As compared to narrative and overview models of theories of change, causal models offer a lot more information that could help improve project planning, as well as monitoring and evaluation. "Through formulating result chains, many hidden assumptions come to light, and additional elements that need to be tackled before outcomes can be expected will become obvious. It would also be a pre-condition to provide impact evaluations that are lacking in the sector of media support for governance."