A precondition for proper evaluation is to precisely define the media programme models and Theories of Change. If the expected results of a project – the outputs, outcomes and overall goals – are well identified, and quantitative and qualitative indicators for all levels are specified, then M&E can be conducted in a systematic way that supports future learning.
Some Theories of Change aim to achieve outcomes strictly within the media sector (for example, efforts to improve media regulations for more freedom, or economic viability of media houses, or improving capacity of journalists) as the stability and development of the media sector is an aim in itself. Other Theories of Change aim to influence areas outside the media sector – for example, to contribute to democratisation, civic participation, social cohesion, reduction of tensions between groups in society, etc. However, expectations about what media interventions can and cannot achieve need to be realistic. As stated earlier, the media can contribute to more inclusive, just and peaceful societies, but they cannot do so alone. In order to enable the media to play their role, the different layers of an enabling media environment need to be addressed according to specific Theories of Change for each layer.
Furthermore, Theories of Change should avoid large ‘gaps’ or ‘miracle boxes’, which should be filled or explained to describe how the project intends to go (for instance) from ‘better reception’ to ‘diminishing tensions’, and link those elements in a causal way. This requires a step-by-step narrative on the Theory of Change and related assumptions on the ‘missing middle’.
Monitoring needs to start early and be conducted regularly, with a view to the Theories of Change, which determine the issues to be monitored. It is also a key ingredient informing adaptation and learning. Evaluations are usually conducted at the end of a programme, but need to be prepared by collecting data before the project starts (baseline study). Evaluation design should remain flexible as project activities have to adapt to realities on the ground. Be aware of the context of the evaluation, encourage the participation of the various stakeholders in the evaluation design, but be careful with who participates in the data analysis and the resulting risks of conflict of interests.
The media assistance field still suffers from a lack of independent, long-term and rigorous evaluation. Investing in such evaluation capacities – for example, in collaboration with academics and in coordination with various donors and implementers – would improve understanding of what level of impact can be expected and demonstrated, how, and with what indicators. However, it should be acknowledged that evaluating the impact of media assistance is particularly challenging. As information flows are so fluid and multiple, it is difficult to isolate the effects of one specific media intervention from other factors (the so-called ‘attribution problem’).
Below are two examples of concrete Theories of Change that give more detail on the causal links between activities, outputs and outcomes, and formulate (various) result chains according to the layers within the media sector.
Example 1: Capacity building of journalists (for example, in elections)
Training of journalists in basic professionalism and/or specific knowledge is one of the most common interventions in media assistance. Let us assume an NGO has opted to train journalists in building basic skills, based on a journalism concept of impartial but critical journalism, and also providing specific knowledge on elections in order to improve reporting during the electoral campaign. A specific goal has been set to ensure that candidates are assessed more for their policy ideas and less for their personality traits. Thus, the result chain in the figure below is an option. In practice, and in a specific context, it is feasible and maybe even recommended to add some interim steps, depending on the conditions on the ground.
Example 2: How to improve economic sustainability
It should be noted that result chains might not always be strictly linear. This example therefore includes different deviations or branches.
