Blog Culture Matters

Mar 28
Culture and societal change – limits and possibilities in evaluation

Cultural projects supported by SDC have often defined societal change as overall goal. How can we evaluate such objectives and what are appropriate tools and methods?  Furthermore is it possible to prove the effects that our project has contributed to this ambitious objective?  Or is the link between culture and societal change in the first place an obvious fact and a matter of belief?
- by Barbara Aebischer, Team Culture and Development, SDC

 

© DEZA/Dominic Nahr
Bamako, Mali, 2016. Residents from the neighbourhood enjoy a performance of Diakoya Dogotoro" le médecin malgré lui de Molière by the theater group Assitan Tangara in Niarela. Their projects have been supported by SDC.

 

Some time ago I had the great opportunity to undertake interviews with three experts in the field of the evaluation of cultural projects in regard to such issues: Céline Yvon, Elisa Fuchs and François Matarasso. The questions dealt with possibilities and limitations of the evaluation of the effects that an independent, diverse and inclusive culture sector can have on societal transformation. Hereby, we excluded the projects that are using culture as a means to a different end. In this article I would like to summarize some aspects of these conversations:

 

How can societal effects of cultural projects be made tangible?

Evaluating societal effects and making them tangible would be a rigorous and cost intensive exercise. For small cultural projects this does not make sense in regard to the available resources. Furthermore, societal changes need a lot of time and it is difficult to assign the causes to the determined effects. In this sense, it can be assumed that SDCs cultural projects can make just a contribution to societal change. In regard to methods to evaluate, there are suggested different kinds of interviews, such as qualitative detailed interviews, surveys and group interviews. It is important that the selected persons include also key persons, who are external of the project. In the role as an interviewer we have to be careful and be aware that there is always a power relation (as donor). In addition, the persons interviewed usually know what the interviewer (or the mandating donor) would like to hear and this can finally influence the answers. In this regard it is recommended to spend informal time with persons involved in the project. Ideally, interviews should be realized at the beginning as well as in the end of the project. In addition, a good way to illustrate the results of a cultural project is by telling anecdotes and success stories.

 

Where do you see the advantages, disadvantages and limits of planning cultural projects with the method of the logical framework?

The logical framework is estimated as a useful tool to think something through as well as a working tool. However, the logical framework should be established at the stage of project definition in collaboration with the partner organization. This in order to define together the specific project goals and indicators and discuss the mutual expectations and individual understanding of the goals. It was underlined, that the simpler the logical framework is, the more effective it will be. Furthermore it is important that the content of the logical framework is instilled by the implementing staff members. One of the dangers of the logical framework is that it can give an unrealistic level of confidence on what we know and what we are doing. For this it is important always to look beyond the logical framework. The causal connections in regard of changes can be assessed more easily concerning an individual, in the group it is more difficult and on the social level it will be quite speculative.  

 

Do you observe an impact on the actual artistic expressions (i.e. their nature, their style or substance), when cause-effect hypotheses are defined in advance?

In the stage of project development, it is possible to develop indicators and goals in collaboration with the selected partner to exchange mutual expectations (as mentioned). However, if the project support depends on specific target criteria, there might be a bias, as project structures and indicators might be constructed artificially in order to receive financial support.

In addition to that, specific goals and target criteria will indirectly always influence the nature and manner of the productions. Furthermore, there is a danger of bringing western norms and expectations into another context. As in many countries only limited money is available for culture promotion, we have to be aware that by supporting a specific art branch or style, it can influence the kind of artistic productions in a region.

 

Is it necessary and feasible, to evaluate cultural projects in regard to effects on social change?

Accountability is as a principle important, among other reasons because there is always pressure to show the legitimacy of support for culture. In addition, different aspects and points of view have been mentioned of what should be considered. On the one hand, a rigorous evaluation on societal change does generally not make sense for very small cultural projects, like the ones often supported by SDC. On the other hand, the societal effects of art and culture are based on rational expectations and beliefs. This is similar to education; where we also cannot foresee what effect it has on the individual child or student. The suggestion is to focus primarily on outputs and in addition to look always at the qualitative aspect of the quantitative results. The level of outputs achieved will give an indication on outreach, outcome and some wider societal impact. But we cannot be sure in advance of what character they will show.

Another advice is to be more modest in the design of cultural projects: for example when we give a contribution to ten artists enabling them to do their work for the next twelve months. After this year results can be reviewed and it is important to recognize the victories that they are achieving with their work, which are sometimes little but at the same time great.

Comments

There are no comments for this post.