10 Recommended Resources on Policy Analysis and Evaluation
The analysis of policy instruments and the evaluation of gender equality measures are crucial aspects for enhancing gender equality in research and innovation. In the following post we are highlighting a number of useful resources on the relevance of policy analysis, as well as on the evaluation of gender equality measures in institutions and projects. These include resources that represent insights into evaluation methodology, and examples of guidelines and handbooks for the analysis and the evaluation of gender equality measures.
The presented resources are far from being exhaustive – they should rather be understood as an invitation to join GenPORT and contribute to its constantly growing body of resources on gender and science.
1. HANDBOOK: UN Women Evaluation Handbook: How to manage gender-responsive evaluation
This Evaluation Handbook is a practical handbook to help those initiating, managing and/or using gender-responsive evaluations by providing direction, advice and tools for every step in the evaluation process: planning, preparation, conduct, reporting, evaluation use and follow up. The primary audience is UN Women staff who manage evaluations or are involved in evaluation processes. However, it may also be useful to international development evaluators and professionals, particularly those working on gender equality, women's empowerment and human rights. The purpose for providing this resource on the Gender Equality Evaluation Portal is to enable the linkage of theory with practice. The Portal contains over 400 evaluations that are focused on gender equality and women’s empowerment providing a space for sharing and learning from gender equality focused and gender-responsive evaluation.
2. PROJECT: Evaluation Framework for Promoting Gender Equality in Research and Innovation
EFFORTI (Evaluation Framework for Promoting Gender Equality in R&I) seeks to analyse and model the influence of measures to promote gender equality on research and innovation outputs and on establishing more responsible and responsive RTDI (research, technology, development, innovation) systems. In the project’s concept and approach the evaluation of gender equality policies with the most recent approaches of RTDI evaluation are combined, in order to make the best use of mutual exchange and learning. Specifically, the project identifies links between initiatives aiming to promote gender equality - through three main gender objectives (more women in R&D, women in leadership positions and integration of a gender dimension in research content and curricula) - and a variety of impacts on research and innovation.
3. BRIEFING NOTE: Integrating Gender Considerations into Energy Operations
This briefing note discusses the key elements of the 'gender in energy' topic and provides specific examples on 'how to' integrate gender considerations in energy policy dialogue and the project cycle. This note draws on recent experience within the World Bank and elsewhere in mainstreaming gender in energy projects. It aims to consolidate this knowledge and make it available to energy practitioners addressing gender aspects of energy projects. This note is complemented by a compendium of online resources to provide energy teams with basic tools, such as sample questionnaires, terms of reference, and screening guidance, as well as reference material on gender and energy. These resources have been developed by drawing on the experience of Energy Sector Management assistance Program's (ESMAP's) Africa Renewable Energy and Access (AFREA) gender and energy program, where gender considerations have been integrated into five energy operations in Senegal, Mali, Benin, Tanzania and Kenya.
The complexity of many cases, such as public projects and programs, requires evaluation methods that acknowledge this complexity in order to facilitate learning. On the one hand, the complexity of the cases derives from their uniqueness and nested nature. On the other hand, there is a need to compare cases in such a way that lessons can be transferred to other (future) cases in a coherent and non-anecdotal way. One method that is making headway as a complexity-sensitive, comparative method is Qualitative Comparative Analysis (QCA). This contribution aims to explain to what extent QCA is complexity-informed, to show how it can be deployed as such, and to identify its strengths and weaknesses as an evaluation method. This publication discusses the properties of complexity, provide an overview of evaluation literature about QCA, and present a simplified step-wise guide for utilizing QCA in evaluation studies.
5. HANDBOOK: Gender-sensitive Design of Criteria and Recruitment, Appointment and Promotion Processes in Academia
Recruitment and appointment processes for senior academic positions or professorships differ among European countries. However, there are important topics which can enhance fairness in general and gender equality in particular. The most important ones are summarized here, including a comparison of the FESTA project partners’ findings regarding recruitment and promotion processes at five research performing organizations in Europe. Guidelines are included which aim to reduce or eliminate biases and lead to gender equality in hiring processes.
Given the increased role of bibliometric measures in research evaluation, it is striking that studies of actual changes in research practice are rare. Most studies and comments on ‘a metric culture’ in academia focus on the ideological and political level, and there is a clear shortage of empirical studies that analyze how researchers handle demands for accountability in context. In adopting a mixed-methods approach involving both bibliometric data and answers form questionnaires, the authors provide an in-depth study of how researchers at the faculty of Arts at Uppsala University (Sweden) respond to the implementation of performance-based research evaluation systems. Publication patterns from 2006 to 2013 show that journal publications, especially English-language ones, are increasing, and the proportion of peer-reviewed publications has doubled. These changes are in line with the incentives of the evaluation systems under study. Answers to the survey confirm that scholars are conscious about this development, and several respondents articulate a disagreement between disciplinary norms and external demands. However, disciplinary background as well as career stage or academic age appears to have a significant influence on how individual researchers react to the instigation of evaluation systems. Finally, responses to national and local evaluation regimes are complex, localized, and dependent on many factors. In-depth contextualized studies of research practices are needed in order to understand how performance-based funding systems influence academic research on the ground.
7. GUIDELINES: Guidelines for evaluating gender equality as a cross-cutting objective
Promoting gender equality as an explicit cross-sectoral goal of the European Social Fund (ESF) is a binding requirement of all ESF-funded programmes and projects. This document provides guidelines for the evaluation of the cross-sectoral gender equality goal of the ESF. The Federal Operational Programme under the European Social Fund for the funding period 2007-2013 pursues a dual strategy, which includes both specific gender-equality measures and the implementation of the cross-sectoral goal. All programmes and projects carried out with ESF funding must therefore be assessed for their implementation of gender mainstreaming and their contribution to promoting gender equality.
These guidelines developed in the INTEGER project aim to supply higher education and research institutions with tools and guidance for the assessment of their Transformational Gender Action Plans. They suggest using evaluation methodology for quality assurance of gender action plans, to support legitimacy and in-house dialogue, and to measure institutional performance of the implementation of plans to foster gender equality.
9. HANDBOOK: Evaluation Standards of the Swiss Evaluation Society
The evaluation standards of the Swiss Evaluation Society (henceforth SEVAL Standards) are meant to contribute to the professionalization of evaluation in Switzerland. Adhering to the SEVAL Standards enhances the credibility, quality, and trustworthiness of evaluations. High quality evaluations can only be created when all involved – that is, the evaluators themselves, those who commission the evaluations, and other persons participating in the evaluation – work together. The SEVAL Standards therefore provide criteria that all persons involved in an evaluation have a duty to uphold.
10. GLOSSARY: OECD Glossary of Key Terms in Evaluation and Results Based Management
The DAC Working Party on Aid Evaluation (WP-EV) has developed this glossary of key terms in evaluation and results-based management because of the need to clarify concepts and to reduce the terminological confusion frequently encountered in these areas.