D1.1 - Current evaluation practices and related challenges

Current evaluation practices and related challenges of policy evaluation and bridging ex-post and ex-ante

The study underlines the importance of the economic evaluation of environmental and climate policies in the process of developing the policies themselves. This ensures that the costs of a policy are reasonable, ensuring that the public and decision makers are more likely to accept it.

The research suggests carrying out economic evaluations both before (ex ante evaluations) and after the implementation of a policy (ex post evaluations), to obtain more effective policies and interventions based on concrete data. Ex post evaluations can help improve the accuracy and reliability of ex ante evaluations, as they will be based on real-world data and experiences rather than assumptions and projections. While ex ante evaluations can inform the design and implementation of a policy and its evaluation principles, to improve its effectiveness and efficiency once implemented.

Linking the two types of assessments can be difficult due to a number of factors. One of these is for example the strong likelihood that policy making differs between the two types of evaluation, due to policy changes that occur during implementation. As a result, the two assessments may be based on different data sources and scales of analysis which are difficult to compare.

The report therefore provides a first attempt to outline multiple areas for improvement regarding the economic evaluation process, such as connecting experts and practitioners ex-ante and ex-post, promoting open access to data and improving the understanding of complex ex-ante models.

D1.2 - Best Practices in Economic Appraisal Methods

Best Practices in economic appraisal methods

The study provides an analysis of economic evaluation methods used in climate and biodiversity policies, aiming to extract the best practices that can be useful in the PATTERN project case studies. It consists of four sections: the first provides an overview of the main approaches related to the analysis proposed in the case studies; the second examines various applied case studies to extract common best practices; the third focuses on the proposed methods for evaluation, such as carbon handprint, Q-method, or life cycle impact assessment; the fourth illustrates the main challenges and needs identified in the analysis. Some of the identified good practices include: (i) a clear definition of the scope of the analysis; (ii) the use of appropriate discount rates; (iii) the use of reliable data and the assessment of all relevant tangible and intangible costs and benefits; (iv) sensitivity analysis to understand how key assumptions change or how variables can influence results; (v) consultation with relevant professionals to ensure that the analysis reflects their concerns and interests.

Overall, the best practice for using economic evaluation methods is to use a combination of approaches and carefully consider the context and specific objectives of the project or policy.

D2.2 - Building a Theory of Change to identify evaluation needs

Building a Theory of Change to identify evaluation needs

Explore PATTERN’s innovative approach to policy evaluation through our latest document on the participatory crafting of Theories of Change (ToCs). This comprehensive endeavor involves stakeholders, policymakers, practitioners, and evaluators, forming a crucial bridge between ex-ante assessments and ex-post evaluations within our diverse Case Studies.

PATTERN’s strategic amalgamation of Utilization-Focused and Realist Evaluation sets the stage for a dynamic understanding of prospective and retrospective policies. Delve into the study to discover the intricate processes involved in policy development, from conceptualization and planning to intervention and retrospective evaluation.

This document not only sheds light on PATTERN’s unique methodology, but also invites you to join the ongoing conversation on rational planning, informed decision-making, and the pivotal role of evaluation in refining policies over their lifecycles. Immerse yourself in the intellectual journey of incremental circularity, where knowledge continually evolves, and evaluation becomes the compass for navigating unforeseeables in policy development.

D2.3 - Social Side of Policy Impact

Assessing the social side of policy impact

PATTERN’s methodological approach extracts a Theory of Change from stakeholder assumptions and introduces the Accountability Ceiling to distinguish direct policy impacts from broader influences. The framework categorizes social impacts into six domains: Capability, Community, Inclusiveness, Labor, Political Participation and Civil Rights, and Way of Life, each with dimensions assessed through specific indicators for thorough evaluation.

The document acknowledges challenges in directly attributing social impacts to policies due to the complex interplay of factors and proposes innovative approaches. The conclusions endorse this integrated, stakeholder-driven approach, urging policymakers and practitioners to adopt the framework to ensure effective and equitable climate and environmental policies.