CES Online Learning Goes Live: Welcome to the CES e-Institute!

Volume 27, 2012 - Special Issue

The Paris Declaration Evaluation Process and Methods

Authors :
Pages :
69-102

The Paris Declaration Evaluation faced the challenge of assessing the effects of a policy compact between nations, an area in which evaluation thinking and practice is still emergent. It also confronted the challenge of attempting to link development results to the implementation of a broad reform agenda in countries of widely differing circumstances. This article describes how the Core Evaluation Team, in consultation with the management and governance structures of the Evaluation and the evaluation teams involved, addressed these challenges. It describes the development of the evaluation questions, the methodological framework, and how the methodology worked in practice. Highlights include ensuring the use of evidence through evaluation teams; the analysis and synthesis processes; and processes for ensuring independence, integrity, quality, and ethics. Limitations and risks encountered are discussed. Finally, the article sets out lessons learned for complex transnational studies from the Core Evaluation Team perspective.

Results of the Paris Declaration Evaluation

Authors :
Pages :
103-128

The Paris Declaration on Aid Effectiveness comprised principles and specific commitments for improving development aid processes, accepted by an unprecedented coalition of developing countries and aid donors. The Synthesis Evaluation used data from a variety of sources to answer questions on (a) the relevance of the Declaration and its implementation for aid effectiveness, (b) the actual record of implementation, (c) contributions to aid effectiveness and development results, (d) changing burdens of aid management, (e) added value of Paris Declaration-style development cooperation, and (f) key implications for future aid effectiveness. Conclusions of the synthesis evaluation and recommendations aimed at policy makers in partner countries and donor agencies are presented. The campaign to make aid programs more effective is relevant and showing results. But improvements are slow and uneven in most developing countries and especially among donor agencies. The Declaration's original aspirations are neither fully implemented nor yet outdated.

Dissemination and Early Use of the Paris Declaration Evaluation

Authors :
Pages :
129-146

Reflecting the long experience of the main organizers of the Paris Declaration evaluation, dissemination and use of the national and synthesis evaluation results were conceived, born, and carried out with a consistent view to the usefulness of both its processes and products for policy. This shaped all aspects of the Evaluation: governance, stakeholder participation, setting the evaluation questions, design and implementation of the processes and methods, and injecting findings and recommendations into the policy bloodstream. The focus on utility imposed the challenges of combining rigorous evaluation with a high degree of transparency and accessibility for the intended users and audiences, while building a network of stakeholders sufficiently committed, engaged, and comfortable with the results to be able to champion them through the tools and channels available to them, individually and collectively. However, the early record of influence illustrates the limits of evaluation in the face of rapidly changing political agendas.

Meta-Evaluation: Evaluating the Evaluation of the Paris Declaration

Authors :
Pages :
147-171

It has become a standard in major high-stakes evaluations to commission an independent review to determine whether the evaluation meets generally accepted standards of quality. This is called a meta-evaluation. Given the historic importance of the Evaluation of the Paris Declaration, the Management Group commissioned a meta-evaluation of the evaluation. The metaevaluation concluded that the findings, conclusions, and recommendations presented in the Paris Declaration Evaluation adhered closely and rigorously to the evaluation evidence collected and synthesized. The meta-evaluation included an assessment of the evaluation's strengths, weaknesses, and lessons. This article describes how the meta-evaluation was designed and implemented, the data collected, and the conclusions reached.

Lessons Learned and the Contributions of the Paris Declaration Evaluation to Evaluation Theory and Practice

Authors :
Pages :
173-200

The final event of the Paris Declaration Evaluation was a lessons- learned workshop. This article first highlights the lessons about joint evaluations identified by participants in that workshop with the resulting report being a model of how to bring closure to a major evaluation. The article then presents 10 contributions of the Paris Declaration Evaluation to evaluation theory and practice. The article closes by recognizing that the Paris Declaration Evaluation received the 2012 American Evaluation Association (AEA) Outstanding Evaluation Award, which noted, "The success of the Evaluation required an unusually skilled, knowledgeable and committed evaluation team; a visionary, well-organized, and well-connected Secretariat to manage the logistics, international stakeholder meetings, and financial accounts; and a highly competent and respected Management Group to provide oversight and ensure the Evaluation's independence and integrity. This was an extraordinary partnership where all involved understood their roles and carried out their responsibilities fully and effectively."

Remarks from the editor of the special issue / Un mot du rédacteur du numéro spécial

The Paris Declaration on Aid Effectiveness: History and Significance

Authors :
Pages :
19-36

The Paris Declaration, endorsed in 2005, commits international development aid donors and recipients to act in accordance with five principles: ownership, alignment, harmonization, results, and mutual accountability. This landmark international agreement was the culmination of several decades of attempts to improve the quality of aid and its impact on development. As context for the evaluation of the Paris Declaration, this article traces the history of international agreements that led up to the Paris Declaration and the significance of the agreement itself. The number of countries and international organizations endorsing the Declaration was unprecedented, reflecting a greater diversity of voices included in international deliberations on aid effectiveness. It provided a practical, action-oriented roadmap with specific targets to be met. The evaluation derives its significance and relevance from the importance of the Paris Declaration and from the innovative, rigorous evaluation approach taken as an exemplar of joint evaluations.

Preparing, Governing, and Managing The Paris Declaration Evaluation

Authors :
Pages :
37-67

Joint or multi-partner evaluations are evaluations of development cooperation policies, programs, and projects in which different donors, development agencies, and partner countries participate. The complexities of a large number of diverse stakeholders and multiple units of analysis in joint evaluations pose significant governance and management challenges to ensure the evaluation's independence, credibility, quality, and utility. This article reports how governance and management were structured and operated to facilitate the evaluation of the Paris Declaration. A common evaluation framework was established to facilitate synthesis. The integrity of national evaluations had to be ensured, including capacity-building and support as needed. National and international reference groups were established to ensure the engagement and buy-in of different stakeholder groups, including input and feedback to the core team that synthesized the results of the national evaluations. The article concludes with three important lessons about complex joint evaluations.