Evaluation Elsewhere: a View from Canada

On November 6, 2015, the CES President, Benoît Gauthier, was invited by the Treasury Board of Canada Centre of Excellence for Evaluation to take part in an EvalConnex event: Making Use of Evaluation Results – International Perspectives on Successful Practices and Opportunities, Here were his remarks.

Thank you for inviting me.

This event is an important opportunity to learn from the experiences of others in evaluation – that is from evaluators from outside Canada but also from Canada but outside the federal evaluation system.

In this talk, I will briefly describe:

• the international evaluation environment and the CES involvement in it
• how evaluation practice differs from place to place (evaluation cultures are not the same from continent to continent but also from one evaluation context to another in Canada)
• the strong international emphasis on professionalization
• the international objectives for evaluation for 2016-2010

International Environment

Coincident with the final year of Milenium Development Goals, 2015 was also declared as International Year of Evaluation (EvalYear) at the Third International Conference on National Evaluation Capacities in Brazil in 2013, endorsed by the United Nations. The aim of designating 2015 as the International Year of Evaluation was to advocate and promote evaluation and evidence-based policy making for continuous program improvements and insights at international, regional, national and local levels.

Practically, EvalYear has been an opportunity to support more than 100 national and international events dedicated to evaluation around the world. It was also a catalyst for important conversations and thinking on the role of monitoring and evaluation in good governance for equitable and sustainable human development.

Let's take stock of the evolution of evaluation organizations internationally. Canada, through the CES, was among the first professional associations to formalize as a not-for-profit bilingual national association of and for evaluators. In the 1980s there were only three national and regional professional evaluation societies serving practitioners through professional development, networking and other services; by the late 1990s there were nine. In 2003, the International Organization for Cooperation in Evaluation (IOCE) was created as an umbrella association of Voluntary Organisations for Professional Evaluation (VOPEs). Over the past 10 years, the number of VOPEs has increased to 158. In 2013, EvalPartners, a world movement to foster evaluation was created by IOCE, UNICEF, UN-Women with strong donor support. The Réseau francophone de l'évaluation was created in 2008; it currently has 22 member associations. The CES is involved in all of these international associations.

There are other international organizations where CES is not involved, such as

• the Active Learning Network for Accountability and Performance in Humanitarian Action (ALNAP) established in 1997
• the Network of Networks for Impact Evaluation (NONIE) established in 2006

There is a number of regional associations as well such as

• the African Evaluation Association
• the European Evaluation Society
• the Asia Pacific Evaluation Association
• the Community of Evaluators from South Asia
• the International Program Evaluation Network from the Commonwealth of Independent States
• the Latin America and the Caribbean Evaluation, Monitoring and Systematization Network (RELAC)
• the Middle East and North Africa Evaluators Network (EvalMENA)

The United Nations, international financing institutions and other international organizations also have networks of their own.

• the United Nations Evaluation Group (UNEG) composed by the heads of evaluation of the UN entities was created in 2002.
• the World Bank is very active; it has its own Independent Evaluation Group as well as a Results, Measurement and Evidence personnel stream. The World Bank sponsored the International Program for Development Evaluation Training (IPDET-PIFED) and to the establishment of Regional Centers for Learning on Evaluation and Results (CLEAR) in Africa, Asia and Latin America.
• the OECD has its Network on Development Evaluation which develops guidance and standards for practical evaluation application.

There is no lack of structure in evaluation at the international level; in fact, as EvalYear comes to an end, various global evaluation bodies are rethinking mandates to ensure differentiation, utility and collaboration.

Many CES members, including its International Working Group, are involved in evaluation and are experts in the international context, including capacity building and sustainable development.

Evaluation practice differs from place to place

Over the past year and a half, I have had the opportunity to be exposed to evaluation practice in different places around the world. I am coming away from that with a renewed sense that the evaluation is very much anchored locally and culturally. Here are some observations.

• Australasia: strong emphasis on the cultural context, the specificity of the program environment as well as stakeholder and human rights
• US: more emphasis on methods
• Canada: more talk of theory-based evaluation, evaluation management, and the credibility of the evaluator
• Africa: more of a grassroots movement with important exceptions where central governments want to institutionalize evaluation
• Europe: focussed on the policy rather than the program level
• Other: the Government of Thailand is the first nation state in the world to develop an evaluation professionalization program in its public sector, drawing on CES competencies and standards.

Canada is seen as a world leader in evaluation. Our professional tools are the envy of many; that includes evaluation policies, results-based management practices, informed decision making processes, emphasis on professional practice, training opportunities, etc. The rigor used by Canadian evaluators is valued internationally. Evaluators are seen as systematic methodologists, asking the right questions and seeking relevant and useful information to produce timely answers.

Within Canada, however, evaluation is certainly not conducted in a monolithic fashion.

• The TB Policy on Evaluation defines much of the federal practice but it leaves more wiggle room than evaluation units typically use
• Only three provinces have policies on evaluation – and one of them does not actually carry out evaluation in practice. Where evaluations are carried out in provinces, they are typically of rather narrow evaluands and they are done because there is a real or anticipated demand for the findings. To me, provincial governments are the next frontier for the promotion of evaluation in Canada.
• Very few local governments implement regular evaluations. This is a sore observation considering what is happening on non-profits.
• The not-for-profit level implements regular evaluation and monitoring efforts, with very limited means and using approaches that are much more participatory, developmental, innovative and embedded than other sectors – with the possible exception of the education sector. Methods are more qualitative and the evidence collected is typically richer than a similar-sized evaluation elsewhere. Program management is much more involved in the evaluation process and the learning (as opposed to accountability) purposes of evaluation are much clearer. There is a lot to learn from the not-for-profit sector in terms of evaluation relevance, use, timeliness, and efficiency.

Very strong international emphasis on professionalization

EvalYear has been an opportunity for professionalization of evaluation to rise as a key concern. In this context, professionalization is a process supported by groups of individuals to allow their members to become professionals. A professional is an individual

• with knowledge, know-how, and judgement (competence)
• to assess a problem, to identify solutions, to select the most appropriate solution, and to implement it successfully (standards) and
• respectfully (ethics)
• in a particular area of human endeavour (self-definition)

The emphasis on professionalization comes from a need to define the practice and the practitioner, to improve quality (broadly defined), to reassure users, and to contribute to professional identity.

CES already offers support and mechanisms to aid evaluators in their professionalization journey. This includes defining the area of human endeavour, offering ethical guidelines, determining the required competencies, and offering standards for the practice of evaluation.

Objectives 2016-2020

As EvalYear is winding down, the global evaluation community wants to ensure that evaluation will play a key role in shaping and contributing to the implementation of policies and programs to achieve sustainable and equitable development. The inspiring 2016-2020 agenda that is being prepared is articulated against four priorities:

• strengthening an enabling environment for evaluation
• strengthening institutional capacities of VOPEs and civil society
• strengthening individual evaluator capacity development
• inter-linkages between enabling environment, institutional capacities and individual capacities

The international evaluation community is adopting an activist vision of evaluation with an emphasis on social justice/equity (compared to social effectiveness and social efficiency). We will have to see how Canadian evaluation fits in that scheme which currently resonates more with the not-for-profit sector than with the government sector.