Volume 36, 2021 - Spring

Toward an Evidence-Based Approach to Building Evaluation Capacity

Authors :

Abstract: Organizations are required to evaluate their programs for both learning and accountability purposes, which has increased the need to build their internal evaluation capacity. A remaining challenge is access to tools that lead to valid evidence supporting internal capacity development. The authors share practical insights from the development and use of the Evaluation Capacity Needs Assessment tool and framework and implications for using its data to make concrete decisions within Canadian contexts. The article refers to validity evidence generated from factor analyses and structural equation modelling and describes how applying the framework can be used to identify individual and organizational evaluation capacity strengths and gaps, concluding with practice considerations and future directions for this work.

View Text

Reflective Practice: Moving Intention into Action

Authors :
Reflection is a pillar of professional practice and can enable evaluators to expand their assumptions, change behaviour, and align practice with their values and beliefs. Reflective Practice is the first domain in the Canadian Evaluation Society’s Competencies for Evaluation Practice, highlighting the importance of professional awareness and growth. However, the literature offers few tools to move reflective thought into action. The authors developed a personalized refl ective approach through a mentoring relationship and the innovative use of a smartphone. Meaningful reflective practice can leverage personal experience, practical tools, and regular dialogue with a significant other to expand assumptions and change behaviour.
 
This is the advanced online version

Review: Darlene F. Russ-Eft and Catherine M. Sleezer. (2020). Case Studies in Needs Assessment.

Authors :
This book generously shares practical approaches to needs assessment, providing both novice and seasoned evaluators with a leg-up on their evaluative thinking when determining how to approach their current situation.
 
This is the advanced online version

Étude exploratoire de la pratique évaluative dans le domaine de la santé au Burkina Faso

Authors :

Abstract: This study provides a reflexive analysis of evaluative practice in the health sector in Burkina Faso. A sample of ten evaluations carried out in the health sector since 2011 was analyzed. The descriptive analysis shows that current evaluation practices vary very widely across the country. Most evaluation reports do not comply with international norms and standards in evaluation, in particular with regard to the definition of terms of reference and evaluation questions, the precision of evaluation criteria and the quality of the methodology. We conclude that a culture of evaluation must be developed and capacity must be built for better capitalization of experiences and for evidence-based decision-making.

This is the advanced online version

L’évaluation évolutive, de la théorie à la pratique : perspectives de praticiens québécois

Authors :

Abstract: Quebec and Canadian organizations, communities, and professionals working collectively to address complex social issues in dynamic contexts are enthusiastically embracing social innovation. Some of this innovation’s limitations have been revealed by formative and summative evaluation approaches. Developmental evaluation (DE) could be a potential solution. This article combines scientifi c literature with the experience of experienced DE practitioners in Quebec. Balancing project management and evaluation, knowledge production and transformation of practices, continuous iterations and setting deadlines is at the very core of an eff ective DE approach.

This is the advanced online version

L’apport du journal de bord en évaluation de programmes dans une perspective de pérennisation : le cas du programme HORS-PISTE

Authors :

Abstract: Over the past few years, multiple anxiety prevention programs for teenagers have been developed and implemented in schools. However, these programs are seldom systematically evaluated and scaled up for use in all schools. When the programs are evaluated, data collection often involves the use of a logbook completed by facilitators. This method’s specific methodological function in a program evaluation involving multiple data sources has not been systematically analyzed to date. This article uses the example of the HORS-PISTE program to show the logbook’s advantages, limitations, and challenges as a data collection method.

This is the advanced online version

Creating Effectiveness Principles for Principles- Focused Developmental Evaluations in Health-Care Initiatives: Lessons Learned from Three Cases in British Columbia

Authors :

Abstract: Principles-focused developmental evaluation is an emergent method of evaluation that is increasingly achieving relevance in initiatives seeking to transform health-care delivery within complex-adaptive systems. Creating meaningful eff ectiveness principles is considered a crucial first step in setting up such evaluations. In this article, we describe four practical steps that we applied in defi ning eff ectiveness principles to align with Patton’s GUIDE criteria. To illustrate our approach, this article features three principles-focused developmental evaluations implemented in British Columbia, highlighting lessons learned through the process of creating eff ectiveness principles.

This is the advanced online version