CES Online Learning Goes Live: Welcome to the CES e-Institute!

Volume 32, 2017 - Spring

Editor's Remarks / Un mot du rédacteur

Authors :
Pages :
i-vii

Articles and practice notes in this issue of the Canadian Journal of Program Evaluation cast light on the value of empirical work in advancing knowledge about evaluation practice. [...]

Log in to post comments or view the article

Contribution Analysis: Theoretical and Practical Challenges and Prospects for Evaluators

Authors :
Pages :
1-24

Contribution analysis (CA) is a theory-based approach that has become widely used in recent years to conduct defensible evaluations of interventions for which determining attribution using existing methodologies can be problematic. This critical review of the literature explores contribution analysis in detail, discussing its methods, the evolution in its epistemological underpinnings to establishing causality, and some methodological challenges that are presented when CA is applied in practice. The study highlights potential adaptations to CA that can improve rigour, and describes areas where further work can strengthen this useful evaluation approach.

Log in to post comments or view the article

Comment évaluer les effets des évaluations d'impact sur la santé : le potentiel de l'analyse de contribution

Authors :
Pages :
25-45

Health impact assessments (HIA) allow the potential impact of nonhealth-related actions (policy, project, program) on health to be analyzed. For example, municipal projects, for urban renewal or urban planning, have an impact on health drivers relating to the created environment, and thus, on public health. In the province of Quebec, the government and affiliated organizations have to comment on potential impacts; they use HIAs to ensure health is factored into public actions. We know little about HIAs' capacity to influence public policies; every policy and program is different and is often implemented only once, greatly complicating evaluation. Our goal is to analyze the potential of contribution analysis for evaluating the impact of HIAs at the municipal level. To this end, we present the HIA process as implemented in Montérégie, we identify evaluation challenges, and put forth, through contribution analysis, an evaluation strategy to analyze effects.

Log in to post comments or view the article

"Advocates Change the World; Evaluation Can Help": A Literature Review and Key Insights from the Practice of Advocacy Evaluation

Authors :
Pages :
46-64

It is only since the new millennium that assessments of policy and systems change initiatives have been given much attention in practice or in research. A comprehensive literature review of advocacy in human services non-profit organizations (NGOs) has found a "lack of systemic, rational evaluation and measurement of the effectiveness of advocacy." Similarly, in a survey of 211 NGOs that undertake advocacy, only one in four (24.6 percent) reported that this work had been evaluated. While the survey was conducted in the United States, it is likely that Canadian NGOs are in a similar situation, not because of a lack of interest or value for evaluation but, rather, because assessing systems change initiatives is challenging territory and NGOs face considerable resource and time constraints that put such evaluation low on the priority list. This article provides a review of key insights from advocacy evaluation practice and research that may help orient and inform NGOs as they decide on evaluation strategies. I will outline the state of the field of systems- and policy-change evaluation in North America as well as its benefits and challenges. Finally, a synthesis of the main steps in advocacy evaluation planning and implementation offers a broad map to NGOs seeking to enhance this practice in their own organizations.

Log in to post comments or view the article

Analyse d'implantation d'une recherche-intervention sur le dépistage rapide du VIH dans la communauté gaie montréalaise

Authors :
Pages :
65-89

Spot is an intervention research project that led to the implementation of a community-based rapid HIV testing service in Montreal's gay community. A case study was undertaken to describe the process through which Spot was implemented and the contextual factors that affected this process. Qualitative analysis allowed an implementation process comprised of several phases to be identified. These phases were shaped by stakeholders' motivations to engage in the project; complex partnership dynamics; challenges with regards to coordinating a multidisciplinary, intersectoral, intervention research project; and the day-to-day management of the intervention team.

Log in to post comments or view the article

La fidélité d'implantation d'un programme probant au-delà de son implantation initiale : l'exemple de Ces années incroyables en protection de l'enfance de 2003 à 2013

Authors :
Pages :
90-108

This article presents a study describing the fidelity of implementation of an evidence-based program and its evolution over a period of 10 years. From this study, the authors establish links between the implementation of programs, their effects, and their sustainability and discuss some of the issues surrounding the measurement of fidelity of implementation. This discussion addresses elements that may influence the fidelity of implementation and the sustainability of the studied program, including adaptations made to it. The advantages and limitations of using a quantitative approach to the evaluation of implementation to produce a portrait of implementation are also presented.

Log in to post comments or view the article

Finding Balance: An Evaluation Governance Model to Ease Tension between Independence and Inclusion

Authors :
Pages :
109-121

Practitioners and theorists have documented the benefits of user engagement and participation in evaluation and, at the same time, the value of neutral and impartial evaluative evidence. Yet producing both an independent and inclusive evaluation is a leading challenge in our field. In this practice note, we present one solution. We describe the design of an evaluation governance structure that was used to find balance between these two themes. We also identify key elements of this experience and present these for adaptation by others, given appropriate tailoring. We have documented our experience as we believe that governance provides currently uncharted potential for this discipline spanning challenge.

Log in to post comments or view the article

Building Evaluation Culture and Capacity in a Community-Level Program: Lessons Learned from Evaluating Youth Futures

Authors :
Pages :
122-130

Youth Futures is a complex program supported by a local partnership aimed at increasing access to post-secondary education among youth from lower-income family backgrounds. Although program stakeholders have recognized the importance of evaluation since the program's inception, embedding an evaluative culture and building capacity for evaluation practice in the context of this partnership-led program has been challenging. We describe our experiences and the lessons that have been learned about generating and sustaining interest in evaluation among a diverse group of stakeholders. The article proposes some recommendations for others who endeavour to conduct evaluation work in community contexts.

Log in to post comments or view the article

L'analyse logique comme outil de soutien à l'application des connaissances dans le domaine psychosocial

Authors :
Pages :
131-142

For fifteen years, social services organizations, such as those working with people with intellectual disabilities, have been deploying numerous efforts to enhance their practices by making them more consistent with scientific knowledge. Logical analysis, a theory-based evaluation approach, is a promising avenue for knowledge translation in practice settings. By describing the experience of applying that particular analysis framework to a social skill program, this article sheds light on the benefits of using that approach, such as implementing a practice consistent with current scientific knowledge and valuing the program designers' experiencebased knowledge.

Log in to post comments or view the article

Book Reviews: Patricia Burch and Carolyn J. Heinrich. (2015). Mixed Methods for Policy Research and Program Evaluation. Thousand Oaks, CA: SAGE. 211 pages.

Authors :
Pages :
143-145

Book Reviews: Patricia Burch and Carolyn J. Heinrich. (2015). Mixed Methods for Policy Research and Program Evaluation. Thousand Oaks, CA: SAGE. 211 pages.

Log in to post comments or view the article

Book Reviews: Thomas A. Schwandt. (2015). Evaluation Foundations Revisited: Cultivating a Life of Mind for Practice. Stanford, CA: Stanford University Press. 215 pages.

Authors :
Pages :
146-149

Book Reviews: Thomas A. Schwandt. (2015). Evaluation Foundations Revisited: Cultivating a Life of Mind for Practice. Stanford, CA: Stanford University Press. 215 pages.

Log in to post comments or view the article