Isabelle Bourgeois

Spring

Editor’s Remarks

Authors:
The entire CJPE editorial team is pleased to note the quality and calibre of the submissions that we continue to receive from the evaluation community, and the papers included in this issue illustrate this quite well. Th e first four articles published in this issue focus on current topics that will be of interest to both researchers and practitioners, such as evaluation capacity building (LaMarre, D’Avernas, Riley, Raffoul, & Jain), modeling program outcomes (LaVelle & Dighe), program complexity and theory of change (Douthwaite, Ahmad, & Shah), and trends in the CES credentialing program (Lawson, Hunter, & McDavid). A fi ft h article, written by Dussault and Duquet, focuses on the lessons learned through an evaluation conducted in Quebec schools. Our four research and practice notes are also sure to generate thoughtful discussion in our community: Roy and Searle move our thinking about developmental evaluation further by discussing scope creep; McDavid, Shepherd, and Morin share their experience with inter-university collaboration in the area of evaluation education; Lahey and his collaborators present us with a description of how evaluation is conducted in provincial and territorial governments across Canada; and Gauthier discusses the professionalization of evaluation practice and its dimensions. Finally, we are pleased to present two book reviews, one by Gómez-Ramírez and one by Sellick, that provide us with a good sense of what new resources are available to us and how they might inform our practice and thinking.
 
One quick correction, my previous Editor’s Remarks misspelled the name of one of the special issue editors. My thanks to Katherine Graham and Rob Shepherd for their important contribution to the CJPE, and apologies to Katherine for the spelling oversight.
 
As always, please share your thoughts with us, and continue to submit your work to the CJPE. We have made great strides in reducing our publication wait times, thanks to the support of CES and the Social Sciences and Humanities Research Council. We look forward to reading your papers soon! 
 
Isabelle Bourgeois
Editor-in-Chief
 

Fall

Editor’s Remarks

Authors:
 
I am pleased to present this issue of CJPE, which contains two theme segments as well as several practice notes and book reviews. Th e first theme segment, which includes an article by Catherine Fallon in French and an article by Lisa Birch and Steve Jacob in English, was guest edited by Jacob and focuses on democracy and evaluation. The second theme segment features a series of keynote addresses on reconciliation and culturally reponsive evaluation from the 2018 CES National Conference and was guest edited by Nan Wehipeihana. Our keynote speakers have generously put their addresses into writing in order to reach all CES members and other evaluators around the world. I would like to take this opportunity to thank Nicole Bowman-Farrell, Larry Bremner, Debbie Delancey, Kate McKegg, and Nan Wehipeihana for their contributions. In addition to these two segments, this issue also features an important update on contribution analysis by John Mayne, an innovative approach to partnership evaluation by Angèle Bilodeau and Gillian Kranias, an inventory of Canadian evaluation education programs by Theresa Hunter and James McDavid, and an evaluation exemplar by Catherine Fréchette-Simard, Jonathan Bluteau and Isabelle Plante. This issue also includes a research and practice note by Jeremy Acree on the collaborative approaches to evaluation principles, and one by Christopher Cook and collaborators on theatre-based evaluation. Finally, we include two book reviews. I thank our authors for their continued submissions and contributions and hope that this issue provides thought-provoking material for all of our readers.
 
Isabelle Bourgeois Editor-in-Chief
 

Spring

Editor’s Remarks

Authors:

This issue of the Canadian Journal of Program Evaluation ( CJPE) is one of our most comprehensive to date. Not only does it include five full articles, five practice notes, and two book reviews, but it also covers a wide range of evaluation-related topics, practices, and studies. I am pleased to note that our editorial team continues to receive high-quality submissions, and I encourage you to keep thinking of the CJPE as an outlet for your work.

The articles and practice notes included in this issue focus on four recurring themes that reflect current topics in our field. First, evaluative thinking and capacity building in non-governmental organizations is the subject of articles by Rogers, Kelly, and McCoy, as well as by Lu, Elliot, and Perlman. Both articles provide insights into the facilitators of, and barriers to, evaluation capacity building as well as the multiple roles played by evaluators in fostering evaluative thinking amongst organizational staff members. Second, process evaluation appears to be of interest to many evaluators and researchers: Leblanc, Gervais, Dubeau and Delame focus on process evaluation for mental health initiatives, while Parrott and Carman provide an example of how process evaluation can contribute to program scaling-up efforts. Chechak, Dunlop, and Holosko also focus on process evaluation and its utility in evaluating youth drop-in programs. Teachers and students of evaluation may be interested in our third theme, which focuses on student contributions to evaluation, both through peer-mentoring—as described in the practice note written by LaChenaye, Boyce, Van Draanen, and Everett—and through the CES Student Evaluation Case Competition—described in a practice note written by Sheppard, Baker, Lolic, Soni, and Courtney. And fourth, we continue to advance our methodological approaches to evaluation, and this is reflected in an article on evaluation in Indigenous contexts by Chandna, Vine, Snelling, Harris, Smylie, and Manson, as well as in an article on the use of an outcome monitoring tool for performance measurement in a clinical psychology setting by Rosval, Yamin, Jamshidi, and Aubry. Czechowski, Sylvestre, and Moreau also feature methods in their practice note on secure data handling for evaluators, a key competency that continues to evolve as our data collection and storage mechanisms adapt to new technology.

In addition to these articles and practice notes, this issue also features two book reviews that are sure to interest our readers. First, Bhawra provides an account of Developing Monitoring and Evaluation Frameworks , by Patrick Markiewicz (2016), and, second, Sellick reviews Collaborative, Participatory, and Empowerment Evaluation: Stakeholder Involvement Approaches , by David Fetterman, Liliana Rodriguez-Campos, Ann Zukoski, and other contributors (2018).

On behalf of the entire editorial team, I hope that these papers stimulate discussion and reflection and support the advancement of our collective knowledge and practice. As always, if you have feedback on this issue, please contact me. I would love to hear your thoughts!

Fall

Editor's Remarks / Un mot de la rédactrice

Authors:
Pages:
v-vii

In reviewing all of the papers included in this issue of the CJPE, I am struck by the fact that all of them, in their own way, focus on people, organizations, and groups. This is not surprising, given that our work as evaluators requires constant contact and communication with stakeholders, clients, managers, and benefi ciaries. This link to others is often what defines our practice and sets us apart from other disciplines. To start us off, Carman and Fredericks co-author a paper on social network analysis, an approach that is gaining traction in our field. They not only provide a description of how, when, and under what conditions social network analysis can be applied in an evaluation context, but they also summarize useful, practice-based examples to illustrate its potential and its challenges.

I am also pleased to introduce a thematic segment that I co-edited with Marthe Hurteau on stakeholder involvement in evaluation, following a colloquium on this topic held in 2016. As evaluators, we are constantly learning how to best involve stakeholders in our work, and the four papers included in this thematic segment provide new insights from research and practice.

L’implication des parties prenantes dans la démarche évaluative : facteurs de succès et leçons à retenir

Authors:
Pages:
236-246

Our cross-cutting overview of the three papers that make up this thematic segment shows that each of the papers addresses the issue of stakeholder involvement quite differently from the others. We focus here on the key messages from each of these papers in order to highlight success factors and lessons learned for stakeholder participation in evaluation. Success factors related to collaborative approaches to evaluation are also presented throughout the analysis, based on the “Principles guiding collaborative approaches to evaluation” recently published by Shulha, Whitmore, Cousins, Gilbert et Al Hudib (2016).

Special Section: Stakeholder Involvement in Evaluation / Implication des parties prenantes en évaluation

Authors:
Pages:
188

En mai dernier, l’Association francophone pour le savoir (ACFAS) était l’hôte d’un colloque sur la place et le rôle des parties prenantes au sein de la pratique évaluative (Montréal, mai 2016). Plusieurs théoriciens et praticiens ont répondu à l’appel et leurs diverses présentations ont été l’occasion de s’informer sur leurs dernières recherches et réflexions ainsi que d’engager un riche dialogue. Trois présentateurs ont décidé de donner suite à cette journée en soumettant un article que nous regroupons dans le présent numéro de la Revue. Il s’agit de Marie-Pier Marchand, de Diane Dubeau et collaborateurs ainsi que de Sylvain Houle et collaborateurs.

Plus précisément, Marie-Pier Marchand revisite le thème de l’implication des parties prenantes en effectuant un survol de la littérature existante à ce jour. Pour sa part, Diane Dubeau et ses collaborateurs documentent les conditions gagnantes favorisant cette participation en décrivant deux modalités différentes, soit la recherche-action et l’accompagnement soutenu et efficace. Finalement, Sylvain Houle et ses collaborateurs traitent de la dimension relationnelle au sein de la démarche évaluative en introduisant le concept de sagesse pratique.

Nous espérons que cette brève présentation des trois articles saura susciter votre curiosité.

Bonne lecture!

Spring

Editor's Remarks / Un mot de la rédactrice

Authors:
Pages:
v-vii

This issue of The Canadian Journal of Program Evaluation will interest evaluators from many different sectors and with many different interests. The articles and practice notes featured in these pages focus on innovative methodological approaches, applied to various practice settings, such as health and education. First, the article by Rusticus, Eva, and Peterson argues for construct-aligned rating scales as one of the evaluator’s tools, specifically in the area of medical education. The article makes an important contribution by helping us to conceptualize scale development to collect data most efficiently. Next, Rosella and her colleagues show that a team-based knowledge brokering strategy was effective in supporting the use of the Diabetes Population Risk Tool (DPorT) in public health settings. The following article, presented by Chen and his co-authors, summarizes the findings of an empirical comparative study of evaluation models using a large-scale education initiative in Taiwan. This article focuses specifically on the usefulness of evaluation models for planning and development purposes. The article by Contandriopoulos, Larouche, and Duhoux will be of interest to evaluators who work closely with research granting institutions or universities. Using social network analysis methods, these authors have found a positive correlation between collaborations and research productivity, and they pushed their investigation further to consider the role played by formal networks in academic collaborations. The following article, by Mediell and Dionne, presents an evaluation design quality control checklist, developed and validated empirically. The checklist will certainly be of interest to novice and experienced evaluators alike as they design and implement future evaluation studies.

Special Issue

Strategic Evaluation Utilization in the Canadian Federal Government

Authors:
Pages:
327-346

Given the potential of the federal program evaluation function to inform decision-making at the highest levels of government, this project sought to investigate the nature and extent to which program evaluation findings are used as part of spending reviews and other reallocation exercises in selected government organizations. The multiple case study design used in this investigation included a qualitative content analysis of evaluation reports published between 2010 and 2013, as well as a series of key informant interviews conducted with evaluation staff and program managers. The findings show very little evidence of strategic evaluation utilization by organizational leaders. This is thought to be due to a few key factors: (a) the requirements of the 2009 Policy on Evaluation that was in effect at the time of the study; (b) the program-level focus of the evaluations; and (c) the public nature of the evaluation reports.

Editor's Remarks / Un mot du rédacteur

Authors:
Pages:
v-vii

I am honoured to address you for the first time as Editor of the Canadian Journal of Program Evaluation (CJPE). As a former Book Review Editor and Associate Editor of the journal, I am tackling my new responsibilities with a solid sense of all that has been accomplished thus far and great enthusiasm for what is to come next. I want to thank Robert Schwartz for his leadership during his 7-year tenure; his influence and impact on the Journal’s quality and reach will be felt for years to come. I have the privilege of being accompanied on this journey by a skilled and dedicated editorial team made up of Emily Taylor, Astrid Brousselle, Jill Chouinard, and Jane Whynot, four women with considerable combined experience in evaluation, academia, government, and consulting. Two student volunteers have recently joined our team: Hélène Lévesque and Michelle Naimi. Their contributions are already much appreciated. We are grateful for the support and advice of our continuing and new Editorial Board members who have all made a 3-year commitment to the CJPE starting this year. Thank you!

Fall

Measuring Evaluation Capacity in Ontario Public Health Units

Authors:
Pages:
165-183

This article presents a study of organizational capacity to do and use evaluation, conducted in 32 public health units in the province of Ontario. Methods include an organizational self-assessment using an instrument developed by Bourgeois, Toews, Whynot, and Lamarche (2013) as well as key informant interviews. Overall, our findings point to the fact that evaluation capacity is still developing in Ontario public health units; factors that support evaluation capacity in these organizations include the presence of an organization-wide evaluation policy, the availability of full-time evaluation staff in a supporting role, greater staff involvement in evaluation, and a standardized evaluation process. These findings highlight the importance of organizational structures and systems to evaluation utilization and provide potential areas of improvement for organizations wishing to improve their evaluation capacity.