Editor's Remarks / Un mot de la rédactrice
In reviewing all of the papers included in this issue of the CJPE, I am struck by the fact that all of them, in their own way, focus on people, organizations, and groups. This is not surprising, given that our work as evaluators requires constant contact and communication with stakeholders, clients, managers, and benefi ciaries. This link to others is often what defines our practice and sets us apart from other disciplines. To start us off, Carman and Fredericks co-author a paper on social network analysis, an approach that is gaining traction in our field. They not only provide a description of how, when, and under what conditions social network analysis can be applied in an evaluation context, but they also summarize useful, practice-based examples to illustrate its potential and its challenges.
I am also pleased to introduce a thematic segment that I co-edited with Marthe Hurteau on stakeholder involvement in evaluation, following a colloquium on this topic held in 2016. As evaluators, we are constantly learning how to best involve stakeholders in our work, and the four papers included in this thematic segment provide new insights from research and practice.
Applications of Social Network Analysis in Evaluation: Challenges, Suggestions, and Opportunities for the Future
As the use of social network analysis in evaluation continues to increase, it is important to understand how, when, and under what conditions social network analysis can add value to evaluation work. In this article, we describe how we have used social network analysis in various evaluation projects. Using the experience of one specific project, we highlight, in greater detail, some challenges we encountered in doing this work, relating to the need for stakeholders to understand the added value of social network analysis, the intricacies of data coding and cleaning, and how changes in the size and scope of the project can have great implications. Finally, we offer some practical suggestions for evaluators considering incorporating social network analysis into their work today, and identify opportunities where evaluators might use social network analysis in the future.
Special Section: Stakeholder Involvement in Evaluation / Implication des parties prenantes en évaluation
En mai dernier, l’Association francophone pour le savoir (ACFAS) était l’hôte d’un colloque sur la place et le rôle des parties prenantes au sein de la pratique évaluative (Montréal, mai 2016). Plusieurs théoriciens et praticiens ont répondu à l’appel et leurs diverses présentations ont été l’occasion de s’informer sur leurs dernières recherches et réflexions ainsi que d’engager un riche dialogue. Trois présentateurs ont décidé de donner suite à cette journée en soumettant un article que nous regroupons dans le présent numéro de la Revue. Il s’agit de Marie-Pier Marchand, de Diane Dubeau et collaborateurs ainsi que de Sylvain Houle et collaborateurs.
Plus précisément, Marie-Pier Marchand revisite le thème de l’implication des parties prenantes en effectuant un survol de la littérature existante à ce jour. Pour sa part, Diane Dubeau et ses collaborateurs documentent les conditions gagnantes favorisant cette participation en décrivant deux modalités différentes, soit la recherche-action et l’accompagnement soutenu et efficace. Finalement, Sylvain Houle et ses collaborateurs traitent de la dimension relationnelle au sein de la démarche évaluative en introduisant le concept de sagesse pratique.
Nous espérons que cette brève présentation des trois articles saura susciter votre curiosité.
Où en sommes-nous avec l’implication des parties prenantes?
This paper focuses on stakeholders’ involvement in program evaluation and provides an overview of the current literature on this topic. The paper aims to conceptualize stakeholders’ involvement from the definition of key dimensions related to this theme, in particular by addressing the concept of stakeholders, discussing different approaches and various forms of stakeholders’ involvement, and identifying criteria for selection. The author tries to highlight the added value of using an evaluation process involving stakeholders, while exposing the issues related to such an exercise.
Les conditions gagnantes et les défis pour une participation active des parties prenantes dans un contexte d’évaluation de programmes d’intervention psychosociale
In order to optimize the appropriateness, utilization, and appropriation of an evaluation process, which is often expensive in terms of time and money, we have recently encouraged the increasingly active involvement of various stakeholders (Cousins et Chouinard, 2012). Beyond this orientation, there are important issues concerning the selection and the degree of participation of these stakeholders, as well as obstacles and conditions facilitating their participation (Hurteau, Houle et Marchand, 2012). Firstly, this article is based on the evaluation of Relais-Pères, carried out as part of participatory action research over more than ten years in order to specify the conditions and obstacles encountered in a framework of services. Secondly, attention is given to issues linked to the specifics of an evaluative approach in the interest of appropriation by partners in this approach.
L’évaluateur et la sagesse pratique : vecteurs essentiels pour assurer la crédibilité de l’évaluation
This article revisits data collected during previous research conducted by Hurteau and Houle (2008). At the time, the researchers were not able to account for the evaluator’s personal skills to pilot an evaluation, the results of which were acceptable to the stakeholders, and to conduct a comprehensive analysis, for lack of an appropriate theoretical framework. By introducing the concept of practical wisdom, House (2015) offers them the opportunity to do so. The author defines this concept as “... doing the right thing in the special circumstances of performing the job” (p. 88). In addition, Schwartz and Sharpe (2010) offer criteria used to establish its presence. The current approach consisted of analyzing the testimony of an experienced police officer. The results helped to improve our understanding of the concept, highlighting the contribution of Telos (moral values) as well as additional criteria such as the importance of time for reflection and the need to supplement the information where necessary.
L’implication des parties prenantes dans la démarche évaluative : facteurs de succès et leçons à retenir
Our cross-cutting overview of the three papers that make up this thematic segment shows that each of the papers addresses the issue of stakeholder involvement quite differently from the others. We focus here on the key messages from each of these papers in order to highlight success factors and lessons learned for stakeholder participation in evaluation. Success factors related to collaborative approaches to evaluation are also presented throughout the analysis, based on the “Principles guiding collaborative approaches to evaluation” recently published by Shulha, Whitmore, Cousins, Gilbert et Al Hudib (2016).
Three Steps Toward Sustainability: Spreadsheets as a Data-Analysis System for Non-Profit Organizations
Many non-profits face barriers developing systems to collect and analyze data that can leverage the type of information that their funders and stakeholders require. Constraints such as limited evaluation expertise, time, and money make this virtually impossible to achieve without a viable solution. In an increasingly competitive environment, it is imperative that non-profits find innovative ways to track and measure their work within their evaluative capabilities. There are different ways in which evaluators can help even the most constrained non-profit organizations capture their reach and make the most of their existing data. This article proposes a three-step framework for the development of a data-collection and -analysis system through the use of spreadsheets. Not only is this proposed system feasible within the constraints of the non-profit sector, but it is also valuable for the sustainability of their services over time.
Readiness in Evaluation: Three Prompts for Evaluators
The notion of readiness in evaluation is often tacit: it can be hidden in the Request for Proposals by a granting agency calling for an evaluation; or, at best, it is hinted at in the winning consultant’s bid. We are practitioners committed to learning-oriented, practical evaluations. However, we have found that the extent to which the client is ready for a collaborative, utilization-focused evaluation (UFE) that enhances organizational learning is often taken for granted. We have learned to address “readiness” early on, as it is a lynchpin that shapes every subsequent step of the process. We bring attention to examples from our practice to highlight three dimensions of readiness: the power to design, the commitment to learn, and building an evaluation culture.
One-Room School: The Summer Institute in Program Evaluation
This article describes the Summer Institute in Program Evaluation conceived and organized in partnership by University of Manitoba faculty members, four partnering organizations in the health and social-service sector, and a university student group. It explains the model of learning, which involves the development of real-life evaluation plans in a week-long intensive course that includes university students and community participants. The article concludes by detailing a series of lessons learned.
Book Review: Chris Fox, Robert Grimm, and Rute Caldeira. (2017). An Introduction to Evaluation . Thousand Oaks, CA: SAGE.
Book Reviews / Comptes rendus de livres
Chris Fox, Robert Grimm, and Rute Caldeira. (2017). An Introduction to
Evaluation . Thousand Oaks, CA: SAGE. 305 pages.
Book Review: David Grembowski. (2016). The Practice of Health Program Evaluation. Thousand Oaks, CA: SAGE.
Book Reviews / Comptes rendus de livres
David Grembowski. (2016). The Practice of Health Program Evaluation.
Thousand Oaks, CA: SAGE. Paperback, 324 pages.
Book Review: Rosalind Eyben, Irene Guijt, Chris Roche, and Cathy Shutt. (2015). The Politics of Evidence and Results in International Development: Playing the Game to Change the Rules?
Book Reviews / Comptes rendus de livres
Rosalind Eyben, Irene Guijt, Chris Roche, and Cathy Shutt. (2015). The Politics of Evidence and Results in International Development: Playing the Game to Change the Rules?
Bourton on Dunsmore, England: Practical Action Publishing Ltd. Kindle Edition.