It is time to vote for your preferred candidate for the Position of Vice-President for 2017-2019.

Volume 30, 2015 - Fall

Editor's Remarks / Un mot du rédacteur

Authors :

Preparing the editor’s remarks for  CJPE  30(2) has brought a big smile to my soul. This issue is jam-packed with gems. Covering a wide range of hot and less hot topics, the issue has more than a little bit of something in it for all readers from the many corners of the evaluation community in Canada and abroad. [...]

Useful Theory of Change Models

Authors :
Pages :
119-142

Although theories of change are frequently discussed in the evaluation literature and there is general agreement on what a theory of change is conceptually, there is actually little agreement beyond the big picture of just what a theory of change comprises, what it shows, how it can be represented, and how it can be used. This article outlines models for theories of change and their development that have proven quite useful for both straightforward and more complex interventions. The models are intuitive, fl exible, and well-defined in terms of their components, and they link directly to rigorous models of causality. The models provide a structured framework for developing useful theories of change and analyzing the intervention they represent.

Meeting at the Crossroads: Interactivity, Technology, and Evaluation Utilization

Authors :
Pages :
143-159
This article is a review and integration of evaluation utilization literature with a new focus on the use of technology to increase evaluation utility. Scholarship on evaluation utilization embodies one of the major and ongoing quandaries in the evaluation profession: What constitutes usefulness and relevance to stakeholders? We think that a constructivist lens is helpful in making sense of the trajectory this literature has taken, where what is "useful" and what culminates in "use" have become much more fl exible notions that are in a constant state of negotiation between evaluators and evaluation stakeholders. We posit that it may be important for evaluators who are closely engaged with stakeholders to pay greater attention to this interactivity to build a common vision of what is "useful" at that moment in time. While this is no small task, we posit that evaluators may have something to gain by exploring the wealth of digital technologies and social media tools that are available. The use of these tools in local level, participatory-oriented contexts may be valuable for encouraging interactivity, potentially encouraging learning, creativity, and ownership. This article aims to stress that integrating technology into everyday evaluation practice, where possible, may ultimately enhance evaluation usefulness and relevance.

Increasing the Utility of AAMC Canadian Graduation Questionnaire Data for Program Evaluation: Subscale Development

Authors :
Pages :
160-176

The AAMC Canadian Graduation Questionnaire (CGQ) is widely used for evaluating undergraduate MD programs; however, the analysis of individual items limits its usefulness. To aid in the management and interpretation of the CGQ for use in program evaluation, this study combined items from the sections on clinical learning experiences, physician competencies, and student services into scales and examined their internal structure and reliability. Factor analyses conducted on data from 517 undergraduate medical students supported combining the items into 15 scales. Two examples illustrate how the scales can be used to evaluate student experiences for different cohorts over time and across sites.

Challenges in Evaluating a Prototype Project in a Large Health Authority: Lessons Learned

Authors :
Pages :
177-194

A developmental evaluation was undertaken to evaluate a prototype test of a new model of perinatal healthcare across acute maternity, public health, and primary care in two hospitals in a large health authority. The project was initiated to bridge gaps in care across the acute and community settings to ensure a seamless perinatal healthcare journey for women. The objective of the evaluation was to support the prototyping process by providing data to inform decisions as the prototype was developed and by documenting decisions as they were made. This article explores challenges faced during the evaluation, including unfamiliarity of the health sector with prototype projects and their inherent uncertainty, a disconnect between the rapid pace of a prototype project and bureaucratic hurdles of working within a large organization, and high leadership turnover throughout the project. How these challenges were addressed, and the lessons learned for future evaluations, are discussed.

Implementation and Evaluation of an Evidence-Based Treatment of Disruptive Behaviour within a Children's Mental Health Program

Authors :
Pages :
195-204

This practice note details the synchronous implementation and developmental evaluation of a novel model of care that integrates an evidence-based treatment of children with disruptive behaviour within a community-based children's mental health centre in Canada. The overall objective of the project was to evaluate the impact and viability of the integrated program. A multidisciplinary team used a "knowledge to action" framework to guide implementation activities and a developmental evaluation approach to support learning and innovation, as well as measure program success. Key insights and lessons learned with regards to building staff capacity and appreciation for evaluation, balancing clinical and evaluation practices, and sustaining evaluation procedures are discussed. Suggestions for future similar efforts within a children's mental health context are provided.

Inspiring Future Program Evaluators through Innovative Curriculum for Undergraduates

Authors :
Pages :
205-215

Instruction in program evaluation is challenging given the inherent interdisciplinary nature of the field. As well, there is no one discipline typically dedicated to evaluation training, and few formal programs and university course offerings exist. Despite these limitations, training and education at the postsecondary level continues to be vital in supporting the professionalization of program evaluation, especially as it is a requirement for credentialing. The current article presents an innovative project comprising both education and hands-on training of program evaluation practices for undergraduate students. The project involved in-class lectures targeting specific program evaluation competencies and a program evaluation assignment in an upper-level undergraduate psychology course. Students were asked to develop a logic model and identify psychometrically sound evaluation measures based on an existing community organization's program or on a theoretical example. At the end of the course, students ( N = 58) completed surveys to assess their achieved evaluation competencies and experience with program evaluation. Overall, students gained evaluation-specific skills and knowledge, and the assignment was successful in promoting interest in program evaluation as a discipline. It is our hope that the current project can support faculty to integrate program evaluation in engaging and meaningful ways into their own curriculum.

Évaluation de la qualité d'un partenariat dans le cadre de la mise en place d'une intervention intersectorielle de type école, famille et communauté

Authors :
Pages :
216-227

The partnership represents a principle of action at the heart of public policies of various kinds. It can be defined as a formal association between organizations that decide to cooperate to achieve a common goal. The formative evaluation of quality conditions of the collaborative action is useful to support partners in improving their practice. This article describes first a tool available to achieve a collective diagnosis by members of a partnership investing voluntarily in an evaluation process and then presents a concrete application of it. It concludes by exposing some knowledge and lessons learned from the experience of the evaluation of a partnership for a comprehensive and integrated cross-sectoral school-community intervention.

Book Reviews: Trevisan, M. S., & Walser, T. M. (2015) Evaluability assessment: Improving evaluation quality and use. Thousand Oaks, CA: Sage. 181 pages. ISBN 978-1452282442.

Authors :
Pages :
228-230

Book Reviews: Trevisan, M. S., & Walser, T. M. (2015) Evaluability assessment: Improving evaluation quality and use. Thousand Oaks, CA: Sage. 181 pages. ISBN 978-1452282442.

Book Reviews: Donaldson, S. L. (Ed.). (2013). The future of evaluation in society: A tribute to Michael Scriven. Charlotte, NC: Information Publishing. ISBN 978-1-62396-451-1.

Authors :
Pages :
231-233

Book Reviews: Donaldson, S. L. (Ed.). (2013). The future of evaluation in society: A tribute to Michael Scriven. Charlotte, NC: Information Publishing. ISBN 978-1-62396-451-1.