CES Online Learning Goes Live: Welcome to the CES e-Institute!

Volume 31, 2016 - Special Issue

Introduction - Setting the Evaluation Use Context

Authors :
Pages :
275-383

Introduction. This special issue honours Dr. Lyn Shulha’s 25-year contributions to the Canadian field of program evaluation by bringing together the perspectives of authors from across North America to identify Dr. Shulha’s influence on their thinking and evaluation practices. Dr. Shulha’s scholarship is best described as a nonlinear influence because the effect of her work on evaluators’ thinking about collaboration, use, standards, and innovation cannot be directly traced.

Log in to post comments or view the article

Developing the Program Evaluation Utility Standards: Scholarly Foundations and Collaborative Processes

Authors :
Pages :
284-304

Developing the third edition of the program evaluation utility standards required multilevel collaborations among task force members, members of the Joint Committee on Standards for Educational Evaluation, sponsoring organizations, and hundreds of involved stakeholders. The scholarship on evaluation use, infl uence, and collaboration was foundational for the utility standards and materials accompanying them and equally important for informing the processes guiding utility standards development. This article emphasizes the foundational role of this recent scholarship and the roles played by all who collaborated in planning and implementing the utility standards development processes.

Log in to post comments or view the article

Optimizing Use in the Field of Program Evaluation by Integrating Learning from the Knowledge Field

Authors :
Pages :
305-327

It has been almost 20 years since Shulha and Cousins (1997) published their seminal paper exploring evaluation use. The paper examined a decade, 1986 to 1996, of theory, practice, and research on evaluation use. Since that time there have been significant developments related to the phenomenon of evaluation use. Outside of evaluation a new and burgeoning field has focused on the use of research in practice and policy; in health care the term knowledge translation has been used and in social sciences knowledge mobilization. Despite the rapidly growing body of research from the knowledge field, the different terminology used in evaluation, health care, and the social sciences has created siloed bodies of knowledge, even when working on similar change processes. This may be one of the factors why the large body of literature on evaluation use has received little attention in health care and vice versa. The aim of this article is threefold: first, to examine the developments in evaluation use since Shulha and Cousins’s (1997) paper; second, to explore how the knowledge fields, focusing on knowledge translation and mobilization, can help to further refine and develop our understanding of use; and third, to imagine what future research that interweaves the knowledge field with the field of program evaluation might look like and how it has the potential to serve the contexts where this research would be conducted.

Log in to post comments or view the article

Reflections on the Meaning of Success in Collaborative Approaches to Evaluation: Results of an Empirical Study

Authors :
Pages :
328-349

How do evaluators using collaborative approaches to evaluation (CAE) define success? This is the core question being asked in a further analysis of data from our previous work (Cousins, Whitmore, & Shulha, 2013 ; Shulha et al., 2016) that developed a set of evidence-based principles to guide collaborative evaluation practice. Probing data from 320 responses to our (2012) survey, we examined what respondents considered "highly successful" and "less successful than hoped" in their collaborative evaluation projects. The results revealed that evaluation use, relationships, and information needs are key factors. We propose a conceptual framework as an aid to thinking about success in CAE.

Log in to post comments or view the article

A Case Study of the Guiding Principles for Collaborative Approaches to Evaluation in a Developmental Evaluation Context

Authors :
Pages :
350-373

Recently, Shulha, Whitmore, Cousins, Gilbert, and al Hudib (2015) proposed a set of evidence-based principles to guide collaboration. Our research undertakes a case study approach to explore these principles in a developmental evaluation context. Data were collected at two points in an 18-month period where an evaluation group collaborated with the program team from a national organization. This article explores the contributions of selected collaborative approaches to evaluation principles as they are applied in a developmental evaluation. The article concludes with a reflection on the implications for collaboration in theory and practice of developmental contexts. Also identified are the practical insights for implementing the principles in evaluation practice.

Log in to post comments or view the article

Influential Mentoring Practices for Navigating Challenges and Optimizing Learning During an Evaluation Internship Experience

Authors :
Pages :
374-396

The increased complexity of contexts that Canadian evaluators work in has important implications for evaluation education. Internship is a valued training component, yet what remains to be identified are empirically based quality indicators of the experience. Analyses of interviews with an intern, mentor, and coordinator supplemented by field notes revealed key features suggesting three influential mentoring practices: orientation to workplace context, autonomy of supervisory approach, and planning for evaluation agility. Implications for evaluation practice and evaluator induction are discussed in light of the Competencies for Canadian Evaluation Practice and three areas influenced by Dr. Lyn Shulha.

Log in to post comments or view the article

The Oral History of Evaluation: An Interview with Lyn Shulha

Authors :
Pages :
397-408

Lyn Shulha is a Canadian evaluator whose scholarship in evaluation use, collaboration in evaluation, and evaluation standards development has shaped evaluation theory and practice in Canada and abroad. This article presents the transcript to an interview conducted with Dr. Shulha conducted in the tradition of oral history studies. Dr. Shulha identifies formative experiences in her development as an evaluator that shaped the trajectory of her thinking on evaluation theory and practice.

Log in to post comments or view the article

In Tribute to Lyn Shulha: The Authentic Evaluator

Authors :
Pages :
409-415

With great honour, I offer a modest commentary on the articles in this special issue of the Canadian Journal of Program Evaluation, complemented by reflections on my decades of interactions with and memories of Lyn. My commentary underscores the enduring legacy of Lyn's significant contributions to the field of evaluation. These contributions largely, though not exclusively, reside in three domains: collaborative approaches to evaluation, the field's deep commitment to evaluation use, and the substance and contributions of meaningful standards for evaluation practice. My reflections honour Lyn's kindness, practical scholarship, integrity, and joyful engagement with the full richness of life.

Log in to post comments or view the article