Introduction - Setting the Evaluation Use Context
Introduction. This special issue honours Dr. Lyn Shulha’s 25-year contributions to the Canadian field of program evaluation by bringing together the perspectives of authors from across North America to identify Dr. Shulha’s influence on their thinking and evaluation practices. Dr. Shulha’s scholarship is best described as a nonlinear influence because the effect of her work on evaluators’ thinking about collaboration, use, standards, and innovation cannot be directly traced.
Developing the Program Evaluation Utility Standards: Scholarly Foundations and Collaborative Processes
Developing the third edition of the program evaluation utility standards required multilevel collaborations among task force members, members of the Joint Committee on Standards for Educational Evaluation, sponsoring organizations, and hundreds of involved stakeholders. The scholarship on evaluation use, infl uence, and collaboration was foundational for the utility standards and materials accompanying them and equally important for informing the processes guiding utility standards development. This article emphasizes the foundational role of this recent scholarship and the roles played by all who collaborated in planning and implementing the utility standards development processes.
Optimizing Use in the Field of Program Evaluation by Integrating Learning from the Knowledge Field
It has been almost 20 years since Shulha and Cousins (1997) published their seminal paper exploring evaluation use. The paper examined a decade, 1986 to 1996, of theory, practice, and research on evaluation use. Since that time there have been significant developments related to the phenomenon of evaluation use. Outside of evaluation a new and burgeoning field has focused on the use of research in practice and policy; in health care the term knowledge translation has been used and in social sciences knowledge mobilization. Despite the rapidly growing body of research from the knowledge field, the different terminology used in evaluation, health care, and the social sciences has created siloed bodies of knowledge, even when working on similar change processes. This may be one of the factors why the large body of literature on evaluation use has received little attention in health care and vice versa. The aim of this article is threefold: first, to examine the developments in evaluation use since Shulha and Cousins’s (1997) paper; second, to explore how the knowledge fields, focusing on knowledge translation and mobilization, can help to further refine and develop our understanding of use; and third, to imagine what future research that interweaves the knowledge field with the field of program evaluation might look like and how it has the potential to serve the contexts where this research would be conducted.
Reflections on the Meaning of Success in Collaborative Approaches to Evaluation: Results of an Empirical Study
How do evaluators using collaborative approaches to evaluation (CAE) define success? This is the core question being asked in a further analysis of data from our previous work (Cousins, Whitmore, & Shulha, 2013 ; Shulha et al., 2016) that developed a set of evidence-based principles to guide collaborative evaluation practice. Probing data from 320 responses to our (2012) survey, we examined what respondents considered "highly successful" and "less successful than hoped" in their collaborative evaluation projects. The results revealed that evaluation use, relationships, and information needs are key factors. We propose a conceptual framework as an aid to thinking about success in CAE.
A Case Study of the Guiding Principles for Collaborative Approaches to Evaluation in a Developmental Evaluation Context
Recently, Shulha, Whitmore, Cousins, Gilbert, and al Hudib (2015) proposed a set of evidence-based principles to guide collaboration. Our research undertakes a case study approach to explore these principles in a developmental evaluation context. Data were collected at two points in an 18-month period where an evaluation group collaborated with the program team from a national organization. This article explores the contributions of selected collaborative approaches to evaluation principles as they are applied in a developmental evaluation. The article concludes with a reflection on the implications for collaboration in theory and practice of developmental contexts. Also identified are the practical insights for implementing the principles in evaluation practice.
Influential Mentoring Practices for Navigating Challenges and Optimizing Learning During an Evaluation Internship Experience
The increased complexity of contexts that Canadian evaluators work in has important implications for evaluation education. Internship is a valued training component, yet what remains to be identified are empirically based quality indicators of the experience. Analyses of interviews with an intern, mentor, and coordinator supplemented by field notes revealed key features suggesting three influential mentoring practices: orientation to workplace context, autonomy of supervisory approach, and planning for evaluation agility. Implications for evaluation practice and evaluator induction are discussed in light of the Competencies for Canadian Evaluation Practice and three areas influenced by Dr. Lyn Shulha.
The Oral History of Evaluation: An Interview with Lyn Shulha
Lyn Shulha is a Canadian evaluator whose scholarship in evaluation use, collaboration in evaluation, and evaluation standards development has shaped evaluation theory and practice in Canada and abroad. This article presents the transcript to an interview conducted with Dr. Shulha conducted in the tradition of oral history studies. Dr. Shulha identifies formative experiences in her development as an evaluator that shaped the trajectory of her thinking on evaluation theory and practice.
In Tribute to Lyn Shulha: The Authentic Evaluator
With great honour, I offer a modest commentary on the articles in this special issue of the Canadian Journal of Program Evaluation, complemented by reflections on my decades of interactions with and memories of Lyn. My commentary underscores the enduring legacy of Lyn's significant contributions to the field of evaluation. These contributions largely, though not exclusively, reside in three domains: collaborative approaches to evaluation, the field's deep commitment to evaluation use, and the substance and contributions of meaningful standards for evaluation practice. My reflections honour Lyn's kindness, practical scholarship, integrity, and joyful engagement with the full richness of life.