I am pleased to introduce this special issue, guest edited by John M. LaVelle and Jill A. Chouinard. Evaluation education is at the heart of what we do, whether we teach in university settings, offer professional development and training courses, or mentor colleagues who are new to our field. Indeed, the work of an evaluator oft en involves educating others on the value, process, and impacts of evaluation; stakeholders, decision-makers, and users are all potential students of evaluation, as are current and future evaluators. Chouinard and LaVelle have successfully brought together evaluation educators from diverse backgrounds and perspectives to describe how evaluation can be taught and learned. This innovative volume is also the first issue of CJPE to be entirely composed of Practice Notes, and it provides easily transferable ideas and lessons from which our community will undoubtedly benefit. I am grateful to the authors of these 15 Practice Notes who have agreed to share their thoughts, experiences, and tools with us and to our two guest editors for managing this ambitious project. Please share with us your key takeaways and all of the ways in which you’ve implemented these ideas.
Why a Special Issue of Practice Notes about How to Teach Evaluation?: Introducing This Special Issue of The Canadian Journal of Program Evaluation
Introducing this special issue of the Canadian Journal of Program Evaluation.
UEval: Bringing Community-Based Experiential Learning to the Evaluation Classroom
This practice note describes the unique features of, and lessons learned from UEval, an innovative one-week ‘evaluation institute’ at the Faculty of Extension, University of Alberta. This initiative responds to an identified community and university need to provide contextual and socially relevant evaluation-focused educational opportunities to learners across disciplines and sectors. UEval is a guided by principles of participatory and experiential learning, and brings together undergraduates, graduates and community professionals as co-learners, for a mutually beneficial and sustainable educational experience. During the week, learners explore and apply evaluation theory through project-based curricula, and develop evaluative responses to community-informed issues.
Competency-Based Evaluation Education: Four Essential Things to Know and Do
Evaluator education must provide robust opportunities to support and assess the progressive, lifelong development of relevant knowledge and skills. If we wish to keep pace with the increasingly complex contexts in which evaluators operate, we need to better align our educational approaches with the global movement towards practice competencies guiding the profession. Key among the challenges is the lack of instructional guidance specific to a competency-based approach to evaluator education. In this practice note, we orient readers to the value of competency-based evaluation education and describe the teaching context using a systems perspective to examine the dynamic learning interactions and experiences. We advance four essential instructional features of the competency-based approach revealed by a study documenting the impacts on learning and student experiences. We conclude with lessons learned from reflecting upon our experiences during the development and implementation of a competency-based doctoral-level evaluation course to highlight the mutual benefits for learners and instructors.
The Role of Evaluative Thinking in the Teaching of Evaluation
In this practice note, I share some reflections on the role of evaluative thinking in the teaching of evaluation. I teach an introductory graduate-level evaluation course to Master’s and Doctoral students and also provide ECB workshops to various community-based education organizations, non-governmental organizations, non-profits, foundations, and groups of emerging evaluators. In this practice note I use a reflective case study augmented with analysis against salient theoretical frameworks to reflect on evaluative thinking as a way: (1) to balance teaching theory and practice; (2) to infuse adult education principles and practices, which can also help learners hone in on the potential for evaluation for social justice; and (3) to equip students for continuing professional development—to help them become life-long learners in evaluation.
Praxis Makes Perfect? Transcending Textbooks to Learning Evaluation Experientially and in Cultural Contexts
The theory-to-practice loop is riddled with gaps, incongruencies, and, at times, trauma when it comes to the professional development and practice of evaluators. Our current system of professional development for evaluators systemically and institutionally reinforces racism, white privilege, and misogyny, thus re-creating harm and the barriers that so many BIPOC and LGBTQ2S evaluators are working hard to overcome. This article provides the reader with an alternative to the field’s valuing and learning evaluation within “institutions of higher education” and other “formal” and “scholarly” learning spaces. Rather, it provides for a balanced approach of experiential learning in the fi eld and within cultural contexts as a much-needed professional design component for developing responsive, effective, and transformative evaluators. Praxis and experience should have at least equal value, merit, and worth for developing current and upcoming evaluators. When done correctly, wisdom to evaluative thinking, development, and practice happens, and not simply reinforcing and generating the same evaluative voices, constructs, and behaviours of the privileged evaluation patriarchy
Establishing and Developing Professional Evaluator Dispositions
A basic principle associated with competent evaluation practice asserts that evaluators should possess the education, abilities, skills, and experience needed to undertake the tasks proposed in an evaluation. For those training evaluators this also means teaching professional dispositions, because an individual’s dispositions (i.e., beliefs and values) influence how they will act as professionals. In order for evaluation educators to teach evaluator competencies, we must understand the underlying dispositions associated with each competency. We must also identify which dispositions are essential for professional practice. This paper discusses the issue of dispositions and the challenges of helping evaluators develop positive professional dispositions.
Strategies for Mentoring and Advising Evaluation Graduate Students of Color
While evaluators have many intersecting identities, ethnicity remains the most salient identity for evaluators of color. As formal graduate training in evaluation continues to expand so too does the number of students from ethnic minoritized populations, who are in need of specialized mentoring and advising. Drawing from previous research on evaluation, higher education literature, and personal reflections from the author, an Afro-Latina faculty member, this practice note outlines five strategies for mentoring and advising evaluation graduate students of color. These include: 1) Consider the impact of vicarious trauma, 2) Assist with the facilitation of peer and mentors ‘squads’, 3) Respect, honor, and celebrate students’ culture, religion, and families, 4) Be vigilant of microaggressions and practice microvalidations, and 5) Develop mentoring competence. Each strategy is presented along with reflections and practical examples for implementation.
Offering Graduate Evaluation Degrees Online: Comparing Student Engagement in Two Canadian Programs
This Practice Note describes and explores the experiences and lessons learned engaging students in two online graduate evaluation programs offered in Canadian universities: the University of Victoria Graduate Certificate and Diploma in Evaluation Program, offered since 2010; and, Carleton University Graduate Diploma in Public Policy and Program Evaluation (DPPPE), offered online since 2016. Both programs are delivered to maximize the geographic accessibility of graduate evaluation education within and outside of Canada. Each program uses different teaching and learning strategies but there are four lessons learned that are common to the two programs: set and meet (or exceed) clearly stated student expectations; set and then model a respectful and inclusive tone in online discussions; stretch online discussions by taking advantage of student expertise and experience; and use adult-oriented and rigorous teaching and learning methods that engage these mature learners.
Teaching Africa-Rooted Evaluation: Using a “Model Client” Innovation to Help Shift the Locus of Knowledge Production
Recent years have seen the emergence in both academic generally and evaluation specifically a strong “Made in Africa” discourse, urging us to critically reflect on how we might integrate African methods, culture and knowledge systems into both teaching and practice. This teaching practice note reflects on one small, but potentially significant step towards this through a curriculum redesign of a core introductory module on University of Cape Town’s Masters in Program Evaluation. Our idea, which we call a “model client” approach, was to bring on board the evaluation client as a co-learner in the classroom environment. Through a series of instructor-facilitated client-student engagements, students and client worked within the classroom environment on understanding the program logic, tailoring evaluation questions, and co-learning about evaluation approach. While not without its challenges, our model client approach made meaningful strides towards moving the locus of evaluation knowledge creation away from a theoretically grounded introductory course which drew predominantly on Western texts and theory, towards an approach where both our understanding of the evaluation process and evaluation capabilities themselves are co-created by (our uniquely African) clients, students, and instructors. Key challenges in implementing this approach included the client’s sense of vulnerability, student inexperience in evaluation theory and practice, and a conspicuous shortage of African-generated evaluation case studies and texts. Reflections for addressing these challenges include the need for teaching instructors to better centre student and clients learning around the objectives of the model client initiative, better communication as to the central principles of Made in Africa evaluation, and continuing to support the development if uniquely “indigenised” African evaluation scholarship and source materials.
If Building Trust Is Important, How Do We Teach Novice Evaluators to Do It?
People and relationships matter in evaluation. While our literature is replete with examples and guidance about how one might go about using interpersonal skills in practice and the reasons why these skills are important, the pedagogy of interpersonal skill development regarding evaluation remains underdeveloped. In this practice note, an evaluation educator shares an intentional, purposeful, and ongoing activity for helping novice evaluators learn to build trust with stakeholders during a semester-long evaluation practice course. An explanation for why the focus is on trust is presented before describing the learning activity itself. Next, a discussion of factors related to the enactment of this learning activity, including implications for evaluation pedagogy, are presented.
Helping Students Reflect on Their Interpersonal Skills: The Team Performance Scale (TPS)
Many instructors of program evaluation incorporate team-based service projects into their courses. What is often overlooked is that it provides a golden opportunity for students to reflect on the interpersonal skills necessary for teamwork. This paper explains how two instructors leveraged service project working groups to include reflection on interpersonal skills. Minor changes were made to the Team Performance Scale (TPS), a tool used for Team-Based Learning in medical education. We found it to be a practical and effective way to engage students in reflecting on behaviours associated with a high functioning team. This article will be of interest to instructors using team learning approaches to teach key competencies in program evaluation.
Using People Styles for Interpersonal Competence: Encouraging Purposeful Reflection on Communication Behaviors
The purpose of this article is to revitalize the idea of soft skills as an important part of evaluator training through reflecting on the use of the People Styles Assessment Inventory (Bolton & Bolton, 2009) as a reflective exercise in a course with masters and Ph. D. level evaluation and measurement students. This paper provides an overview of the recent conversation in the literature surrounding interpersonal competency in evaluation, highlighting the need for and lack of emphasis on interpersonal skills in evaluation training, then describes the teaching practice context for engaging in the People Styles assessment and reflective activity. The People Styles model is described in detail along with key reflections on the use of the model for training future evaluation practitioners.
Fieldwork Experience as Cultural Immersion: Two International Students and Their Professor Reflect on a Recent Evaluation Practicum
In this Practice Note, two international students (one from Latin America and the other from West Africa), reflect on their first evaluation experience gained through an evaluation practicum course. The paper includes a reflection on four main cultural challenges faced by these international students related to immersion in the culture and program context, interpersonal and communication skills, learning the language of evaluation and telling the story. The course professor, responsible for selecting projects and providing guidance to students throughout the semester, provides further reflections about the students’ perspectives and the challenges of teaching evaluation to international students.
Collaborative Evaluation Designs as an Authentic Course Assessment
Strategies for optimizing evaluator education is an ongoing discussion in the field of evaluation. While several options exist, formal post-secondary courses are limited in Canada. Graduate courses offered at post-secondary institutions must navigate institutional structures while creating learning opportunities that bring together the theoretical and practical competencies required for evaluators. In this practice note, we advance evaluation designs as a useful authentic assessment. First, we provide a brief look at the literature related to authentic assessments, followed by a description of the contexts where we apply the evaluation design. Next, we offer details of four processes spanning planning and implementation through to final evaluation. Finally, we conclude with insights from our experiences of utilizing the evaluation design as an authentic assessment.
Pinpointing Where to Start: A Reflective Analysis on the Introductory Evaluation Course
This reflective analysis details four approaches to an introductory course for evaluation learners within a methodologically focused graduate-level program on statistics, measurement, and research design. Evidence of student learning outcomes, or SLOs, was utilized within Gibbs’ reflective cycles to redesign the course using Fink’s integrated course design process. The purpose of each approach varied along a theory-practice continuum, including theory, theory-to-practice, practice, and evidence building. The purpose, SLOs, and learning experiences of each approach are accompanied by longitudinal reflections on evaluation learners, course purposes, and the creation of a multi-course learning progression. This exploration offers perspectives and lessons learned that may assist new and experienced instructors in determining how an introductory course may best fit the learning needs of their students.
In Their Own Words: Student Key Learning Experiences in an Introductory Evaluation Course
Student reflections on their key learning experiences in evaluation courses can be useful data for educators seeking to understand which aspects of their teaching practice are most impactful. This practice note describes how we analyze students’ reflections on their key learning experiences using 1) grounded theory, 2) Fink’s taxonomy of significant learning (2013), and 3) the American Evaluation Association’s Competencies framework (2018), and how we use the frameworks and data to understand our teaching of evaluation practice.
Re-Envisioning Evaluation Pedagogy with a Community of Scholar Teachers
Concluding chapter for the special issue
Peer Reviewers for Volume 35 and Manuscripts Submitted in 2020