The Canadian Journal of Program Evaluation

 

34.3 Special Issue 2020 

I am pleased to introduce this special issue, guest edited by Robert Shepherd and Katherine Graham.

The guest editors have assembled a number of important contributions for this volume, which provide a legal and epistemological framing for evaluating in Indigenous contexts as well as examples of how this translates into practice. There is still much to learn and to construct in this domain of evaluation, and Shepherd and Graham articulate quite clearly what actions are neededto advance our collective knowledge and practice. I extend my heartfelt thanksto the guest editors, authors, and reviewers who made this special issue possible, and I hope that our readers take away its important messages.

 

Isabelle Bourgeois Editor-in-Chief

Abstract:

This paper argues that theory of change can be used to help stakeholders

in agricultural research for development projects collectively agree on problems and

visions of success. This helps them feel greater ownership for their project, motivation

to achieve outcomes, and understanding of how to do so. However, the dynamic is

damaged if projects are pushed to be too specific too early about the outcomes for

which they are to be held accountable. This is most likely to happen when system

response to project intervention is uncertain, as opposed to projects that work with

existing pathways and partnerships where the role of research is well established.

 

 

 

Abstract:

In Quebec, sexual education contents were made mandatory for students

 

 

Abstract:

Despite a great deal of discussion about the notion of professionalizing

evaluation practice around the world, many associated concepts are not clearly defi

ned. This practice note provides operational definitions of the concepts of profession,

professionalism, professional and professionalization, and presents methodology

designed to reflect on the ins and outs of a national process of professionalization.

In conclusion, the author analyzes the prospects for the professionalization of the

practice of evaluation at an international level.

 

 

 

Abstract:

Evidence-based decision-making and managing for results are terms oft en heard from politicians and senior government offi cials at both federal and provincial levels of government in Canada. But, while there is some level of under- standing at the federal level in terms of the role and use of evaluation in measuring results, there is significantly less information readily available about the extent to which evaluation is being used at other levels of government. This paper provides a cross-Canada synopsis on the capacity and use of systematic evaluation at the provincial and territorial levels of government. Authors from nine provinces and two territories provide a succinct analysis of the extent to which evaluation is being used in their provincial/territorial government, as well as a description of the challenges and opportunities that lie ahead for evaluation. There is a paucity of published information on this subject, but the paper uses research conducted in 2001 as a benchmark to compare the state of aff airs for evaluation within provincial/territorial governments. With limited progress over the past two decades, the paper offers an overview of findings and some proposed actions for the way ahead.

This is the advanced online version.

Abstract:

Evaluation is a transdiscipline with a focus upon asking and answering

important questions about programs, policies, and interventions. A unifi ed taxonomy

of evaluation-specific program outcomes would be a helpful tool to evaluators,

program designers, implementers, and policymakers, but one has not yet been

proposed and validated in an evaluation context.This study builds from a grounded

analysis of 125 programs and over 850 individual program outcomes, from which

was developed a taxonomy of nine outcomes specific to evaluating programs and

interventions: attitude, affect, behavior, cognition, status, relationship, biological,

environmental, and economic. The article defines key terms, directionality, and flexible

timeframes for measurement, and suggests specifi c fields that provide a helpful

lens for understanding programs and improving evaluation practice.

 

 

 

 

Abstract:

The CES membership database was analyzed in order to determine the

socio-demographic profi le that best defi nes Credentialed Evaluators (CE). In general,

those currently holding CE status tend to be employed in the private sector, have a PhD-level education, have long-term experience in evaluation, have a major focus

on evaluation within their professional activities, and be located within the Atlantic region, Western Canada, or the National Capital Chapter region. Implications of

these trends for the sustainability of the CE designation program and options for

broadening the scope of uptake within the CES membership are discussed.

 

 

Abstract:

This practice note describes and reflects on an inter-university collaboration

to deliver a graduate credential in evaluation to Government of the Northwest

Territories (GNWT) staff . The collaboration involved faculty members from two

member institutions of the Consortium of Universities for Evaluation Education

(CUEE). We describe the process of negotiating a contractual agreement, developing

and delivering the program, and offer our reflections and lessons learned. Overall

this collaboration is worthwhile but is challenging to do in an environment where

universities are focused on their own academic missions and programs.

 

 

 

Abstract:

This practice note illustrates a situation where, as program evaluators,

we crept beyond the provisional boundaries set by our Developmental Evaluation

(DE) goals to facilitate learning. Our DE was initially focused on one program which

was being designed for online delivery in higher education. During the DE of this

program, questions and themes arose which had larger organizational applicability;

we were asked to help design a strategic learning session that addressed a large group

of stakeholders within which existed the tiny subset of stakeholders engaged in the

original DE. This practice note describes how we negotiated the emergent purposes

of the DE with the need for intentional pivots within the strategic learning session

to serve our intended subset of stakeholders with their project as well as stimulate

evaluative thinking within the larger stakeholder group.

 

In this textbook, Robinson and Leonard take the reader on a deep dive into

current research and knowledge about best practice, as well as Likert’s foundational

work on scales, to correct common misconceptions about survey design. Th e

book is organized into three sections comprising eight chapters and supported by

a glossary, an appendix, and a comprehensive list of references.

 

 

 

 

Evaluation Failures is an edited volume detailing the tribulations and hard-earned

lessons of 23 evaluation professionals. Kylie Hutchinson has compiled a compelling

page-turner whose self-reflective spirit can be summarized as an antidote to

full-blown failure.

Divided into eight parts, the book takes the reader along a typical evaluation

cycle, from managing the evaluation and defining stakeholders to reporting useful

findings.