Guest Editor's Remarks / Un mot du rédacteur invité
Program evaluation in Canada seen through the articles published in CJPE
In honour of the 20th anniversary of The Canadian Journal of Program Evaluation, this article presents a retrospective of publication trends from the fi rst edition in Spring 1986 to Fall 2004. This review entailed creating a database by coding all of the abstracts and, when necessary, the full article, according to criteria such as the authors, their origins, the purpose of the articles, the public policy sectors presented, the methodology, and the type of evaluation discussed. As anticipated, the publication trends tend to reflect broader developments in the field of evaluation practice and theory in Canada and abroad. From utilization debates to capacity-building issues, from auditing to empowerment evaluation and beyond, the articles in CJPE sometimes lead and sometimes follow new trends in the evaluation field. Over the years, authors affiliated with universities have progressively increased their share of articles.
Two decades of the Canadian Journal of Program Evaluation: a content analysis
This year, the Canadian Journal of Program Evaluation (CJPE) is celebrating its 20th anniversary. To commemorate this milestone, this study provides a content analysis of the full population of articles and research and practice notes from the Journal's 20-year history. Using a framework that draws from an extensive literature review of similar studies in other fields, the CJPE papers were systematically analyzed and presented through descriptive statistics and graphic representations. In addition to providing an opportunity to reflect upon the history of the Journal, it is anticipated that this study will assist readers, and the Journal's editor and editorial board, to consider possible future directions as the Journal heads into its next decade of publication.
Patient satisfaction with health care: recent theoretical developments and implications for evaluation practice
Using client or patient satisfaction instruments to assess the quality of services and programs is an integral component of much program research and evaluation. But however methodologically straightforward and programmatically useful, the constructs and theory underlying patient or client satisfaction measures are not particularly simple, and the informativeness and usefulness of these satisfaction measures have been repeatedly called into question. To help program evaluators more effectively incorporate service users' perspectives into program evaluations, this article reviews major approaches to, and research on, satisfaction theory, including disconfirmation, fulfillment and consumer models, and sociological perspectives, and discusses emerging approaches and implications for evaluation practice.
Randomized and quasi-experimental evaluations of program impact in child welfare in Canada: a review
Respondents to a recent survey identified the evaluation of service effectiveness as the most pressing priority for child welfare research in Canada. After a comprehensive search, we located and reviewed 10 peer-reviewed impact evaluations, published during 1995–2005, of interventions in Canadian child welfare. Four evaluations were based on randomized controlled trials, and six on non-randomized, quasi-experimental designs. After a critical review of each study, we formulated the implications of the review, for the design and evaluation of child welfare interventions in Canada, in terms of three main needs: for more high-quality impact evaluations; for the evaluation of the effectiveness of a wider range of interventions; and for the implementation and evaluation, in the Canadian context, of interventions of mainly U.S. origin that incorporate the principles of the increasingly influential perspective of evidence-based practice.
Evaluation in Canada's social services: progress, rifts, and challenges
This article provides a brief retrospective on three major developments in evaluation within the social services sector over the last decade. These developments include the development of a performance measurement orientation in social service organizations, the growth of empowerment evaluation, and emerging capacity challenges that threaten the use of evaluation within the sector. The author argues that performance measurement has been a positive and necessary development for these organizations, but a new level of outcomes conceptualization is required. This new level is based upon creating community-level outcomes that aggregate data into a more systematic picture of social wellbeing in communities. Empowerment evaluation and investment in the organizational learning capacity of social service organizations are seen as complementary strategies that can help to further the growth of evaluation within social services.
How do you evaluate a network? A Canadian Child and Youth Health Network experience
Over the past decade, approximately 20 child and youth health networks have been initiated in Canada. The value of any network depends on its effectiveness in achieving stated goals; however, description and measurement of network effectiveness is challenging, given the complex multi-sectoral and/or multidisciplinary relationships involved and the relative dearth of evaluation methods specific to networks. Since its inception in 2001, the Southern Alberta Child and Youth Health Network (SACYHN) has reviewed the network evaluation literature and developed and implemented an evaluation framework. This article describes key findings from the literature on networks and their evaluation, and the experience and learning from network evaluation activities conducted by SACYHN. These may inform evaluation efforts in other networks or similar inter-organizational initiatives.
Evaluating school improvement in Canada: a case example
This article gives a brief description of school improvement in Canada over several decades and describes the difficulties — conceptual, technical and communicative — associated with evaluating school improvement. Through a case example, the authors draw attention to the problems associated with the "live-action" world of school improvement and highlight the responsibility of researchers and evaluators to choose the appropriate method, consider the consequences of their work and be aware of the policy implications that accompany any evaluation that addresses current policy issues.
Program evaluation in the Government of Canada: plus ça change...
This article describes the evolution of evaluation policy and practice from 1977, when the first policy on program evaluation was issued by Treasury Board, to the current state of evaluation in the Government of Canada. It discusses the changing nature of evaluation policy, resources, and conceptions of the function as well as a number of concerns about federal evaluation that have been expressed during this period. The article concludes by providing suggestions that, if implemented, might enhance the status and role of evaluation in the federal government.
Interview with Joe Hudson, Founding Editor of CJPE