Simon Roy

Spring

The Lay of the Land: Evaluation Practice in Canada in 2009

Authors:
Pages:
1-49

A group of 12 evaluation practitioners and observers takes stock of the state of program evaluation in Canada. Each contributor provides a personal viewpoint, based on his or her own experience in the field. The selection of contributors constitutes a purposive sample aimed at providing depth of view and a variety of perspectives. Each presentation highlights one strength of program evaluation practiced in Canada, one weakness, one threat, and one opportunity. It is concluded that Canadian evaluation has matured in many ways since 2003 (when a first panel scan was conducted): professional designation is a reality; the infrastructure is stronger than ever; organizations are more focused on results. Still, evaluation is weakened by lacunas in advanced education and professional development, limited resources, lack of independence, rigidity in evaluation approaches, and lack of self-assessment. While the demand for evaluation and evaluators appears on the rise, the supply of evaluators and the financial resources to conduct evaluations are not. The collective definition of the field of evaluation still lacks clarity. There is also reassurance in looking toward the future. With increased appetite for evaluation, evaluators could make a real difference, especially if evaluators adopt a more systemic view of program action to offer a global understanding of organizational effectiveness. The implementation of a Certified Evaluator designation by CES is a major opportunity to position evaluation as a more credible discipline.

Special Issue

Evaluation practice in Canada: results of a national survey

Authors:
Pages:
1-42

This article reports on the results of a national survey that describes the professional and practice profiles of program evaluators in Canada, their views of their working conditions, and their sense of belonging to the field of evaluation. The data were collected between May and July 2005 via a Web survey, and 1,005 respondents filled out questionnaires. Among them, 647 indicated that they were internal or external evaluation producers, the others being evaluation users, students, or researchers. The results raise several issues. First, much of the evaluation work being done in Canada appears to be driven by accountability requirements, and secondarily by an appetite for program improvement or reconsideration. Second, voluntary certification, while quite widely supported, may create or encounter significant challenges in attempting to achieve professionalization goals. Third, the survey documents the need for professional training and the low levels of satisfaction with the training received to meet the requirements of evaluation positions. Finally, based on the current configuration of the population of active evaluators, on the intent of a majority of young evaluators to leave the field in the next few years, and on the training required in evaluation, the profession is not currently in a position to sustain itself through the renewal of a stable, capable, and committed workforce. Taken together, these results suggest a need for reflection and action on the future development of the profession.

Spring

The lay of the land: evaluation practice in Canada today

Authors:
Pages:
143-178

A group of 12 evaluation practitioners and observers takes stock of the state of program evaluation in Canada. Each of the contributors provides a personal viewpoint, based on their own experience in the field. The selection of contributors constitutes a purposive sample aimed at providing depth of view and a variety of perspectives. Each presentation highlights one strength of program evaluation practiced in Canada, one weakness, one threat, and one opportunity. It is concluded that evaluators possess skills that other professions do not offer; they are social and ecc researchers versed in using empirical data collection and analysis methods to provide a strong factual foundation for program and policy assessment. However, program evaluation has not acquired an identity of its own and, in fact, has tended to neglect key evaluation issues and to lose emphasis on rigour. Today's program evaluation environment is dominated by program monitoring, the lack of program evaluation self-identity, and insufficient connection with management needs. But evaluation is not without opportunities — resultsbased and outcome-based management, advocacy and partnership efforts, individual training and development, and bridging between program management and policy development represent some. But first, evaluators must self-define to communicate to others what their specific contribution is likely to be. The article concludes with implications for the practice of evaluation in Canada and the blueprint of a workplan for evaluators individually and collectively, in their organizations and in their professional association.