Le magicien du temps: approche participative axée sur le développement d'un projet et l'utilisation des résultats d'une évaluation
A community organization in rural Québec, together with various inter-sectoral partners, wanted to make it possible for families to take their destiny into their own hands and improve their quality of life, in particular, their children's quality of life. The goal of this article is to present the three main innovative aspects in the evaluation of this community project, namely the value of a developmental evaluation approach, the relevance of a multipleneeds analysis framework, and the contribution of the continuous dissemination of the evaluation results to the project's development. This case study likewise sheds light on the factors that facilitate the use of evaluation results.
Editors' Remarks / Mot des rédacteurs
Les dispositifs de la participation aux étapes stratégiques de l'évaluation
A growing number of tools are now available to enhance the validity and social relevance of knowledge produced through the participatory spaces that program evaluation makes possible. Tools that facilitate a shared understanding of a program and its results are of this nature. However, evaluation teams are generally less equipped to support participation through the strategic processes of planning an evaluation or of redesigning a program based on evaluation results. This article presents social and technical processes that can enhance the participation of actors from the program system in these strategic choices. In addition to the usual tools that support program modeling mechanisms and validation of research results, the instruments presented address the implementation of a participatory space, choice of evaluation questions, assessment of findings, and purposeful deliberation on future program directions. These instruments enhance the social contribution of evaluation by introducing greater rationality in participatory evaluation.
Evaluation practice in Canada: results of a national survey
This article reports on the results of a national survey that describes the professional and practice profiles of program evaluators in Canada, their views of their working conditions, and their sense of belonging to the field of evaluation. The data were collected between May and July 2005 via a Web survey, and 1,005 respondents filled out questionnaires. Among them, 647 indicated that they were internal or external evaluation producers, the others being evaluation users, students, or researchers. The results raise several issues. First, much of the evaluation work being done in Canada appears to be driven by accountability requirements, and secondarily by an appetite for program improvement or reconsideration. Second, voluntary certification, while quite widely supported, may create or encounter significant challenges in attempting to achieve professionalization goals. Third, the survey documents the need for professional training and the low levels of satisfaction with the training received to meet the requirements of evaluation positions. Finally, based on the current configuration of the population of active evaluators, on the intent of a majority of young evaluators to leave the field in the next few years, and on the training required in evaluation, the profession is not currently in a position to sustain itself through the renewal of a stable, capable, and committed workforce. Taken together, these results suggest a need for reflection and action on the future development of the profession.
Synthèse - Empowerment et évaluation: quelques réflexions
Guest Editor's Introduction (Thematic Segment: Growing Evaluation: Are We Missing the Boat?)
Will evaluation prosper in the future?
The main thesis of this article is that program evaluation and program evaluators have largely missed out on the movement, now into its second decade, to make performance measurement the centrepiece of public sector management and accountability. If these developments are not strategically faced by evaluators, program evaluation runs the risk of becoming less and less relevant to public sector and nonprofit organizations. Evaluators, divided by epistemological and methodological differences, have collectively not been willing to embrace professionalization as a way to reclaim territory lost to the audit and management consulting professions. It is essential that evaluators develop and implement strategies that are intended to create a professional practice space that includes designing and implementing performance measurement systems. Evaluators are well-positioned to do so, if they act collectively.
Evaluation can cross the boundaries: the case of Transport Canada
This article examines how an evaluation unit in a federal government department transformed itself from a traditional function focusing on a few time-consuming and resource-intensive evaluation studies per year to a more service-oriented unit that has added results measurement to its core functions. It went down this road to increase the impact it was having on individuals, programs, and the department as whole. Along the way it discovered a new breed of evaluator: outgoing people interested in teaching, coaching, and facilitating. It found that there is more results measurement work out there than one unit can handle, and it learned to go where impact is highest. It had to cope with good evaluators being lured away by programs, and it recognized threats to evaluation independence. In the end, it has become a champion of the evaluator as results measurement specialist and is reaping the benefits of better results data, more astute evaluators, a more dynamic and stimulating work environment, and a larger and more visible impact.
Studies are not enough: the necessary transformation of evaluation
The authors contend that developments in the public and notfor- profit sectors over the last decade or so have profound implications for the profession of evaluation, implications that are not being adequately addressed. They argue that evaluation needs to transform itself if it wishes to play a significant role in the management of organizations in these sectors. Beyond traditional evaluation studies, evaluators working in public and not-for-profit organizations need to (a) lead the development of results-based management systems, (b) using this and all available evaluative information, strengthen organizational learning and knowledge management, and (c) create analytic streams of evaluative knowledge. Failing to grasp these challenges will result in a marginalized and diminished role for evaluation in public and not-for-profit sector management.
The role of culture and the future of the evaluation function: considerations and key questions
The author draws connections across the three articles in this thematic segment and poses a series of questions on the future of the evaluation function. He concludes that an enduring role for evaluation within Canadian public administration requires the adoption of a culture change agenda within the evaluation community and, by extension, within the organizations in which we work. Central to such a change agenda is sustained movement toward increasing the professionalism of the function.
Introduction du rédacteur invité (Segment thématique : Programmes communautaires et innovations méthodologiques : participation, accompagnement, et empowerment)
L'animation de groupe : une pratique à redécouvrir afin de développer le pouvoir d'agir des individus!
Based on action research conducted with community kitchens in the Bois-Francs region of Quebec, the article examines how group facilitation contributed to the process of empowerment of individual members within these groups. The authors explain how the research team realized the multiple elements of the conceptual framework for community empowerment to achieve this outcome. In conclusion, the role of social facilitation as a community practice to be rediscovered is briefl y discussed.
Former des intervenants pour intervenir en faveur du renforcement du pouvoir d'agir
The development of Palabres sur le pouvoir d'agir—Outil d'accompagnement sur l'empowerment [Discussions on the Power to Act—A Coaching Tool for Empowerment] resulted from an approach implemented by the AIDS 3 Project with sex trade workers in West Africa. The tool has two interrelated sections that focus on both practice and theory in a highly dynamic process. It enables front-line workers to integrate the approach more easily in their work, to understand their role as coaches, and to review their own practices from an empowerment perspective.
Un outil d'évaluation de l'empowerment: une tentative en Haïti
For many years, numerous attempts have been made to define empowerment, to identify its processes, and to suggest translations. Recently the debate has focussed on the complexity raised by its measurement and evaluation. The purpose of this article is to illustrate the contribution of Ninacs's framework (2002) in evaluating empowerment, by presenting its application to the evaluation of a health promotion program in Haiti. The instrument designed based on this framework was valued by the evaluation participants, making possible a simple and useful visual representation of a complex and multidimensional concept.
Le développement du pouvoir d'agir (empowerment) appliqué à une expérience d'insertion par l'écque: suggestion d'indicateurs d'évaluation
A social intervention experiment was conducted to try to apply an approach developing the capacity to act, better known as empowerment, in an ecc inclusion framework with individuals who have been socio-eccally withdrawn, sometimes for a long time. The experiment involved the design, implementation, and testing of a new resource intended to support unemployed individuals carrying out a community job-creation project in working class districts in Quebec City. The experiment is of interest, from an evaluation standpoint, for its development of a series of indicators to measure the impact of an undertaking of that kind. Although the evaluation could not be completed, the purpose of this article is to present the evaluation planning process and the list of indicators selected.
Porter et gérer une action transformatrice: quelques outils de discernement pour quand on est dans ça avec d'autres
Is it possible to systematically learn from an individual's experience as a leader in a coalition of groups and make use of it in a shared management situation? I wrote this account after more than nine years as coordinator and spokesperson for the Collectif pour un Québec sans pauvreté. When asked once about how I dealt with the complexity of the leadership function, an answer came to me in the form of a set of discernment tools on a key theme, transformative action. The instrument panel described in this article includes 10 images with comments and empty space for the necessary degree of freedom, introduced and followed by considerations for individuals and the organization on a number of issues surrounding the shared exercise of authority.
Collectif pour un Québec sans pauvreté: un processus d'évaluation continue des «fruits» obtenus et des «chemins» parcourus
From the start of Projet AVEC, the Collectif pour un Québec sans pauvreté adopted the obligation to track the project's development. Far from the traditional methods of evaluation, principles were established and constantly revisited to ensure a rigorous evaluation process appropriate for the project involving individuals in a situation of poverty. Measurement tools were developed with those principles in mind. One, called the "fruits" tool, was created to evaluate activities and another, the "route" tool, was designed to facilitate the evaluation of the project in relation to its objectives.
Du carré à la spirale: réflexions sur quelques années de participation du comité avec du Collectif pour un Québec sans pauvreté
This article presents the refl ections of two academics on their participation in a committee of the Collectif pour un Québec sans pauvreté, which brings together members from a variety of backgrounds, including some experiencing poverty. The authors explain how they joined the committee, the challenges encountered, and the lessons learned both personally and professionally. This brief testimony is intended to raise awareness of the importance of adopting a refl ective approach as a researcher or evaluator and the opportunity to succeed in doing so in community groups.