A Framework for Characterizing the Practice of Evaluation, with Application to Empowerment Evaluation
As the evaluation profession experiences a widening diversity of theory, method, and practice, there is a growing need for some means of characterizing significant similarities and differences in evaluation approaches. To that end, a framework is presented which consists of six aspects: context, purpose and social role, phna of interest, procedural rules, methods of justification, and sanctions. The utility of the framework is illustrated through its application to a current debate concerning the theory and practice of Empowerment Evaluation.
Facilitating Development of Organizational Productive Capacity: A Role for Empowerment Evaluation
Efficacy of community-based social programs is highly dependent on the development of organizational and program capacities - capacities which include the domains of (a) a core-empowered group, (b) internal resources organization, (c) external resources mobilization and integration, (d) strategy comprehensiveness and logic, and (e) monitoring, evaluation and feedback. A framework is offered in which organizational productivity and program outcomes are conceptualized as a product of a transformational process, in which organizational capacities transform vision into productive activity, using the resources of an organization and activated through an environment of collective empowerment. Collective empowerment is defined by the authors as an energy force of mutual commitment, cohesiveness, and conscientiousness that activates the development of increasing organizational capacities, through a cyclical process that builds increasing commitment, "small wins," and expanded membership. Collective empowerment contributes to the expansion of the program effort with concomitant increased activity accomplishment and achievement of intended outcomes. The authors argue that empowerment evaluation is a philosophy and set of practices that contributes to the development of collective empowerment as well as to the development of organizational and productive capacities. Empowerment evaluation must simultaneously keep in mind the intended program outcomes (which need to be reliably measured) in order to justify the original goals of the program and ensure continuous learning and benefit. Guidelines for evaluators using the empowerment evaluation model are provided.
Organizational Development and Evaluation
This article explores some of the opportunities open to evaluators as organizational development practitioners and the particular competencies and comparative advantages evaluators can bring to such initiatives. Examples include: (1) making systems connections between program culture and organizational culture (and therefore between program effectiveness and organizational effectiveness); (2) organizational mission fulfillment assessment (in contrast to and in relation to program goal attainment (3) results -oriented, reality-testing leadership development as an organizational development effort to enhance the appropriate, effective, and ongoing use of outcomes evaluation at senior organizational levels; (4) reflective practice and action research built on evaluation logic and processes for organizational development; (5) knowledge management for learning organizations through identification and use of high quality empirically supported and triangulated) lessons learned: (6) evaluation facilitation as a way of enhancing organizational communications; (7) process uses of evaluation for organizational learning and capacity building; and (8) developmental evaluation as a form of organizational development in teams.
Communities Evaluating Community Level Interventions: The Development of Community-Based Indicators in the Colorado Healthy Communities Initiative
This paper reports on two communities that are developing and using community-based indicators to evaluate their progress toward becoming a healthier community. These communities are part of the Colorado Healthy Communities Initiative (CHCU, a project involving 28 diverse communities in the state of Colorado. Following a description of CHCI and the 28 communities involved in it, the article explains the evaluation components for the initiative, one of which is community-based indicators, The community indicators component is illustrated by two case studies. The process the communities used to develop their indicator sets, and the indicator reports they produced Lire described, with illustrations of specific domains, dimensions, and indicators. The article concludes with a discussion of the indicators evaluation component and the need for responsive design in developing indicators.
Enhancing Managers' Evaluation Capacity: A Case Study from Ontario Public Health
Enhancing capacity is an important facet of empowerment evaluation. This article describes an initiative designed to help public health managers in Ontario improve their knowledge and skills in program evaluation. The initiative involved the development of a self-directed learning resource called the Program Evaluation Tool Kit and an accompanying workshop. The development of the Program Evaluation Tool Kit embraced five principles: taking stock of what was needed, building on shared values, valuing different perspectives, integrating planning and evaluation into routine program management, and maximizing adult learning.
Building Capacity for School Improvement Through Evaluation: Experiences of the Manitoba School Improvement Program Inc.
The evolution of the Manitoba School Improvement Program's (MSIP) approach to evaluation as a means of building schools' capacity for continuous improvement will be described from the perspective of tile MSIP evaluation consultant. and in the context of the difficulties inherent in secondary school reform. Drawing on the experiences of the MSIP schools and the data that have been collected by and from the schools over the last seven years, this article will attempt to address two key questions: What have we learned about how to move schools into an empowerment model of evaluation? What are the benefits of empowerment evaluation to educational reform?
Empowerment Goes Large Scale: The Canada Prenatal Nutrition Experience
This article describes a large-scale federal program evaluation which employed empowerment strategies in its design and implementation. Bandura's concept of group efficacy is important for empowerment evaluation and can enhance ownership of an evaluation by identifying conditions that foster powerlessness and removing them through good evaluation practice. Conger and Kanungo's five-stage empowerment process is described in relation to the evaluation of the Canada Prenatal Nutrition Program which provides food supplementation, nutrition counselling, support, education, referral, and lifestyle counselling to pregnant women at risk who are likely to have babies of unhealthy birth weight. Specific strategies described include the use of visual metaphor, participant generation of survey items, variable survey design to meet stakeholder needs, decentralized evaluation funds, use of an evaluation help line, and different levels of evaluation reporting. Early examples of empowerment outcomes are provided in terms of program improvement and heightened community awareness and ownership of program goals.
Epilogue - comments on the special issue
Editor's Introduction to the Issue
Reflections on Empowerment Evaluation: Learning from Experience
Empowerment evaluation (EE) is the use of evaluation concepts, techniques, and findings to foster improvement and self-determination. It employs both qualitative and quantitative methodologies. Although it can be applied to individuals, organizations, communities, and societies or cultures, the focus is usually on programs. A wide range of programs use EE, including substance abuse prevention, indigent health care, welfare reform, battered women's shelters, adolescent pregnancy prevention, individuals with disabilities, doctoral programs, and accelerated schools. This approach has been institutionalized within the American Evaluation Association since its introduction in 1993 and is consistent with the spirit of the standards developed by the Joint Committee on Standards for Educational Evaluation. Empowerment evaluation has become a worldwide phnon, its acceptance in part a function of timing. Evaluators were already using forms of participatory self-assessment or were prepared to use it because it represented the next logical step. Funders and clients were focusing on program improvement and capacity building. A critical match between people and common interests was made with an underlying and often implicit commitment to fostering self-determination. Widespread use of this evaluation approach was also a result of its appearance on the Internet.