Casing out evaluation: expanding student interest in program evaluation through case competitions
This paper describes the authors' experience in organizing bilingual evaluation case competitions for the National Capital Chapter of the Canadian Evaluation SOciety in 1996, 1997 and 1998. It covers the structure of competitions, eligibility rules, and the judging of student presentations. The authors also share lessons about recruiting contestants and preparing cases.
Assessing the Organizational Impact of Development Cooperation: A Case from Agricultural R&D
This paper reports on an evaluation of the organizational impacts of the International Service for National Agricultural Research (ISNAR), a non-profit agency with a mandate to strengthen national agricultural research organizations and systems. For the purposes of this study, "organizational impacts" are defined as the effects of ISNAR's activities and outputs on the external environment, organizational motivation, and capacity of client organizations. The article outlines the context and objectives of the evaluation and the conceptual framework developed for it. A set of component evaluation studies, data collection and analysis procedures and some salient findings are described. Overall results are contrasted with the initial evaluation objectives.
Repenser l'évaluation des partenariats
This article reflect on the evaluation of partnerships through an examination of the actual evaluation frameworks designed for a participatory setting. By concentrating particularly on the underlying paradigms of these frameworks, the article first shows how processs evaluation is problematic because its methodology is based on a limiting worldview, when compared to the worldviews expressed by participants in an actual partnership. Second, the article shows that empowerment evaluation is more suitable, because it is based on a paradigm that allows for a broader definition of partnerships.
Assessing the Effectiveness of Industrial Pollution Prevention Programs
In response to social, market, and regulatory pressures, companies are adopting innovative approaches to environmental management. Pollution prevention (P2), which advocates avoiding, eliminating, or reducing the use of polluting substances, is one such innovation. This paper proposes that program evaluation techniques can be readily adapted to guide private-sector organizations in reviewing and assessing the effectiveness of P2 programs on both a periodic and ongoing basis. A P2 program - Effectiveness Review Framework that draws upon the Twelve Attributes of Effectiveness - developed by the Canadian Comprehensive Auditing Foundation - is outlined. A self-assessment approach, based on the Environmental Self Assessment Program developed by the Global Environmental Management Initiative, is then presented. The overall model detailed in this paper is designed to assist managers with incorporating P2 program effectiveness reviews into their management cycle.
A Confirmatory Factor Analytic Model of Evaluation Use
The purpose of this study was to develop and test a second-order factor analytic model of conceptual, instrumental, and symbolic use of evaluation information. The participants were 38 elementary and secondary school teachers who participated in the British Columbia School Accreditation Program. Structural equation modeling was used to test the fit of the model. The results confirmed a three factor structure of conceptual, instrumental, and symbolic use. Cross-validation indicated that the model would likely replicate in other independent samples. The model in this study is unique in that it is a first attempt at confirming the factor structure of conceptual, instrumental, and symbolic use in a measure of evaluation use.
Repenser le concept d'évalution de l'efficacité d'une organisation
The effectiveness of an organization is a widely criticized notion. A review of the literature reveals the confusion and the strong conceptual ambiguity that currently surrounds this phnon. Effectiveness appears to be a subject of study that is complex, unclear, unstable, polymorphous and polysemous all at the same time. Moreover, Anspach (1991) characterizes effectiveness as elusive and resistant to definition, conceptualization and measure. Some authors have tried to conceptualize effectiveness and have proposed various evaluation models. Most often, however, these models have proven divergent, difficult to reconcile and unsatisfactory. Taken individually, the models seem incapable of identifying and explaining the whole effectiveness phnon. This study presents a new way to approach the evaluation of the effectiveness of an organization. The proposed analysis framework reflects a dynamic conceptualization of effectiveness. Effectiveness is perceived more as a continuous process than an end in itself and is considered both a means and a result of the actors' behaviour. The analytic framework identifies five dimensions of organizational effectiveness, structural, operational, systemic, strategic and specific dimensions, which could potentially be examined based on the context and needs of an evaluation. This new way to conceptualize effectiveness merits empirical review.
Evaluating the Ontario Academic Course Teacher In-Service Program
The general purpose of this program evaluation was to provide formative information to the Education Quality and Accountability Office (EQAO) (and through the office to the Ministry of Education and Training) for improving the Ontario Academic Course Teacher In-Service Program (OAC-TIP). The evaluation was designed and conducted in collaboration with EQAO's Assessment Advisory Committee, representing the key partner organizations, and a steering committee of individuals experienced with the program. Data for the study were collected by means of questionnaires, focus-group consultations, and a review of Ministry of Education and Training documents and records. The study generated 25 recommendations for consideration in the development/redesign of an OAC-TIP-like program.
Measuring the Consistency in Change of Hepatitis B Knowledge Among Three Different Types of Tests: True/False, Multiple Choice, and Fill in the Blanks Tests
This Research and Practice Note assesses grade 7 students' recall of hepatitis B information using three different types of tests: short answer, true-false, and multiple choice. It concludes that short answer testing is the most reliable and recommends that evaluators be consistent with the type of test used to assess pre- and post-knowledge of students.