Standards for program evaluation in Canada: a discussion paper
Des normes en évaluation de programme au Canada: une proposition pour fins de discussion
L'évaluation d'un programme innovant : le programme de soutien et de renforcement des liaisons universites-industries et de recherche dite de transfert (LUI)
This article discusses the evaluation of an experimental Quebec government program (LUI) aimed at increasing the transfer of scientific and technical knowledge between universities and industry by means of grants to research centers. The evaluation covers the relevance, effectiveness, performance and impact of the program. Evaluation problems, methodology and results of the quasi-experimental study are examined and recommendations are made concerning the (LUI) program.
The Delphi Method: A Review of Methodology and an Application in the Evaluation of a Higher Education Program
The first section of this article details the history of the Delphi method, explains the method, and describes when it should be used. The advantages and disadvantages of the method are reviewed. The second section discusses the application of a Delphi method in a review of a higher education program.
Evaluation of Research and Development Accomplishments: Northern Forestry Centre
A retrospective, or hindsight, model was used to evaluate impacts and benefits of completed research and development (R&D) at the Northern Forestry Centre (NoFC) over a recent 10-year period. The evaluation included: (1) an inventory of R&D accomplishments in use with estimates of benefits; (2) client (forestry users) validation; (3) management rating of 62 accomplishments; (4) time and cost assessments; and, (5) quantified benefit compilations for a sample of six accomplishments. A conservative three-to-one benefit-to-cost ratio, over the entire NoFC R&D program costs (including services), for the 10-year period has been identified for the sample alone. The most successful R&D was client driven, with emphasis on applied R&D and technology transfer.
Performance Auditing and Policy Evaluation: Discussing Similarities and Dissimilarities
For a long time, policy evaluation and performance auditing, were quite distinct activities. In several supreme audit institutions in the industrialized world, there now appears to be a growing collaboration between auditors and evaluators. This paper focuses on possible similarities and dissimilarities of both professional activities. It also points at spin-offs from evaluation to auditing and from auditing to evaluation. The paper is based on the few case studies that are available, as well as on personal experiences of the author, both with auditing and evaluation inside the Netherlands.
Une strategie de triangulation pour l'évaluation des effets d'un programme de soutien aux nouveaux parents
Combined use of qualitative and quantitative approaches to program evaluation is a strategy which is attracting some interest in evaluation circles. However, its applications are few and its rules, related to the choice of devices and data analysis, are still poorly defined. In this article, the authors present an application of a triangulation strategy evaluating the effects of a child abuse and neglect prevention program. Using objectivation and generalisation techniques, the qualitative data then become comparable to the quantitative data. This comparison makes it possible to better interpret the results, raise new hypotheses and identify some unanticipated results.
valuation des effets d'un système d'information clinique sur l'activité medico-hospitaliere
We have analyzed the impact of a clinical information system designed to give feedback on profiles of medical practice in two hospitals. The system was planned to use group influence to induce change among deviant physicians. The information system failed to modify the physician's practice profiles. A large part of this failure is linked to the small variation recorded in utilization patterns among physicians. Medical practice appears to be quite homogeneous in a setting such as a hospital where the clinical behavior of physicians is highly visible to colleagues.
The Social Discount Rate
Recently the discount rate recommended by Treasury Board Canada for use in program evaluations and project assessments has been challenged. This article reviews the theory and evidence supporting various estimates of the Canadian discount rate, and includes a comparison of rates used by the U.S. government and the World Bank. The article was written with the support of Training and Development Canada.
The Usefulness of a Standardized Form for the Process Analysis Stage: A Case Study
Process analysis is useful in determining how, why, and under what conditions a program succeeds or fails in creating certain outcomes. Process analysis is conceptually appealing because the results can be used for program replication, improvement and pruning. As well, process analysis aids in the identification of confounding variables operating in a program that may effect outcomes. Unfortunately, a major practice obstacle in conducting process analysis is determining measures and methods for collecting process data. This article describes the Standardized Tracking Form (STF) used to record information useful for process analysis. A case study of how the STF is used by a health unit is presented and discussed.
Multicultural evaluation in Canada and the United States