Needs assessment, a tool for program planning, involves collecting data from service agencies, key informants, and potential clients, and drawing inferences from indicators of community needs. As portrayed in the literature, this approach to evaluation should only be used prior to program development. Both the utility and the timing of the needs assessment approach have been restricted by this portrayal, as case studies will illustrate. Needs assessments should be conducted not only in the initial planning stage but periodically after a program has been implemented.
This article examines an unanticipated problem associated with the successful implementation of the "utilization-focused" method of evaluation described by Patton. The study involved the development and evaluation of a suicide screening instrument for use by the Alberta Solicitor General's Department. To maximize the possibility that these results would be useful, a utilization-focused approach was adopted. We will describe the evolution of the project over a series of stages, from identifying stakeholders to preparing a final report.
Naturalistic program evaluation strategies are gaining acceptance within the evaluation profession. They are especially useful in the examination of the transaction, or process, components of programs. One of their disadvantages is that they demand extensive quantities of evaluator time and effort which are difficult to arrange. Guba refers to evaluation conducted within the naturalistic paradigm (as distinct from the use of naturalistic techniques) as fourth-generation evaluation.
Participating in program evaluations is a new part of the classroom teacher's role. If teachers are to embrace the opportunities provided by this participation, they need to understand the process of program evaluation and to appreciate its potential to empower them in their professional lives. This article discusses these issues in the context of Saskatchewan's current curriculum change and offers an illustration of one instance where participating in a proper evaluation study did enhance the professionalism of a teacher.
Estimates of ecc activity generated and jobs created that are derived using input-output analysis are often presented in program evaluations and confused with the benefits of the resulting from the program. Two such cases are presented as examples. We argue that for two main reasons this type of analysis constitutes a misuse of input-output analysis. First, input-output estimates generated using the Keynesian closed versions of models are biased upwards because they ignore the price and financial feedbacks that tend to reduce multipliers in macroecc models.
Evaluation studies by the Government of Canada follow standard procedural guidelines but have been retrospective and idiosyncratic in their methodologies. Therefore, they do not enable comparisons of the likely future effectiveness of competing programs. In contrast, the World Bank uses a combination of benefit/cost analysis and risk analysis to produce standard "bottom line" measures of program effectiveness. The latter approach makes evaluation studies more useful in decision making.
Quantifying Qualitative Decision Trees For Multi-Respondent Decision-Making: Theory And A Program Evaluation Application
A method for quantifying qualitative decision trees is presented and applied within a program evaluation setting. The approach combines Likert type scales with decision trees and develops a mathematical curve based on the associated set of means and standard deviations. Depending upon where the respondent's result plots, one of six different decision conditions are suggested.