Recent Advances in Questionnaire Design for Program Evaluation

Evaluation uses questionnaires as a central data-gathering technique, yet researchers often appear unaware of recent developments in questionnaire design. This article reviews issues beyond the creation of standardized questions and the basic rules researchers find useful in data collection. These elementary guidelines remain robust for much evaluation research and should not be abandoned hastily. However, rapid change in the theory underlying questionnaire design has important implications for evaluation. Three themes illustrate these changes. First, magnitude scales and their use in client satisfaction scales show how response categories can improve individual questions. Second, decision theory sees respondents as selecting "correct" responses from a portfolio of potential answers. In this view, the answer to a question is conditioned by the values held by the respondent and his or her perception about the risks of revealing true feelings. Third, in some cases it is possible to cast the entire questionnaire into a framework that replicates how choices are made by respondents. In one application, the questionnaire simulates the decision-making of a consumer in the market place. Each of these three themes is applied to problems evaluators face in data collection involving surveys and questionnaires.

La prévention du SIDA chez les jeunes non intégrés au réseau scolaire : évaluation du projet tete à queue

This paper presents both the procedure and the results of an evaluation of Tête à Queue, a condom promotion magazine for youth outside the school system. Three studies were conducted: a survey among practitioners, an experimental study to test the effects of the magazine on youths' attitudes and intentions regarding condom use, and a critical analysis of the controversy that surrounded the diffusion of the magazine. Results show that aspects related to the definition of the need to the definition of the target population, diffusion strategy, and some specific contents account for some of the difficulties encountered during the course of the project.

Toward Effectiveness Assessment of Public Management Education in Canada: Convergence and Complementarity

A growing theme in management education literature is that of' effectiveness assessment; Integrated effectiveness assessment systems represent a growing important challenge in public management education in Canada. This paper examines the service effectiveness framework that evolved from comprehensive auditing developments in the public domain. It presents an integrated model of effectiveness assessment systems and the current issues that impede its full development and implementation in Canada. Finally, the paper examines the evolving effectiveness assessment challenges that could fundamentally change public management education in Canada.

Prevalent Department of Social Services' Client Satisfaction: A Research and Practice Note

This successive cross-sectional or panel study evaluated the City of' Windsor's Department of Social Services, specifically the Special Services and Children's Services Branches from 1990 to 1994. Their programs were assessed with quantitative (consumer satisfaction questionnaire, CSQ-8, score range = 8 to 32 (Larsen, Attkisson, Hargreaves, & Nguyen, 1979) and qualitative (e.g., open-coded queries about what they liked most and least about the social services they received) measures of client satisfaction.

Application ofnal Group Technique to Evaluate a Community Health Status Report

In 1992, the City of North York Public Health Department released the Community Health Status Report and Social Profile Report on the residents in North York, Ontario, Canada. These reports are a resource that is used by departmental staff to assess local needs, plan and evaluate programs, and assist in departmental strategic planning. This paper describes the application of thenal group technique (NGT) among staff in a public health department to identify, discuss, and prioritize ideas to improve the reports.

Political Context and Program Evaluation: The Inextricable Link

Program evaluations have assumed a much greater importance as pressure on public expenditure increases. This article eschews evaluation designs that are politically vacuous by drawing attention to the importance of contextual variables in a case study of a community relations program based in Northern Ireland. The research illustrates the dilemma now faced by evaluators in the United Kingdom. The demands of short-term, context-stripped, value-for-money evaluations are often in conflict with context-bound qualitative inquiries that can uncover the unique mix of social, political, and cultural influences on program delivery. The dominance of macro-political variables are illustrated as one example of a unique mix within this program.

Impact de l'entente Cartier-Champlain sur certains indicateurs relatifs à la sécurité routière

In this article, the impact of a Cartier-Champlain police service agreement with a federal agency and the Sûreté du Québec is evaluated. The purpose of this agreement was to improve road safety by increasing police patrols in the area involved. In order to quantify the impact of the agreement,, an evaluation model with control groups and a chronological series was used. This model involves taking a series of monthly measures in the area covered by the agreement and comparing them to two control areas. The results of the evaluation lead us to question the positive impact of the agreement and whether external factors could have been responsible for the reduction in accidents.

Setting Achievement Standards/Expectations for Large-Scale Student Assessments

This article discusses a variety of issues associated with establishing formal standards or expectations for student achievement on large-scale assessment programs. The article reviews the educational literature on the subject of standard-setting methods, describes two approaches to standard setting that have been used in province-wide assessments in Saskatchewan, and evaluates the effectiveness of these approaches. Although the two procedures described were generally successful, numerous issues are raised and recommendations are made to support future standard-setting activity and research.

An Attempt to Integrate Empowerment Evaluation and Cross Cultural Evaluation

Western countries currently donate large sums of money to developing countries. This money is used to fund programs that provide humanitarian assistance, protect the environment build democracy, and encourage ecc growth. In the implementation of such programs, donor agencies often adopt a top-down approach in which intended beneficiaries have little involve ment. Empowerment evaluation could be used as a tool for over coming this problem. Empowerment evaluation is, as the name implies, a collection of evaluation techniques that promote the empowerment process. The empowerment process has been thought of as having five stages, and evaluation can play an important role in most of these stages.