Measuring Program Effects in the Presence of Selection Bias: The Evolution of Practice
The author applies a variety of methods for estimating the effects of employment and training programs on labor market behavior. This article traces how these methods have evolved over the past 10 years, focusing on the problem of selection bias. It discusses the context of measuring effects related to labor market activity, and then presents the essential elements of selection bias modeling and the specific changes that led to methods currently in use. It also provides an assessment of the benefits and potential weaknesses of these techniques.
The Holistic Perspective in the Evaluation of Public Programs: A Conceptual Framework
Public programs should be considered as components of the overall societal welfare system in which social, ecc, political, and biophysical factors interact. Evaluation of the effectiveness of public programs should recognize that they are an integral part of the overall societal welfare system. Evaluating programs as individual entities with specific objectives is not likely to yield information of critical importance to public decision-makers. With the continuing fiscal restraints, there is a need for strategic information on the overall performance of public programs. Existing techniques for broadening the focus of evaluations are not sufficient. This article discusses a holistic approach for evaluating public programs and examines a national performance databank for managing the results of such evaluations.
Le contexte français de l'évaluation : approche comparative
In 1990, the French government created an institutional tool aimed at promoting interdepartmental policy evaluation, and favouring development of expertise in this field. Three years after the start of this experiment, the author attempts to identify the features of the French model of evaluation compared to an Anglo-Saxon model. A promising tool to regulate complex public systems, policy evaluation fits less easily into decision making processes concerning public resources allocation.
A Study of the Relationship Between Evaluation Report Characteristics and Measures of Use in Two Countries
This article reports the results of secondary analysis research done in order to determine if there were differences in evaluation utilization among major cultural groups. First, the evaluation utilization literature is reviewed. Second, original research and secondary analysis methods are discussed. Finally, the results of four cross-cultural canonical correlation analyses are presented. The results show that, among the American subjects, there was a strong correlation between evaluation utilization and a high-order construct called evaluation implementation. This relationship did not exist for the Mexico subjects. These results are discussed in a cross-cultural context and conclusions are drawn.
All Pulling Together: Working Toward a Successful Evaluation
Utilization of results is an important aspect of program evaluation. Naturalistic methods aim at increasing the utilization of research findings by providing stakeholders with a rich understanding of the situation being studied. In this article, we report on a study that caused us to examine the assumption of the power of naturalistic methods. We found that one key to utilization is to narrow the gap between the type of information provided by the study and the expectations of the stakeholders.
L'évaluation en France: a une forte prescription d'évaluation correspond un faible essor des pratiques
Three factors have helped evaluation to develop in France: (1) contractual policies, which demand a partnership between several institutions, have been set up; (2) by organizing competence transfers, decentralization has given communities the major share of responsibility for implementing policies, something which had previously been done by the State; and, finally, (3) a public service reform policy has been set up within the State administration. The author, after giving a brief description of the various institutions that have been set up, analyzes the discrepancies noted between the interest sparked by the politico-theoretical debate preceding evaluation institutionalization and the paltriness of obtained results, characterized by the meager number of evaluations and the limited promotion of practices.
Ethical Considerations when using Qualitative Methods in Evaluation
This article discusses some of the ethical dilemmas that can arise when evaluators utilize qualitative methodologies. The dilemmas discussed are those not easily dealt with by referring to one of the existing codes of research ethics. The author does not offer solutions to these complex dilemmas, but suggests that their identification and the ensuing discussion will assist evaluators in designing an evaluation study.
The Australian Government's Evaluation Strategy: A Perspective from the Center
In the six years since the Australian government's evaluation strategy was set in place there has developed a substantial and growing emphasis on the conduct and use of evaluation. It is notable that Cabinet places considerable priority on having evaluation findings available to better inform its deliberations and decision-making. The article focuses on the attitudinal changes, institutional arrangements, and formal requirements that have led to this emphasis on evaluation. The central role of the Department of Finance as catalyst and coordinator is also analyzed.
The Municipal Comprehensive Audit Function
Many Canadian municipalities have established comprehensive audit systems with diverse structural characteristics. This article examines the comprehensive audit functions in Edmonton, Calgary, and Regina, focusing on their evolution and their strengths and weaknesses. The author proposes a model for a comprehensive audit system that would achieve balance and effectiveness.