CES Online Learning Goes Live: Welcome to the CES e-Institute!

Volume 26, 2011 - Special Issue

Logic Analysis: Testing Program Theory to Better Evaluat e Complex Interventions

Authors :
Pages :
61-89

Evaluating complex interventions requires an understanding of the program's logic of action. Logic analysis, a specific type of program theory evaluation based on scientific knowledge, can help identify either the critical conditions for achieving desired outcomes or alternative interventions for that purpose. In this article, we outline the principles of logic analysis and its roots. We then illustrate its use with an actual evaluation case. Finally, we discuss the advantages of conducting logic analysis prior to other types of evaluations. This article will provide evaluators with both theoretical and practical information to help them in conceptualizing their evaluations.

Trois conceptions de la nature des programmes: implications pour l'évaluation de programmes complexes en santé publique

Authors :
Pages :
91-104

This article discusses three perspectives on the nature of programs to clarify the purposes of an evaluation. An empirical realist program design views programs as real objects that allow dysfunctional and problematic objects to be repaired. In that view, the purpose of the evaluation is to determine the efficacy of programs to solve problems. An idealist design views both programs and the problems they address as representations, which leads to evaluation intended to reconcile the players' representations. In a critical realist design both programs and problems are events resulting from causal mechanisms activated by the program players. From that perspective, the purpose of the evaluation is to support innovation and program transformation.

Discussion: Practice-Based Evaluation as a Response to Adress Intervention Complexity

Creating a dialogue between the concepts of complexity paradigms and the pragmatic approaches proposed for evaluating complex interventions

Guest Editors' Remarks: How Does Complexity Impact Evaluation? / Un mot des rédacteurs invités : En quoi la complexité a-t-elle des répercussions sur l’évaluation?

Évaluer une intervention complexe : enjeux conceptuels, méthodologiques, et opérationnels

Authors :
Pages :
1-16

Theoretically, evaluation should help decision-makers address contemporary health system challenges. Paradoxically, the use of evaluation results by decision-makers remains poor, despite rapid development in the evaluation field. The level of use depends on the evaluator's ability to account for the complexity of health-care systems. The complex nature of an intervention often compels evaluators to adopt unconventional approaches to account for the roles of the players. The evaluation of a complex intervention raises conceptual, methodological, and operational challenges the evaluation has to overcome to increase the level of use of its findings by decision-makers.

L'évaluation des interventions complexes: Quelle peut être la contribution des approches configurationnelles?

Authors :
Pages :
17-35

This article shows the added value brought by configurational approaches in the field of evaluation of complex interventions. First, we analyze the work in the scientific literature that has addressed this issue to identify epistemological considerations. The first part shows our finding that the evaluation of complex interventions needs to go further in terms of theorization. We then explain how productive the configurational approach is in the theorization of evaluation. Third, we discuss methodological issues and the operationalization of this approach.

L'approche Realist à l'épreuve du réel de l'évaluation des programmes

Authors :
Pages :
37-59

The Realist approach suggests a way to grasp the complexity of social interventions, through questioning what works, for whom, and under what circumstances. We present here some challenges and discussion on the use of this methodological innovation, based on an analysis of our own experience and an examination of the scientific literature. Although the Realist approach seems The Canadian Journal of Program Evaluation Vol. 26 No. 3 Pages 37–59 ISSN 0834-1516 Copyright © 2012 Canadian Evaluation Society 38 The Canadian Journal of Program Evaluation intellectually attractive, building its "Context-Mechanism-Outcome" configurations proves problematic. Finding the necessary information about context and outcome, differentiating contextrelated elements from those related to mechanisms, and adopting an accurate level of analysis are among the challenges in using the Realist approach.