Advocacy Evaluation Theory as a Tool for Strategic Conversation: A 25-year Review of Tobacco Control Advocacy at the Canadian Cancer Society
Advocacy evaluation has been considered difficult, even in specific and uncomplicated situations. This article describes a short evaluative review of a series of tobacco control policy advocacy activities involving multiple parties over 25 years. Despite its obvious limitations, the effort suggests that the retrospective development of a structured contribution analysis and "key event" stories from documentary and key informant information can generate compelling circumstantial evidence for impact.
Tobacco control advocates have played an important role in Ontario's tobacco control efforts. As with other advocacy work, their efforts have been mostly unexamined.
Evaluating school-based tobacco control programs and policies: an opportunity gained and many opportunities lost
In the domain of school-based tobacco control, there is a large gap between the research being performed and the type of research needed to inform school-based prevention activities. The Tobacco Module of the School Health Action Planning and Evaluation System is a data collection and feedback system designed to support school-based tobacco control planning and evaluation.
The challenge of reconciling scientific rigour with feasibility is central to the goals of policy and program evaluation for tobacco control. Evaluations conducted in various settings are held to high standards of performance, and must also be considered feasible by program authorities and stakeholders. This article describes three recent examples from the field of tobacco control. Issues of context, relevance, and stakeholder participation in planning evaluation designs are central to successful reconciliation.
Utilité de l'évaluation de l'évaluabilité des politiques gouvernementales de lutte contre le tabagisme: l'expérience québécoise des centres d'abandon du tabagisme
The government's tobacco control policies are often multifaceted and large-scale in nature. The process and outcome evaluation of such programs generally requires a significant investment in time, money, and effort. A critical but often neglected step prior to conducting a successful evaluation is the evaluability assessment. This tool is a method for examining a given program to determine whether it meets the criteria for a meaningful evaluation. Even when it fails to do so, it still contributes to the improvement of the program.
Common measures to advance the science and practice of population intervention for chronic disease prevention: The promise and two early experiences in tobaco control
Pertinent evidence to inform population interventions for chronic disease prevention is sparse. The use of common measures across multiple jurisdictions is a promising approach to study "natural experiments" that can dually advance research/knowledge development and evaluation/practice improvement for population intervention.
We explore key lessons arising out of target-setting agendas in tobacco control. Two real-world examples are described that illustrate the pitfalls of choosing an omnibus target for a comprehensive strategy: (a) a target to reduce wholesale cigarette sales in Ontario and (b) a First Ministers' target to reduce current smoking. Changing contexts brought about by shifts in illicit tobacco sales made it problematic to interpret success in both cases.
Challenges to evaluating comprehensive complex strategies revolve around addressing comprehensiveness, attribution, and complexity. The latter requires attention to synergies amongst interventions, feedback loops, and other forms of nonlinearity. The article reviews and assesses how well six approaches to evaluating strategies succeed in dealing with these challenges, including one developed in light of the initial review. None of the approaches offer ideal solutions to the challenges of complexity.