Do evaluator and program practitioner perspectives converge in collaborative evaluation?

Interest in collaborative and participatory forms of evaluation -- evaluation that involves evaluators working directly with non-evaluator program practitioners or stakeholders -- has increased substantially in recent years. Yet research-based knowledge about such approaches remains limited. Moreover, empirical studies have focussed almost exclusively on the perspectives of evaluators or to a lesser extent non-evaluator stakeholders associated with the program.

English

Modeling success: articulating program impact theory

This study was undertaken to articulate program impact theory for the Comprehensive Home Option of Integrated Care for the Elderly (CHOICE) program. The study showed that CHOICE combines elements found in a traditional Health Maintenance Organization with elements and process components drawn from primary care and case management to deliver a broad range of home support, day program, and social and health services to its participants and their informal care givers.

English

Softly, softly catch the monkey: innovative approaches to measure socially sensitive and complex issues in evaluation research

Many government program evaluations require information to be captured that is hard to measure, of a sensitive nature and difficult for the respondent to articulate. This paper suggests research designs and methodologies to assist in overcoming such problems in evaluation research. Our discussion is illustrated by three evaluation case studies. Suggestions for research design focus on increasing reliability through intersubjective certifiability and the use of triangulated respondent groups, as well as varying the composition of the research team at different stages of the research.

English

Insurance claimants working while on claim

Subject to certain restrictions, the Canadian unemployment insurance ( UI ) system permits insurance benefit recipients to work and supplement the UI benefits. Although this provision relating to the treatment of earnings during the benefit period under the UI (now called EI) Act had been in operation for many years, hardly anything is known about the extent to which this provision is utilised and what impact this has had on their UI benefit period.

English

Accountability, rationality and new structures of governance: making room for political rationality

This article argues that the new institutional arrangements and results or performance based accountability run the risk of overlooking several political realities and considerations. The new arrangements and processes represent a preference for ecc rationality over bureaucratic rationality. They give little consideration to political rationality, which is concerned with collective values and power relationships. In managerial terms, it is not clear that the new arrangements will be any more durable than their numerous predecessors.

English

Do distinct servqual dimensions emerge from mystery shopping data — a test of convergent validity

Service quality is commonly though to comprise five generic dimensions - responsiveness, assurance, tangibles, empathy and reliability. These dimensions form the basis for service measurement tools such as SERVQUAL. Research in this area, using tools such as SERVQUAL has predominantly focused on customer perceptions of quality. However, another approach used by many organisations is to send trained raters into the service environment, posing as customers to evaluate service levels.

English

Methodological challenges in evaluating mobile crisis psychiatric programs

Mobile crisis psychiatric programs (MCPPs) are innovative community interventions that have gained acceptance in the health, social and political environments. In Canada they are becoming widely implemented and the need to evaluate them is pressing. Unfortunately, there has been very little formal evaluation of them and virtually no data as to their effectiveness. Part of the reason for this deficiency may be the methodological challenges inherent in these programs.

English

Using the right tools to answer the right questions: the importance of evaluative research techniques for health services evaluation research in the 21st century

Because of the marked changes in health care expenditures in recent years, there has been a call for greater accountability in the area of health education, policy, services and reform. In recent years, evaluative research has been conducted in health arenas under the rubric of health services research. Health services research methods have not evolved from evaluation research methodology, but rather have adopted methods that often do not lend themselves to deriving appropriate causal inferences in multi-causal environments.

English

An Evaluation Framework For The Maison Decision House Substance Abuse Treatment Program

An evaluation framework was prepared for Maison Decision House (MDH) to guide its future examination of the success of its substance abuse treatment program. MDH is a halfway house accommodating substance-abusing male federal offenders. This residential intervention is based on a cognitive-behavioral model and has incorporated specific program activities to achieve the ultimate objective of lifelong abstinence.

English

The Politics And Practice Of Empowerment Evaluation And Social Interventions: Lessons From The Atlantic Community Action Program For Children Regional Evaluation

Empowerment or participatory evaluation has gained significant popularity in the last few years. However, there is considerable variation in the manner in which the terms are used and in the politics that underlie and inform such practices. This article, which reflects upon the authors' collective experience with the Atlantic Community Action Program for Children (CAPC) regional evaluation, argues for empowerment-oriented evaluation strategies informed by emancipatory politics.

English