Scott G. Chaplowe and J. Bradley Cousins. (2016).
Monitoring and Evaluation Training: A Systematic Approach . Thousand Oaks, CA: SAGE. Hardcover, 439 pages. (ISBN 978-1-4522-8891-8)
Reviewed by Michele Tarsilla , UNICEF, West and Central Africa, Regional Offi ce, Dakar, Senegal

When I first looked at the title of this book, I confess that I had mixed feelings. First, the book’s focus on “monitoring and evaluation” seemed a bit too broad: monitoring and evaluation certainly complement each other; however, their respective specificities risk being neglected when the “M” and the “E” are bundled together. Next, the book’s focus on “training” seemed too narrow to tackle the complexity of the M&E capacity realms. Specialized literature confi rms that training by itself is only one of the many strategies one could pursue to strengthen M&E capacity. I have argued for years that the excessive attention given to training (a phenomenon that I refer to as the objectification of capacity) has undermined efforts made thus far to strengthen national M&E capacity. To the contrary, it is through interventions and strategies geared toward capacity strengthening at not only the individual but also the organizational and systemic levels that sustainable and contextually relevant Evaluation Capacity Development (ECD) occurs (Tarsilla, 2014).

Despite efforts for greater patient engagement in health care quality improvement, evaluation practice in this context remains mostly conventional and noncollaborative. Following an explication of this problem we discuss relevant theory and research on patient-centred care (PCC) and patient engagement and then consider potential benefits of collaborative and participatory approaches to evaluation of such initiatives. We argue that collaborative approaches to evaluation (CAE) are logically well-suited to the evaluation of PCC initiatives and then suggest contributions that the evaluation community can offer to help advance patient engagement. Finally, we outline a research agenda that identifies important areas that are in need of further examination.

How do evaluators using collaborative approaches to evaluation (CAE) define success? This is the core question being asked in a further analysis of data from our previous work (Cousins, Whitmore, & Shulha, 2013 ; Shulha et al., 2016) that developed a set of evidence-based principles to guide collaborative evaluation practice. Probing data from 320 responses to our (2012) survey, we examined what respondents considered "highly successful" and "less successful than hoped" in their collaborative evaluation projects. The results revealed that evaluation use, relationships, and information needs are key factors. We propose a conceptual framework as an aid to thinking about success in CAE.

Through a guided discussion, this article explores a five-year cross-cultural evaluation relationship comprising multiple projects involving an evaluator from Canada and a group of Indian colleagues working on educational reform in India. The initiative was funded through a multilateral consortium of donors and involved Western evaluation specialists working in collaboration with Indian colleagues to (a) develop evaluation capacity within the country and (b) produce evaluative knowledge about education quality initiatives associated with large-scale educational reform. This article is based on a conversation between the principal investigator from Canada and three Indian colleagues who had been involved in all phases of the work. It focuses on their respective perspectives and experiences, including the benefits obtained and the challenges encountered in the process of bridging Western and Indian knowledge systems. The article begins with background about the initiative and continues with a conversation among the participants about their cross-cultural evaluation experience. It concludes with an analysis of the issues that emerged and generation of lessons learned for evaluators interested in cross-cultural evaluation.

This article provides commentary for the thematic segment titled "Applying a variety of methods to the evaluation of various efforts aimed at transferring knowledge generated from research." The authors revisit arguments supporting inquiry that takes up the challenge of connecting cognate fields of evaluation utilization and the broader domain of knowledge utilization. The central contribution of each of the foregoing articles is identified and situated within the context of ongoing inquiry in this domain.

Despite increasing interest in the integration of evaluative inquiry into organizational functions and culture, the availability of empirical research addressing organizational capacity building to do and use evaluation is limited. This exploratory descriptive survey of internal evaluators in Canada asked about evaluation capacity building in the context of organizational characteristics (learning, support structures), evaluative activity and use, and variables that mediate use. We received a total of 340 usable responses to an online survey. This article provides a descriptive account of the findings with a cursory look at differences across respondent role, organization type, and self-reported perceived level of evaluation knowledge. Results showed a pattern of moderately high ratings of organizational learning and support functions, the extent to which evaluation is being conducted and used, and stakeholder involvement in evaluation. Some differences across respondent roles, organization type, and evaluation knowledge were observed. Results are discussed in terms of an agenda for future inquiry.

According to the literature published on the topic, the development of an organization's capacity to do and use evaluation typically follows four stages: traditional evaluation, characterized by externally mandated evaluation activities; awareness and experimentation, during which organizational members learn about evaluation and its benefits by participating in a number of evaluation-related activities; evaluation implementation, the stage at which the role of evaluation is more clearly defined in the organization; and evaluation adoption, which occurs when evaluative inquiry becomes a regular and ongoing activity within the organization through the allocation of continued financial and human resources. In this article we argue that this perspective is oversimplified and that it is essential to understand the complexity of an organization's evaluation capacity in order to better understand how it might proceed with evaluation capacity building (ECB). We present an analysis of four Canadian federal government organizations' self-assessment of their organizational evaluation capacity using a profile conceptual framework developed as part of our larger study. We then integrate the resulting multidimensional profiles of observed levels of organizational evaluation capacity with the aforementioned stages of ECB to provide added value in thinking about the direction of organizational ECB.

The special issue is devoted to the examination of organizational capacity for evaluation and evaluation capacity building (ECB) through empirical inquiry. The compilation consists of two quantitative surveys of evaluators and seven single or multiple case studies across a broad array of organizations in a diverse contexts (e.g., east-central Ontario, California, Hawaii, Minnesota, and Israel). In this final article, the authors look across the collection of studies to identify emerging themes and trends with implications for ECB. The emergent themes are defining ECB; conceptualizing ECB outcomes; organizational context; ECB implementation issues; and enabling factors and barriers to organizational evaluation capacity development.