Dr. Roland Chrisjohn, Associate Professor, St. Thomas University
Empirical research typically categorizes data as quantitative (continuous variation) and qualitative (discrete variation). The distinction is neither hard nor fast, nor does it entail a great divide between the mathematical and philosophical sophistication demanded by each approach. Contemporary "qualitative analysis," though borrowing its name from discrete data analysis, is, in actuality, a form of hermeneutics or narratology, presenting itself as an empirical yet non-quantitative approach to research.
This presentation will not delve into justifying this interpretation, but rather, will challenge the pervasive portrayal of narratology as an essentially Indigenous form of empirical inquiry.
The notion that narratology reflects how Native peoples conducted research before contact is problematic for several reasons: it implies that Native peoples may dispense with more formal research methods, suggests racial incapability, and may serve to simply legitimize previously excluded voices rather than promote genuine empirical inquiry. Moreover, misrepresenting narratology as science to Native scholars undermines their sincere efforts to address important issues, relegating their work to irrelevance.
Dr. Roland Chrisjohn is an Oneida of the League of the Haudenausaunee. He has been both practitioner and academic for well over 50 years. He received his PhD in Psychology from the University of Western Ontario (Personality, Measurement, & Multivariate Analysis) in 1981 and obtained clinical certification in 1983. From 1982 until 1988 he worked in the Crisis Unit of Toronto East General Hospital doing front-line suicidology work. He has participated in or conducted numerous evaluation projects, including the Cariboo Tribal Council’s research on lasting effects of Indian Residential Schooling, the Kainaiwa Nation’s evaluation of student achievement, and the first panel of Canada’s Longitudinal Aboriginal Health Survey. He is the author of The Circle Game: Shadows and Substance in the Indian Residential School Experience in Canada; Dying to Please You: Indigenous Suicide in Contemporary Canada (with Shaunessy McKay); and the forthcoming “...and Indians, too!” Indigenous Peoples and the Canadian Form of Racism.
For the last 25 years he has been an associate professor in the Department of Native Studies at St. Thomas University in Fredericton, New Brunswick. He lives in Lower Jemseg with his wife of 47 years and their two cats.
Patricia Rogers, Founder, BetterEvaluation
Successfully navigating an increasingly uncertain and unpredictable future requires significant changes to evaluation theory and practice to better support the collaboration, learning and adaptation needed.
This session will highlight some of the important ideas, principles and examples that need to be brought together, including theories of change that better represent interventions in complex systems; prioritizing evaluative inquiry as part of planning and implementation, including more rapid and real-time evaluation; more support to choose an appropriate combination of methods, processes and evidence for evaluation; using rubrics and global scales to synthesize diverse evidence; and embedding attention to equity and environmental sustainability in all evaluation.
Patricia will set out suggestions for action by individual evaluators, evaluation managers, and professional networks.
Patricia Rogers is an independent consultant, formerly Professor of Public Sector Evaluation at the Royal Melbourne Institute of Technology (RMIT University), Australia, and founder of BetterEvaluation, the global open access knowledge platform on evaluation methods, processes and approaches, now part of the Global Evaluation Initiative. She is also a co-founder of the Footprint Evaluation Initiative, which is working to ensure all evaluation includes environmental sustainability.
Patricia is an experienced evaluator, advisor, educator and researcher who has worked for more than thirty years with a wide range of sectors, countries and organizations, including national and sub-national government departments, UN agencies, development banks, NGOs and philanthropic organizations. Her work, especially on theories of change, addressing complexity, and situationally appropriate evaluation design, has been widely used in evaluations and monitoring and evaluation systems and been recognised by the American Evaluation Association (Gunnar Myrdal Evaluation Practice Award) and the Australian Evaluation Society (Outstanding Contribution to Evaluation Award).
Patricia completed a PhD in evaluation from the University of Melbourne, and a post-doctoral fellowship in evaluation at Harvard University. She is a Fellow of the Australian Evaluation Society. She is the co-author (with Sue Funnell) of the book, Purposeful Program Theory.
This interactive plenary panel raises critical questions about what matters most - ethically and morally - in evaluation practice. Panelists will draw on their practices, research, and experiences to highlight key ethical challenges and priorities in evaluation practice today. This will include discussion of the evolution of CES ethics noting key developments in mapping core professional values integral to ethical evaluation practice in Canada. Panelists will facilitate interactive discussion on practical applications of tools and frameworks that can help evaluators identify and address ethical concerns.
Harry Cummings, CE: Harry Cummings is a past president and treasurer of CES national. He presently directs Harry Cummings and Associates Inc doing approximately 20 evaluations each year and employing from 3-5 evaluation professionals. He taught program evaluation at the University of Guelph for 20 years and supervised over 100 MSC and PHD students.
Harry's areas of practice have included Canadian and International NGOs involved in humanitarian and social service support more directly. A subspecialty is economic impact assessment. He has also worked for the federal government, provincial governments, regulated colleges of health, and municipalities in the health, food, agricultural, municipal planning, youth mental health and related areas. More recently the equity diversity and inclusion agenda has been important including a focus on the immigrant and refugee issue. The ethical conduct of research has been an important element of his work.
Harry has a PhD in geography, is a Registered Professional Planner and a Credentialed Evaluator (CE). He is fluent in French, English, and Bahasa Indonesia.
Betty Onyura: Following her PhD in organizational psychology, Betty Onyura began her career helping organizations use evaluation to gain the insight needed to optimize or sustain programs and innovations. For the past decade, she has held multiple roles as a leader, educator, and scholar at the intersection of education and health care. Presently, she is the Director, Knowledge Mobilization within the Centre for Addiction & Mental Health (CAMH).
Betty identifies strongly as a scientist-practitioner (or practitioner-scientist) and holds a scientist appointment at the Wilson Centre at the University of Toronto. There, she leads a SSHRC-funded, program of research that is focused on the science of evaluation, with particular interest in the socio-political, and ethical tensions associated with evaluation as a social practice.
Josephine Watera, CE: Josephine Watera is a seasoned evaluator with sixteen years working experience in the evidence-informed decision-making space. She is an Assistant Director, Research Services and previous Head of the Monitoring and Evaluation Division in the Parliament of Uganda. She supported the development of guidance for responsible evaluation for the Independent Evaluation Group (IEG). In 2021, she led the development of the African Evaluation Principles by the Africa Evaluation Association (AfrEA). She is a member of the Independent Evaluation Panel of the Global Fund, the CES Ethics Guidance Working Group, and the National Monitoring and Evaluation Technical Working Group. She is a former Board Member of the International Development Evaluation Association (IDEAS).
Josephine has mentored a number of Young and Emerging Evaluators under the EvalYouth International Mentorship Program of EvalPartners and Search for Common Ground Global Mentorship Programme. In 2023, Josephine received the African Evidence Leadership Award-Evidence User Category by the African Evidence Network and her Credentialed Evaluator designation. In 2017, she was recognized by the American Evaluation Association as the most promising evaluator from developing countries.
Josephine is a PhD candidate in Programme Evaluation at the University of Cape Town.
Holly Zapreff: Holly Zapreff has worked in the not-for-profit sector for almost 18 years where she has been passionate about empowering the most vulnerable populations in order to reduce inequities. More recently, she was introduced to evaluation, and quickly realized her interest in the field. She has since completed a Graduate Diploma in Public Policy and Program Evaluation through Carleton University and will begin her Masters in Public Policy and Administration at York University this Fall.
Throughout her work in the not-for-profit sector, Holly has often balanced the needs of funders and other stakeholders with the well-being of the diverse communities which she has served. As such, she recognizes the complexities inherent in navigating ethical dilemmas, and has actively engaged in initiatives promoting transparency and ethical practices in social services. As a student and emerging evaluator, Holly is interested in further exploration of ethical principles, and how they intersect with the practical challenges faced by professionals in the field of evaluation.
Beth Snow, CE: Beth Snow is the Head of Evaluation at the Centre for Advancing Health Outcomes and a Clinical Assistant Professor in the School of Population & Public Health at the University of British Columbia. She has been an evaluator in the healthcare sector in British Columbia for more than a decade. She also teaches program planning and evaluation in the UBC Master of Public Health and Master of Health Administration programs and statistics at the Justice Institute of British Columbia.
Beth holds the Credentialed Evaluator (CE) designation and is the Vice President of CES, the chair of the Ethical Guidance Working Group, and a Student Evaluation Case Competition Coach.