What is important to understand about the evaluation process in the First Nations context?

Emilie Grantham, Patricia Montambault and Céline Yon,
FNQLHSSC Research Agents, Wendake (Quebec) Canada

Composed of 41 communities present throughout the territory of Quebec, the First Nations are divided into ten nations: Abenaki, Algonquin, Atikamekw, Cree, Innu, Maliseet, Micmac, Mohawk, Naskapi and Huron-Wendat. Each of them has a history, beliefs, lifestyles and know-how of its own. In Quebec, the First Nations represent about 1% of the population.

The First Nations of Quebec and Labrador Health and Social Services Commission (FNQLHSSC) is a non-profit organization responsible for supporting the efforts of the First Nations in Quebec, which include planning and delivering culturally adapted and preventive health and social services. Its mission is to promote the physical, mental, emotional and spiritual well-being of First Nations and Inuit people, families and communities in respect for their cultures and local autonomy. One of the mandates of the FNQLHSSC is to conduct research, evaluations and population surveys, so that First Nations communities and organizations can possess data enabling them to improve services to their populations.

In this article, we discuss some of the specifics of an evaluative approach in the context of the First Nations, issues to consider and solutions that are available to evaluators.

In recent decades, the First Nations have developed and employed tools to enable them to exercise oversight and control of the processes of research and evaluation. Among them are the First Nations in Quebec and Labrador’s Research Protocol (Assembly of First Nations Quebec-Labrador, 2014),[1] designed for the use of First Nations communities and organizations. It echoes the movement of affirmation that has evolved in the world of research and information management for First Nations individuals, communities and territories. It offers an ethical framework, but does not replace the existing tools within First Nations communities and organizations. Rather, it highlights the various aspects that First Nations who are asked to participate in a research project or evaluation should clarify in order to adequately grasp the issues and implications of the project. Although the protocol is focussed on research, its principles and values ​​apply equally to evaluations.

No single approach is fundamentally favored by the protocol. Rather, it is considered that the values ​​of respect, equity and reciprocity as well as the principles of ownership, control, access and possession (OCAP™)[2] must be the common threads that inspire and guide research teams at each step of the process. The research protocol and the OCAP™ principles are also therefore indirectly addressed to the scientific community.

The OCAP™ principles were developed in the 1990s by the First Nations. They are statements of values ​​that promote the establishment of a relationship of mutual trust between First Nations, the scientific community, governments and others.

Ownership refers to the fact that a community or group collectively owns the information associated to its culture, in the same way that a person owns its personal information.

The principle of control reflects the aspirations and inherent rights of First Nations members in terms of taking and maintaining control of all aspects of their lives and their institutions.

The principle of access affirms that First Nations must have physical access to information and data that concerns themselves and their communities, regardless of where the data is stored.

Possession is not an essential condition to data ownership. It provides a mechanism which helps confirm and protect ownership.

The protocol strongly encourages collaboration, dialogue, knowledge transfer and the protection and recognition of First Nations knowledge. It invites First Nations to elaborate their own priorities for research and evaluation and to define methodological and conceptual approaches appropriate to the local culture. To accomplish these objectives, several tools support the research protocol, including a research agreement template. Training is currently being developed to promote the document to the largest number of First Nations possible and will be offered on request to external partners.

In addition to considering the research protocol and OCAP™ principles, evaluators carrying out evaluations in a First Nations context must be familiar with the various elements affecting the relationship of communities with the evaluative process. Few First Nations communities or organizations introduce evaluative mechanisms on their own initiative. The evaluation of a project, program or initiative is frequently in response to a demand for accountability. Frequently, one or more stages of the evaluation process are carried out by external consultants. It is evident that there is very little involvement of First Nations in planning and conducting evaluations.

The same issues regularly arise with evaluations conducted at the FNQLHSSC. During an evaluation, it is not uncommon that personnel turnover in the communities has an impact on data collection. In addition, the timetables imposed by donors and the terms of the funding they provide for the evaluation seldom align with realities on the ground. For example, the local cultural calendar can sometimes be at odds with the established schedule, and budgets relating to necessary travel to and from the community are not always sufficient, given the degree of remoteness of many communities. Another commonly encountered issue is the belief that the role of an evaluation is to judge the work of interveners and to determine whether or not to continue funding.

All these factors increase the risk of misunderstandings, negative perceptions of the evaluation process and suboptimal use of the results.

In response to these challenges, several approaches are recommended that benefit everyone concerned. Adopting a collaborative approach is beneficial, especially when the objective of the evaluation goes beyond simple reporting. A collaborative approach leads to evaluations that are useful and used, especially in support of First Nations processes of self-determination. This approach requires establishing a relationship of trust between the participating communities and the evaluator that proceeds from their needs, makes explicit the conduct of the evaluation and demonstrates openness and flexibility throughout the process. Other solutions emerge from the use of tools that situate research in the context of the First Nations and must be anticipated by the evaluators before undertaking an evaluation process.

Evaluators who want to work with First Nations must be sensitive to the fact that one of the objectives of the evaluation process is the strengthening of local capacities.

For further information on evaluations conducted by the FNQLHSSC and the tools that are used, visit the following website: http://www.cssspnql.com/en/fnqlhssc.

[1] Available at the FNQLHSSC Documentation Center: http://www.cssspnql.com/publications

[2] Refer to the website of the First Nations Information Governance Center (FNIGC): http://fnigc.ca/