4

Until the lions have their storytellers, the story of the hunt will always glorify the hunter. (Ngugi wa Thiong’o)

In order to learn about decolonizing evaluation, I first had to understand what colonialism looks like, and think about how it shows up in evaluation. I had to learn to see the Colonial Matrix of Power (CMP), which I described in Blog 1: Before the beginning …, as the modern structures and institutions of power and control based in colonialism that privilege Eurocentric ways of knowing and being as superior, progressive and universal, though they are actually racist, capitalist, misogynist, hetrosexist and patriarchal. Tiffeny Jiménez and Gordon Lee point out that epistemic hegemony (the domination of one view/form of knowledge and the subordination of all others) is rarely called into question and is the glue that sustains the CMP.

Seeing Colonialism

The thing about the CMP is that it is invisible to people who are privileged by it. So how do we go about learning to see it? In other words, how do we begin to unlearn what we have absorbed by osmosis, growing up in the dominant paradigm of thinking?

One way to start recognizing colonialism is to have something shake you awake. For many people in Canada - but not enough - the Truth and Reconciliation Commission of Canada (TRC) served this purpose.

The TRC was established as a result of the largest class-action settlement in Canadian history, the Indian Residential Schools Settlement Agreement. Over a period of 150 years, 150,000 First Nations, Inuit and Métis Nation children were taken from their homes and families and forced to attend Indian Residential Schools (IRS); many never returned. An IRS operated in the Sipekni’katik district of Mi’kma’ki, in Nova Scotia from 1930 to 1967, where about 1,000 students were forced to attend.

Between 2007 and 2015, the TRC traveled across Canada hearing the stories and experiences of 6,500 witnesses whose lives were impacted by IRS. The TRC concluded that the Indian Residential School system was a systematic, government-sponsored cultural genocide that tried to destroy Indigenous cultures and languages and assimilate Indigenous peoples so that they no longer existed as distinct peoples. The TRC hosted events to educate the Canadian public about the history and legacy of IRS. More people in Canada - but still not enough - have started to recognize colonialism and its impacts over the past couple of years with the discovery of mass graves of children’s remains at the sites of several former IRS.

Similarly, although the story of the oppression of people of African descent in Nova Scotia has been well documented, not enough people are paying attention and waking up to see the impact of the CMP on African Nova Scotians.

Decades of research show that in Canada, police stop, search, and question Black Canadians more frequently than people in other racial groups. In 2019, the Halifax Nova Scotia Street Checks Report found that African Nova Scotians were six times more likely than White people to be subjected to street-checks by police. Black people are more likely to be arrested and charged with criminal offenses than others with the same behaviours - in Nova Scotia Black men made up 2.1% of the population, but 14% of admissions to provincial custody. They are also much more likely than others to be seriously injured and killed during encounters with police.

Data from Ontario’s Special Investigation Unit show that from 2013-2017, Black people made up 8.9% of Toronto’s population, and were involved in 28.8% of cases where police use force, 36% of police shootings, 61.5% of cases where police used force that resulted in civilian death, and 70% of police shootings that result in civilian death. As well as being over-represented in the justice system, Black Canadians are the primary targets of racially motivated hate crimes in Canada.

The May 2020 murder of George Floyd by police in Minneapolis sparked demonstrations and protests against police brutality and racial injustice in support of the Black Lives Matter movement in more than 60 countries around the world, including in dozens of towns and cities across Canada. Black Lives Matter Canada is leading a public call to defund policing and reinvest funds to ‘Create a world where we all feel safe, and we all get what we need to live a life of dignity’. Some people were shaken awake by these events, but again not enough, and not for long enough.

Unpacking colonialism

Tiffeny Jiminez and Gordon Lee emphasize that we need to understand how colonial thoughts and behaviours that perpetuate harm are embedded in the way we make sense of reality, how we view our relationships with other humans and the rest of the world, and what we see as possible.

My personal work over the last several years has focused largely on seeing and unlearning the Colonial Matrix of Power. I have come to understand that in Canada all of our institutions and systems are inherently and implicitly racist, capitalist, misogynist, hetrosexist and patriarchal because they are the product of the colonial mindset. We point to past manifestations of colonialism - slavery, Indian Residential Schools - so we can separate and distance ourselves from it so we don’t have to feel bad. We fail to see the current manifestations of colonialism - the child welfare system; the arrest, harsh sentencing and incarceration of Black men; murdered and missing Indigenous women and girls - and we don’t see our complicity in perpetuating these colonial systems. We need to unforget, as Patty Krawec would say, acknowledging our willful intent to not remember the impacts of colonialism.

Unpacking all of this could create space for reimagining what we would like to manifest in the world. We might think about and work toward building mutually respectful and caring relationships with people who see things differently than we do, learning from each other, caring responsibly for the natural world we are all part of. Perhaps we could focus on cultivating a genuine understanding of what works well in communities, and how we can collectively strengthen communities in meaningful ways.

Colonialism in Evaluation

In the field of evaluation, some people are slowly starting to recognize that evaluation is embedded within a colonial paradigm, and taking steps to begin to address this. Kate McKegg calls on the evaluation field to recognize the role it plays in maintaining the dominant system of White privilege and power and its dominant use of western evaluation frameworks that exclude culturally distinct ways of knowing and being. Caldwell and Bledsoe suggest that the field of evaluation has been complicit in perpetuating theories, methods, and practices that exclude diverse ways of knowing and aren’t dominant culture focused. Hilliard and Stanfield both noted that the field is neither race nor culture neutral, and that its practices and approaches reflect a singular Eurocentric worldview, ignoring culturally diverse questions, definitions of data, and interpretations of findings - and jeopardizing the lives of groups of people who see the world through another lens.

Lived experience of colonial evaluation

JamBoard sheet from January 2022 Learning Pathway gathering

In one of our first Learning Pathway gatherings, participants described the most colonial evaluation they could imagine. Many were able to draw on lived experiences of colonial evaluation work in their communities. The notes from the group JamBoard sheet shown above suggest that colonial evaluations originate with an outside body (e.g., as a requirement of funders) and do not respect or take into consideration the historical knowledge and context of the community, do not involve the community in determining what is important to evaluate, are constrained by tight timelines, do not look at long-term impacts, and do not serve the community.

The language used to talk about evaluation is full of jargon, and can be intimidating to community members. The evaluation questions asked are narrowly focused on predetermined outcomes and what is prescribed by the funder, the questions are not determined by the community, and do not address community needs or value the community’s story. Data collection is limited to information about grants and does not draw on community expertise. Data analysis does not involve community or value different ways of knowing and being and interpreting meaning, success is defined by numbers and not by real impacts, and is not determined by the community.

Decolonized Evaluation

I think decolonized evaluation is evaluation that is free from colonial shackles. It has room for different ways of knowing and being, centres communities, and reflects their world views.

Bagele Chilisa and her colleagues describe the principles of an evaluation framework based on African values:

The value system of most African societies is built around respect for others and oneself. This respect is built around the concept “humanness or personhood” (Segobye, 2000 , p.3) or respect. A relational axiology is embedded in the Ubuntu relational ontology principles of (a) I am we, I am because we are; (b) relations of people with the living and the nonliving; and (c) spirituality, love, harmony, and community building (Chilisa, 2012 ). The emphasis is on values grounded on collective responsibilities, cooperation, interdependence, and interpersonal relationships among people (Carroll, 2008 ). From these principles, an ethical framework emerges that is focused on the responsibilities of researchers and evaluators and on the creation of respectful relationships between researchers, evaluators, and participants, and that takes into account the participants’ web of relationships with the living and the nonliving. Community spirit, cooperation, collectiveness, democracy, and consensus building are the values espoused through this philosophy (Chilisa & Preece, 2005). These value orientations also influence the evaluation theory of change, criteria or standards, indicators of success or failure of projects, and conclusions about the worth or merit of programs, policies or projects.

Joan Lafrance and Richard Nichols (2010) outline the core values and Indigenous evaluation practices of an Indigenous Evaluation Framework, which are summarized in the table below.

Drawing on these worldview approaches, a decolorized evaluation process – and its tools and methodologies - would be based on community ways of knowing and being, centre relationships, be grounded in community context, honour community values and beliefs, and serve the community.

Learning Pathway participants shared their wisdom about their evaluation practice in their communities based on some of the things we were learning together:

“I like that we’re decolonizing evaluation by using our own voices and perspectives, by being able to use our own symbols or share ideas amongst each other. I like the idea of this. In our small group, we talked about wampum belts. And how we use decorative beads to communicate important messages. It was not only an important message, but it was also art at the same time.

That [Jara Dean Coffey] article really spoke to me about using art like evaluation, and how we have to get out of our own heads, like everybody. It's easy to follow a framework that was already set but who made the framework? Why did they make it? And does it still fit? What we're doing today, it challenges things that no, maybe that's not the way we want to do it. Or maybe that's not an appropriate way to do it for us.. But at the same time, we need some type of a framework to keep everything organized”. (Learning Pathway participant)

——

“The braiding concept from the cultural artifacts we discussed two sessions ago is staying with me. I’m thinking about how when we bring the different perspectives and organizations together it builds that beautiful braid, that connection. That is some of the most powerful stuff, I’ve never thought about evaluation that way before - that was really really cool.

I think what that means is for me to understand that there are different perspectives. And when you do bring all of that together, when you allow for those different perspectives to be showcased, that it creates a beautiful evaluation that is more thorough. When we have different perspectives of things, there's not a one size fits all model. So when you look at different evaluation methods, and bring them all together you create a more accurate and more thorough evaluation”. (Learning Pathway participant)

——

I know how that played out for me and my grouping and how I framed those questions with intention to get at the heart of what was happening in practice. And that, to me, is a decolonized framework. You know, someone from community doing the work and understanding what they're looking for. Not just any old person can just come in and understand. (Learning Pathway participant)

——

“I think hosting the [final gathering] in a Black space was great; you knew that anyone walking into that space, even for Indigenous people, that was going to be an open space, because it wasn’t an inherently White space. That space was built for historically excluded people. So that space inherently and by historically excluded people, so it will inherently have safety, right? The way that people are able to move through it, or be in it or speak in it”. (Learning Pathway participant)

——

“For me, it was that holding the information sacred, and really understanding that that knowledge has come from not just this one individual, but this knowledge has been either passed on or come through a community or a lived experience. And really making sure that when you walk into that space, and you take this information when it's honoured, but also you understand the weight of the information that you have received. And you're then able to present that data and that information in an entirely different way.. And, you know, I've always had the mindset of it's not the numbers, it's the impact, especially in the work that I do that it is very impact driven, but how do you get that across in terms of reporting this data, or sharing the information, and that really shifted the weight of the kind of type of language I used in terms of conversation, and really shifted who's holding this. It’s not just “Okay, here's the data, you take it”, it's really more of a sharing, rather than losing it. That part really supported that decolonization where I have worked in such a colonized space for such a long time, but also then integrating it into how I do things in my everyday life” (Wisdom Circle member)

Decolonizing Evaluation

I'm now realizing that what I have been thinking of as decolonizing evaluation is actually practicing decolonized evaluation (evaluation that is not coming from a colonial frame of reference). From my place of privilege I have been thinking more about better ways to do evaluation, and not enough about how to change evaluation so that it can be practiced in those better ways.

“Decolonization is a big umbrella. And, you know, we didn't even talk about anything related to capitalism or colonialism or anything like that. It was more about spiritual practices, which I thought was not a complete picture of how to do decolonizing” (Learning Pathway participant)

The Learning Pathway experience has shown me that we can't practice evaluation differently without doing the work to change it. Decolonizing evaluation is the work of dismantling the colonial structures and removing the shackles so that communities can practice evaluation in ways that work for them - with respect and support from the rest of the evaluation ecosystem

“We're talking about decolonization practices, those are words that don't mean anything to community. I think that people understand their own circumstances and experiences. And to use that type of language kind of limits the conversation about how to actually engage, I think it would have been better to discuss barriers, or have brought them information prior, and instead of brainstorming what we could do. What are already established best practices, what sort of work and research has already been done? How does it connect to capitalism? Talk about what the actual tangible economic and social problems mean to people”. (Learning Pathway participant)

A Role for All Parts of the System in Decolonizing Evaluation

In Blog 2: Emergence and learning, I talked about looking at all parts of a system and how they interact, and paying attention to the consequences of those interactions as part of systems change work. That means thinking about communities, funders/government who require evaluations from communities, and evaluators.

Communities

African Nova Scotian and Mi’kmaw communities have to learn and own doing evaluation in their own ways that reflect their world views and values:

“And until you hear something like what Marlene (Dr. Marlene Ruck) presented … I didn't even know I thought the way I thought until I heard her speak. And I think that's a big part of the decolonizing work. Yesterday on LinkedIn, I saw somebody talking about how the concept of professionalism is a colonial way of locking people into certain ways of being right. So what are what's the equivalent to that in evaluation? What's locking us in? For me, a lot of it is the whole objectivity of data and things like that. Her presentation just really turned that on its head really.” (Wisdom Circle member)

Government

Government and other funders have to learn to see the Colonial Matrix of Power, and they have to figure out how they can help dismantle it within their spheres of influence. They also have to learn from communities about meaningful evaluation and how they can support it.

At our final gathering, one of the Learning Pathway participants who is a provincial government champion for communities and for evaluation talked about the most significant change she experienced being part of the Learning Pathway:

“My most significant change is in the context of looking at evaluation, from an academic perspective, and my work environment, which is government, and there's this table [Learning Pathway]. And the most significant part is how much this table exposes the limitations of what happens in the academic and the institutional environment around evaluation. And that's been really hard for me because I live in that world. And I don't know how to change it. And it's really difficult, but I think when Marlene (Dr. Marlene Ruck) was talking that night, I got it. I think we can read all the articles we want but until somebody like Marlene speaks up from the heart, and it's relational, then I think that change is really difficult to make. I feel like it would be amazing if Marlene could speak to government. But I also feel like it puts her in a space where she has to reveal something so close to her to a bunch of people she doesn't even know. So there's that real conflict there.

So my question that I came in and I'm leaving with is ‘How do we find not just the results that you'll find in your evaluations (of community projects) but it's also the insights that you bring to what evaluation is like, how can we share those? I want to hold on to that thread so that those in the government or in the academic world can start to understand that statement that the information you're working with is sacred. It's not some scientific exercise, or you know, it's not something you just do from nine to five, and then you turn it off. It's something that people are living every day. So thank you for being that third piece in my way of thinking”. (Government observer in Learning Pathway)

Evaluators

White evaluators in Nova Scotia have to help each other learn to recognize the Colonial Matrix of Power, and unlearn colonial practices of evaluation. Reflecting on Canada’s Truth and Reconciliation Commission’s Final Report and Calls to Action, Larry Bremner suggests that evaluators and professional associations can contribute to truth and reconciliation by understanding the past realities of Indigenous communities and the roles we have played in perpetuating these realities. He suggests that through self reflection and a deeper understanding, evaluators can move toward facilitating and supporting Indigenous peoples controlling and developing their own evaluation agendas.

The Canadian Evaluation Society is working in a couple of new areas that could help people see how the evaluation ecosystem interacts. Its course, Truth and Reconciliation for Evaluators is designed to help evaluators recognize the impacts of colonialism on Indigenous peoples in Canada, recognize how colonial world views impact the evaluation process, incorporate cultural safety principles into evaluation practice, and to actively practice evaluation.

The Canadian Journal of Program Evaluation has a new journal section called Roots and Relations. Its purpose is to “honour our lineage, grow our kinship, and sustain our intergenerational legacies of Indigenous wisdom and practices in and through evaluation”.

Lessons About Decolonization from the Learning Pathway Process

Queen’s University’s Centre for Teaching and Learning in Kingston, Ontario cautions:

People disagree about what the end goal of decolonization and Indigenization is or should be. We suggest that rather than focusing on the end goal, you consider two elements. First, it’s important to think about the reasons you’re decolonizing: who are you doing it for and why are you doing it? This helps avoid issues of tokenism and recolonization. Second, remember that decolonization is a process, not a product. Instead of wondering where the finish line is, take a step along the journey and see where it leads you.

My Most Significant Change

Here is the most significant change I experienced through the Learning Pathway process, which I shared at our final gathering:

“The most significant change that I've experienced being part of this Learning Pathway is that I am very drawn to the braid image. However, I don't feel like I deserve or have earned the braid image. I think I feel the knot of my colonial mindset loosening as I open myself to thinking about and understanding things in other ways. So I picked up this little piece of climbing rope that I had, and I tried to undo the knot to show it to you and then I realized the knot is really hard to undo. The knot of my colonial thinking, while I have loosened it, is still there. And I hope I can untangle my knot enough to contribute a meaningful strand to the braid”.

Reflection & Implications

First we have to understand colonization: In order to understand decolonization, we first have to understand colonization. It is a White supremacist, capitalist, misogynist, hetrosexist and patriarchal.frame of reference from which all of our current structures and systems were built. It is alive and well, implicit in our current systems and infrastructures, with many different manifestations.

People need a wake-up call to see colonialism: The people who are privileged by this frame of reference do not see it and in some cases refuse to see it even with a big wake up call like the Truth and Reconciliation’s conclusions about the government of Canada committing cultural genocide of the Indigenous peoples who lived on Turtle Island for thousands of years before contact, or the murder of George Floyd by police, and the continuing police brutality against Black men. Even if some of us wake up to these truths briefly, many of us eventually slide back into our privilege where we are not reminded every day about the impacts of colonialism - an experience Black and Indigenous people  cannot escape because they are faced with it and impacted by it every day.

Evaluation is rooted in a colonial mindset: Evaluation is an ecosystem that is based on a colonial mindset. We must recognize this truth as a field so that we can begin to undo the colonial structures and manifestations in evaluation and make way for decolonized evaluation that is free from colonial shackles and has room for different ways of knowing and being, centres communities, and reflects their world views.

Decolonizing evaluation is the act of dismantling the colonial structures: Decolonizing evaluation is the act of dismantling the colonial structures and removing the shackles so that communities can practice evaluation in ways that work for them. It requires that communities build their understanding of evaluation and appropriate practices. It requires that White evaluators recognize our part in holding up and perpetuating colonial beliefs and practices and that we change evaluation so that it serves communities. Decolonizing evaluation means that governments and funders recognize the colonial power structure and work to change the way they view evaluation so they can support evaluation that truly serves communities. All parts of the evaluation ecosystem have to do the work of changing evaluation so that it can transform communities and systems and serve racial justice. 

This work is an act of love, humility, and faith in humanity: With the Learning Pathway we were attempting what Paulo Freire would call an act of love, humility. and faith in humanity that requires hope, mutual trust, and critical thinking people who are in dialogue together.  Although we have different levels of power and privilege, and very different experiences of the oppressive system we are all part of (the CMP), we have to work together to humanize each other and lift each other up.

Black and Indigenous evaluators are the ones to teach Indigenous and Africentric evaluation approaches: Our experience aligned with Paulo Freire's argument that people who are oppressed have to lead the struggle for liberation. Only Black and Indigenous evaluators can offer authentic teachings about framing evaluation practice based on their worldviews and values. The role of White colonial evaluators and government champions is in part to support the creation of spaces where Black and Indigenous communities can come together and grow their own evaluation approaches. Government champions have to work to dismantle clonal mindsets within their spheres of influence; they must also listen to what makes evaluation meaningful to communities. 

Critical Reflection Questions:

Am I doing my work to recognize the Colonial Matrix of Power and see its manifestations in current systems and practices? Am I truly open to unlearning what I have been taught from my place of privilege? Where do I need to dig in deeply to build my understanding?
How can I learn to see the manifestations of colonialism in my practice? How can I have conversations about colonialism with my peers, co-workers, colleagues, teachers, and students? So that we can help each other learn and begin to decolonize our practices?
Where do I have influence, and how can I use my influence to support others in seeing and unlearning colonial practices?
Am I listening deeply to understand and respect the views of Mi’kmaw and African Nova Scotian community members? How can I act on what I hear if it is different than what I think?
Where is colonial thinking showing up in my work? How can I address it?
What is making me feel uncomfortable, and where is my discomfort coming from? Do I need to let go of the way I am thinking about something?
What is my role in decolonizing evaluation? What is mine to take on? How will I know? How can I support others in their roles to decolonize evaluation?
How am I building and strengthening respectful relationships among everyone in the evaluations I am conducting?
How much do I understand about the lived reality of the communities I am working with? How can I build my understanding without burdening community members? How does this understanding affect my approach to evaluation?
How can I best support the creation of space for Black and Indigenous communities to build their own practices and approaches to evaluation based on their worldviews?
Where do I need to get out of the way?

Here is a description of the Learning Pathway. It’s set up as a stand-alone piece so readers can refer to it easily throughout the blog series.

The main purpose of this blog series is to share learnings (and perspectives) from the Learning Pathway experience that could be valuable to others interested in transforming evaluation so that it can better serve racialized communities, and influencing the evaluation ecosystem to support racial justice.

In the blogs series we will be exploring:

Search the Learning Pathway

4