Alt-Assessment: But what if the Bridge Collapses?
Debunking a common defense of traditional assessment
I’m writing up my research on faculty members’ experiences with alt-assessment across disciplines, and I’m glad I found space to address a very common refrain in support of the status quo.
This argument has come up so often that people in the audience can anticipate it easily whenever I talk about grading because they’ve heard it, too.
Here’s the argument (and I’m sorry STEM folks, but it often comes from your people1):
If I don’t measure students’ ability to do X in my 100-level mathematics (or engineering or anatomy or whatever) course in the way I’ve always done it, they’ll go on to build a bridge that collapses (or kill a patient on the table or whatever).
You’ve heard this? The first problem with the argument is basic: It assumes that what we’ve been doing for 60 years is the right/best/only way and that we have data to support it, a flawed premise.
I address the argument from a different angle because it came up in one of the interviews when a faculty member shared her experiences in hearing from traditional assessors. I contextualized it this way:
…those of us operating in power-aware spaces will argue that power and profit are far more responsible for matters such as bridge failures than individual knowledge of trigonometry: We can look to disasters such as the explosion of the Challenger space shuttle (Weick, 1997), the gas leak in Bhopal, India (Shrivastava, 1994), the Three Mile Island nuclear disaster (Walker, 2004), and bridge (Mazzetti, 2019) and building collapses (Baker & LaForgia, 2021) in Florida for further understanding of how those dynamics influenced tragic outcomes.
Each of these situations includes thick strands of power that influenced outcomes as evidenced by the post-event analyses. Individuals made decisions within the context of those power dynamics. What we often attribute to individual decision-making or knowledge is more often a product of a constellation of power dynamics, most of which have material roots.
In addition to those crises, I also think about Betty Vinson at WorldCom back at the turn of the century. She knew the creative accounting she was asked to implement was wrong, so her actions weren’t the result of a lack of knowledge. Her accounting instructors are free and clear!
Instead, it was power that influenced her choices, not only the power of her direct superiors, but the power of contextual factors that include identity and material consequences.
In this WSJ article, the writer uncritically shares details that power-informed readers can immediately pick up. The following are direct quotations from the article:
Asked by her bosses there to make false accounting entries, Ms. Vinson balked -- and then caved. (In other words, not a problem of knowledge.)
And when an employee's livelihood is on the line, it's tough to say no to a powerful boss. (Yeah. Material conditions.)
Ms. Vinson developed a reputation for being hardworking and diligent. She was known as a loyal employee who would "do anything you told her,” says a former colleague. (Researchers in both sensemaking and critical sensemaking [Helms Mills et al., 2010] assert that our making sense of a situation flows through identity: We make sense of situations based on who we think we are.)
The first two bullets indicate the impact of hard power, that is, the influence of powerful people who can change the trajectory of an entire family’s existence in very real ways.
The third is the influence of identity in enacting our next steps: Someone who sees herself as a team player, who prioritizes the good of the org, who is known as “loyal,” will likely make decisions in support of that identity. That’s the one individual factor in this constellation of dynamics, but I don’t think it’s the heaviest one.
Here is the kicker for me:
But after further thought, Ms. Vinson decided against quitting, says the person close to her. She was the family's chief breadwinner, earning more than her husband's roughly $40,000 a year compensation. The Vinsons depended on her insurance. She was anxious about entering the job market as a middle age worker.
The problem of seeing individual decision making strictly as the result of individual dynamics is that it ignores the impact of power and assumes a type of rationality that doesn’t play out in real time: That particular rationality assumes that doing the right thing is weighted heavier in the personal cost/benefit analysis than supporting my family.
In this case, Vinson was constrained by both formative context and material conditions: The fact that her family’s health and well-being hinged on her employment. Health insurance is generally tied to one’s employment here in these great United States (I hope that at this point we see how messed up that is).
She also engaged prospective sensemaking about another formative context: The fact that ageism is rampant in the hiring process, especially for women (true then and true now).
We are a culture that often views individual choices as simply individual and simply rational, ignoring context and the effects of intersecting strands of power.
I think we’re also not adequately knowledgeable about how those intersecting strands of power exert unequal pressure. For example, I love seeing when people take a stand and walk away from situations that compromise them ethically; at the same time, I recognize the privilege of the ability to do so.
My question when considering human decision making is “what are the conditions that influenced this outcome?” I always want to know how the field of plausible options looks to an actor. Whether the question is about conspiracy to commit crimes, bridge collapses, or student plagiarism, I want to know about the dynamics that frame decisions2. What makes that choice the least worst?
All of that is to say, then, that the idea that a bridge will fall down if we change how we’re assessing student learning doesn’t hold up3. Bridges fall down, but at no point will that outcome be relegated to one engineer’s knowledge of one concept because if that one concept is enough to bring down a bridge, clearly it would have been reinforced across a curriculum.
More likely, catastrophes happen when people find themselves making what they believe is a least worst choice from their field of possible options as they see them. Very often that field is constrained by material matters, as evidenced by the FIU project where the full scale of inspections and peer review was not supported by the project’s budget.
It wasn’t that the reviewer didn’t know something; it was that they didn’t have the opportunity to put that knowledge to use because of material constraints. Period.
I suggest that in addition to knowing X concept, students also really need to know how systems of power influence individual behavior. They need a vocabulary to understand these dynamics so that they can recognize them when they experience them (and they will).
Alternative assessment is an ideal avenue into conversations about power, which — even more than understanding the details of differential equations — can influence one’s view of a situation as well as inform their identification of what needs to change to create conditions that drive desired outcomes.
Someone who understands that ethical behavior is driven — in part — by material conditions and formative contexts can work to dismantle incentives for acting unethically. They can work toward ensuring that safety is not subverted for the sake of profit. They can better understand the significance of dynamics that facilitate and constrain decision makers.
Discussing power matters is important because power matters.
Interested in a unique conference experience that focuses on intersecting influences of power, progressive educational practices, and school transformation? Please consider attending the Human Restoration Project’s upcoming conference:
Conference to Restore Humanity! is an international invitation for K-12 and college educators to engage in a human-centered system redesign: centering the needs of students and educators toward a praxis of social justice.
I love the conference design:
This conference is designed from the ground up to best fit a virtual format. We recognize that the traditional conference format can be overwhelming, costly, and difficult to learn within.
And cheap! Only $150 with early-bird registration by June 30, $200 thereafter.
Full disclosure: It comes from the writing people sometimes, too: “If I don’t grade students on their use of the language of power (habits of white language), they won’t have access to power, and I’ll be doing them a disservice.”
I think the sentiment comes from a good place, but two things: First, we can do both. We can teach students how to activate levers of power as they exist while also interrogating those systems, situating them as the gate-keeping mechanisms they are, and showing students how to use power in more positive ways to make positive change. We can teach the dominant discourses without punishing the languaging students bring with them into the classroom (hence my focus on feedback and not grades).
Second, just knowing “the language of power” is not enough. Knowing how to navigate a busted system is not enough. In fact, “I can’t breathe” is a perfectly grammatically sentence, and yet . . .
These dynamics directly inform faculty decisions about alt-assessment, too, and I’ll share more about that next week.
Further, that position assumes that we can’t adequately measure student learning outside of traditional approaches, and that’s just not true. These guys teach math and will tell you all about it.