Every time an AI screening tool produces a discriminatory outcome, the framing is "AI bias." I want to push back on that framing — not to defend the outcome, but because the wrong diagnosis leads to the wrong treatment. The AI isn't the problem. The data is. And fixing the data is a lot harder than switching vendors.
---
Let's start with what the research actually shows.
A University of Washington study found that AI resume screeners preferred candidates with white-associated names 85% of the time. A separate study by VoxDev found that LLMs systematically favor female candidates for certain roles while disadvantaging Black males — a pattern of intersectional bias that varies by role type and framing. These are real, documented, reproducible findings. They matter.
But here's what I think gets missed in the coverage: these models didn't independently develop preferences for certain names or demographic characteristics. They learned patterns from historical data — hiring decisions made by humans, over decades, that encoded existing discrimination into the training signal. The AI is, in a narrow technical sense, doing exactly what it was designed to do: identify patterns in data and generalize from them. The pattern it found was human bias.
This distinction matters enormously for how you fix it.
If the problem is "AI bias" — something that exists in the model itself, separate from its training — then the solution is a better model, or no AI at all. Some advocates land here: ban AI in hiring because AI is biased.
If the problem is data bias amplified by AI — historical discrimination encoded in training data, then applied at scale and velocity by a system that doesn't second-guess itself — then the solution is data governance. It's auditing your training sets. It's validating model outputs against real-world demographic distributions. It's building the human review processes that actually catch and correct for drift. It's not a simple fix, but it's a tractable one.
The UW research adds a layer that most people gloss over. When the study provided humans with AI recommendations — even ones the study participants were told were biased — those humans mirrored the AI's bias roughly 90% of the time in severe cases. The ostensible safeguard, human review, largely failed because humans default to deferring to the system in front of them. This is automation bias: the cognitive tendency to trust algorithmic outputs over independent judgment.
This finding has significant implications. The two-part argument that most employers make — "we use AI, but humans make the final call" — breaks down when the human is effectively rubber-stamping whatever the AI recommended. The human in the loop is only meaningful if that human has both the information and the organizational permission to disagree with the AI.
So what does actually work?
Training data governance is the starting point. If you're buying a third-party AI screening tool, the first thing you should audit — or require the vendor to audit — is the demographic composition of the training data and the historical hiring outcomes that were used as ground truth. If the tool was trained primarily on successful hires from organizations with documented diversity gaps, it will replicate those gaps.
Adverse impact testing at deployment is non-negotiable. Before you run any AI model on real candidates, you need baseline testing that measures selection rates across protected classes. If the model selects Black candidates at 80% or less of the rate it selects white candidates (the four-fifths rule), that's a legal and ethical problem you need to address before go-live — not after your first complaint.
Human override design matters as much as human override existence. The bias dropped 13% when study participants completed an implicit association test before reviewing AI recommendations. That's not a large effect, but it demonstrates that structuring the human review process — rather than just inserting a human into it — changes outcomes.
The industry conversation is stuck on AI bias as a vendor problem. It's actually an organizational capability problem. The vendors building the best tools are doing the training data work. But the employers deploying those tools often have no governance infrastructure to detect when they're working as intended versus when they've drifted.
Build the infrastructure. The vendors can't do it for you.
---
Quick Hits
VoxDev Research on Intersectional Bias
The VoxDev/An et al. research is worth reading in full. The finding that LLMs favor female candidates on average obscures a more complex pattern: those same models disadvantage Black male candidates specifically, producing intersectional outcomes that can't be understood by looking at gender or race in isolation. This is a preview of the complexity that AI bias audits will need to handle — single-axis analysis misses intersectional patterns almost by definition.
UNESCO's LLM Gender Bias Findings
UNESCO published research showing that LLMs trained on internet-scale text data encode gender stereotypes from that text in ways that persist into deployment — associating certain professions, competencies, and personality traits with gender in ways that reflect cultural biases, not objective signal. The implication for HR tech: any model that uses general-purpose LLMs as a foundation inherits those biases unless explicit debiasing work has been done. Ask your vendors specifically about this.
Automation Bias as a Cognitive Trap
Automation bias — the tendency to over-rely on algorithmic recommendations — is well-documented in aviation, medicine, and financial services, and it's now clearly present in hiring contexts. The UW data puts it in stark relief. Designing against automation bias requires more than adding a human approval step; it requires giving that human independent information that doesn't simply echo the AI recommendation. Interface design and process design matter here, not just policy.
---
The Operator's Take
I'll be direct: the "ban AI in hiring" argument treats the status quo as neutral, and it isn't. Human recruiters are also biased — provably, consistently, and often in ways they'd deny if asked. The question is whether AI-assisted hiring, with proper data governance and auditing, produces better or worse demographic outcomes than unassisted human hiring. The honest answer is: it depends entirely on how the AI was built and how it's deployed. A well-built, well-audited AI screening tool can outperform human screeners on demographic fairness. A poorly built or poorly governed tool will be worse. The solution is better AI, not no AI. But "better AI" requires doing the hard work of data governance, which most organizations are not currently doing.
---
Running an actual bias audit on your AI screening tools requires a structured methodology — not just looking at overall selection rates, but testing across demographic intersections, comparing against a baseline, and documenting findings in a way that supports legal defensibility. The AI Bias Audit Checklist walks through that methodology step by step, including the specific statistical tests you need and how to interpret the results.
Get it here → AI Bias Audit Checklist
---