Theme: “racist AI” isn’t a robot with a slur in its code. It’s old bias, trained on an unequal world and fired back into it at machine speed — then laundered as objective math. The government and companies like Palantir build the layer that aims it.

People hear “racist AI” and picture a cartoon — some robot in a white hood. That’s not it, and the cartoon is exactly why it’s so easy to wave off. Real algorithmic racism doesn’t announce itself. It wears a lab coat. It shows up as a number, a match, a risk score — something that looks neutral because a computer produced it. And “the computer said so” is one of the most powerful sentences in the English language, because it ends the argument before it starts.

Here is the mechanism, and it isn’t complicated. You train a system on the past. The past is unequal. The system learns that inequality as if it were a law of nature, and then applies the “law” to the future — faster, cheaper, and at a scale no human bureaucracy could ever reach. It doesn’t invent the bias. It automates it. That’s the dangerous part: it takes a prejudice that used to need a biased human in the room and turns it into infrastructure that runs while everyone’s asleep.

It’s not a theory. It’s measured.

This isn’t a vibe. It’s in the government’s own numbers.

In 2019 the National Institute of Standards and Technology — a federal agency, not an activist group — tested 189 face-recognition algorithms from 99 developers against roughly 18 million photos. The finding: in one-to-one matching, Black and Asian faces threw false positives at rates 10 to 100 times higher than white faces. In the one-to-many searches police actually use to generate suspects, the worst false-positive rate landed on Black women — the precise failure that turns an innocent person into a “match.”

And it did. Robert Williams, a Black man in Detroit, was arrested in front of his daughters for a crime he didn’t commit, because software picked his face out of a database. He wasn’t the last. The short, grim list of people wrongly arrested off a face-recognition hit has one thing in common: the faces are Black.

Go up a layer, to risk scores. In 2016 ProPublica examined COMPAS, an algorithm used in courtrooms to predict who will reoffend. Black defendants were nearly twice as likely to be falsely flagged high-risk; white defendants were more often mislabeled low-risk and waved through. (The vendor disputed the analysis — that fight is real, and I’ll come back to it.) And predictive policing has a feedback loop built into its bones: send more police where you’ve policed before, you log more arrests there, the model “learns” that’s where crime lives, and it sends more police. The map of “crime” quietly becomes a map of where you already looked.

Where the government and Palantir come in

An algorithm is just math until someone points it at people. That aiming — the deployment layer — is where the government and its contractors live, and it’s where a company like Palantir matters.

Palantir builds the plumbing that fuses scattered data into a single lens. In April 2025, ICE awarded it a $30 million contract — about $60 million with the follow-on, and with no competitive bid, because ICE declared Palantir the “only source” that could do it — to build ImmigrationOS: a system to identify and prioritize deportation targets, provide “near real-time visibility” into where people are, and streamline the machinery of detention and removal. It braids together IRS tax records, Social Security data, State Department passport and visa files, DMV photos, license-plate readers, and border biometrics — the loose threads of your life, pulled into one view.

Nobody voted on it. Congress found out from procurement records. There’s no published error rate and no independent audit — even though the American Immigration Council warns that a database mistake here doesn’t cost you a late fee. It costs you your freedom, inside a system aimed overwhelmingly at brown people.

To be fair — and I want to be — Palantir pushes back hard. The company says it builds data-integration tools, not surveillance; that every interaction is logged, which it argues makes its platforms “exceptionally poor tools for abuse”; that there is no secret “master database”; that it all runs under data-sharing agreements and government oversight. Take that seriously. A tool is not a motive. Palantir doesn’t write immigration policy, or the sentencing guidelines, or the patrol routes. It builds the lens. The hand that aims it is the government’s.

But that is exactly the thing. “We just build the tool” is a real answer and an incomplete one. When you build a lens this powerful and hand it to a system with a documented habit of pointing at the same people, you don’t get to act surprised about where it looks.

Where this honestly breaks

Now the honest part, because a one-sided version of this is its own kind of lie. “Racist” usually implies intent, and most of these systems have none — no engineer typed a slur. The sharper term is disparate impact: the outcome lands on race even when the intent didn’t aim there. That distinction matters, legally and morally, and flattening it is lazy.

And the tech isn’t uniformly broken. That same NIST study found the best algorithms were far more equitable — which means bias here isn’t destiny, it’s a training-data and accountability problem, and those are fixable. Some crime is real; some of these tools genuinely help find missing kids and actual predators. The point was never “ban the math.” The point is narrower and harder: a system that fails worst on the people already failed worst by everything else, and then hides that failure behind the word “objective,” is not neutral. It’s the old bias with better PR.

Why I build the way I build

This is why I care so much about things you can see the inside of — open, local, auditable, owned by the people they touch, not a black box some agency rents and aims at your block. If a machine is going to make a decision about a human being, the human being should be able to open it up and ask it why.

“The computer said so” should be the start of the argument. Never the end of it.

---

This is analysis and opinion — my read of the public record, not a legal finding. Companies named here dispute some of these characterizations, and I’ve tried to say so plainly.

Sources: NIST, “Study Evaluates Effects of Race, Age, Sex on Face Recognition Software” (Dec. 2019); ProPublica, “Machine Bias” (2016); ACLU, on the wrongful arrest of Robert Williams; American Immigration Council, “ICE to Use ImmigrationOS by Palantir”; State of Surveillance, “ICE Paid Palantir $30 Million to Build a Deportation Operating System” (2025); Palantir, “Correcting the Record” (company response, Jan. 2026); The Hill, “Palantir courts major federal contracts — and controversy — in the Trump era.”