Ethics Series — Racial Bias and AI: What Nonprofit Leaders Cannot Afford to Ignore

When you spend a career working in human services, racial bias stops being abstract very quickly.

I have worked with unhoused individuals in New York City, where race and poverty are so deeply intertwined that you cannot talk about one without the other. I have worked in public housing in Chicago — one of the most dramatic and systematic examples of racial bias in our nation's history, where entire communities of color were deliberately concentrated in underfunded, underserved developments while resources flowed elsewhere. I have worked with public school students in Chicago and Seattle, where the disparities in outcomes between white students and students of color are consistent, measurable, and persistent. And I have worked with refugees in Seattle, people who have survived unimaginable hardship only to encounter new systems that were not built with them in mind.

Across all of that work, one thing has been undeniable: racial bias is not a relic of the past. It is a living, breathing feature of nearly every system our clients navigate — healthcare, housing, criminal justice, education, employment.

And now, it is woven into AI.

Before I go further, I want to acknowledge that I write this as someone who has witnessed these systems from the outside — and that is a fundamentally different vantage point than living inside them. The people who understand racial bias in AI most deeply are the people who have experienced algorithmic discrimination firsthand. Their voices should be centered in this conversation far more than mine. What I can offer is what I've seen, what the research shows, and a call to action for the nonprofit leaders reading this.

What AI Is Trained On — and Why It Matters

AI models are trained on enormous datasets drawn from the internet, books, academic papers, news articles, court records, medical literature, and other human-generated text. They are a vast mirror of human knowledge — and human history, including all of its inequities.

When that training data reflects decades of discriminatory policing, biased medical research, racially skewed hiring practices, and exclusionary housing policies — the AI learns from all of it. Not maliciously. Not deliberately. But consequentially.

The result is that the same racial biases embedded in our institutions show up in our AI tools. Often in ways that are invisible until someone looks carefully. Sometimes in ways that are devastating.

What the Research Shows

The evidence is not anecdotal. It comes from peer-reviewed research, major academic medical centers, and international human rights bodies.

A study published in Nature found that AI language models show covert racial bias when processing African American Language (AAL). The models are more likely to suggest that speakers of AAL be assigned less prestigious jobs, be convicted of crimes, and be sentenced to death. Importantly, the researchers found that current efforts to reduce this bias — such as human preference alignment — actually make it worse by superficially obscuring racism that the models maintain at a deeper level.¹

A Cedars-Sinai study examining AI in psychiatric care found disturbing patterns in how race affected treatment recommendations. Two large language models omitted medication recommendations for ADHD cases when race was explicitly stated — but suggested them when race was absent. Another suggested guardianship for depression cases with explicit racial characteristics.²

In housing, AI tenant screening tools routinely return incorrect, outdated, or misleading information that landlords use to disproportionately deny applications to Black and Latino renters — drawing on data about credit scores, eviction records, and criminal backgrounds that reflect longstanding racial disparities.³

In criminal justice, AI tools used to predict the likelihood of reoffending have been shown to over-predict risks for Black people compared to their white counterparts. Predictive policing creates a feedback loop: increased police presence in over-policed communities generates more data that justifies more police presence.⁴

And perhaps most troubling of all — research published in Nature found that people who complete tasks assisted by a biased AI system reproduce that bias in their own decisions, even after they stop using the tool.⁵ The bias doesn't stay in the machine. It transfers to the humans who use it.

Why Nonprofit Leaders Cannot Look Away

The communities most harmed by racial bias in AI are the same communities that nonprofit organizations exist to serve. The unhoused person being screened out of housing by a biased algorithm. The Black patient receiving worse psychiatric care recommendations. The formerly incarcerated person whose risk score follows him into every new system he encounters.

These are not edge cases. These are the people in your programs, your intake forms, the impact stories you write in your grant applications.

If your organization is using AI — to write communications, analyze program data, screen volunteers or staff, or generate images for your website — those tools carry these same biases. Which means checking for racial bias is not optional. It is part of the job.

What You Can Do

You don't need to be a data scientist to start catching racial bias in AI output. You need to be paying attention and asking the right questions.

Ask AI to check its own work. Paste any piece of AI-generated content back into your tool and ask: "Does this content reflect any racial bias? Does it make assumptions about the race of the people described? Does it use language that could be harmful or othering to people of color? Please revise with a racial equity lens."

Watch how your clients are portrayed. When AI generates impact stories, program descriptions, or client communications, read carefully. Are the people you serve portrayed as active agents of their own lives, or as passive recipients defined by their struggles? Are they described with dignity and complexity?

Check your images. AI-generated portraits of people in STEM professions are almost exclusively depicting male, white individuals.⁶ If you're using AI image tools, look critically at who appears and who doesn't.

Be specific in your prompts. When AI defaults to assumptions that don't reflect your community, push back explicitly. "Please write this assuming the client is a Black woman in her forties." Specificity is one of the most effective tools against default bias.

Question the data behind any AI tool you're evaluating. Ask vendors directly what data the tool was trained on, whether it has been audited for racial bias, and what the error rates look like across different demographic groups. If they can't answer those questions, that is your answer.

The Bigger Picture

Racial bias in AI is not a technology problem with a technology solution. It is a human problem that technology has inherited and amplified.

The organizations in our sector have something that the tech companies building these tools often lack — deep, sustained relationships with the communities most harmed by algorithmic bias. We know what the data misses. We know whose stories don't get told. We know when something doesn't add up.

That knowledge is not a small thing. It is one of the most important contributions we can make to how AI gets built and deployed.

Use it.

Follow us on LinkedIn and Instagram @nonprofitAIstudio so you never miss a post.

Footnotes

¹ Hofmann, V., et al. "Dialect prejudice predicts AI decisions about people's character, employability, and criminality." Nature, August 2024. nature.com/articles/s41586-024-07856-5

² Aboujaoude, E., et al. "Cedars-Sinai Study Shows Racial Bias in AI-Generated Treatment Regimens for Psychiatric Patients." Cedars-Sinai Newsroom, November 2025. cedars-sinai.org

³ "The Discriminatory Impacts of AI-Powered Tenant Screening Programs." Georgetown Law Poverty Journal, 2025. law.georgetown.edu

⁴ Christian, G. "Racial bias in AI should be the immediate concern." Policy Options, December 2024. policyoptions.irpp.org

⁵ Vicente, L., and Matute, H. "Humans inherit artificial intelligence biases." Scientific Reports / Nature, October 2023. ncbi.nlm.nih.gov/pmc/articles/PMC10547752

⁶ Messingschlager, T.V., and Appel, M. "Algorithmic bias in image-generating artificial intelligence: prevalence and user perceptions." Journal of Communication, November 2025. tandfonline.com