Algorithmic bias refers to machine-learning systems producing systematically unequal outcomes across groups of people. Because most models learn statistical patterns from data drawn from the world, they tend to reproduce — and can amplify — biases present in that data, even when no bias is explicitly programmed in.
Two primary studies make the mechanism concrete. In “Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification” (Joy Buolamwini and Timnit Gebru, Proceedings of the 1st Conference on Fairness, Accountability and Transparency, 2018), the authors evaluated commercial gender-classification systems and found large accuracy gaps: they reported that “darker-skinned females are the most misclassified group (with error rates of up to 34.7%)” while “the maximum error rate for lighter-skinned males is 0.8%.” In “Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings” (Bolukbasi, Chang, Zou, Saligrama, and Kalai, 2016), the authors showed that “even word embeddings trained on Google News articles exhibit female/male gender stereotypes to a disturbing extent,” and proposed methods to reduce those associations geometrically while preserving useful word relationships.
These results illustrate two distinct points: that bias can appear in a deployed commercial product’s measured accuracy, and that it can be embedded in the numerical representations models use internally. Beyond research datasets, algorithmic bias has been alleged in high-stakes settings such as criminal-justice risk scoring — most prominently in reporting by the news organization ProPublica on the COMPAS recidivism tool — but that case rests on journalism rather than a primary research source and is noted here only as reported by ProPublica.