Lesson 1 · Definition
What Is Algorithmic Bias?
Every time an AI system makes a decision — whether it's ranking a job application, approving a loan, suggesting a news article, or diagnosing a medical image — it's following patterns learned from data. Algorithmic bias occurs when those patterns systematically favor or disadvantage certain groups of people in ways that are unfair, inaccurate, or harmful.
📖 Core Definition
Algorithmic bias is a systematic and repeatable error in an AI system that produces unfair outcomes — privileging or penalizing individuals based on characteristics such as race, gender, age, disability, or socioeconomic background.
The word "algorithmic" simply means the error is baked into the automated process — not a one-time human mistake. This is what makes it powerful and dangerous: the system repeats the same unfair outcome thousands or millions of times, at scale, and often invisibly.
Key insight: Bias in AI isn't always about bad intentions. Most algorithmic bias is unintentional — the result of flawed data, narrow development teams, or overlooked assumptions baked into the system design.
Think of it this way: if a teacher only graded essays from students who attended private school, they'd build mental expectations based on that narrow sample. An AI does the same — if it learns from incomplete or skewed data, its "expectations" will reflect those gaps. The difference is the AI will apply those skewed expectations to millions of people, instantly.
⚠ Common Misconception
Many people assume that because AI uses math and data, it must be objective and neutral. This is one of the most dangerous myths in AI literacy. Data reflects the world as it was — including historical inequalities, exclusions, and prejudices. An AI trained on that data will reproduce and often amplify those patterns.
Lesson 2 · Origins
Where Does Bias Come From?
Algorithmic bias doesn't appear out of nowhere. It enters AI systems through identifiable pathways — and understanding those pathways is the first step toward spotting it. Here are the six most common sources:
Biased Training Data
If the data used to teach the AI reflects historical inequalities, the AI will learn those inequalities as "normal." Garbage in, bias out.
Example: A hiring AI trained on 20 years of male-dominated tech hiring will learn to prefer male-sounding resumes.
Underrepresentation in Data
When certain groups appear rarely in training data, the AI has too few examples to learn their patterns accurately — leading to worse performance for those groups.
Example: Facial recognition trained mostly on lighter-skinned faces performs poorly on darker skin tones.
Flawed Problem Design
The way a problem is defined shapes everything. If the team asks the wrong question, or defines "success" in a skewed way, bias is built in from the start.
Example: Defining a "good credit risk" using ZIP code — a proxy for race and wealth — bakes in discrimination.
Homogeneous Development Teams
Teams that lack diversity in gender, race, background, and life experience are less likely to notice blind spots that affect people unlike themselves.
Example: A pain-assessment AI built by a team without chronic pain patients failed to detect certain pain patterns common in women.
Feedback Loop Bias
When an AI's past decisions feed back into its future training, initial biases can compound over time and become harder to correct.
Example: A predictive policing tool that directs more patrols to certain neighborhoods generates more arrests there — "confirming" its own predictions.
Proxy Variables
AI systems sometimes use seemingly neutral variables that are actually stand-ins (proxies) for protected characteristics like race or gender.
Example: Using "years of unbroken employment" as a factor disadvantages people who took time off for caregiving — disproportionately women.
Lesson 3 · Classification
Types of Algorithmic Bias
Not all algorithmic bias looks the same. Understanding the different types helps you recognize bias in the wild — even when it's subtle. Expand each type below to explore it.
This bias exists in the data because the world it was collected from was already biased. Even if data is collected perfectly, if the underlying reality was unfair, the data will encode that unfairness. This is perhaps the hardest type to eliminate because it requires acknowledging and actively correcting for historical injustice.
📌 Example
A mortgage AI trained on 50 years of lending data will have "learned" from an era when redlining and discriminatory lending were legal — encoding those patterns as normal lending behavior.
Occurs when the data collected measures something differently or less accurately for certain groups. This happens when the tools, methods, or definitions used to collect data don't work equally well for everyone.
📌 Example
Early pulse oximeters were calibrated primarily on lighter skin tones and are less accurate for patients with darker skin, leading to overestimates of blood oxygen levels — a critical medical measurement.
Occurs when a single AI model is applied to multiple groups that actually have meaningfully different patterns or needs. By treating everyone the same, the model performs well "on average" but poorly for specific subgroups.
📌 Example
A diabetes prediction model trained on a general population may perform well overall but poorly for specific ethnic groups who show different symptom patterns — leading to missed diagnoses.
Arises when an AI model is used in a context or for a purpose different from what it was designed and tested for. A model may be "unbiased" in its original setting but produce discriminatory outcomes when applied elsewhere.
📌 Example
A mental health chatbot designed for North American college students, deployed globally without adaptation, may give culturally inappropriate responses that miss important cultural context around mental health.
Occurs when training data systematically underrepresents or misrepresents certain groups, leading the AI to have weaker or skewed performance for those populations.
📌 Example
Voice recognition systems trained primarily on standard American English accents perform significantly worse for speakers with Southern, non-native, or regional accents — limiting access for those users.
Remember: These types often overlap. A single AI system can exhibit multiple types of bias simultaneously, which is why comprehensive bias detection requires looking at data collection, model design, and real-world deployment together.
Lesson 4 · Case Studies
Real-World Examples
Algorithmic bias isn't theoretical — it has affected real people in consequential decisions. These documented cases illustrate the range of domains where bias has surfaced and the real-world harm it can cause.
Domain · Criminal Justice
COMPAS Recidivism Algorithm
A risk-scoring algorithm used in U.S. courts to predict the likelihood of reoffending was found to assign higher risk scores to Black defendants than white defendants with similar criminal histories. Judges used these scores to make decisions about bail, sentencing, and parole.
⚡ Impact: Affected sentencing for tens of thousands of peopleDomain · Employment
Amazon's Hiring Algorithm (2018)
Amazon built an AI résumé screener trained on 10 years of hiring data — predominantly male applicants. The system learned to penalize résumés that included words like "women's" (e.g., women's chess club) and downgraded graduates of all-women's colleges. The tool was scrapped before deployment.
⚡ Impact: Systemic gender bias in automated candidate screeningDomain · Healthcare
Healthcare Resource Allocation Algorithm
A widely used commercial healthcare algorithm was found to assign lower risk scores to Black patients than to equally sick white patients. The algorithm used healthcare costs as a proxy for health needs — but Black patients historically spend less on healthcare due to systemic barriers to access, not because they are healthier.
⚡ Impact: Affected care access for approximately 200 million U.S. patientsDomain · Financial Services
Apple Card Credit Limits (2019)
Customers reported that the Apple Card algorithm assigned significantly lower credit limits to women than to men — even when the women had higher credit scores and shared finances with male partners. The algorithm's criteria, undisclosed to users, appeared to encode gender-correlated financial patterns.
⚡ Impact: Triggered New York State Department of Financial Services investigation📌 What These Cases Have in Common
Each of these systems was built by credentialed teams at major organizations. None of the bias was deliberate. In each case, the bias was invisible to the system's designers — and only became apparent through external scrutiny, investigative journalism, or academic research. This is why bias detection skills matter for everyone, not just AI engineers.
Lesson 5 · Relevance
Why This Matters to You
You might be thinking: "I'm not an AI engineer — why do I need to understand this?" The answer is that AI systems are now involved in decisions that affect your daily life — and understanding bias gives you the ability to recognize, question, and push back.
❌ Myth
"AI is always more objective than humans." Because it uses data and math, it can't be biased — only humans are biased.
✅ Reality
AI systems reflect and often amplify the biases in their training data. Scale makes AI bias far more dangerous than individual human bias — one biased algorithm can affect millions of decisions simultaneously.
❌ Myth
"Bias only affects marginalized groups." This is someone else's problem — it won't affect me.
✅ Reality
Algorithmic bias affects loan decisions, hiring, healthcare, insurance pricing, and more. Anyone can be disadvantaged by a biased system — and understanding bias helps you advocate for yourself.
❌ Myth
"You need to be a developer to spot bias." Only technical people can identify or challenge AI systems.
✅ Reality
Many of the most important bias discoveries came from non-technical people — journalists, community advocates, and affected individuals who noticed patterns and asked hard questions.
AI literacy isn't just a career skill — it's a civic skill. As AI systems take on greater roles in healthcare, education, hiring, and justice, the ability to recognize when something "feels off" about an automated decision, and knowing what questions to ask, becomes increasingly essential.
Your next skill: In the following sections of this module, you'll learn concrete techniques for identifying algorithmic bias in real situations — including what questions to ask, what patterns to look for, and what to do when you spot something that doesn't seem right.
📚 Key Terms
Glossary for This Section
Algorithm
A set of rules or instructions a computer follows to complete a task or make a decision. In AI, algorithms learn these rules from data.
Algorithmic Bias
Systematic, repeatable unfairness in an AI system that disadvantages or privileges certain groups of people.
Training Data
The dataset used to teach an AI model. Its quality and composition directly determine the model's behavior and potential biases.
Proxy Variable
A data point that seems neutral but correlates with a protected characteristic like race or gender, allowing indirect discrimination.
Feedback Loop
A cycle where an AI system's outputs feed back into its training, potentially amplifying initial biases over time.
Protected Characteristic
Attributes — such as race, gender, age, or disability — that are legally protected from discrimination in many jurisdictions.
Recidivism
The tendency of a previously convicted person to reoffend. Recidivism prediction algorithms are used in criminal sentencing.
Fairness (in AI)
A property of AI systems where outcomes are equitable across different demographic groups. Multiple mathematical definitions of fairness exist and can conflict.
Knowledge Check
Test Your Understanding
Answer the questions below to check your grasp of the key concepts from this section. Select your answer for each question, then submit.
Question 1 of 5
Which of the following best describes algorithmic bias?
Question 2 of 5
Amazon's AI hiring tool was shut down primarily because it learned to:
Question 3 of 5
What is a "proxy variable" in the context of algorithmic bias?
Question 4 of 5
The healthcare algorithm that assigned lower risk scores to Black patients used healthcare costs as a proxy for health needs. What made this problematic?
Question 5 of 5
Which type of bias occurs when an AI model is applied in a context it wasn't designed or tested for?
Questions answered correctly
Section Complete
You've covered the foundational concepts of algorithmic bias — what it is, where it comes from, its types, real-world impact, and why it matters. You're ready to move to the next section: Detecting Bias in AI Outputs.
Continue to Section 2 →