Top 10 AI Failures and Controversies

Amazon scrapped a secret AI recruiting tool in 2018 after discovering it systematically downgraded résumés that included the word "women's." The model had been trained on a decade of male-dominated hiring data and replicated historical biases. The case highlighted the dangers of training on unexamined real-world datasets.

In 2015, Google Photos automatically tagged photos of two Black individuals as "gorillas," causing public outrage. Google's fix years later was to simply remove "gorilla," "chimp," and related labels entirely from the app. The incident remains a stark illustration of racial bias embedded in image classifiers.

An Uber autonomous test vehicle struck and killed pedestrian Elaine Herzberg in Tempe, Arizona — the first recorded fatal crash involving a self-driving car and a pedestrian. Investigations revealed the system had detected the woman but categorized her as an "unknown object" and was not designed to stop. The crash triggered widespread regulatory scrutiny of autonomous vehicle testing.

In 2023, a New York lawyer submitted court filings containing AI-fabricated case citations produced by ChatGPT, none of which existed. The judge sanctioned the attorneys and the case drew global media attention. It became the canonical example of the hallucination problem in high-stakes professional contexts.

ProPublica's 2016 investigation found that the COMPAS risk-scoring algorithm used in US courts was nearly twice as likely to falsely flag Black defendants as future criminals compared to white defendants. The company disputed the methodology, sparking an ongoing debate about fairness metrics in predictive justice systems. COMPAS became the most cited example of algorithmic discrimination in criminal justice.

In 2017, Facebook's translation AI rendered an Arabic post saying "good morning" as "attack them" in Hebrew, leading to the arrest and detention of a Palestinian construction worker in Israel. The error was swiftly corrected but not before causing real harm. The incident underscored the life-altering consequences of poor NLP in multilingual moderation.

By 2024, AI-generated deepfake explicit images of celebrities and private individuals reached epidemic proportions online, with images of Taylor Swift going viral in early 2024. Platforms scrambled to respond and the US Congress introduced emergency legislation. The crisis accelerated global calls for enforceable deepfake laws.

Internal documents from 2017 revealed that IBM Watson for Oncology had recommended "unsafe and incorrect" cancer treatment options at multiple hospitals. The AI had been trained primarily on hypothetical cases rather than real patient outcomes. IBM quietly wound down Watson Health in 2022 after years of overpromising and underdelivering.

In November 2023, OpenAI's board abruptly fired CEO Sam Altman citing a loss of confidence in his candor, triggering a corporate crisis that saw nearly all employees threaten to resign. Altman was reinstated within five days and the board was largely replaced. The saga raised profound questions about AI governance and the difficulty of overseeing the world's most powerful AI lab.

Amazon scrapped a secret AI recruiting tool in 2018 after discovering it systematically downgraded résumés that included the word "women's." The model had been trained on a decade of male-dominated hiring data and replicated historical biases. The case highlighted the dangers of training on unexamined real-world datasets.

In 2015, Google Photos automatically tagged photos of two Black individuals as "gorillas," causing public outrage. Google's fix years later was to simply remove "gorilla," "chimp," and related labels entirely from the app. The incident remains a stark illustration of racial bias embedded in image classifiers.

An Uber autonomous test vehicle struck and killed pedestrian Elaine Herzberg in Tempe, Arizona — the first recorded fatal crash involving a self-driving car and a pedestrian. Investigations revealed the system had detected the woman but categorized her as an "unknown object" and was not designed to stop. The crash triggered widespread regulatory scrutiny of autonomous vehicle testing.

In 2023, a New York lawyer submitted court filings containing AI-fabricated case citations produced by ChatGPT, none of which existed. The judge sanctioned the attorneys and the case drew global media attention. It became the canonical example of the hallucination problem in high-stakes professional contexts.

ProPublica's 2016 investigation found that the COMPAS risk-scoring algorithm used in US courts was nearly twice as likely to falsely flag Black defendants as future criminals compared to white defendants. The company disputed the methodology, sparking an ongoing debate about fairness metrics in predictive justice systems. COMPAS became the most cited example of algorithmic discrimination in criminal justice.

In 2017, Facebook's translation AI rendered an Arabic post saying "good morning" as "attack them" in Hebrew, leading to the arrest and detention of a Palestinian construction worker in Israel. The error was swiftly corrected but not before causing real harm. The incident underscored the life-altering consequences of poor NLP in multilingual moderation.

By 2024, AI-generated deepfake explicit images of celebrities and private individuals reached epidemic proportions online, with images of Taylor Swift going viral in early 2024. Platforms scrambled to respond and the US Congress introduced emergency legislation. The crisis accelerated global calls for enforceable deepfake laws.

Internal documents from 2017 revealed that IBM Watson for Oncology had recommended "unsafe and incorrect" cancer treatment options at multiple hospitals. The AI had been trained primarily on hypothetical cases rather than real patient outcomes. IBM quietly wound down Watson Health in 2022 after years of overpromising and underdelivering.

In November 2023, OpenAI's board abruptly fired CEO Sam Altman citing a loss of confidence in his candor, triggering a corporate crisis that saw nearly all employees threaten to resign. Altman was reinstated within five days and the board was largely replaced. The saga raised profound questions about AI governance and the difficulty of overseeing the world's most powerful AI lab.