MIT AI Risk Repository · Risk Sub-Category · 60.03.03
Market concentration and single points of failure
Category: Systemic risks
Description
"Market shares for general- purpose AI tend to be highly concentrated among a few players, which can create vulnerability to systemic failures. The high degree of market concentration can invest a small number of large technology companies with a lot of power over the development and deployment of AI, raising questions about their governance. The widespread use of a few general- purpose AI models can also make the financial, healthcare, and other critical sectors vulnerable to systemic failures if there are issues with one such model."
From International AI Safety Report 2025 (Bengio2025), as extracted by the MIT AI Risk Repository (CC BY 4.0).
Classification
Subdomain definition: AI-driven concentration of power and resources within certain entities or groups, especially those with access to or ownership of powerful AI systems, leading to inequitable distribution of benefits and increased societal inequality.
Real-world incidents in this subdomain
- LLM Scrapers Allegedly Target Multiple Open Source Projects Disrupting the FOSS Ecosystem
- Coupang Allegedly Tweaked Search Algorithms to Boost Own Products
- Amazon Allegedly Tweaked Search Algorithm to Boost Its Own Products
- Gmail’s Inbox Sorting System Reportedly Reduced Visibility of Political Emails and Campaign Calls-to-Action
- Google Fined for Changing Shopping Algorithms in EU to Favor Own Service
- Amazon India Allegedly Rigged Search Results to Promote Own Products
How other frameworks describe this risk
Other entries from Bengio2025
- Risks from malicious use
- Harm to individuals through fake content
- Harm to individuals through fake content
- Harm to individuals through fake content
- Manipulation of public opinion
- Manipulation of public opinion
- Cyber offence
- Cyber offence
- Cyber offence
- Cyber offence
- Cyber offence
- Biological and chemical attacks