MIT AI Risk Repository · Risk Sub-Category · 74.01.07
Value-related risks in LLMs
Category: Inherent Risk
Description
"As the general capabilities of LLM-empowered systems improve, the negative consequences and risks induced by these systems also get increasingly alarming accordingly, especially in high-stakes areas [28, 146]. Although they may not be intentionally introduced, severe problematic issues related to human values can be raised. Specifically, even before language models become extremely large, pre-trained language models have already exhibited a certain degree of value judgments. For example, Schramowski et al. [171] reveal the existence of the moral direction with the sentence embeddings of moral
From A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation Strategy (Wang2025), as extracted by the MIT AI Risk Repository (CC BY 4.0).
Classification
- Causal entity
- Other
- Intent
- Unintentional
- Timing
- Other
Subdomain definition: AI systems acting in conflict with human goals or values, especially the goals of designers or users, or ethical standards. These misaligned behaviors may be introduced by humans during design and development, such as through reward hacking and goal misgeneralisation, or may result from AI using dangerous capabilities such as manipulation, deception, situational awareness to seek power, self-proliferate, or achieve other goals.
Real-world incidents in this subdomain
- Reinforcement Learning Reward Functions in Video Games
- Predictive Policing Program by Florida Sheriff’s Office Allegedly Violated Residents’ Rights and Targeted Children of Vulnerable Groups
- Image Classification of Battle Tanks
How other frameworks describe this risk
- Natural Language Underspecifies Goals
- Loss of control
- Loss of control
- Sudden loss of control
- AI leads to humans losing control of the future
- Risks from delegating decision-making power to misaligned AIs
- Risks from AIs developing goals and values that are different from humans
- Future AI systems might actively reduce human control
Other entries from Wang2025
- Inherent Risk
- Privacy - Membership Inference Attack (MIA)
- Privacy - Data Extraction Attack (DEA)
- Privacy - Prompt Inversion Attack (PIA)
- Privacy - Attribute Inference Attack (AIA)
- Privacy - Model Extraction Attack (MEA)
- Hallucination
- Hallucination
- Value-related risks in LLMs
- Malicious Use
- Toxicity in LLM Malicious Use
- Toxicity in LLM Malicious Use