MIT AI Risk Repository · Risk Sub-Category · 33.02.05
Prompt engineering
Category: Technology concerns
Description
"With the wide application of generative AI, the ability to interact with AI efficiently and effectively has become one of the most important media literacies. Hence, it is imperative for generative AI users to learn and apply the principles of prompt engineering, which refers to a systematic process of carefully designing prompts or inputs to generative AI models to elicit valuable outputs. Due to the ambiguity of human languages, the interaction between humans and machines through prompts may lead to errors or misunderstandings. Hence, the quality of prompts is important. Another challenge i
From Generative AI and ChatGPT: Applications, Challenges, and AI-Human Collaboration (Nah2023), as extracted by the MIT AI Risk Repository (CC BY 4.0).
Classification
- Causal entity
- Human
- Intent
- Other
- Timing
- Post-deployment
Subdomain definition: Challenges in understanding or explaining the decision-making processes of AI systems, which can lead to mistrust, difficulty in enforcing compliance standards or holding relevant actors accountable for harms, and the inability to identify and correct errors.
Real-world incidents in this subdomain
- Uber Launched Opaque Algorithm That Changes Drivers' Payments in the US
- Israeli Tax Authority Reportedly Used an Opaque Automated System to Issue a Fine, Declining to Explain or Disclose the Underlying Calculation
- Uber Allegedly Wrongfully Accused Drivers of Fraud via Automated Systems
- Houston ISD's EVAAS Teacher-Evaluation System Reportedly Put Teachers' Jobs at Risk Through Unverifiable Scores
- Dutch City Court Defended Home Value Generated by Black-Box Algorithm
How other frameworks describe this risk
- Intelligibility
- Attributing the responsibility for AI's failures
- Model Evaluations (Interpretability/Explainability)
- Model outputs inconsistent with chain-of-thought reasoning
- Lack of understanding of in-context learning in language models
- General Evaluations (Difficulty of identification and measurement of capabilities)
- Lack of transparency and interpretability
- Opacity (the black box problem)