MIT AI Risk Repository · Risk Category · 05.14.00

Evaluation - Auditing

Open the MIT Risk Navigator All entries from Hagendorff2024

Description

Closely related to other clusters like AI safety, fairness, or harmful content, papers stress the importance of evaluating generative AI systems both in a narrow technical way as well as in a broader sociotechnical impact assessment focusing on pre-release audits as well as post-deployment monitoring. Ideally, these evaluations should be conducted by independent third parties. In terms of technical LLM or text-to-image model audits, papers furthermore criticize a lack of safety benchmarking for languages other than English.

From Mapping the Ethics of Generative AI: A Comprehensive Scoping Review (Hagendorff2024), as extracted by the MIT AI Risk Repository (CC BY 4.0).

Classification

Domain: —
Subdomain: —
Causal entity: Not coded
Intent: Not coded
Timing: Not coded

Other entries from Hagendorff2024

Fairness - Bias Risk Category · 1.1
Safety Risk Category · 7.1
Harmful Content - Toxicity Risk Category · 1.2
Hallucinations Risk Category · 3.1
Privacy Risk Category · 2.1
Interaction risks Risk Category · 5.1
Security - Robustness Risk Category · 2.2
Education - Learning Risk Category · 4.3
Alignment Risk Category · 7.1
Cybercrime Risk Category · 4.2
Governance - Regulation Risk Category · 6.5
Labor displacement - Economic impact Risk Category · 6.2