AI governance glossary

A machine-based system that operates with some autonomy, may adapt after it is deployed, and infers from the input it receives how to generate outputs such as predictions, content, recommendations or decisions that can influence physical or virtual environments. This is the EU AI Act's definition, which follows the OECD's; whether software is an "AI system" decides whether the Act applies to it at all.

Source: EU AI Act (Regulation (EU) 2024/1689) EU AI Act →AI system inventory control →

An AI model, typically trained on a large amount of data with self-supervision at scale, that shows significant generality, can perform a wide range of distinct tasks and can be built into many downstream systems. Models used only for research or prototyping before release are excluded. Providers of these models have their own duties under the EU AI Act, separate from the duties attached to AI systems.

Source: EU AI Act (Regulation (EU) 2024/1689) Duties of AI model providers →

Under the EU AI Act, a general-purpose AI model has systemic risk when it has high-impact capabilities, which is presumed when the cumulative compute used to train it exceeds 10²⁵ floating-point operations, or when the Commission designates it. Such models carry extra duties: model evaluation including adversarial testing, assessing and mitigating systemic risks, reporting serious incidents, and cybersecurity protection.

Source: EU AI Act (Regulation (EU) 2024/1689) Frontier model safety framework control →Red-team testing control →

In the EU AI Act, an AI system that is a safety component of (or is itself) a product covered by the EU product-safety laws listed in Annex I and needing third-party conformity assessment, or one used in an area listed in Annex III, such as biometrics, critical infrastructure, education, employment, access to essential services, law enforcement, migration and the administration of justice. A provider may document that an Annex III system is not high-risk when it does not pose a significant risk of harm.

Source: EU AI Act (Regulation (EU) 2024/1689) Risk management duties →EU AI Act role and risk classifier →

Uses of AI the EU AI Act bans outright, including manipulative or deceptive techniques that cause significant harm, exploiting vulnerabilities, social scoring, predicting crime from profiling alone, untargeted scraping of facial images, emotion recognition in workplaces and schools (with narrow exceptions), biometric categorisation by sensitive traits, and most real-time remote biometric identification by police in public spaces.

Source: EU AI Act (Regulation (EU) 2024/1689) Prohibited practice duties →Prohibited-use screening control →

The use a provider intends an AI system for, including the context and conditions of use, as stated in its instructions, marketing and technical documentation. Classification as high-risk, and most provider duties, are assessed against the intended purpose; using a system outside it can shift provider duties onto the user.

Source: EU AI Act (Regulation (EU) 2024/1689)

A change to an AI system after it is placed on the market that the provider's original conformity assessment did not foresee, and that affects its compliance or changes its intended purpose. Under the EU AI Act, whoever makes a substantial modification to a high-risk system can become its provider, with the provider's duties.

Source: EU AI Act (Regulation (EU) 2024/1689) Release and change management control →

The process of showing that a high-risk AI system meets the EU AI Act's requirements before it is placed on the market or put into service, either by the provider's own internal check or with a notified body. It ends with an EU declaration of conformity and CE marking, and for most Annex III systems a registration in the EU database.

Source: EU AI Act (Regulation (EU) 2024/1689) Conformity assessment duties →Conformity assessment and registration control →

An independent conformity assessment body designated by an EU member state to assess high-risk AI systems where the EU AI Act requires a third party, for example certain biometric systems.

Source: EU AI Act (Regulation (EU) 2024/1689)

The documented policies, procedures and instructions a provider of a high-risk AI system keeps to ensure compliance across the system's life: design, testing, data management, risk management, post-market monitoring, incident reporting and accountability. Required of providers by the EU AI Act.

Source: EU AI Act (Regulation (EU) 2024/1689) Quality management duties →Quality management system control →

The activities a provider carries out to collect and review data on how its AI systems perform once in use, so that it can spot the need for corrective or preventive action. High-risk providers must have a documented post-market monitoring plan.

Source: EU AI Act (Regulation (EU) 2024/1689) Post-market monitoring duties →Post-market monitoring plan template →

An incident or malfunction of an AI system that directly or indirectly leads to death or serious harm to a person's health, a serious and irreversible disruption of critical infrastructure, an infringement of obligations that protect fundamental rights, or serious harm to property or the environment. Providers of high-risk systems must report serious incidents to the market surveillance authority.

Source: EU AI Act (Regulation (EU) 2024/1689) Incident duties →AI incident response playbook →

Design and operating measures that let people effectively oversee an AI system while it is in use: understand its capabilities and limits, watch for automation bias, interpret its output, and decide not to use it, override it or stop it. Required for high-risk systems under the EU AI Act and a common expectation in other frameworks.

Source: EU AI Act (Regulation (EU) 2024/1689) Human oversight duties →Human oversight procedure template →

An assessment, before first use, of how a high-risk AI system could affect the people it is used on: who is affected, the specific risks of harm, the human oversight in place and what happens if the risks materialise. The EU AI Act requires it of public bodies, private bodies providing public services, and deployers of systems for credit scoring and for life and health insurance pricing.

Source: EU AI Act (Regulation (EU) 2024/1689) Impact assessment duties →FRIA template →

The skills, knowledge and understanding that let providers, deployers and affected people use AI systems in an informed way and be aware of their opportunities, risks and possible harm. The EU AI Act asks providers and deployers to take measures to ensure a sufficient level of AI literacy among the staff who deal with AI systems.

Source: EU AI Act (Regulation (EU) 2024/1689) AI literacy duties →AI literacy training plan →

AI-generated or AI-manipulated image, audio or video content that resembles existing people, objects, places or events and would falsely appear authentic. The EU AI Act requires deployers to disclose that such content is artificial, with lighter rules for evidently artistic or satirical work.

Source: EU AI Act (Regulation (EU) 2024/1689) Transparency duties →Synthetic content labelling control →

A controlled framework run by a competent authority in which providers can develop, train, validate and test innovative AI systems for a limited time under regulatory supervision. The EU AI Act requires each member state to set up at least one.

Source: EU AI Act (Regulation (EU) 2024/1689)

The policies, objectives and processes an organisation puts in place to develop, provide or use AI systems responsibly, structured like other ISO management systems (plan, do, check, act). ISO/IEC 42001 sets the requirements for one and can be certified by an accredited body.

Source: ISO/IEC 42001:2023 ISO/IEC 42001 record →ISO 42001 gap assessment →

A voluntary framework from the US National Institute of Standards and Technology for managing the risks of AI systems, organised in four functions: Govern (culture and accountability), Map (context and risks), Measure (analysis and tracking) and Manage (prioritising and acting on risks).

Source: NIST AI 100-1 NIST AI RMF record →NIST, EU and ISO crosswalk →

Under New York City Local Law 144, an impartial evaluation by an independent auditor of an automated employment decision tool, reporting selection or scoring rates and impact ratios by sex and race or ethnicity. Employers must have one done within a year before using the tool, publish a summary, and notify candidates.

Source: NYC Department of Consumer and Worker Protection Local Law 144 record →

Structured adversarial testing in which testers try to make an AI system fail: produce harmful output, leak data, ignore its instructions or be misused. It is one of the evaluations expected of the most capable general-purpose models and a common control for generative AI.

Source: EU AI Act (Regulation (EU) 2024/1689) Red-team testing control →AI red team test plan →

A short document published with a machine-learning model that states its intended use, the data it was evaluated on, its performance across groups and conditions, and its known limitations. The format comes from Mitchell et al. (2019) and now meets part of the documentation duties several laws place on model providers.

Source: Mitchell et al., Model Cards for Model Reporting (2019) Technical documentation control →AI model card template →

Develops an AI system or model, or has it developed, and places it on the market or puts it into service under its own name.

Obligations that apply to this →

Uses an AI system under its own authority (other than personal, non-professional use).

Obligations that apply to this →

Places on a market an AI system that bears the name of a provider established outside that market.

Obligations that apply to this →

Makes an AI system available on a market without being its provider or importer.

Obligations that apply to this →

Person or entity mandated by a non-established provider to carry out obligations on its behalf.

Obligations that apply to this →

Natural person interacting with, or affected by, an AI system.

Obligations that apply to this →

Public-sector body procuring, deploying or overseeing AI systems.

Obligations that apply to this →

Provider of a general-purpose AI model, including foundation and frontier models.

Obligations that apply to this →

A model trained on broad data that can perform a wide range of tasks and be integrated into many systems.

System that produces text, images, audio, video or code.

System that makes or materially supports decisions about people.

System processing biometric data to identify, verify or categorise people, or recognise emotions.

System that ranks, recommends or personalises content, products or people.

AI used as a safety component of machinery, vehicles, medical devices or similar regulated products.

System that interacts with people through natural language.

System that scores, profiles or predicts behaviour, risk or eligibility.

Use banned outright (terminology used by the EU AI Act; other jurisdictions may differ).

Obligations that apply to this →

Use subject to the strictest requirements before and after deployment.

Obligations that apply to this →

Use permitted with disclosure or labelling duties (for example chatbots, synthetic content).

Obligations that apply to this →

No AI-specific obligations beyond general law.

Obligations that apply to this →

General-purpose models with high-impact capabilities subject to additional duties.

Obligations that apply to this →

The instrument does not use a risk tiering.

Obligations that apply to this →

An approved, versioned statement of intent and rules, signed off by accountable management.

A documented, repeatable process with steps, roles and inputs.

A row in a maintained inventory: AI systems, vendors, models or datasets.

A maintained list of identified risks with owner, rating, treatment and review date.

A completed assessment of one system's risks, with method, findings and treatment decisions.

A completed assessment of effects on people, groups or rights, with mitigations.

An assessment required by data-protection law for high-risk processing of personal data.

Provenance, collection, labelling, representativeness and known limitations of training, validation and test data.

Intended purpose, architecture, training, evaluation and limitations of a model.

The assembled documentation a regulator or notified body would review.

Results of testing accuracy, robustness, bias, safety or security against defined criteria.

Logs, dashboards or reports from ongoing performance and drift monitoring.

A recorded incident with timeline, impact, root cause and corrective action.

A submission made to an authority: a registration, a report, a declaration.

Evidence that an accountable person approved a decision, with date and scope.

Minutes or decisions of a board, committee or review body.

Who was trained on what, when, with completion evidence.

Agreed terms with a vendor or customer allocating AI duties, data rights and notification.

A completed due-diligence questionnaire or audit of a third party.

The text shown to a person: an AI interaction notice, a content label, an explanation of a decision.

System-generated records of who did what, retained for a defined period.

An internal or external review of whether a control operates as described.

A signed declaration or third-party certificate of conformity with a rule or standard.