AIPolicyTracker
Menu

AI incident ·

South African Legal Team for Northbound Processing Reportedly Admits Legal Genius AI Produced Purportedly Non-Existent Case Law in Urgent Court Filing

4 news reports Snapshot 7 Sep 2026

In brief

An AI system built by Legal Genius and deployed by Giles Barclay Beuthin, Arnold Subel and 1 other allegedly harmed Northbound Processing, Giles Barclay Beuthin and 2 others.

Risk domain
Misinformation False or misleading information
Occurred
Coverage
4 reportsJul 2025

What happened

In South Africa, the legal team for Northbound Processing reportedly admits that non-existent case citations in an urgent licensing dispute with the South African Diamond and Precious Metals Regulator were purportedly produced by the Legal Genius AI tool. The Johannesburg High Court found the fake references undermined the credibility of Northbound's submissions and referred the lawyers' conduct to the Legal Practice Council for investigation.

Laws that address this harm

Policy angle: Classified under Misinformation (False or misleading information) in the MIT AI Risk Repository taxonomy; 5 recorded instruments address this use case.

Matched from the record's risk domain and country to the instruments recorded here. A reviewer can correct the match in the repository (data/external/incident_overrides.yaml).

News reports (4)

Titles link to the original publisher; report text is not reproduced here.

  1. Lawyers face probe for using ‘hallucinating’ GenAI in court
    itweb.co.za · Admire Moyo
  2. When using Artificial Intelligence goes wrong: Judge slams lawyers for legal bungle
    msn.com · Zelda Venter
  3. Another episode of fabricated citations, real repercussions: South African courts show no tolerance for AI-hallucinated cases
    cliffedekkerhofmeyr.com · Nadeem Mahomed, Safee-Naaz Siddiqi
  4. AI Blunder in Court: Fake Case Law Lands Junior Lawyer Before Legal Council
    thehighstreetjournal.com · Isaac Kofi Tsoenamawu

Who was involved

Alleged developer
Legal Genius
Alleged harmed party
Northbound Processing, Giles Barclay Beuthin, Arnold Subel, A Legal Team Representing Northbound Processing

Classification (MIT AI Risk Repository taxonomy)

Risk domain
Misinformation
Causal entity
AI
Intent
Unintentional
Timing
Post-deployment
Harm level
—
Sectors
—
Countries
—

Risk entries describing this failure mode

Entries from the MIT AI Risk Repository coded to subdomain 3.1.

  • Pursuing Consistent Context

    "LLMs have been demonstrated to pursue consistent context [129]–[132], which may lead to erroneous generation when the prefixes contain false information. Typical examples include sycophancy [129], [130], false demonstra...

    Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

  • Defective Decoding Process

    In general, LLMs employ the Transformer architecture [32] and generate content in an autoregressive manner, where the prediction of the next token is conditioned on the previously generated token sequence. Such a scheme...

    Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

  • Noisy Training Data

    "Another important source of hallucinations is the noise in training data, which introduces errors in the knowledge stored in model parameters [111]–[113]. Generally, the training data inherently harbors misinformation....

    Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

  • Hallucinations

    "LLMs generate nonsensical, untruthful, and factual incorrect content"

    Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

  • Faithfulness Errors

    "The LLM-generated content could contain inaccurate information" which is is not true to the source material or input used

    Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

  • Factuality Errors

    "The LLM-generated content could contain inaccurate information" which is factually incorrect

    Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

  • Untruthful Content

    "The LLM-generated content could contain inaccurate information"

    Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

  • Knowledge Gaps

    "Since the training corpora of LLMs can not contain all possible world knowledge [114]–[119], and it is challenging for LLMs to grasp the long-tail knowledge within their training data [120], [121], LLMs inherently posse...

    Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

Incidents in the same risk subdomain

All incidents in this subdomain

Source record: incident #1139 on the AI Incident Database · all 4 reports