AIPolicyTracker
Menu

AI incident #1324 ·

Pieces Technologies' Clinical AI Systems Allegedly Marketed With Misleading Performance Claims

Open on the AI Incident Database 4 news reports Synced from the AIID API · record last edited 3 Sep 2026

What happened

The Texas Attorney General announced a settlement with Pieces Technologies following allegations that the company misrepresented the accuracy of its healthcare AI systems used for clinical documentation. The state concluded that marketing claims about low error and hallucination rates may have misled hospitals and clinicians relying on the tools in patient care settings. The settlement imposed restrictions on future claims and required greater transparency around performance and risk.

Editor's notes (AI Incident Database)

Timeline note: The incident ID date of 09/18/2024 is taken from the date the Texas Attorney General's Office announced its settlement with Pieces Technologies, Inc. The incident ID was created on 01/02/2026.

Only the incident metadata is stored here. The underlying news reports are on the AI Incident Database (CC BY-SA 4.0); use the links above to read them.

News reports (4)

Coverage catalogued by the AI Incident Database. Titles link to the original publisher; the text is not reproduced here.

  1. Attorney General Ken Paxton Reaches Settlement in First-of-its-Kind Healthcare Generative AI Investigation
    texasattorneygeneral.gov · Attorney General of Texas · AIID #6866
  2. Texas Attorney General’s Office Reaches Settlement with AI Company Over Deceptive Claims
    wilmerhale.com · Kirk J. Nahra, Ali A. Jessani, Sarah Litwin · AIID #6766
  3. Rising AI Enforcement: Insights From State Attorney General Settlement and U.S. FTC Sweep for Risk Management and Governance
    sidley.com · Sidley Austin · AIID #6865
  4. Smarter Technologies Acquires Pieces Technologies and Launches SmarterNotes
    businesswire.com · John Hallock · AIID #6864

Who was involved

Alleged deployer
Pieces Technologies
On AIID: Pieces Technologies
Alleged developer
Pieces Technologies
On AIID: Pieces Technologies
Alleged harmed party
Public health Patients Healthcare institutions Epistemic integrity Clinicians
On AIID: Public health, Patients, Healthcare institutions, Epistemic integrity, Clinicians

AI systems implicated

Unknown generative AI systemsPieces Technologies generative AI systemsHealthcare decision-support systemsEnterprise AI systemsClinical documentation AIAI-enabled decision support systems

Classification (MIT AI Risk Repository taxonomy)

Risk domain
Misinformation
Causal entity
Human
Intent
Intentional
Timing
Post-deployment
Harm level
—
Sectors
—
Countries
—

Risk entries describing this failure mode

Entries from the MIT AI Risk Repository coded to subdomain 3.1.

  • Pursuing Consistent Context

    "LLMs have been demonstrated to pursue consistent context [129]–[132], which may lead to erroneous generation when the prefixes contain false information. Typical examples include sycophancy [129], [130], false demonstra...

    Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

  • Knowledge Gaps

    "Since the training corpora of LLMs can not contain all possible world knowledge [114]–[119], and it is challenging for LLMs to grasp the long-tail knowledge within their training data [120], [121], LLMs inherently posse...

    Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

  • Hallucinations

    "LLMs generate nonsensical, untruthful, and factual incorrect content"

    Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

  • Faithfulness Errors

    "The LLM-generated content could contain inaccurate information" which is is not true to the source material or input used

    Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

  • Defective Decoding Process

    In general, LLMs employ the Transformer architecture [32] and generate content in an autoregressive manner, where the prediction of the next token is conditioned on the previously generated token sequence. Such a scheme...

    Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

  • Untruthful Content

    "The LLM-generated content could contain inaccurate information"

    Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

  • Factuality Errors

    "The LLM-generated content could contain inaccurate information" which is factually incorrect

    Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

  • Noisy Training Data

    "Another important source of hallucinations is the noise in training data, which introduces errors in the knowledge stored in model parameters [111]–[113]. Generally, the training data inherently harbors misinformation....

    Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

Incidents in the same risk subdomain

All incidents in this subdomain