Texas Attorney General Resolves Deceptive Marketing Allegations Against Healthcare AI Company | Manatt, Phelps and Phillips, LLP

On September 18, 2024, the Texas Attorney General released a statement: residential with Pieces Technologies, a Dallas-based artificial intelligence technology company that markets its software to hospitals and other healthcare facilities.

This solution is important because:

• This is the first state attorney general settlement involving deceptive marketing of artificial intelligence software.
• Although the agreement did not include a monetary payment, the Texas AG imposed higher requirements on Pieces’ marketing of its AI software in the future.
• The agreement is an important reminder that states do not need new legislation to regulate the use of artificial intelligence.

Allegation that AI Software Company Exaggerated the Accuracy of the Software

Under the agreement, Pieces designed its generative AI product to assist medical staff in patient care, including summarizing, charting and drafting clinical notes. Pieces provided its software to at least four hospitals in Texas after Pieces allegedly made misleading statements about the accuracy and safety of the products.

Specifically, the Texas Attorney General alleged that Pieces advertised measurements to indicate that their product was “extremely accurate” and had no error rate or “severe hallucination rate.”

Agreement Requires Limitations and Disclosures for Ongoing Marketing

Pieces has agreed to a voluntary assurance of conformity (AVC) that imposes higher requirements on its future marketing, including:

• All Pieces advertisements will clearly and conspicuously disclose the definition of measurements and methods of calculating measurements that it uses to support its accuracy claims.
• Pieces is prohibited from making any false, misleading or unfounded representations about the accuracy, functionality, purpose or other characteristics of its products.
• Pieces will clearly and conspicuously disclose in writing to all current and future customers any known or reasonably knowable harmful or potentially harmful uses or misuses of its products.

The purpose of these disclosures is to ensure that hospital staff understand the extent to which they can trust Pieces’ generative AI products. Under Texas law, a breach of the covenants in the AVC constitutes prima facie evidence of a violation of the Texas Consumer Protection Act.

The Agreement Has Significant Implications for Artificial Intelligence Companies

The settlement is an important reminder that states do not need new legislation to regulate the use of AI because consumer protection, privacy and anti-discrimination laws already apply to AI systems. In June, the Texas AG announced A new data privacy and security initiative focused on the application of data privacy and artificial intelligence under various new and old laws. enforcement scan It covers more than 100 companies. Massachusetts Attorney General’s Office published guidance In April, it warned that making false statements about the reliability, manner of performance, security or status of an AI system, including unsubstantiated claims about its capabilities, was a violation of Massachusetts consumer protection law.

Manatt will continue to follow We are closely monitoring these developments as state regulators provide additional guidance and other states, including CaliforniaAdvancing AI-related legislation that has a significant impact on healthcare.