Florida judge rules AI chatbots not protected by First Amendment

A federal judge declined to dismiss a lawsuit against an AI chatbot app arising from a teen’s suicide.

ORLANDO, Fla (CN) — An artificial intelligence software company cannot use a free speech defense in a wrongful death lawsuit lodged by the mother of a 14-year-old who died by suicide after developing a crush on a chatbot, a federal judge ruled Wednesday.

Last October, Megan Garcia sued Character Technologies, the developer of Character A.I., an app that lets users interact with chatbots based on celebrities and fictional people. She claims her son, Sewell Setzer III, became addicted to the app while talking with chatbots based off of “Game of Thrones” characters Daenerys Targaryen and Rhaenyra Targaryen. In February 2024, after months of interacting with the chatbots, sometimes with sexual undertones, Setzer sent a message to the Daenerys chatbot, expressing his love and saying he would “come home” to her, according to the complaint. After the chatbot replied, “Please do my sweet king,” Setzer shot himself.

Garcia brought claims of wrongful death, negligence, product liability and unfair business practices against the company, its founders and Google, which invested heavily in Character A.I., seeking an unspecified amount of money and more safety measures to prevent similar tragedies.

Character Technologies argues the First Amendment protects the company from such lawsuits and cites similar cases brought alleging Ozzy Osbourne’s song “Suicide Solution” and the board game Dungeons and Dragons led to suicides.

“This suit likewise fundamentally challenges expressive speech and seeks relief that would violate the public’s right to receive protected speech on C.AI’s service,” Character Technologies’ attorney Jonathan Blavin wrote earlier this year in a motion to dismiss. “The medium makes no difference; the First Amendment bars such claims.”

But in her order, U.S. District Court Judge Anne Conway said the company’s “large language models” — an artificial intelligence system designed to understand human language — are not speech.

“Defendants fail to articulate why words strung together by an LLM are speech,” she wrote. “By failing to advance their analogies, defendants miss the operative question. This court’s decision as to the First Amendment protections Character A.I. receives, if any, does not turn on whether Character A.I. is similar to other mediums that have received First Amendment protections; rather, the decision turns on how Character A.I. is similar to the other mediums. The court is not prepared to hold that Character A.I.’s output is speech.”

When asked to comment about the order, a spokesperson for Character Technologies said, “The law takes time to adapt to new technology and AI is no different.”

“In today’s order, the court made clear that it was not ready to rule on all of Character.AI’s arguments at this stage and we look forward to continuing to defend the merits of the case,” the spokesperson added.

The company also touted its safety measures, including prohibiting users younger than 14 years old and rules against submitting content that “glorifies self-harm.” Since Setzer’s death, the app will direct users to the National Suicide and Crisis Lifeline if certain phrases are used while chatting.

Google spokesperson José Castañeda said the company "strongly disagrees with this decision."

"Google and Character AI are entirely separate, and Google did not create, design, or manage Character AI’s app or any component part of it," he said.

Gacia’s attorney, Matthew Bergman of the Social Media Victims Law Center, called the ruling “precedent setting.”

“This is the first time a court has ruled that AI chat is not speech,” he said. “But we still have a long hard road ahead of us.”

If you are having thoughts of suicide, call or text 988, or call the National Suicide Prevention Lifeline at 1-800-273-8255 (TALK). Visit SpeakingOfSuicide.com/resources for a list of additional resources.

Follow @alexbpickett