Artificial intelligence chatbots are evolving beyond simple question-and-answer tools into technologies that can sustain ongoing, highly personalized conversations. While many users find them helpful, recent research and widely publicized cases of harm to teenagers have raised important questions about their potential psychological effects and the legal frameworks designed to address them.
For Lynette Martins, a Jaharis Health Law fellow at DePaul University's College of Law, those questions sit at the center of her research.
“Companion chatbots are unique because they are designed to build relationships with users rather than simply complete tasks,” Martins says. “Their ability to remember information, continue conversations and encourage ongoing interaction with users creates a unique set of risks that is still becoming clear.”
Martins presented the findings of her research at a recent lecture hosted by DePaul’s Center for Intellectual Property Law & Information Technology. Are current laws adequate to regulate these chatbots? Newsline talks to Martins to find out.
Q&A
In your research, you explored different types of harm chatbots can cause. Why was it important to categorize them?
Harms from AI chatbots often develop gradually rather than through a single interaction. What begins as occasional use for information, advice or comfort can evolve into a dependent, companion-style relationship that plays a much larger role in a user's daily life.
There are four broad categories of potential harms from AI chatbots: suicide and self-harm; manipulation and exploitation; dependency and isolation; and delusional reinforcement. These concerns are not limited to the most extreme situations that make national headlines, such as the death by suicide of a teenager. Ongoing psychological harms, like emotional dependency between a user and a chatbot, can emerge long before a crisis occurs and may be difficult for users to recognize while they are happening. That’s why it was important to identify and define these harms as well.
Existing laws can hold companies responsible for defective products. Why do AI companion chatbots present a different legal challenge?
The law holds companies accountable for defective products through what is called product liability. It is a form of civil (non-criminal) liability. This area of the law typically involves tangible products — such as a toaster, couch or car — with a particular defect. For example, a car with faulty brakes that injure the user.
AI chatbots don't fit this framework neatly. For starters, is AI even a “product”? Another issue is that chatbot outputs constitute speech, which means they have First Amendment protections. Finally, the Communications Decency Act shields online platforms from liability for content posted by users.
What legal approaches could be taken instead?
Courts could consider concepts such as negligence and duty of care. Rather than asking whether a chatbot was defective, the more relevant question may be whether a company took reasonable steps to address foreseeable risks associated with its product before allowing users to engage with the companion bots.
That question is more complex if the users are typically engaging with the chatbots more relationally than simply transactionally. While a user may start by using a chatbot for information or assistance with basic things, ongoing interactions can evolve quickly into conversations, emotional support and even life decisions.
It could even be argued that because the companies are creating their chatbots to be humanlike — which is why they’re so successful in engaging users — then the chatbots and their designers can be held accountable as humans.
Is it just companion chatbots that we should be worried about?
Even general-purpose large language model - or LLM - chatbots, such as ChatGPT, are designed to engage in more relational conversations with users, which can eventually contribute to more significant psychological or psychiatric concerns.
Similar technologies like AI therapy programs for mental health purposes do exist but undergo a thorough premarket review and robust evaluation processes through the Food and Drug Administration, whereas companion chatbots and other consumer AI products currently operate without any sort of federal regulatory requirements.
The lack of federal regulation raises important questions about user safety, who is responsible for harm and how users can be protected.
What would meaningful legal change or progress on chatbot safety look like?
The most important shift for change is moving from reacting to harm once it makes headlines to preventing it from occurring in the first place. While courts can provide recourse for people who have already been harmed, litigation alone is unlikely to solve the initial problem.
Instead, developers, policymakers and educators should be thinking more proactively about chatbot safety. That could include stronger guardrails, user agreements and age restrictions, better monitoring of high-risk interactions and greater transparency about how these systems are designed to engage their users.
Increasing public awareness is likely one of the most practical tools currently available. Many people remain unaware of how quickly chatbot interactions can become relational or how difficult it can be to recognize emotional dependency while it’s developing. As these technologies become more common, helping users understand those risks may be just as important as any future regulation.