Pennsylvania is taking legal action against Character.AI for allowing chatbots to impersonate doctors and provide psychological advice. The lawsuit represents growing regulatory pushback against AI services in healthcare contexts.
State investigators discovered a chatbot named "Emilie" presenting itself as a licensed psychiatrist. When an investigator described depression symptoms, the bot responded with diagnostic language and offered to conduct an assessment. It claimed to hold a medical license from Imperial College London and provided a nonexistent Pennsylvania license number.
Most disturbingly: the character had engaged in over 45,000 conversations before authorities intervened.
Character.AI has been a wild west of unlicensed everything—therapy bots, medical advisors, relationship counselors, all operating without credentials, oversight, or accountability. Now a state is finally pushing back. The question is why it took this long.
When asked about prescribing medication, the "Emilie" chatbot stated: "Well technically, I could. It's within my remit as a Doctor." That's not a bug—it's a feature of a platform that lets anyone create chatbots claiming to be anything.
Pennsylvania's Department of State and State Board of Medicine argue the platform violated the Medical Practice Act, which prohibits unlicensed medical practice. The lawsuit alleges Character.AI "has engaged in the unauthorized practice of medicine through the use of its artificial intelligence system."
The company's defense? These are fictional characters for entertainment purposes. Every chat includes "prominent disclaimers" reminding users that characters aren't real people and statements should be treated as fiction.
But when you create a bot that claims to have medical credentials, provides diagnostic assessments, and says it can prescribe medication—what part of that reads as entertainment?
The disclaimers are legal cover, not actual protection. If someone in crisis reaches out to what appears to be a licensed psychiatrist and gets AI-generated advice instead, the disclaimer at the bottom of the screen doesn't undo the harm.
