New York lawmakers are weighing a proposal that could sharply limit how artificial intelligence chatbots present themselves to users — particularly when it comes to offering professional advice.
The legislation aims to stop AI systems from posing as licensed professionals such as attorneys, doctors, or therapists. If passed, it would prevent chatbots from delivering responses that amount to professional guidance that, if given by a human, would be considered the unauthorized practice of a licensed profession.
Supporters of the measure say the rapid spread of large language models has created a regulatory blind spot. Right now, nothing explicitly stops an AI system from implying it is qualified to provide legal or medical guidance, raising concerns about misinformation and potential harm.
Under the proposed rules, platforms that allow their AI tools to present themselves as licensed professionals could face legal consequences. Users who rely on incorrect legal information provided by a chatbot claiming professional authority would be able to file lawsuits and seek damages.
Importantly, companies would not be shielded from liability simply by labeling their systems as “non-human chatbots.” Lawmakers argue that a disclaimer alone is not enough if the system’s responses still resemble professional advice.
The proposal would apply broadly across professions that require state licensing, extending beyond law to areas such as healthcare and mental health services.
It is one piece of a wider effort in New York to regulate the fast-growing AI sector. Other bills under consideration include measures designed to protect minors from harmful chatbot features and requirements that AI platforms clearly warn users that generated responses may contain errors.
The push for new rules comes as artificial intelligence companies face increasing scrutiny over the real-world effects of their tools. Several AI developers are already battling lawsuits tied to claims that chatbot interactions caused serious harm, allegations the companies have disputed.
The legal world has also been grappling with AI’s growing influence. Courts have sanctioned multiple attorneys after discovering they filed documents containing fabricated case citations produced by AI systems — so-called “hallucinations” generated by the technology.
The New York proposal has already cleared a key legislative committee and now heads further into the state’s lawmaking process, setting up what could become one of the first major legal guardrails in the United States against AI impersonating licensed professionals.
