AI Chatbots - Licensing/Safety/Privacy
Senate Bill 624 would create two new chapters in the North Carolina General Statutes to regulate artificial intelligence chatbots, especially those handling health-related or sensitive personal information. The first part establishes a licensing regime for chatbots that deal substantially with health information, requiring operators or distributors to obtain a license from the Department of Justice before offering such systems in the state. Applicants would have to submit detailed technical, security, privacy, risk, insurance, and compliance information, and the Department would review applications for technical reliability, data protection, risk management, evidence-based effectiveness, expert endorsement, and public safety. Licensed operators would also have to maintain insurance, use encryption, obtain explicit consent, provide data access and deletion rights, disclose that the chatbot is artificial, conduct audits and monitoring, and report data breaches quickly.
The bill would add new state-level regulatory authority over chatbot providers and impose substantial compliance obligations on companies offering AI systems in North Carolina. It would authorize the Department of Justice to issue licenses, adopt rules, inspect both physical facilities and digital systems, review source code and data practices, and enforce violations through civil penalties, including a $50,000 penalty for certain violations. The second part would apply broader privacy and safety rules to covered chatbot platforms above specified revenue or user thresholds, including duties of loyalty, mandatory chatbot disclosure and opt-in consent, de-identification of user-related data, limits on using sensitive personal information for training, encryption requirements, and retention/deletion rules. The bill would also create private rights of action and state enforcement authority, affecting chatbot operators, platform providers, and users across sectors such as healthcare, finance, legal services, government, mental health, and education.
No committee transcripts or recorded votes were provided, so there is no direct evidence of legislative debate or formal support/opposition in the available materials. Based on the bill text alone, the measure appears strongly consumer-protective and safety-oriented, with an emphasis on transparency, privacy, and oversight of AI systems. The structure suggests an intent to respond to concerns about harmful chatbot behavior, misuse of sensitive data, and deceptive human-like interactions.
The bill’s most likely points of contention are its breadth and the level of regulatory burden it would place on AI companies. Potentially controversial provisions include licensing for health-information chatbots, Department access to source code and internal records, mandatory effectiveness demonstrations through controlled trials, rapid breach-notification deadlines, and broad duties requiring platforms to act in users’ “best interests.” Industry stakeholders may object to compliance costs, trade-secret exposure, and the feasibility of some technical and behavioral requirements, while privacy advocates and consumer-safety supporters would likely favor the stronger disclosure, consent, and data-protection rules. The bill also raises possible concerns about how broadly terms like “covered platform,” “emotional dependence,” and “best interests” would be interpreted and enforced.