This bill would create a new article in the General Business Law called the “New York Stealth Crawler Prohibition Act.” It defines a “crawler” broadly to include software, bots, AI agents, spiders, fetchers, and similar tools that access websites or other internet sources. The bill focuses on “covered news sources,” meaning qualifying journalism outlets that publish regularly, perform a public-information function, and have at least 1,000 monthly active New York users, viewers, listeners, or subscribers.
Under the bill, any crawler that accesses a covered news source must disclose its identity and purpose before or at the time of access. That disclosure must include a valid user-agent string identifying the software product, version, and company behind it, as well as the specific nature and purpose of the crawler’s use. The bill also prohibits operators from deploying a “stealth crawler” in a way that damages, impairs, burdens, or economically harms a covered news source. The Attorney General could enforce the law through injunctions and civil penalties of up to $15,000 per day per violation, and journalism providers would have a limited subpoena process to identify alleged violators through service providers.
The bill would add new statutory protections for news organizations in New York and impose new compliance obligations on operators of web crawlers, including AI-related tools and automated scraping systems. It would also create a new enforcement mechanism in the Attorney General’s office and a pre-action subpoena pathway for journalism providers seeking to identify anonymous actors through internet service or communications providers. The act would take effect 90 days after becoming law.
The general sentiment reflected by the bill text is protective of journalism and skeptical of undisclosed automated access to news content. Because there are no committee transcripts or recorded votes provided, there is no direct evidence of debate or opposition in the available record. The bill’s framing suggests support for news publishers’ concerns about scraping, indexing, and AI training uses, while potentially raising concerns for technology companies, search tools, AI developers, and other operators that rely on automated crawling.
The main point of contention is likely the breadth of the definitions and the scope of the disclosure and prohibition requirements. Supporters would likely view the bill as a safeguard against hidden scraping and economic harm to news outlets, while critics may argue that it could chill legitimate indexing, research, and AI-related data collection, or create uncertainty about what counts as a covered news source, a crawler’s “purpose,” or economic harm.
The bill would amend the General Business Law by adding a new article 48 regulating automated web access to qualifying news websites and digital publications. It would impose disclosure duties on crawlers, prohibit stealth crawlers that harm covered news sources, authorize Attorney General enforcement with civil penalties and injunctive relief, and allow journalism providers to use a subpoena process to identify alleged violators through service providers. It would directly affect operators of bots, spiders, scrapers, AI agents, and similar tools, as well as journalism providers and internet/service providers involved in identifying anonymous actors.
The available materials suggest a generally pro-news-media, anti-stealth-scraping posture. The bill is framed as a consumer- and journalism-protection measure aimed at transparency and preventing economic harm to news sources. No committee transcript or vote record is provided, so there is no documented floor or committee debate to indicate formal support or opposition, but the structure of the bill indicates likely support from publishers and likely concern from technology and AI stakeholders.
The likely contention centers on whether the bill is too broad in defining crawlers and covered news sources, and whether requiring disclosure of identity and purpose before access is workable for ordinary indexing, research, or AI systems. News organizations and supporters are likely to favor the bill as a response to undisclosed scraping and bot activity that can burden websites and reduce revenue. Technology companies, search engines, AI developers, and digital platforms may object that the bill could restrict legitimate automated access, create compliance burdens, and invite disputes over what constitutes economic harm or a stealth crawler.