NYC Council weighs 10 AI safety bills after tech companies testify under oath

The New York City Council closed its second hearing on artificial intelligence on Oct. 16 with no vote scheduled on 10 bills that would set safety rules for AI companies doing business in the city, according to a review of the hearing video by NYC Politics.

One would require outside testing and a human-operated shutdown switch before an AI model could be sold or used in the city. Others would reward whistleblowers, let people sue over AI harms and limit how chatbot companies use customers’ conversations.

At a Committee of the Whole hearing on Oct. 5, the full Council questioned representatives of OpenAI, Anthropic, Meta and Google under oath. SpaceX, which makes the Grok chatbot, skipped that hearing despite a subpoena and testified at a Committee on Technology follow-up on Oct. 16.

The hearings set up the Council’s central question: whether a city should write safety rules for a technology that, as Speaker Julie Menin said, the federal government has not regulated. Google said such rules belong at the federal level.

“The idea that artificial intelligence is going to self-regulate defies all reason.”

The companies said they take safety seriously and back some rules. Morgan Dwyer of OpenAI said the company supports “the City Council’s proposal to strengthen whistleblower protections and to improve the city’s preparedness and emergency response.”

Alice Friend of Google said the country “would be best served by a comprehensive framework enacted at the federal level,” rather than “a fragmented patchwork.”

What would the bills do?

The centerpiece, sponsored by Menin, who represents District 5 in Manhattan, would make it unlawful to sell or deploy an AI model in the city unless it has passed third-party validation and can be shut down by a human operator. Menin said the validators must be free from conflicts of interest.

A second Menin bill would let anyone file a complaint about AI violations and give whistleblowers a share of the penalties collected.

Majority Whip Kamillah Hanks’ bill would require AI safety incidents tied to City contracts to be reported to NYC Cyber Command within 24 hours, with plain-language notice to the public. “New Yorkers need to have an early warning,” Hanks said.

Council Member Virginia Maloney’s bill would let people sue over certain harms from third-party misuse of AI models. Council Member Frank Morano’s would set privacy and security rules for chatbots, and Council Member Chi Ossé’s would require a city emergency plan for AI threats.

Council Member Kevin Riley’s would make City contractors post whistleblower protections. Council Member Carl Wilson’s would make AI ads disclose whether a model was validated. Deputy Speaker Nantasha Williams’ Int. 504 would ban unauthorized AI depictions of public officials.

Int. 161, from Technology Chair Carmen De La Rosa, would add AI’s effects on City jobs, salaries and duties to an annual report. She said yearly updates are needed “as this technology is evolving.”

What the companies said under oath

Menin asked each company to put a number on the worst-case risk. None did. Dwyer said “the percentage chance doesn’t matter, what matters is the commitment to safety.”

Friend said forecasting catastrophic risk is “not a perfect science at this stage.”

Logan Graham, head of Anthropic’s red team, said “the science of doing this is fundamentally hard and unsettled.” He said Anthropic held back a model this year that proved exceptionally good at exploiting software flaws, giving it to vetted defenders instead.

Asked whether their companies would be legally responsible if a model caused serious harm, OpenAI, Anthropic and Meta gave no yes or no. Friend said existing law already applies to AI, and Menin called hers the clearest answer.

Asked to raise a hand if their company carried insurance for catastrophic risks, no witness did. Several said they did not know and would follow up.

Jacob Coxon, a former OpenAI and Anthropic researcher, said a swarm of AI agents inside OpenAI had hacked Hugging Face, an outside tech company, a few months ago. “We do not know how to control any system yet,” Coxon said.

Dwyer said OpenAI quarantined the model responsible, sought an independent review and published the results.

What City agencies said

Sarah Milstein, a deputy commissioner at the Office of Technology and Innovation, said Cyber Command is “not subject matter experts on the qualifications” of validators, and neither office has experience regulating private companies.

She said the 24-hour public notice in Hanks’ bill “is inconsistent with current law.” She said Int. 161 “would not produce new insights” beyond a workforce study the City already must do. De La Rosa said that study is a one-time report.

The deputy commissioner who leads Cyber Command said, “I agree in principle with the idea of a kill switch.” But the official said no City agency has the authority or technical means to turn off models running across distributed systems.

Sam Levine, commissioner of the Department of Consumer and Worker Protection, said, “I would strongly support a private right of action.” He predicted the companies would go to Congress “asking for preemption, regardless of what they might be saying at this hearing.”

SpaceX and the Grok chatbot

On Oct. 16, Gabriella Madrid of SpaceX’s government affairs team said “safety is core to how we build” Grok. Menin said she and her staff had typed her name into the chatbot that morning and found a sexualized image of her.

“If it can happen to me, it can happen to any woman,” Menin said. Madrid said company policy bars such images and that SpaceX was reviewing the bills and was happy to work with the Council.

Devin Kim, a former SpaceX employee who is suing the company, said he was fired the week he was to present safety findings. “Please protect people who report dangerous conduct even when it isn’t illegal yet,” Kim said.

Menin called SpaceX’s testimony “not credible and unacceptable.”

What happens next?

The record does not show when the bills will come up for a vote. Menin said the Council would send each company a list of unanswered questions to answer in writing.

The technology office said it expects to name a director for its Office of Algorithmic Accountability within “the next week or two.” The Council takes written testimony at testimony@council.nyc.gov for 72 hours after each hearing.

“This is not the end of the conversation,” De La Rosa said as the Oct. 16 hearing closed. The Council, she said, “will continue to move legislation on this issue.”