Engineer Jacob Cookson warned the New York City Council on Monday that artificial intelligence labs prioritize speed over safety, creating a high likelihood that humanity will lose control of advanced systems and face catastrophic outcomes as companies race to build unmanageable technology.
A former researcher who left Anthropic in September stepped before the New York City Council in Manhattan to deliver a warning about the future of artificial intelligence. All 51 members of the council attended the full-committee meeting alongside remote testimony from representatives of Anthropic, OpenAI, Google, and Meta.
The proceedings exposed a deep rift between municipal lawmakers and the tech industry over safety protocols, corporate transparency, and the pressure of the global technology race.
Engineers Warn New York Lawmakers of Extinction Risks
Testifying before the council, Jacob Cookson critiqued the engineering culture driving the development of frontier models. He argued that the industry relies on a playbook borrowed from software startups that is unsuited for systems of this magnitude.
“Move fast and fix things later”
Jacob Cookson, former Anthropic researcher, via Aljazeera
He warned lawmakers that humanity does not yet know how to control the systems being developed.
Jacob Cookson told Shorouknews that on the current path, I think the most likely outcome is that humanity loses control of AI systems, and this could end in the extinction of the human race.
Other former industry researchers echoed these concerns during the hearing. Daniel Kokotajlo, who left OpenAI, called on the sector to increase transparency and slow down the rapid development of leading models, according to CNBC coverage. Alex Turner, a former researcher at Google DeepMind, characterized unmitigated AI development as an internal threat that could eclipse geopolitical competitors.
“China is not our only potential enemy”
Alex Turner, former Google DeepMind researcher, via Aljazeera
SpaceX Fails to Comply with Legal Subpoena
Julie Menin revealed that out of five companies contacted between September 15 and 17, only Meta agreed to participate voluntarily. Google and Anthropic initially rejected the deadline, prompting Menin to authorize legal subpoenas for the remaining firms. OpenAI and Google reversed course on Sunday, while Anthropic confirmed its attendance late that night. SpaceX, owned by Elon Musk, failed to appear entirely.
According to Aljazeera, Julie Menin stated that SpaceX, owned by the richest man in the world, Elon Musk, was the only one that failed to comply with the legal subpoena issued by the council, even after the city issued a summons for Musk,
confirming that the council intends to pursue the matter in court. Representatives from Meta, Anthropic, OpenAI, and Google attended the session via video link, while SpaceX was absent. Menin appeared frustrated at times during the session, a report noted.
When questioned by lawmakers via video link, corporate representatives defended their safety practices. Morgan Dwyer, head of policy development and operations at OpenAI, stated that she did not believe a distinction mattered between a 1% or 20% chance of catastrophe, emphasizing that companies should not train models unless they can prove they can keep them under control. Alice Friend of Google urged the creation of a federal framework to handle national security concerns.
Proposed New York City Regulations Target Safety Gaps
The municipal hearings take place as local governments step into a policy vacuum left by federal authorities. While President Donald Trump dismissed concerns about artificial intelligence as a hoax and signed a late-September voluntary agreement allowing companies to self-regulate, New York lawmakers are weighing a suite of ten legislative proposals.
- Third-party verification requirements before any artificial intelligence system can be sold within the city.
- Mandatory human oversight switches for deployed models.
- Financial incentives and whistleblower protections for employees reporting safety violations, including a share of recovered fines.
- A strict 24-hour reporting mandate for city contractors utilizing AI tools.
- Empowering the city to file lawsuits when compromised AI tools cause financial or physical harm.
Lawmakers Confront Unresolved Questions Over Liability
Major technology firms continue to emphasize the daily economic and medical benefits their models provide, arguing that innovation and safety can advance in tandem. When pressed by Julie Menin on legal liability for severe financial damage or loss of life caused by AI systems, corporate representatives repeatedly deflected direct answers.
With the municipal council preparing to challenge SpaceX in court over its unheeded subpoena, lawmakers and whistleblowers are waiting to see whether local mandates can successfully bind an industry hurtling forward under the banner of rapid progress, or if legal liability will remain entirely undefined.
