Key points
- Newsom orders California to develop an AI "kill switch"
- An expert panel has two months to recommend rules
- He vetoed a similar mandate two years ago
California Governor Gavin Newsom signed an executive order on Friday. It directs state agencies to develop an AI "kill switch" and tighter safety rules for the most advanced AI models. "We're not waiting to act," Newsom said in the announcement, adding that "the stakes are too high to wait or delay action."
The order does not create the kill switch or the rules on its own. It tells the Government Operations Agency, working with the Governor's Office of Emergency Services, to bring in national experts and report back within two months. Newsom's office published the order, numbered N-9-26, on Friday.
What the order asks for
The expert panel is asked to recommend several changes to California's AI safety laws. The order lists a verified kill switch for frontier AI models, independent organizations to check company safety plans on site, and a wider definition of the incidents companies must report.
On the kill switch, the order asks the state to "[a]dvance the creation of a 'kill switch' for frontier models, with the efficacy of the switch verified on an ongoing basis by an independent verification organization." On reporting, it wants California to "[u]pdate definitions of critical safety incidents to include loss-of-control incidents such as the Hugging Face attack."
The order also tells agencies to move faster on two other state AI laws, SB 813 and AB 1405. It builds on SB 53, the Transparency in Frontier Artificial Intelligence Act signed last year, which the order calls "the nation's first AI safety law." SB 53 already requires frontier AI developers to disclose their safety frameworks and report critical incidents to the state.
A reversal from two years ago
In 2024, Newsom vetoed SB 1047, a bill from state Senator Scott Wiener that would have required makers of the largest AI models to build in a full shutdown capability, a kind of kill switch. CalMatters described Friday's order as coming after he rejected the tougher law.
The order lands while AI companies argue over how far state rules should go. OpenAI has shifted which California bills the company backs and is asking Congress for national rules. Nvidia (NVDA) chief executive Jensen Huang has called new antitrust rules meant to slow AI "completely unnecessary," while Anthropic chief executive Dario Amodei has urged the industry to slow down.
Newsom framed the order as California acting where Washington has not. "While Washington abdicates its responsibility to protect Americans, California is building on the strongest AI regulatory framework in the nation," he said. "California has already built a national model, and our policy should be the national baseline."
The rules would target frontier AI developers, the small group of companies training the most advanced models. The expert group's recommendations are due within two months. Any new mandate would then need action by the legislature or the agencies, not the order by itself.
