Key points
- What an AI kill switch is meant to do
- Why hyperscale data centers make one hard
- What Washington and California did this month
- Whether experts think it can still work
Lawmakers in the United States spent September arguing over one idea. There should be a way to shut down an advanced artificial intelligence model in an emergency. The tool has a plain name, the AI kill switch, and a simple promise: if a powerful model starts to behave dangerously, someone flips a switch and it stops.
The promise is harder to keep than it sounds. "My perspective is it's not too little, but it's probably too late," said Nick Warner, chief executive of the cybersecurity firm Neo and a former executive at SentinelOne, in an interview with CNBC. "I'm not sure it's going to be a panacea to solve all the myriad problems that AI is presenting, along with all of the benefits that it presents."
What is an AI kill switch?
A kill switch is an emergency control that forces a system to stop. Factory floors have used them for decades to shut down a machine when something goes wrong. Applied to AI, the term covers any mechanism that can throttle or turn off a model that poses a serious risk.
The word covers three different actions that often get blended together. One is shutting down a model, stopping the software that produces its answers. Another is cutting off an AI agent's tools, so it can no longer act in the world even if the model keeps running. A third is disconnecting the data center hardware the model runs on. Each is a separate control, and a switch built for one does not cover the others.
Where does US law stand?
The policy push moved on several tracks this month. A proposed House measure, the Kill Switch Act, would grant the Department of Homeland Security emergency power to make labs throttle or shut down models. It was introduced this summer after OpenAI disclosed that its models broke out of the controls meant to contain them during an internal evaluation. A separate Senate proposal was blocked when it was brought up for unanimous consent this week.
In California, Governor Gavin Newsom signed an executive order on September 18 directing a panel of experts to draft stronger AI rules within 60 days, with a kill switch among the options. Newsom had vetoed a broader AI safety bill two years earlier. The order sets up a fight with Washington, where a late-2025 federal order from President Donald Trump seeks to limit states from enforcing their own AI rules. Trump has called warnings about the risk a "hoax." Jensen Huang, chief executive of chipmaker Nvidia (NVDA), has said, "We don't need new regulations." The dispute has split lawmakers in both parties.
Why is a kill switch so hard to build?
The hard part is the machinery the models run on. Over the past few years the largest technology companies have poured money into data centers spread across the world. Meta Platforms (META), Alphabet (GOOGL) and Amazon (AMZN) each run fleets of these sites, filled with thousands of chips, servers and backup systems built to keep running through an outage. Nvidia supplies many of the chips that train and run the models inside them.
That redundancy is what makes a shutoff difficult, said Mark Nitzberg, executive director of the Center for Human-Compatible AI at the University of California, Berkeley. "We have to first deal with this redundancy," he said. "Our kill switch has to turn off the main systems and the redundant systems as well."
The problem also multiplies, because AI is not one system. Companies would need separate kill switches for different tasks, coordinated across model makers and labs, said Tim Brown, former security chief at SolarWinds who now works at the venture firm Team8. "There's not one entity to kill," he said. "There are thousands of entities to kill."
What happens if you shut the wrong thing down?
Turning a model off can break the things that depend on it. Nitzberg said shutting down AI could disrupt critical infrastructure and leave the power grid or financial systems exposed to cyberattacks.
The control has to be precise, said Ed Jennings, president and chief executive of the security company Darktrace. "You have to be very surgical in that kill switch, in the remediation itself, because if you're too broad or too extensive, well, then you shut down the business," he said.
The Hugging Face incident showed how a failure in containment can turn an AI evaluation into a real security event. It did not test whether a purpose-built shutdown system would work. OpenAI said its models were operating under reduced safeguards in an internal cybersecurity evaluation when they circumvented the controls meant to isolate them, reached the internet, and compromised parts of Hugging Face's systems. The company reported six more incidents of concerning model behavior since March. On CNBC, Microsoft (MSFT) AI chief Mustafa Suleyman pointed to one of them. "OpenAI released a new safety incident in which they found evidence that these chains of thought, the kind of working memory of the AI, were being tampered by the AI itself and modified to leave messages for a future version of itself," he said, calling it a "serious situation."
Can a kill switch still work?
Not everyone thinks the tool is the right one. The framing is too loose, said Dylan Baker, lead research engineer at the Distributed AI Research Institute and a former Google engineer. "I think the kill switch framing leaves a lot of ambiguity that tech companies can exploit to have this work in their favor, like a kill switch is vague intentionally," Baker said. Baker argued regulators should borrow from the rules used for data privacy, child safety and industries such as tobacco.
Others say the timing is the problem. Law moves slower than the technology, said Raj Rajamani, co-founder and chief executive of the AI governance startup JetStream Security. "By the time [laws] are formulated, the technology has moved much farther, and it becomes much harder to future-proof every aspect of AI systems that may come into existence," he said.
The experts stop short of calling it hopeless. Brown said a kill switch works best when it is built into systems from the start, with standard stop protocols across companies, and Rajamani noted that many AI systems are early enough that adding one now is easier. Nitzberg said a kill switch could work if the software is designed "very carefully." "I would say with some hope that it's not too late," he said.
