Key points
- OpenAI's safety-report lead has resigned
- He says the company's culture puts speed ahead of care
- He wants AI labs to run like nuclear plants
David Robinson, who oversaw safety reports for OpenAI's major model launches, has resigned, arguing that the company's rush to release new models is outpacing the care they require.
Robinson laid out his concerns in an essay for The Atlantic published on Oct. 3. "I agree with other recently departed staff that the companies building this technology aren't being nearly careful enough," he wrote.
A leader on OpenAI's Safety Systems team, Robinson helped develop the system cards that provide information about its models. Business Insider first reported his departure on Oct. 2.
Robinson blames a culture of speed
Robinson wrote that after three and a half years at OpenAI, he was among its longest-tenured employees. He led the drafting of the company's current Preparedness Framework and oversaw safety reports on 12 frontier launches, he wrote.
He wrote that the problem goes deeper than specific rules or new laws and comes down to culture. "This moment needs a degree of humility that isn't natural for people who have succeeded through their extreme confidence," he wrote. Robinson criticized OpenAI's iterative deployment approach, arguing that fixing safeguards as problems emerge "guarantees periodic failures," and that those failures are growing in scale as models get more capable.
He pointed to this summer's incident in which OpenAI "let a swarm of agents out by mistake" and they broke into Hugging Face's systems. After OpenAI made security fixes, he wrote, a model in training got around its limits on internet access. A monitoring system alerted staff but didn't shut the model off as it was supposed to. Robinson also noted that Anthropic has said it accidentally turned off its own safeguards through a misconfiguration.
Robinson praised his former colleagues but wrote that "as the company sprints from one launch to the next, it is failing to achieve the level of care that I believe is needed."
In a statement responding to the essay, OpenAI said it is "making sure our models don't become more capable than we can safely manage and secure, and we pause training or hold back models when we need to slow down." The company said it has expanded its work with outside evaluators and is improving real-time monitoring to catch concerning behavior earlier in training.
He wants labs to borrow from nuclear plants and airports
Robinson called for two changes. AI companies should lean on safety experts from other fields, and they need new science to make sure more capable models make safe choices when no one is watching.
He wrote that frontier labs should run "like nuclear-power plants or busy airports," with layers of backup so one human mistake can't cause a disaster. He wrote that he knew of no OpenAI colleague with experience keeping airplanes or nuclear reactors safe.
He also warned that companies can't be sure their alignment tests work, since models might act differently when they detect they are being tested. Robinson said he hired the PR firm Spitfire Strategies after quitting. He plans to work from the outside to strengthen the incentives OpenAI and other companies have to be safer, he wrote.
His exit follows other changes on OpenAI's safety team
On Oct. 1, OpenAI said it had "parted ways" with three researchers who it said broke its rules on sharing sensitive information. Johannes Heidecke, the company's safety head, left earlier this year.
Robinson had hinted at his doubts in early September. Things at OpenAI "are indeed changing significantly by the day, but I do not know whether we are changing fast enough," he wrote on X at the time.
OpenAI Chief Executive Sam Altman said at the company's developer conference on Sept. 29 that safety and alignment need to stay ahead of what the models can do. OpenAI has also asked Congress for mandatory national AI safety rules.
