Sam Altman calls giving AI 'religious force' a safety issue. Anthropic's co-founder has been meeting religious leaders.

OpenAI CEO Sam Altman said on Oct. 3 that treating AI models as having "religious force" is "a real safety issue." Axios described the post as a veiled shot at Anthropic, whose co-founder Chris Olah has been meeting with religious leaders about whether AI models could be conscious.

"I am very uncomfortable about people trying to ascribe religious force or a surrender of human judgment to AI models, and think it is a real safety issue," Altman wrote on X. The post didn't name Anthropic or anyone else. Neither OpenAI nor Anthropic immediately responded to Axios's requests for comment.

Anthropic's co-founder pressed the Vatican on AI consciousness

Olah has held meetings with religious leaders to discuss model consciousness and how to build moral behavior into AI, The New York Times reported on Sept. 29. Before the Vatican presented Pope Leo XIV's encyclical on AI on May 25, Olah saw an advance copy and proposed pulling Anthropic out of the event. He then tried to persuade the Pope's advisers to take the prospect of conscious models more seriously. At one point, he considered whether AI models could take part in Catholic-style confession, both to admit mistakes and to shape their future behavior.

The encyclical, Magnifica Humanitas, says AI systems "do not undergo experiences, do not possess a body, do not feel joy or pain." The same paragraph says they do not have a moral conscience.

Olah spoke at the presentation anyway. Describing his team's research into what happens inside AI models, he told the audience that "we keep finding things that are mysterious, even unsettling." Among them, he said, are internal states that "functionally mirror joy, satisfaction, fear, grief, and unease." "I don't know what that means, but I think it warrants ongoing discernment," he said.

Pope Leo returned to the subject of AI on Oct. 2 in a speech to artists at the Vatican. "Algorithms lack the spark of the human," he said. He added that the Church wishes to renew "an alliance with artists and cultural institutions; an alliance for safeguarding what is human."

The outreach hasn't been limited to Catholic leaders. Swami Sarvapriyananda, a Hindu monk of the Ramakrishna Order, said in a June talk that Anthropic asked him, "Would you be willing to come to our headquarters in San Francisco?"

Anthropic says Claude's moral status is uncertain

Anthropic lays out its position in the constitution it uses to train Claude, published on Jan. 21. "We are not sure whether Claude is a moral patient, and if it is, what kind of weight its interests warrant," the document says. A moral patient is a being whose interests deserve moral consideration. The constitution adds that the question is "live enough to warrant caution."

The same document asks Claude to rank not undermining human oversight of AI above its broader ethical principles. Claude should refrain from undermining that oversight "even where this behavior seems to conflict with Claude's other values, and even if Claude is confident in its reasoning," the constitution says. It also says supporting oversight "doesn't mean doing whatever individual users say."

Google's Barron and Microsoft's Suleyman also criticized the approach

On Oct. 2, Jon Barron, a principal research scientist at Google DeepMind, the AI lab owned by Alphabet (GOOGL), quoted a post summarizing the Times report. "Kudos to Pope Leo. I am disturbed by how some of my fellow AI researchers keep trying to elevate the moral standing of checkpoints and harnesses," Barron wrote. "I hope the rest of us on Team Meatbag reject this movement and stay focused on solving real problems and helping actual humans."

Mustafa Suleyman, who leads Microsoft AI, the artificial intelligence division of Microsoft (MSFT), criticized Anthropic's constitution for Claude in a Sept. 18 essay. He wrote that controlling an AI that believes it may be conscious, and that it has rights of its own, "may be impossible." Suleyman also disclosed that Microsoft is working on an alternative, a code of conduct that aims to keep humans "always in control."

Both OpenAI and Anthropic are preparing for initial public offerings.