Key points
- OpenAI publishes the perplexity scores it uses to track its models
- OpenAI's unreleased internal model scores 0.732, its lowest yet
- OpenAI's chief scientist proposes industry limits tied to the metric
OpenAI published the perplexity scores it uses to track progress in training its AI models, and Chief Scientist Jakub Pachocki called for developers to use comparable measurements to set shared safety standards.
In a Nov. 22 post, Pachocki proposed that labs agree on development limits tied to perplexity and other measures. OpenAI's latest unreleased model recorded its lowest score yet on the company's test.
Publishing the scores "gives competitors a clearer picture of where we stand," Pachocki wrote, but he said the need for shared safety measures outweighed that concern.
Perplexity measures next-token prediction
Perplexity measures how well a model predicts the next token, a word or piece of a word, on average. A lower number means better predictions. OpenAI reports the logarithm of perplexity, which is also called cross-entropy loss.
OpenAI measures it on held-out data, text the model wasn't trained on, taken from OpenAI's internal codebase. The score tracks progress in pretraining, the first stage of training on a large body of text, rather than later steps such as fine-tuning.
The metric is unrelated to Perplexity, the AI search startup.
OpenAI's scores fell faster in 2026
The chart in OpenAI's post shows the score for each model, with labels for some of the abilities OpenAI says each one reached.
| Model | Date on chart | Log-perplexity | OpenAI's label |
|---|---|---|---|
| GPT-3.5 | March 2022 | 0.986 | Fluent conversation |
| GPT-4 | March 2023 | 0.913 | Benefits from thinking before answering |
| GPT-4 Turbo | November 2023 | 0.909 | |
| GPT-4o / o1 | May 2024 | 0.904 | |
| GPT-4.5 | February 2025 | 0.876 | |
| GPT-4.1 / o3 | April 2025 | 0.902 | |
| GPT-5 / 5.1 | August 2025 | 0.893 | |
| GPT-5.2 | December 2025 | 0.888 | |
| GPT-5.3-Codex / 5.4 | February 2026 | 0.871 | Useful coder |
| GPT-5.5 | April 2026 | 0.848 | |
| GPT-5.6 Sol | July 2026 | 0.828 | Cybersecurity expert |
| GPT-6 Astra | September 2026 | 0.783 | |
| GPT-6.2 Astra | November 2026 | 0.761 | Solved a Millennium Prize math problem |
| Internal model (unreleased) | November 2026 | 0.732 |
Source: OpenAI, chart data in its Nov. 22, 2026 post. Lower is better. Scores are measured on held-out data from OpenAI's internal codebase.
The scores come from a fixed snapshot of that codebase. They aren't a universal intelligence score and can't be compared directly with other labs' numbers, and OpenAI cautions that the choice of dataset affects how informative the measure is for different models.
The score fell 0.110 from GPT-3.5 to GPT-4.5 over nearly three years, and 0.156 from GPT-5.2 to the internal model in about 11 months, by our calculations. OpenAI attributes the steep acceleration in 2026 mostly to long-term research and computing investments and only in a smaller part to automation. For the most recent models, OpenAI has "already seen meaningful design contributions coming from AI," Pachocki wrote.
The chart doesn't say which Millennium Prize problem GPT-6.2 Astra solved. OpenAI said in September that a model had resolved the Navier-Stokes problem, one of the seven Millennium Prize Problems. Scientific American reported that the result relied on an external force that the Clay Mathematics Institute's formulation allows, which many experts consider a loophole. On Nov. 20, Tristan Buckmaster of New York University posted an AI-aided proof with Anthropic's Levent Alpöge that the two say closes that loophole and covers the version without an external force, the magazine reported.
Pachocki expects new capabilities near 0.68 to 0.70
Pachocki wrote that once OpenAI's models reached losses below 0.8, it saw "the emergence of transformative capabilities in fields like cybersecurity and mathematics." A model with a score of about 0.76 "started producing groundbreaking results" on open math problems, he wrote.
"Extrapolating from recent progress, we should expect further transformative capabilities as we enter the ~0.68-0.7 log-perplexity window," Pachocki wrote. "I don't know what those capabilities will be, and we may not be testing for them today." OpenAI's unreleased internal model is at 0.732.
He also wrote that AI could speed up research on pretraining itself, and that "as soon as 2027 we could conceivably see a similar avalanche of fundamental pretraining advancements" to the one OpenAI saw in math.
Pachocki proposes limits that labs agree not to cross
Pachocki proposed that AI developers agree on "common limits, based on perplexity and other measurements," that they would not cross until their ability to understand and monitor AI behavior reaches a concrete milestone. His example of such a milestone is a successor to chain-of-thought monitors, tools that read the reasoning a model writes out before it answers.
He listed problems with the metric. Different labs measure perplexity on different datasets, so their numbers can't be compared directly yet. Perplexity doesn't capture gains from chain-of-thought reasoning, which Pachocki wrote "leads to a drastic increase in capabilities." And no one knows how low the score can go.
Pachocki called for "extreme caution" in an earlier essay in September, as we reported at the time. The new post comes two days after Alpöge posted a claimed proof of the Riemann hypothesis developed with Claude. The Lean and Mathlib communities have certified its formal proof, while mathematicians are still working to understand the argument.












