Key points
- DeepSeek shares the software behind its API
- What DeepSeek kept out of DeepInfer
- How DeepSeek's prices compare with Anthropic's
DeepSeek released DeepInfer on Oct. 20, giving developers access to much of the software behind its V4.1 Flash AI service. The framework lets companies run the model on their own servers, although DeepSeek has kept some infrastructure code private.
The company said the release aims to help developers cut computing costs and energy use. The code is available on GitHub under the Apache 2.0 license, which allows commercial use. The repository had attracted more than 12,000 stars by Oct. 20.
DeepSeek on X
DeepInfer supports Nvidia and Huawei chips
DeepInfer currently runs only DeepSeek-V4.1-Flash and loads its Oct. 17 update by default. DeepSeek plans to add inference code for future models.
The framework supports Nvidia (NVDA) SM100 GPUs from its Blackwell family and Huawei's Ascend 950 chips, according to the README. Batch-invariant inference and deterministic sampling, two features designed to make results reproducible, are available only on Nvidia hardware.
Ascend support follows DeepSeek's Sept. 30 release of programming tools for Huawei chips.
DeepSeek says one cluster serves 155 trillion tokens a day
A single cluster running the V4.1 Flash service handles 155 trillion tokens a day and brings in $38 of revenue per GPU per hour at off-peak prices, according to DeepSeek's deployment Q&A. Tokens are the small chunks of text a model reads and writes.
The document describes several ways the framework cuts computing work. In a technique DeepSeek calls DSpark, a smaller draft model proposes five tokens at a time and the main model checks them. DeepSeek said a confidence score on each draft token reduces the number checked per step by about 30%, which works out to a 4% to 7% throughput gain at its production load.
A second change recomputes only the last 128 tokens of each prompt in part of the model. DeepSeek said that nearly halves total computation in prefill, the stage where a model processes the prompt before it starts answering.
The figures are DeepSeek's own. The README's speed charts measure each stage on its own, prefill on four SM100 GPUs and decoding, the stage that generates the answer, on 64. DeepInfer was originally built mainly on vLLM, an open-source inference project, with ideas from SGLang and Nvidia's TensorRT-LLM, the README says.
DeepSeek held back two parts of its system
DeepSeek hasn't fully released its cache management or load-balancing code. The company said those parts are "tightly coupled to our internal infrastructure" and described how they work in a Zhihu blog post instead. DeepSeek's own service runs prefill and decoding on separate groups of machines. The released code doesn't include that split yet, though the README says most of the engine work needed for it is already in place.
For a split setup like its own, the smallest unit DeepSeek describes as efficient uses 64 GPUs for prefill and 64 for decoding. The company recommends at least 64 GPUs for decoding and says the two stages run at roughly a 1-to-1 ratio in practice.
In April 2025, DeepSeek decided against releasing its full inference engine, which was then an early fork of vLLM. "As a small research team focused on developing better models, we lack bandwidth to maintain a large open-source project," the company wrote at the time.
How DeepSeek's API prices compare with Haiku
Matthias Bastian of The Decoder argued that the release adds pricing pressure on Western AI providers such as OpenAI and Anthropic.
DeepSeek charges $0.30 per million uncached input tokens and $1.20 per million output tokens for V4.1 Flash during peak hours, and half that off-peak. Anthropic lists Claude Haiku 5.5 at $0.10 per million input tokens and $0.50 per million output tokens for prompts up to 100,000 tokens, and Claude Opus 5.5 at $4 and $20, according to its pricing page.
Haiku's rates are also below DeepSeek's off-peak prices. DeepSeek is cheaper on cached input, charging $0.006 per million tokens at peak against $0.01 for Haiku 5.5. The models differ in size and capability, so the prices aren't a direct comparison.
DHS orders DeepSeek out of federal civilian agencies
The release came a day after the Department of Homeland Security made public an order barring DeepSeek products from federal civilian agency purchases. The order, published in the Federal Register on Oct. 20, also requires agencies to remove DeepSeek products, as well as third-party products and services developed using DeepSeek implementations, from their IT systems. The signed copy lists an original release date of July 24.
