Does Self-Hosting DeepSeek Pay Off for Mid-Size Companies

On October 20, 2026, DeepSeek released its inference engine "DeepInfer" under Apache 2.0. The Q&A document inside the repository contains operating figures such as "155 trillion tokens per day on a single cluster" and "US$38 of revenue per GPU-hour (at valley pricing)", along with storage and hardware requirements. Using this public information, this post estimates what it costs for a mid-size company to run DeepSeek-V4.1-Flash on its own hardware, and compares it against the pay-per-token API to find out at what scale self-hosting breaks even.

Key Takeaways

• The speed and efficiency techniques behind DeepSeek’s production service are almost fully open-sourced; only KV cache management and load balancing remain closed
• Hardware is over 90% of the investment: 400-550k USD for an 8×B300 node, while the KV cache SSDs (tens of TB) are a rounding error
• Self-hosting breaks even only when purchased outright with sustained demand above ~300B tokens/month; renting never pays off at any utilization

What the DeepInfer release actually revealed

The short answer: nearly the entire speed-and-efficiency stack is now open source, and the two pieces left closed are KV cache management and load balancing. Those two are the plumbing of a real service, and they remain unpublished.

17:We have previously open-sourced TileLang and a series of inference kernel
    libraries. **Today we are open-sourcing DeepInfer, our inference framework
    for large-scale cloud services.** It powers the high-throughput, low-latency
    large-scale inference service behind DeepSeek V4.1 Flash. A single cluster
    currently serves 155 trillion tokens per day, with overall revenue of
    US$38 per GPU-hour (at valley pricing).

• This is a revenue figure, not profit. No cost side is disclosed.
• It is qualified with "at valley (off-peak) pricing". The time-averaged rate should be higher than 38, which makes the claim conservative.
• The figures are self-reported. No third-party verification exists yet.

What is open and what is not, in one table:

Infrastructure layout reverse-engineered from the public info

Conclusion first: inference runs on commodity 8-GPU servers, while the KV cache lives on dedicated SSD storage servers. Treating GPU cost and storage cost as separate line items is the right way to read this architecture.

The requirements stated in the Q&A (excerpts from the repository document):

L77:  We recommend 20 × 30.73 TB SSDs per KV cache storage server,
      with each server providing at least 40 GB/s read and 5 GB/s write throughput ...
L81:  In practice, about 14.4 TB per GPU measured as logical size ...
L153: ... roughly 100 PiB of SSD storage for the KV cache,
      with a TTL of 3 days for agent traffic and 7 days for chatbot traffic ...
L217: Main KV: per-16, e4m3 — like NVFP4 but without the global scale.
L219: Indexer KV: per-32, ue8m0, MXFP4.
L67:  ... the smallest efficient deployment unit is 64 GPUs of prefill
      plus 64 GPUs of decode.

Three takeaways:

• DeepSeek’s own smallest *efficient* unit is 64 prefill + 64 decode GPUs. But the Q&A also says "1P + 1D is deployable", so technically an 8-GPU node can go below that.
• The KV cache is designed to live on disk. The production system keeps roughly 100 PiB of SSD glued together by 3FS, with garbage collection at TTLs of 3 to 7 days.
• KVs are stored in FP4. That is the core reason storage stays cheap, as shown below.

GPU cost estimate

Conclusion: one 8-GPU node runs 400-550 thousand USD, and hardware is over 90% of the initial investment. Storage and networking are rounding errors. What matters is the price of the whole server (CPU, power, chassis), not a bare GPU.

Market prices as of October 2026 (from multiple vendors’ published rates):

Two hard constraints:

• DeepInfer supports NVIDIA SM100 (B200/B300 class) only, as stated in the README. It does not run on H200.
• Rental pricing kills the economics. At 7.10 USD/GPU-hr, eight B300s cost 56.8 USD/hour, and as the simulation below shows, renting never breaks even. A purchase-based amortization is the only viable framing.

KV cache storage cost is a rounding error

Conclusion: a few hundred users need tens of TB, worth a few thousand to tens of thousands of USD — under 10% of the GPU investment. Storing KVs in FP4 is the core enabler.

Capacity per context, derived from the disclosure. The V3 generation (MLA, FP8) is known to need about 35KB per token. V4.1-Flash stores its main KV in FP4 (per-16 e4m3) and uses a CED (cross-layer encoder-decoder) structure that avoids persisting full decoder KV, so what actually lands on disk is mostly the encoder-side KV plus the indexer. Assuming roughly 5-15KB per token:

For reference, enterprise NVMe drives (U.2, 7.68TB) sell for about 3,900-6,300 USD each, and the cheapest in-stock enterprise price tracked in October 2026 was about 354 USD/TB. Sixty TB fits in one to three tens of thousands of USD.

Performance is not a bottleneck either. The Q&A asks for "at least 40 GB/s read and 5 GB/s write per storage server" — with 20 drives per server, that is only 2 GB/s per drive. A single modern consumer NVMe already sustains over 5 GB/s sequentially, so bundling many cheap drives beats buying fast ones, and the drive generation barely affects overall throughput. KV reads are large sequential block reads (about 1-4.5MB per 128-token block), a very different load profile from the random 4K I/O that dominates databases.

Break-even simulation against API pricing

Conclusion: with three conditions — buying the hardware, sustained demand above ~300B tokens/month, and people to operate it — self-hosting enters break-even territory. Renting never breaks even, even at 100% utilization.

Assumptions, stated openly (all figures below are derived estimates from public information, not measurements):

Monthly cost comparison under these assumptions:

Reading the break-even point:

• Purchase with 4-year amortization reaches parity at ~300-500B tokens/month. At 100M tokens per person, that is 2,000-4,000 heavy users.
• Eight rented B300s cost 56.8 USD/hour while the same load billed through the API tops out in the tens of USD/hour. Renting loses money even at full utilization.
• If you already own idle GPUs, marginal cost drops to power and operations, which is a different, much friendlier calculation.

One more structural point: raising utilization does not close the gap. The 38 USD/GPU-hr figure comes not from high utilization but from per-GPU token throughput that is 10-20x a self-built vLLM setup, stacked from speculative decoding × FP4 × cache hits × hand-tuned kernels. Keeping EP64+ batches constantly full is how that intensity gets earned, and that requires demand depth.

Why it may still not pay off

Conclusion: operations, not the spreadsheet, is the real wall. Beyond the cost table:

• The unpublished pieces must be self-built: KV cache management and load balancing are not open-sourced. 3FS itself is public, but the operational design — TTL, GC, cross-machine cache hits — has to be built and tuned in-house. This is the biggest engineering effort, and a simplified version lowers hit rates, eroding the API gap.
• Model lock-in: DeepInfer runs DeepSeek-V4.1-Flash only. Whether the engine tracks future models depends on DeepSeek’s updates, a risk baked into any 4-year amortization.
• Operations staffing is easy to undercount: node maintenance, capacity planning, failure response, security patching. One or two dedicated engineers belong in the amortization math.
• Regulatory direction: in 2026 the US DHS ordered DeepSeek products removed from federal civilian agencies and barred their procurement (published in the Federal Register). No law currently bans private-sector use, but the order extends to "third-party products and services built using DeepSeek implementations", which means companies selling to government or public-sector customers cannot adopt this strategy even self-hosted. Further regulatory expansion would act as a discount factor on any multi-year GPU investment.
• Data sovereignty as the contrarian upside: conversely, for companies with no government business and data that cannot leave the building, being able to keep the weights local is the single largest benefit — a gain that never shows up in a cost table.

Summary

• The infrastructure disclosure revealed "155 trillion tokens/day", "38 USD/GPU-hour", and an FP4-based KV cache layout. The speed comes from a stack of published techniques; only cache management and load balancing remain closed.
• For a mid-size company, GPUs (400-550k USD for 8×B300) are over 90% of the investment, while KV cache SSDs are tens of TB and a rounding error. Against API pricing, break-even requires 300B+ tokens/month and a purchase-based amortization plan.
• Renting never pays off at any utilization. Before asking "does it break even", ask whether you can absorb the self-built cache infrastructure, the operations staffing, and whether data sovereignty matters enough to justify it in the first place.