DeepSeek article exposes the cost..etc

LINK

quote:

---

The most proximate announcement to this weekend’s meltdown was R1, a reasoning model that is similar to OpenAI’s o1. However, many of the revelations that contributed to the meltdown — including DeepSeek’s training costs — actually accompanied the V3 announcement over Christmas. Moreover, many of the breakthroughs that undergirded V3 were actually revealed with the release of the V2 model last January.

---

Now we get to the truth:

The key implications of these breakthroughs — and the part you need to understand — only became apparent with V3, which added a new approach to load balancing (further reducing communications overhead) and multi-token prediction in training (further densifying each training step, again reducing overhead): V3 was shockingly cheap to train. DeepSeek claimed the model training took 2,788 thousand H800 GPU hours, which, at a cost of $2/GPU hour, comes out to a mere $5.576 million.

DeepSeek is clear that these costs are only for the final training run, and exclude all other expenses; from the V3 paper:

DeepSeek is clear that these costs are only for the final training run, and exclude all other expenses; from the V3 paper:


once more


DeepSeek is clear that these costs are only for the final training run, and exclude all other expenses; from the V3 paper:


We do not know anything other than the final training run cost them 5.5 million.

A training run does not, I repeat, DOES NOT encompass all the costs.