Artificial intelligence · 2024

DeepSeek-V3

A 671B-parameter open MoE model trained for about $5.6 million in GPU time.

Released December 26, 2024, DeepSeek-V3 activated 37B parameters per token and used multi-head latent attention and FP8 training on 2.788 million H800 GPU hours. It matched leading closed models on many benchmarks and reset expectations of training cost.

See it in the living treeWikipedia