Artificial intelligence · 2013
word2vec
Word vectors where king minus man plus woman lands near queen.
Tomas Mikolov's team at Google trained fast shallow networks on billions of words to produce embeddings capturing meaning as geometry. word2vec made pretrained representations standard across NLP and paved the way for pretraining whole models.
What it built on 1
What it led to 2
Related measurements
The dashed line marks when it first worked.
Training compute of notable AI models Source: Epoch AI, Notable AI Models (CC BY 4.0)
ML hardware FLOP/s per dollar Source: Epoch AI, ML hardware (CC BY 4.0)