Artificial intelligence · 2015
ResNet
Skip connections let networks grow to 152 layers and beat human ImageNet error.
Kaiming He and colleagues at Microsoft Research Asia added identity shortcuts so each layer learns a residual. ResNet won ImageNet 2015 at 3.57% top-5 error. Residual connections are now in every transformer.
What it built on 1
What it led to 3
Related measurements
The dashed line marks when it first worked.
Training compute of notable AI models Source: Epoch AI, Notable AI Models (CC BY 4.0)
ML hardware FLOP/s per dollar Source: Epoch AI, ML hardware (CC BY 4.0)