Artificial intelligence · 2024
Scaling Monosemanticity
Millions of interpretable features are extracted from inside a production model.
Anthropic's May 2024 work trained sparse autoencoders on Claude 3 Sonnet and found features for concepts like the Golden Gate Bridge, which could be amplified to steer behaviour. It was a major step for mechanistic interpretability of frontier models.
What it built on 2
Related measurements
The dashed line marks when it first worked.
Training compute of notable AI models Source: Epoch AI, Notable AI Models (CC BY 4.0)
ML hardware FLOP/s per dollar Source: Epoch AI, ML hardware (CC BY 4.0)