Artificial intelligence · 2023
Vision-language-action models
Robots controlled by a web-trained vision-language model that outputs actions.
Google DeepMind's RT-2 (July 2023) fine-tuned vision-language models to output robot actions as tokens, transferring web knowledge to unseen objects and instructions. It started the robotics foundation model approach.
What it built on 2
What it led to 6
Related measurements
The dashed line marks when it first worked.
Training compute of notable AI models Source: Epoch AI, Notable AI Models (CC BY 4.0)
ML hardware FLOP/s per dollar Source: Epoch AI, ML hardware (CC BY 4.0)