Artificial intelligence · 2023
llama.cpp
Quantised large models run on a laptop CPU.
Georgi Gerganov's C/C++ inference engine, released in March 2023, ran LLaMA on a MacBook using 4-bit quantisation. With its GGUF file format it became the backbone of local AI tools such as Ollama and LM Studio.
What it built on 3
Related measurements
The dashed line marks when it first worked.
Training compute of notable AI models Source: Epoch AI, Notable AI Models (CC BY 4.0)
ML hardware FLOP/s per dollar Source: Epoch AI, ML hardware (CC BY 4.0)