QuantizeLlama-3 8B
72% smaller
Weights
4-bit/ weight
4 shade levels, fewer bits store coarser values.
Size on disk4.5 GB
VRAM needed6.0 GB
Throughput88 tok/s
Quality kept96%
Fits comfortably on 16 GB6.0 GB needed · 10.0 GB free
Q4_K_M: best size / quality balance