Latest Article
Q4 vs Q5 vs Q8: Which Quantization Fits Your GPU VRAM?
Q4_K_M, Q5_K_M, Q6_K, Q8_0 and FP16 compared by memory use. See what each quantization means for an 8B model and which one fits 8, 12, 16 and 24 GB.
Q4_K_M, Q5_K_M, Q6_K, Q8_0 and FP16 compared by memory use. See what each quantization means for an 8B model and which one fits 8, 12, 16 and 24 GB.
