Best AI Models for 8 GB of VRAM (What Actually Fits)

An 8 GB GPU such as the RTX 4060, RTX 3070 or RTX 5060 runs 4B to 8B models well at Q4_K_M. Anything larger will not fit once the context is added.

Check your own card with the Can My GPU Run This AI Model? calculator.

What fits in 8 GB (Q4_K_M, 8K context)

Best picks

• Qwen3 8B or Llama 3.1 8B: the best quality that fits. Keep context to 8K and close other GPU apps.
• Qwen3 4B: about 4.7 GB, so it leaves room for long context and runs very fast.

Tips for 8 GB

1. Use Q4_K_M, not Q6 or Q8.
2. Keep context at 8K or less. A Q8 KV cache helps if you need more.
3. Close games and browsers using the GPU.
4. Check that the model is fully on the GPU (in Ollama, ollama ps shows this).

When 8 GB is not enough

If you want 12B and larger models such as Gemma 3 12B (about 9.3 GB) or Qwen3 14B (about 11.4 GB), you need 12 GB or more. The RTX 3060 12GB is a low-cost route in memory terms.

Frequently asked questions

Can 8 GB run Llama 3.1 8B?

Yes at Q4_K_M, with about 7 GB used at 8K context.

Can 8 GB run a 13B or 14B model?

Not on the GPU alone.

What speed should I expect?

It depends on bandwidth: roughly 33 to 44 tokens per second on an RTX 4060 and 55 to 73 on a 448 GB/s card.

Sources and method

• Model architecture: each model’s published config.json on Hugging Face (Llama 3.1 8B and Gemma 3 are read from the unsloth mirrors because the original repositories require a login).
• Community speed reference: LocalScore.ai reports 38.3 tokens per second for Llama 3.1 8B Q4_K on an RTX 4060 (crowdsourced).

Estimates: weights = parameters × bits per weight; KV cache from the model’s published architecture; about 1 GB of runtime overhead. “Fits” means at most 92 percent of VRAM, “Tight” is 92 to 100 percent. Speed is memory bandwidth divided by model size at 60 to 80 percent efficiency. These are planning estimates, not our own benchmarks.

Last updated 3 Nov 2026.