Short answer: Gemma 3 12B Instruct at Q4_K_M needs about 9.3 GB of memory with an 8K context, so you want a GPU with at least 12 GB of VRAM. Context length, the quantization you choose and other GPU apps all change that number.
Check your own setup with the Can My GPU Run This AI Model? calculator. The numbers below come from the same data.
How much memory does Gemma 3 12B Instruct need?
Gemma 3 12B Instruct has 12.19 billion parameters, 48 layers and 8 KV heads of size 256. Weights take parameters × bits per weight ÷ 8. The KV cache adds memory as context grows, and the runtime adds about 1 GB.
| Quantization | Bits per weight | Weights | Total at 8K context (est.) |
|---|---|---|---|
| Q4_K_M | 4.9 | 7.5 GB | 9.3 GB |
| Q5_K_M | 5.7 | 8.7 GB | 10.6 GB |
| Q6_K | 6.6 | 10.1 GB | 11.9 GB |
| Q8_0 | 8.5 | 13.0 GB | 14.8 GB |
Which GPUs can run it?
At Q4_K_M and an 8K context:
| GPU | VRAM | Result | Speed (est., tok/s) |
|---|---|---|---|
| RTX 5090 | 32 GB | Fits | 144 to 192 |
| RTX 5080 | 16 GB | Fits | 77 to 103 |
| RTX 5070 Ti | 16 GB | Fits | 72 to 96 |
| RTX 5070 | 12 GB | Fits | 54 to 72 |
| RTX 5060 Ti 16GB | 16 GB | Fits | 36 to 48 |
| RTX 5060 Ti 8GB | 8 GB | No | – |
| RTX 5060 | 8 GB | No | – |
| RTX 5050 | 8 GB | No | – |
| RTX 4090 | 24 GB | Fits | 81 to 108 |
| RTX 4080 Super | 16 GB | Fits | 59 to 79 |
| RTX 4080 | 16 GB | Fits | 58 to 77 |
| RTX 4070 Ti Super | 16 GB | Fits | 54 to 72 |
| RTX 4070 Super | 12 GB | Fits | 41 to 54 |
| RTX 4070 | 12 GB | Fits | 41 to 54 |
| RTX 4060 Ti 16GB | 16 GB | Fits | 23 to 31 |
| RTX 4060 Ti 8GB | 8 GB | No | – |
| RTX 4060 | 8 GB | No | – |
| RTX 3090 | 24 GB | Fits | 75 to 100 |
| RTX 3080 10GB | 10 GB | Tight | 61 to 81 |
| RTX 3070 | 8 GB | No | – |
| RTX 3060 Ti | 8 GB | No | – |
| RTX 3060 12GB | 12 GB | Fits | 29 to 39 |
| RTX 3050 8GB | 8 GB | No | – |
| RX 7900 XTX | 24 GB | Fits | 77 to 103 |
| RX 7900 XT | 20 GB | Fits | 64 to 86 |
| RX 7800 XT | 16 GB | Fits | 50 to 67 |
| RX 7600 XT | 16 GB | Fits | 23 to 31 |
| RX 7600 | 8 GB | No | – |
| RX 9070 XT | 16 GB | Fits | 51 to 69 |
| RX 9060 XT 16GB | 16 GB | Fits | 26 to 34 |
Estimates: weights = parameters × bits per weight; KV cache from the model’s published architecture; about 1 GB of runtime overhead. “Fits” means at most 92 percent of VRAM, “Tight” is 92 to 100 percent. Speed is memory bandwidth divided by model size at 60 to 80 percent efficiency. These are planning estimates, not our own benchmarks.
Our pick
The smallest cards that fit it comfortably are the RTX 5070, RTX 4070 Super, RTX 4070, RTX 3060 12GB (12 GB). If you want longer context or a higher-quality quantization, step up to the next memory size. For other models, see the best AI models for 8 GB of VRAM and Q4 vs Q8.
Frequently asked questions
How much VRAM does Gemma 3 12B Instruct need?
About 9.3 GB at Q4_K_M with an 8K context; about 14.8 GB at Q8_0.
Can I run Gemma 3 12B Instruct on a CPU or with system RAM?
Yes, layers can be offloaded to system RAM, but generation gets much slower than keeping everything in VRAM.
Does longer context need more VRAM?
Yes. At 32K context the KV cache alone is about 2.5 GB for this model.
Sources and method
- Model architecture: Gemma 3 12B Instruct config.json on Hugging Face.
- GPU specs: see our calculator data, taken from manufacturer line cards and published reviews.
- VRAM and speed figures are estimates, not our own benchmarks.
Last updated 10 Nov 2026.