The RTX 4080 Super has 16 GB of GDDR6X and 736 GB/s of bandwidth. It runs 8B to 14B models quickly and can load gpt-oss-20b, but like every 16 GB card it stops short of 24B and larger models at Q4.
Check your own setup with the Can My GPU Run This AI Model? calculator. The numbers below come from the same data.
RTX 4080 Super specs that matter for local AI
- Memory: 16 GB GDDR6X on a 256-bit bus
- Memory bandwidth: 736 GB/s
- Board power: 320 W
- Same family: the original RTX 4080 has 716.8 GB/s, so speeds are nearly identical.
VRAM decides which models fit. Bandwidth decides how fast they generate text, because every new token means reading the whole model from memory.
Which AI models fit on the RTX 4080 Super?
The table assumes Q4_K_M quantization (about 4.9 bits per weight) and an 8K context. Totals include the KV cache and about 1 GB of runtime overhead. Speed is an estimate: memory bandwidth divided by model size, at 60 to 80 percent efficiency, which matches published llama.cpp results on several RTX cards.
| Model | Weights | KV cache (8K) | Total (est.) | On 16 GB | Speed (est.) |
|---|---|---|---|---|---|
| Qwen3 4B | 2.5 GB | 1.2 GB | 4.7 GB | Fits | 179 to 239 tok/s |
| Llama 3.1 8B Instruct | 4.9 GB | 1.1 GB | 7.0 GB | Fits | 90 to 120 tok/s |
| Qwen3 8B | 5.0 GB | 1.2 GB | 7.2 GB | Fits | 88 to 117 tok/s |
| Gemma 3 12B Instruct | 7.5 GB | 0.9 GB | 9.3 GB | Fits | 59 to 79 tok/s |
| Qwen3 14B | 9.0 GB | 1.3 GB | 11.4 GB | Fits | 49 to 65 tok/s |
| OpenAI gpt-oss-20b | 12.8 GB | 0.2 GB | 14.0 GB | Fits | Varies (MoE) |
| Mistral Small 3.2 24B | 14.7 GB | 1.3 GB | 17.0 GB | No | – |
| Qwen3.6 27B | 17.0 GB | 0.5 GB | 18.6 GB | No | – |
| Gemma 3 27B Instruct | 16.8 GB | 1.1 GB | 18.9 GB | No | – |
| Qwen3 32B | 20.1 GB | 2.1 GB | 23.2 GB | No | – |
| Qwen3.6 35B-A3B | 22.0 GB | 0.2 GB | 23.2 GB | No | – |
“Fits” means the estimate is at most 92 percent of VRAM. “Tight” means it is between 92 and 100 percent, so close other GPU apps. Mixture-of-Experts models (gpt-oss-20b and Qwen3.6 35B-A3B) need their full weights in memory, but they only use part of them per token, so speed is not estimated.
Best models for the RTX 4080 Super
- Qwen3 14B (Q4_K_M): about 11.4 GB at 8K, with space for a larger context.
- gpt-oss-20b: about 14 GB total at 8K.
- Llama 3.1 8B and Qwen3 8B: Q8_0 fits (about 10.6 GB at 8K) for higher quality.
- Gemma 3 12B: about 9.3 GB at Q4.
How context length changes things
The KV cache grows with every token of context, so a model that fits at 4K can fail at 32K.
| Model (Q4_K_M) | Context | KV cache | Total (est.) | On 16 GB |
|---|---|---|---|---|
| Llama 3.1 8B Instruct | 4K | 0.54 GB | 6.5 GB | Fits |
| Llama 3.1 8B Instruct | 8K | 1.07 GB | 7.0 GB | Fits |
| Llama 3.1 8B Instruct | 16K | 2.15 GB | 8.1 GB | Fits |
| Llama 3.1 8B Instruct | 32K | 4.29 GB | 10.2 GB | Fits |
| Qwen3 14B | 4K | 0.67 GB | 10.7 GB | Fits |
| Qwen3 14B | 8K | 1.34 GB | 11.4 GB | Fits |
| Qwen3 14B | 16K | 2.68 GB | 12.7 GB | Fits |
| Qwen3 14B | 32K | 5.37 GB | 15.4 GB | Tight |
A Q8 KV cache (an option in Ollama and llama.cpp) roughly halves the cache cost if you need more context.
What does not fit
Mistral Small 3.2 24B (about 17 GB), the 27B models (about 19 GB) and Qwen3 32B (about 23 GB) are too big for 16 GB at Q4.
A fast 16 GB card. It is a strong choice for 8B to 20B models. Buy it for the speed; buy a 24 GB card if you want 24B to 32B models.
Verdict
What speed to expect
At 736 GB/s an 8B Q4 model has a ceiling near 150 tokens per second. A community llama.cpp dataset (May 2024) measured the closely related RTX 4080 at 108 tokens per second on Llama 3 8B Q4_K_M and 40.6 on FP16. Our estimate for the 4080 Super is about 90 to 120 tokens per second for 8B. Crowdsourced LocalScore.ai lists 78.7 for the 4080 Super, but individual results vary a lot with software and settings, so we weigh the larger dataset more.
Frequently asked questions
Can the RTX 4080 Super run Llama 3.1 8B at full precision?
The FP16 weights are about 16 GB, so no. Q8_0 fits and is almost lossless.
Is 16 GB enough for AI?
For 8B to 20B models, yes. For 24B and larger at Q4, no.
RTX 4080 vs 4080 Super for AI?
Both have 16 GB, and bandwidth is 716.8 vs 736 GB/s, so AI speed is nearly the same.
Sources and method
- RTX 4080 Super specs: GamersNexus’ RTX 4080 Super, 4070 Ti Super and 4070 Super specs.
- Measured speeds: XiongjieDai’s GPU-Benchmarks-on-LLM-Inference on GitHub (llama.cpp, May 2024 snapshot).
- Crowdsourced speed: LocalScore.ai Llama 3.1 8B leaderboard.
- Linear-attention and sliding-window layers are handled with reduced KV cache; small fixed states in some newer models are not counted.
- Model architecture: each model’s published
config.jsonon Hugging Face (Llama 3.1 8B and Gemma 3 are read from the unsloth mirrors because the original repositories require a login). - VRAM and speed figures are estimates from our calculator, not our own benchmarks. Last updated 3 Nov 2026.
