The RTX 5080 combines 16 GB of GDDR7 with 960 GB/s of bandwidth, so it generates text roughly twice as fast as a 16 GB RTX 5060 Ti. The catch is the memory size: 16 GB is the limit that decides which models you can run, and it is the same as much cheaper cards.
Check your own setup with the Can My GPU Run This AI Model? calculator. The numbers below come from the same data.
RTX 5080 specs that matter for local AI
- Memory: 16 GB GDDR7 on a 256-bit bus
- Memory bandwidth: 960 GB/s
- Board power: 360 W
VRAM decides which models fit. Bandwidth decides how fast they generate text, because every new token means reading the whole model from memory.
Which AI models fit on the RTX 5080?
The table assumes Q4_K_M quantization (about 4.9 bits per weight) and an 8K context. Totals include the KV cache and about 1 GB of runtime overhead. Speed is an estimate: memory bandwidth divided by model size, at 60 to 80 percent efficiency, which matches published llama.cpp results on several RTX cards.
| Model | Weights | KV cache (8K) | Total (est.) | On 16 GB | Speed (est.) |
|---|---|---|---|---|---|
| Qwen3 4B | 2.5 GB | 1.2 GB | 4.7 GB | Fits | 234 to 312 tok/s |
| Llama 3.1 8B Instruct | 4.9 GB | 1.1 GB | 7.0 GB | Fits | 117 to 156 tok/s |
| Qwen3 8B | 5.0 GB | 1.2 GB | 7.2 GB | Fits | 115 to 153 tok/s |
| Gemma 3 12B Instruct | 7.5 GB | 0.9 GB | 9.3 GB | Fits | 77 to 103 tok/s |
| Qwen3 14B | 9.0 GB | 1.3 GB | 11.4 GB | Fits | 64 to 85 tok/s |
| OpenAI gpt-oss-20b | 12.8 GB | 0.2 GB | 14.0 GB | Fits | Varies (MoE) |
| Mistral Small 3.2 24B | 14.7 GB | 1.3 GB | 17.0 GB | No | – |
| Qwen3.6 27B | 17.0 GB | 0.5 GB | 18.6 GB | No | – |
| Gemma 3 27B Instruct | 16.8 GB | 1.1 GB | 18.9 GB | No | – |
| Qwen3 32B | 20.1 GB | 2.1 GB | 23.2 GB | No | – |
| Qwen3.6 35B-A3B | 22.0 GB | 0.2 GB | 23.2 GB | No | – |
“Fits” means the estimate is at most 92 percent of VRAM. “Tight” means it is between 92 and 100 percent, so close other GPU apps. Mixture-of-Experts models (gpt-oss-20b and Qwen3.6 35B-A3B) need their full weights in memory, but they only use part of them per token, so speed is not estimated.
Best models for the RTX 5080
- Qwen3 14B (Q4_K_M or Q5_K_M): about 11.4 GB at Q4 with 8K context, with room for a longer context.
- gpt-oss-20b: about 14 GB, with a very small KV cache.
- Gemma 3 12B: about 9.3 GB at Q4, so Q8_0 also fits.
- 8B models at Q8_0: about 10.6 GB at 8K for close-to-original quality, and very fast.
How context length changes things
The KV cache grows with every token of context, so a model that fits at 4K can fail at 32K.
| Model (Q4_K_M) | Context | KV cache | Total (est.) | On 16 GB |
|---|---|---|---|---|
| Llama 3.1 8B Instruct | 4K | 0.54 GB | 6.5 GB | Fits |
| Llama 3.1 8B Instruct | 8K | 1.07 GB | 7.0 GB | Fits |
| Llama 3.1 8B Instruct | 16K | 2.15 GB | 8.1 GB | Fits |
| Llama 3.1 8B Instruct | 32K | 4.29 GB | 10.2 GB | Fits |
| Qwen3 14B | 4K | 0.67 GB | 10.7 GB | Fits |
| Qwen3 14B | 8K | 1.34 GB | 11.4 GB | Fits |
| Qwen3 14B | 16K | 2.68 GB | 12.7 GB | Fits |
| Qwen3 14B | 32K | 5.37 GB | 15.4 GB | Tight |
A Q8 KV cache (an option in Ollama and llama.cpp) roughly halves the cache cost if you need more context.
What does not fit
Mistral Small 3.2 24B (about 17 GB), 27B models (about 19 GB) and Qwen3 32B (about 23 GB) need more than 16 GB at Q4. For them, you need a 24 GB or 32 GB card.
Fast, but memory-limited. For 8B to 20B models it is excellent. If your plan includes 24B to 32B models, the extra memory of a 24 GB card will serve you better than the extra speed of this one.
Verdict
What speed to expect
At 960 GB/s, an 8B Q4 model has a ceiling near 195 tokens per second, so expect about 117 to 156 tokens per second, and 64 to 85 for a 14B model. We did not find a reliable measured result for this card, so these are estimates. For reference, the RTX 4080 (717 GB/s) was measured at 108 tokens per second on Llama 3 8B Q4_K_M in a community dataset from May 2024.
Frequently asked questions
Can the RTX 5080 run a 27B model?
Not at Q4 entirely on the GPU. It needs about 19 GB at 8K context.
Is the RTX 5080 faster than the RTX 4080 for AI?
Its bandwidth is higher (960 vs 717 GB/s), so token generation should be faster for models that fit in memory.
Can it run gpt-oss-20b?
Yes, with about 14 GB used at 8K context.
Sources and method
- RTX 5080 specs: PNY’s NVIDIA GeForce linecard, January 2026 (16 GB, GDDR7, 960 GB/s, 360 W).
- RTX 4080 reference: XiongjieDai’s GPU-Benchmarks-on-LLM-Inference on GitHub (May 2024 snapshot).
- Linear-attention and sliding-window layers are handled with reduced KV cache; small fixed states in some newer models are not counted.
- Model architecture: each model’s published
config.jsonon Hugging Face (Llama 3.1 8B and Gemma 3 are read from the unsloth mirrors because the original repositories require a login). - VRAM and speed figures are estimates from our calculator, not our own benchmarks. Last updated 3 Nov 2026.
