The Radeon RX 7900 XTX is the main AMD option for local AI because it offers 24 GB of GDDR6 and 960 GB/s of bandwidth. On paper it fits the same models as an RTX 4090. The difference is software: AMD GPUs are supported by llama.cpp through its HIP and Vulkan backends, but setup and speed can vary more than on NVIDIA.
Check your own setup with the Can My GPU Run This AI Model? calculator. The numbers below come from the same data.
RX 7900 XTX specs that matter for local AI
- Memory: 24 GB GDDR6 on a 384-bit bus
- Memory bandwidth: 960 GB/s
- Board power: 355 W
VRAM decides which models fit. Bandwidth decides how fast they generate text, because every new token means reading the whole model from memory.
Which AI models fit on the RX 7900 XTX?
The table assumes Q4_K_M quantization (about 4.9 bits per weight) and an 8K context. Totals include the KV cache and about 1 GB of runtime overhead. Speed is an estimate: memory bandwidth divided by model size, at 60 to 80 percent efficiency, which matches published llama.cpp results on several RTX cards.
| Model | Weights | KV cache (8K) | Total (est.) | On 24 GB | Speed (est.) |
|---|---|---|---|---|---|
| Qwen3 4B | 2.5 GB | 1.2 GB | 4.7 GB | Fits | 234 to 312 tok/s |
| Llama 3.1 8B Instruct | 4.9 GB | 1.1 GB | 7.0 GB | Fits | 117 to 156 tok/s |
| Qwen3 8B | 5.0 GB | 1.2 GB | 7.2 GB | Fits | 115 to 153 tok/s |
| Gemma 3 12B Instruct | 7.5 GB | 0.9 GB | 9.3 GB | Fits | 77 to 103 tok/s |
| Qwen3 14B | 9.0 GB | 1.3 GB | 11.4 GB | Fits | 64 to 85 tok/s |
| OpenAI gpt-oss-20b | 12.8 GB | 0.2 GB | 14.0 GB | Fits | Varies (MoE) |
| Mistral Small 3.2 24B | 14.7 GB | 1.3 GB | 17.0 GB | Fits | 39 to 52 tok/s |
| Qwen3.6 27B | 17.0 GB | 0.5 GB | 18.6 GB | Fits | 34 to 45 tok/s |
| Gemma 3 27B Instruct | 16.8 GB | 1.1 GB | 18.9 GB | Fits | 34 to 46 tok/s |
| Qwen3 32B | 20.1 GB | 2.1 GB | 23.2 GB | Tight | 29 to 38 tok/s |
| Qwen3.6 35B-A3B | 22.0 GB | 0.2 GB | 23.2 GB | Tight | Varies (MoE) |
“Fits” means the estimate is at most 92 percent of VRAM. “Tight” means it is between 92 and 100 percent, so close other GPU apps. Mixture-of-Experts models (gpt-oss-20b and Qwen3.6 35B-A3B) need their full weights in memory, but they only use part of them per token, so speed is not estimated.
Best models for the RX 7900 XTX
- Mistral Small 3.2 24B (Q4_K_M): about 17 GB at 8K.
- Gemma 3 27B and Qwen3.6 27B: about 19 GB at Q4.
- Qwen3 32B (Q4_K_M): about 23 GB, a tight fit.
- gpt-oss-20b and 8B to 14B models: comfortable.
How context length changes things
The KV cache grows with every token of context, so a model that fits at 4K can fail at 32K.
| Model (Q4_K_M) | Context | KV cache | Total (est.) | On 24 GB |
|---|---|---|---|---|
| Llama 3.1 8B Instruct | 4K | 0.54 GB | 6.5 GB | Fits |
| Llama 3.1 8B Instruct | 8K | 1.07 GB | 7.0 GB | Fits |
| Llama 3.1 8B Instruct | 16K | 2.15 GB | 8.1 GB | Fits |
| Llama 3.1 8B Instruct | 32K | 4.29 GB | 10.2 GB | Fits |
| Qwen3 14B | 4K | 0.67 GB | 10.7 GB | Fits |
| Qwen3 14B | 8K | 1.34 GB | 11.4 GB | Fits |
| Qwen3 14B | 16K | 2.68 GB | 12.7 GB | Fits |
| Qwen3 14B | 32K | 5.37 GB | 15.4 GB | Fits |
A Q8 KV cache (an option in Ollama and llama.cpp) roughly halves the cache cost if you need more context.
What does not fit
70B-class models do not fit on one 24 GB card at Q4.
A capable 24 GB card if you accept AMD software. If you want the least friction, an NVIDIA card with the same memory is simpler. If you are comfortable with llama.cpp and the Vulkan or HIP builds, the 7900 XTX gives the same model capacity.
Verdict
What speed to expect
At 960 GB/s the memory limit is the same as the RTX 3090 or slightly higher, so the theoretical range for an 8B Q4 model is about 117 to 156 tokens per second. We did not find a reliable measured result for this card, and real speed on AMD depends more on the backend you use, so treat these as upper-end estimates and expect to land lower.
Frequently asked questions
Does AMD work for local AI?
Yes, llama.cpp supports AMD GPUs through HIP and Vulkan backends, though setup is less uniform than on NVIDIA.
Is the 7900 XTX as fast as an RTX 4090?
Its memory bandwidth is close (960 vs 1,008 GB/s), but real speed depends on software, and we have no reliable measured comparison.
Can it run a 27B model?
Yes at Q4, about 19 GB at 8K context.
Sources and method
- RX 7900 XTX specs: Tom’s Hardware, AMD RDNA 3 / RX 7000 everything we know (24 GB, 960 GB/s, 355 W).
- Linear-attention and sliding-window layers are handled with reduced KV cache; small fixed states in some newer models are not counted.
- Model architecture: each model’s published
config.jsonon Hugging Face (Llama 3.1 8B and Gemma 3 are read from the unsloth mirrors because the original repositories require a login). - VRAM and speed figures are estimates from our calculator, not our own benchmarks. Last updated 3 Nov 2026.
