Can I run Gemma 4 26B-A4B on an RTX 4080 (16 GB)?
Yes. UD-IQ4_XS (12.7 GB) fits fully on the GPU, with room for 64K tokens of context.
Every quant on RTX 4080
| Quant | Size | VRAM at 8K context | On this card |
|---|---|---|---|
| Q8_0 | 25.0 GB | 15.4 GB | experts of 16/30 layers in RAM (12.8 GB) |
| UD-Q6_K | 21.6 GB | 15.0 GB | experts of 14/30 layers in RAM (9.7 GB) |
| UD-Q5_K_M | 19.7 GB | 15.1 GB | experts of 12/30 layers in RAM (7.6 GB) |
| UD-Q4_K_M | 15.8 GB | 15.3 GB | experts of 6/30 layers in RAM (3.4 GB) |
| MXFP4_MOE | 15.4 GB | 15.4 GB | experts of 5/30 layers in RAM (2.9 GB) |
| UD-IQ4_XS | 12.7 GB | 14.4 GB | fits, up to 64K context |
| UD-Q3_K_M | 11.9 GB | 13.6 GB | fits, up to 64K context |
| UD-Q2_K_XL | 9.8 GB | 11.5 GB | fits, up to 128K context |
Try it
- Install LlamaForge (one line, no admin). It runs this same math on the file before loading, then measures the real footprint and tok/s on your machine.
- Already running it? Open UD-IQ4_XS in LlamaForge's Discover tab (it shows the quant; the download still takes your click).
- The files: huggingface.co/unsloth/gemma-4-26B-A4B-it-GGUF
How this is worked out
This is a prediction, not a benchmark. The quant sizes and the model's attention layout come from the real GGUF headers in unsloth/gemma-4-26B-A4B-it-GGUF (read 2026-11-01). VRAM counts every layer on the GPU, an f16 KV cache, about 0.7 GB of CUDA overhead, and about 0.5 GB already taken by the desktop. Speed isn't predicted here: it depends on far more than memory.
The same model on: RTX 3060, RTX 3060 Ti, RTX 3070, RTX 3080, RTX 3090, RTX 4060, RTX 4060 Ti 16GB, RTX 4070, RTX 4070 Super, RTX 4070 Ti Super, RTX 4090, RTX 5060 Ti 16GB, RTX 5070, RTX 5070 Ti, RTX 5080, RTX 5090, Radeon RX 7900 XTX, Radeon RX 9070 XT. Every model and card.