Can I run Qwen3.8 Flash Next on an RTX 4090 (24 GB)?
Yes, with some experts in system RAM. UD-IQ4_XS keeps the expert weights of 34 of its 48 layers in RAM (about 66.6 GB of system RAM) and the rest on the GPU. That's the usual way to run a mixture-of-experts model bigger than the card, since only a few experts work on each token.
Every quant on RTX 4090
| Quant | Size | VRAM at 8K context | On this card |
|---|---|---|---|
| Q8_0 | 175.3 GB | 22.8 GB | experts of 42/48 layers in RAM (155.9 GB) |
| UD-IQ4_XS | 87.2 GB | 23.2 GB | experts of 34/48 layers in RAM (66.6 GB) |
| UD-Q3_K_XL | 83.8 GB | 23.2 GB | experts of 33/48 layers in RAM (63.1 GB) |
| UD-Q2_K_XL | 73.5 GB | 23.0 GB | experts of 28/48 layers in RAM (52.3 GB) |
Try it
- Install LlamaForge (one line, no admin). It runs this same math on the file before loading, then measures the real footprint and tok/s on your machine.
- Already running it? Open UD-IQ4_XS in LlamaForge's Discover tab (it shows the quant; the download still takes your click).
- The files: huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF
How this is worked out
This is a prediction, not a benchmark. The quant sizes and the model's attention layout come from the real GGUF headers in unsloth/Qwen3.8-Flash-Next-GGUF (read 2026-11-01). VRAM counts every layer on the GPU, an f16 KV cache, about 0.7 GB of CUDA overhead, and about 0.5 GB already taken by the desktop. Speed isn't predicted here: it depends on far more than memory.
The same model on: RTX 3060, RTX 3060 Ti, RTX 3070, RTX 3080, RTX 3090, RTX 4060, RTX 4060 Ti 16GB, RTX 4070, RTX 4070 Super, RTX 4070 Ti Super, RTX 4080, RTX 5060 Ti 16GB, RTX 5070, RTX 5070 Ti, RTX 5080, RTX 5090, Radeon RX 7900 XTX, Radeon RX 9070 XT. Every model and card.