DeepSeek · 33B · Q4_K_M · 18.5 GiB
DeepSeek-R1 Distill Qwen 32B at Q4_K_M needs about 22.0 GiB at 8K context, and the smallest listed machine that fits it is GeForce RTX 5090.
Reasoning distill on the Qwen2.5 32B architecture.
ollama run deepseek-r1:32b
Weights: DeepSeek-R1-Distill-Qwen-32B-Q4_K_M.gguf. Architecture: config source. Hugging Face repo created 2025-01-20. License field: mit.
Quants with a measured file size
| Quant | File | Size |
|---|---|---|
| Q4_K_M · recommended | DeepSeek-R1-Distill-Qwen-32B-Q4_K_M.gguf | 18.5 GiB |
| Q5_K_M | DeepSeek-R1-Distill-Qwen-32B-Q5_K_M.gguf | 21.7 GiB |
| Q6_K | DeepSeek-R1-Distill-Qwen-32B-Q6_K.gguf | 25.0 GiB |
| Q8_0 | DeepSeek-R1-Distill-Qwen-32B-Q8_0.gguf | 32.4 GiB |
Machines
| Machine | Memory | Total | Verdict | Speed |
|---|---|---|---|---|
| Apple M1, 8 GB | 8 GB unified | 22.0 GiB | No | — |
| Apple M2, 8 GB | 8 GB unified | 22.0 GiB | No | — |
| Apple M3, 8 GB | 8 GB unified | 22.0 GiB | No | — |
| GeForce RTX 3060 Ti | 8 GB GDDR | 22.0 GiB | Offload | 1 tok/s |
| GeForce RTX 3070 | 8 GB GDDR | 22.0 GiB | Offload | 1 tok/s |
| GeForce RTX 4060 | 8 GB GDDR | 22.0 GiB | Offload | 1 tok/s |
| GeForce RTX 4060 Ti 8GB | 8 GB GDDR | 22.0 GiB | Offload | 1 tok/s |
| GeForce RTX 5060 | 8 GB GDDR | 22.0 GiB | Offload | 2–3 tok/s |
| GeForce RTX 5060 Ti 8GB | 8 GB GDDR | 22.0 GiB | Offload | 2–3 tok/s |
| Radeon RX 6600 | 8 GB GDDR6 | 22.0 GiB | Offload | 1 tok/s |
| Radeon RX 7600 | 8 GB GDDR6 | 22.0 GiB | Offload | 1 tok/s |
| GeForce RTX 3080 10GB | 10 GB GDDR | 22.0 GiB | Offload | 1–2 tok/s |
| GeForce RTX 3060 12GB | 12 GB GDDR | 22.0 GiB | Offload | 1–2 tok/s |
| GeForce RTX 4070 | 12 GB GDDR | 22.0 GiB | Offload | 1–2 tok/s |
| GeForce RTX 4070 Super | 12 GB GDDR | 22.0 GiB | Offload | 1–2 tok/s |
| GeForce RTX 4070 Ti | 12 GB GDDR | 22.0 GiB | Offload | 1–2 tok/s |
| GeForce RTX 5070 | 12 GB GDDR | 22.0 GiB | Offload | 2–4 tok/s |
| Intel Arc B580 | 12 GB GDDR6 | 22.0 GiB | Offload | 1–2 tok/s |
| Radeon RX 6700 XT | 12 GB GDDR6 | 22.0 GiB | Offload | 1–2 tok/s |
| Radeon RX 7700 XT | 12 GB GDDR6 | 22.0 GiB | Offload | 1–2 tok/s |
| Apple M1 Pro, 16 GB | 16 GB unified | 22.0 GiB | No | — |
| Apple M1, 16 GB | 16 GB unified | 22.0 GiB | No | — |
| Apple M2 Pro, 16 GB | 16 GB unified | 22.0 GiB | No | — |
| Apple M2, 16 GB | 16 GB unified | 22.0 GiB | No | — |
| Apple M3, 16 GB | 16 GB unified | 22.0 GiB | No | — |
| Apple M4, 16 GB | 16 GB unified | 22.0 GiB | No | — |
| GeForce RTX 4060 Ti 16GB | 16 GB GDDR | 22.0 GiB | Offload | 2–3 tok/s |
| GeForce RTX 4070 Ti Super | 16 GB GDDR | 22.0 GiB | Offload | 2–3 tok/s |
| GeForce RTX 4080 | 16 GB GDDR | 22.0 GiB | Offload | 2–3 tok/s |
| GeForce RTX 4080 Super | 16 GB GDDR | 22.0 GiB | Offload | 2–3 tok/s |
| GeForce RTX 5060 Ti 16GB | 16 GB GDDR | 22.0 GiB | Offload | 4–5 tok/s |
| GeForce RTX 5070 Ti | 16 GB GDDR | 22.0 GiB | Offload | 4–6 tok/s |
| GeForce RTX 5080 | 16 GB GDDR | 22.0 GiB | Offload | 4–6 tok/s |
| Intel Arc A770 16GB | 16 GB GDDR6 | 22.0 GiB | Offload | 2–3 tok/s |
| Legion Go | 16 GB LPDDR5X-7500 | 22.0 GiB | No | — |
| Radeon RX 6800 XT | 16 GB GDDR6 | 22.0 GiB | Offload | 2–3 tok/s |
| Radeon RX 7800 XT | 16 GB GDDR6 | 22.0 GiB | Offload | 2–3 tok/s |
| Radeon RX 9060 XT 16GB | 16 GB GDDR6 | 22.0 GiB | Offload | 3–5 tok/s |
| Radeon RX 9070 XT | 16 GB GDDR6 | 22.0 GiB | Offload | 4–6 tok/s |
| ROG Ally | 16 GB LPDDR5-6400 | 22.0 GiB | No | — |
| Steam Deck LCD | 16 GB LPDDR5-5500 | 22.0 GiB | No | — |
| Steam Deck OLED | 16 GB LPDDR5-6400 | 22.0 GiB | No | — |
| Apple M3 Pro, 18 GB | 18 GB unified | 22.0 GiB | No | — |
| Radeon RX 7900 XT | 20 GB GDDR6 | 22.0 GiB | Offload | 7–11 tok/s |
| Apple M2, 24 GB | 24 GB unified | 22.0 GiB | No | — |
| Apple M3, 24 GB | 24 GB unified | 22.0 GiB | No | — |
| Apple M4 Pro, 24 GB | 24 GB unified | 22.0 GiB | No | — |
| Apple M4, 24 GB | 24 GB unified | 22.0 GiB | No | — |
| GeForce RTX 3090 | 24 GB GDDR | 22.0 GiB | Tight | 23–38 tok/s |
| GeForce RTX 4090 | 24 GB GDDR | 22.0 GiB | Tight | 25–41 tok/s |
| Radeon RX 7900 XTX | 24 GB GDDR6 | 22.0 GiB | Tight | 24–39 tok/s |
| ROG Ally X | 24 GB LPDDR5X-7500 | 22.0 GiB | No | — |
| Apple M1 Max, 32 GB | 32 GB unified | 22.0 GiB | Tight | 10–16 tok/s |
| Apple M1 Pro, 32 GB | 32 GB unified | 22.0 GiB | Tight | 5–8 tok/s |
| Apple M2 Max, 32 GB | 32 GB unified | 22.0 GiB | Tight | 10–16 tok/s |
| Apple M2 Pro, 32 GB | 32 GB unified | 22.0 GiB | Tight | 5–8 tok/s |
| Apple M4, 32 GB | 32 GB unified | 22.0 GiB | Tight | 3–5 tok/s |
| GeForce RTX 5090 | 32 GB GDDR | 22.0 GiB | Fits | 45–72 tok/s |
| Apple M3 Pro, 36 GB | 36 GB unified | 22.0 GiB | Fits | 4–6 tok/s |
| Apple M4 Max 14-core, 36 GB | 36 GB unified | 22.0 GiB | Fits | 10–17 tok/s |
| Apple M3 Max 16-core, 48 GB | 48 GB unified | 22.0 GiB | Fits | 10–16 tok/s |
| Apple M4 Max 16-core, 48 GB | 48 GB unified | 22.0 GiB | Fits | 14–22 tok/s |
| Apple M4 Pro, 48 GB | 48 GB unified | 22.0 GiB | Fits | 7–11 tok/s |
| Apple M1 Max, 64 GB | 64 GB unified | 22.0 GiB | Fits | 10–16 tok/s |
| Apple M2 Max, 64 GB | 64 GB unified | 22.0 GiB | Fits | 10–16 tok/s |
| Apple M3 Max 16-core, 64 GB | 64 GB unified | 22.0 GiB | Fits | 10–16 tok/s |
| Apple M4 Max 16-core, 64 GB | 64 GB unified | 22.0 GiB | Fits | 14–22 tok/s |
| Apple M4 Pro, 64 GB | 64 GB unified | 22.0 GiB | Fits | 7–11 tok/s |
| Apple M2 Max, 96 GB | 96 GB unified | 22.0 GiB | Fits | 10–16 tok/s |
| Apple M3 Max 16-core, 128 GB | 128 GB unified | 22.0 GiB | Fits | 10–16 tok/s |
| Apple M4 Max 16-core, 128 GB | 128 GB unified | 22.0 GiB | Fits | 14–22 tok/s |
FAQ
Why Q4_K_M?
Q4_K_M is the quant this page recommends. The file is DeepSeek-R1-Distill-Qwen-32B-Q4_K_M.gguf at 18.5 GiB. Other single-file quants are in the table when a real size was available.
Where do the architecture numbers come from?
Official config.json. Source: https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B/raw/main/config.json
Does this include a long context?
The fit uses 8,192 tokens, or less when that is all the fast memory can hold. The model file allows up to 131,072 tokens, and KV cache grows as you use more of that.
What is the Ollama command?
ollama run deepseek-r1:32b. The library tag is Q4-class or the vendor quant. Compare it with the GGUF linked on this page before you treat the sizes as identical.