NVIDIA · 8 GB GDDR · 448 GB/s

Yes. GeForce RTX 5060 Ti 8GB fits Qwen3.5 9B at Q4_K_M, the largest catalog model that clears the fit line at 8K context.

NVIDIA compare table: 5060 Ti is 16 GB or 8 GB GDDR7, 128-bit, 448 GB/s. Source.

Recommended quant, 8,192 tokens.
ModelQuantWeightsTotalVerdictSpeedOllama
Qwen3.5 9BQ4_K_M5.29 GiB6.39 GiBFits41–63 tok/sollama run qwen3.5:9b
Qwen3 8BQ4_K_M4.68 GiB6.62 GiBFits40–71 tok/sollama run qwen3:8b
Llama 3.1 8B InstructQ4_K_M4.58 GiB6.39 GiBFits41–73 tok/sollama run llama3.1:8b
Qwen2.5 7B InstructQ4_K_M4.36 GiB5.56 GiBFits48–77 tok/sollama run qwen2.5:7b
Gemma 4 E4BQ4_K_M4.64 GiB5.66 GiBFits47–72 tok/sollama run gemma4:e4b
Gemma 4 E2BQ4_K_M2.89 GiB3.69 GiBFits76–115 tok/sollama run gemma4:e2b
Gemma 3 4B InstructQ4_K_M2.32 GiB3.24 GiBFits89–144 tok/sollama run gemma3:4b
Phi-4-mini InstructQ4_K_M2.32 GiB4.01 GiBFits69–144 tok/sollama run phi4-mini
Llama 3.2 3B InstructQ4_K_M1.88 GiB3.42 GiBFits83–177 tok/sollama run llama3.2:3b
SmolLM3 3BQ4_K_M1.78 GiB2.99 GiBFits98–187 tok/s—
Llama 3.2 1B InstructQ4_K_M0.75 GiB1.58 GiBFits229–444 tok/sollama run llama3.2:1b
Gemma 4 12BQ4_K_M6.63 GiB7.95 GiBTight32–50 tok/sollama run gemma4:12b
Qwen3.6 35B-A3BUD-Q4_K_M20.6 GiB22.4 GiBOffload18–26 tok/s—
Qwen2.5 32B InstructQ4_K_M18.5 GiB22.0 GiBOffload2–3 tok/sollama run qwen2.5:32b
DeepSeek-R1 Distill Qwen 32BQ4_K_M18.5 GiB22.0 GiBOffload2–3 tok/sollama run deepseek-r1:32b
Qwen3 32BQ4_K_M18.4 GiB22.0 GiBOffload2–3 tok/sollama run qwen3:32b
Gemma 4 31BQ4_K_M17.1 GiB19.9 GiBOffload2–3 tok/sollama run gemma4:31b
Qwen3 30B-A3BQ4_K_M17.3 GiB19.5 GiBOffload19–27 tok/sollama run qwen3:30b-a3b
Qwen3.8 27BUD-Q4_K_M15.3 GiB17.3 GiBOffload3–4 tok/s—
Gemma 3 27B InstructQ4_K_M15.4 GiB17.8 GiBOffload3–4 tok/sollama run gemma3:27b
Gemma 4 26B-A4BUD-Q4_K_M15.8 GiB17.5 GiBOffload19–28 tok/sollama run gemma4:26b-a4b
Mistral Small 3.2 24B InstructQ4_K_M13.3 GiB15.9 GiBOffload3–4 tok/s—
Devstral Small 2 24B InstructQ4_K_M13.3 GiB15.9 GiBOffload3–4 tok/s—
gpt-oss-20bMXFP411.3 GiB12.6 GiBOffload25–37 tok/sollama run gpt-oss:20b
Qwen2.5 14B InstructQ4_K_M8.37 GiB10.9 GiBOffload8–11 tok/sollama run qwen2.5:14b
DeepSeek-R1 Distill Qwen 14BQ4_K_M8.37 GiB10.9 GiBOffload8–11 tok/sollama run deepseek-r1:14b
Qwen3 14BQ4_K_M8.38 GiB10.6 GiBOffload8–12 tok/sollama run qwen3:14b
Phi-4Q4_K_M8.43 GiB11.0 GiBOffload7–11 tok/sollama run phi4:14b
Mistral Nemo 12B InstructQ4_K_M6.96 GiB9.15 GiBOffload15–22 tok/sollama run mistral-nemo:12b
Gemma 3 12B InstructQ4_K_M6.80 GiB8.52 GiBOffload25–37 tok/sollama run gemma3:12b
gpt-oss-120bMXFP459.0 GiB62.8 GiBNo—ollama run gpt-oss:120b
Qwen2.5 72B InstructQ4_K_M44.2 GiB49.5 GiBNo—ollama run qwen2.5:72b
Llama 3.3 70B InstructQ4_K_M39.6 GiB44.7 GiBNo—ollama run llama3.3:70b

FAQ

What does the verdict for GeForce RTX 5060 Ti 8GB mean?

Memory is the GGUF file, plus KV cache at 8,192 tokens, plus a 0.5 GiB engine allowance, then 5% headroom. Fits means that total is at most 85% of fast memory. Tight means it still fits. Offload means it needs system RAM. No means it does not fit in VRAM plus system RAM after an 8 GB Windows reserve. The formula is on the methodology page.

What memory did you count as usable?

Fast memory is the full 8 GB. System RAM is assumed to be 32 GB DDR5-5600, with 8 GB kept for Windows, and you can change that on the homepage calculator.

Why is speed a range?

Decode speed is memory bandwidth divided by the bytes read per token. The high end is 80% of that roofline on the weights alone. The low end is 55% once the KV cache at the working context is included. It is not a benchmark.

How should I run Gemma 4 12B?

The table uses Q4_K_M. Ollama's matching command is `ollama run gemma4:12b`. That blob is the library default and can differ from the cited GGUF by a few hundred MB.