Local models for the rig you already built

An RTX 4060 fits Qwen3 8B at Q4_K_M, and an RTX 4070 barely fits Qwen3 14B, both at 8K context.

Your gaming PC can already run AI. The number below is the whole answer: weights, KV cache, overhead, then a yes, a tight, an offload, or a no.

Gaming PC with RGB fans and liquid cooling beside a monitor showing an AI chat

Calculator

RTX 4070 · Qwen3 8B · Q4_K_M · 8K

Yes. A GeForce RTX 4070 runs Qwen3 8B at Q4_K_M with room left at 8,192 tokens, roughly 44–80 tokens per second.

Verdict
Fits
Total
6.62 GiB
Weights
4.68 GiB
KV cache
1.13 GiB
Speed
44–80 tok/s
Ollama
qwen3:8b

What an RTX 4070 runs

Recommended quant, 8,192 tokens, 32 GB system RAM. The full list is on the RTX 4070 page.

ModelQuantWeightsTotalVerdictSpeedOllama
Llama 3.1 8B InstructQ4_K_M4.58 GiB6.39 GiBFits46–82 tok/sollama run llama3.1:8b
Qwen2.5 7B InstructQ4_K_M4.36 GiB5.56 GiBFits54–86 tok/sollama run qwen2.5:7b
Llama 3.2 3B InstructQ4_K_M1.88 GiB3.42 GiBFits94–200 tok/sollama run llama3.2:3b
Llama 3.2 1B InstructQ4_K_M0.75 GiB1.58 GiBFits258–499 tok/sollama run llama3.2:1b
Qwen2.5 14B InstructQ4_K_M8.37 GiB10.9 GiBTight26–45 tok/sollama run qwen2.5:14b
Qwen2.5 32B InstructQ4_K_M18.5 GiB22.0 GiBOffload1–2 tok/sollama run qwen2.5:32b
Qwen2.5 72B InstructQ4_K_M44.2 GiB49.5 GiBNo—ollama run qwen2.5:72b
Llama 3.3 70B InstructQ4_K_M39.6 GiB44.7 GiBNo—ollama run llama3.3:70b

Find your setup

Compact mini PC and a laptop on a desk, both running a local AI chatMacs and laptopsHands installing a graphics card into a PCGPU upgradesGaming PC beside a monitor running an AI chatBest model for your VRAM

Start with a machine

GeForce RTX 3060 12GB12 GB GDDR · 360 GB/sGeForce RTX 407012 GB GDDR · 504 GB/sGeForce RTX 409024 GB GDDR · 1008 GB/sGeForce RTX 509032 GB GDDR · 1792 GB/sSteam Deck OLED16 GB LPDDR5-6400 · 102.4 GB/sApple M4, 16 GB16 GB unified · 120 GB/s

Buying questions

Same formula, aimed at the card you are about to buy. VRAM tiers assume 32 GB of system RAM.

Best model for 8 GBLargest model that fits at 8KBest model for 12 GBLargest model that fits at 8KBest model for 16 GBLargest model that fits at 8KBest model for 24 GBLargest model that fits at 8KBest model for 32 GBLargest model that fits at 8K4070 to 5070Same 12 GB, more bandwidth3060 12 GB to 5070Same VRAM, newer card4070 Ti Super to 508016 GB to 16 GB4090 to 509024 GB to 32 GB7800 XT to 9070 XT16 GB AMD

Models in the catalog

ModelParametersQuantFileContext
Llama 3.2 1B Instruct1.2BQ4_K_M0.75 GiB131,072
Llama 3.2 3B Instruct3.2BQ4_K_M1.88 GiB131,072
Llama 3.1 8B Instruct8.0BQ4_K_M4.58 GiB131,072
Llama 3.3 70B Instruct71BQ4_K_M39.6 GiB131,072
Qwen2.5 7B Instruct7.6BQ4_K_M4.36 GiB32,768
Qwen2.5 14B Instruct15BQ4_K_M8.37 GiB32,768
Qwen2.5 32B Instruct33BQ4_K_M18.5 GiB32,768
Qwen2.5 72B Instruct73BQ4_K_M44.2 GiB32,768
Qwen3 8B8.2BQ4_K_M4.68 GiB40,960
Qwen3 14B15BQ4_K_M8.38 GiB40,960
Qwen3 32B33BQ4_K_M18.4 GiB40,960
Qwen3 30B-A3B31BQ4_K_M17.3 GiB40,960
Qwen3.5 9B9.0BQ4_K_M5.29 GiB262,144
Qwen3.8 27B27BUD-Q4_K_M15.3 GiB262,144
Qwen3.6 35B-A3B35BUD-Q4_K_M20.6 GiB262,144
Gemma 3 4B Instruct3.9BQ4_K_M2.32 GiB131,072
Gemma 3 12B Instruct12BQ4_K_M6.80 GiB131,072
Gemma 3 27B Instruct27BQ4_K_M15.4 GiB131,072
Gemma 4 E2B4.6BQ4_K_M2.89 GiB131,072
Gemma 4 E4B7.5BQ4_K_M4.64 GiB131,072
Gemma 4 12B12BQ4_K_M6.63 GiB262,144
Gemma 4 26B-A4B25BUD-Q4_K_M15.8 GiB262,144
Gemma 4 31B31BQ4_K_M17.1 GiB262,144
gpt-oss-20b21BMXFP411.3 GiB131,072
gpt-oss-120b117BMXFP459.0 GiB131,072
Phi-415BQ4_K_M8.43 GiB16,384
Phi-4-mini Instruct3.8BQ4_K_M2.32 GiB131,072
Mistral Nemo 12B Instruct12BQ4_K_M6.96 GiB131,072
Mistral Small 3.2 24B Instruct24BQ4_K_M13.3 GiB131,072
Devstral Small 2 24B Instruct24BQ4_K_M13.3 GiB393,216
DeepSeek-R1 Distill Qwen 14B15BQ4_K_M8.37 GiB131,072
DeepSeek-R1 Distill Qwen 32B33BQ4_K_M18.5 GiB131,072
SmolLM3 3B3.1BQ4_K_M1.78 GiB65,536

FAQ

What is RigRunsAI?

A static fit guide for open local models on the GPU, handheld, or Mac you already have. Each answer is weights plus KV cache plus overhead, from published files, not a vibe check.

Does the calculator work without JavaScript?

The verdict, tables, and FAQs are HTML. The manual VRAM, RAM, and Mac form needs JavaScript. Without it, open a GPU or Mac page and read the model table.

Are these affiliate links?

No. There are no shopping links yet. The affiliate disclosure page says so.

Can I reuse the data?

Yes. /data/models.json and /data/hardware.json are CC-BY 4.0. Cite RigRunsAI and keep the source URLs with the specs.