24 GB at 1008 GB/s → 32 GB at 1792 GB/s

GeForce RTX 5090 newly fits Qwen2.5 32B Instruct, Qwen3 32B, and Qwen3.6 35B-A3B at the recommended quant and 8K context, which GeForce RTX 4090 does not.

Hands installing a graphics card into a PC motherboard slot

GeForce RTX 4090 and GeForce RTX 5090, same 8K context and recommended quant, 32 GB system RAM.

ModelGeForce RTX 4090GeForce RTX 5090Speed beforeSpeed after
Llama 3.2 1B InstructFitsFits515–998 tok/s916–1775 tok/s
Llama 3.2 3B InstructFitsFits187–399 tok/s333–710 tok/s
Llama 3.1 8B InstructFitsFits92–164 tok/s164–291 tok/s
Llama 3.3 70B InstructOffloadOffload1 tok/s2–3 tok/s
Qwen2.5 7B InstructFitsFits108–172 tok/s191–306 tok/s
Qwen2.5 14B InstructFitsFits52–90 tok/s93–159 tok/s
Qwen2.5 32B InstructTightFits25–41 tok/s45–72 tok/s
Qwen2.5 72B InstructNoOffload—2 tok/s
Qwen3 8BFitsFits89–160 tok/s158–285 tok/s
Qwen3 14BFitsFits54–90 tok/s95–159 tok/s
Qwen3 32BTightFits25–41 tok/s45–73 tok/s
Qwen3 30B-A3BFitsFits195–397 tok/s347–706 tok/s
Qwen3.5 9BFitsFits92–142 tok/s164–252 tok/s
Qwen3.8 27BFitsFits32–49 tok/s57–87 tok/s
Qwen3.6 35B-A3BTightFits233–376 tok/s414–668 tok/s
Gemma 3 4B InstructFitsFits199–324 tok/s355–576 tok/s
Gemma 3 12B InstructFitsFits68–110 tok/s121–196 tok/s
Gemma 3 27B InstructFitsFits31–49 tok/s56–87 tok/s
Gemma 4 E2BFitsFits171–260 tok/s304–461 tok/s
Gemma 4 E4BFitsFits106–162 tok/s188–288 tok/s
Gemma 4 12BFitsFits73–113 tok/s130–201 tok/s
Gemma 4 26B-A4BFitsFits210–357 tok/s374–635 tok/s
Gemma 4 31BFitsFits28–44 tok/s50–78 tok/s
gpt-oss-20bFitsFits211–333 tok/s375–592 tok/s
gpt-oss-120bNoNo——
Phi-4FitsFits52–89 tok/s92–158 tok/s
Phi-4-mini InstructFitsFits155–324 tok/s276–575 tok/s
Mistral Nemo 12B InstructFitsFits63–108 tok/s112–192 tok/s
Mistral Small 3.2 24B InstructFitsFits35–56 tok/s63–100 tok/s
Devstral Small 2 24B InstructFitsFits35–56 tok/s63–100 tok/s
DeepSeek-R1 Distill Qwen 14BFitsFits52–90 tok/s93–159 tok/s
DeepSeek-R1 Distill Qwen 32BTightFits25–41 tok/s45–72 tok/s
SmolLM3 3BFitsFits220–421 tok/s391–748 tok/s

FAQ

What counts as an upgrade here?

A model newly reaching Fits, or a higher tokens-per-second range when VRAM does not change. Gaming FPS is a different question.

Are the two cards measured the same way?

Both use 8,192 tokens, the recommended quant, and 32 GB of assumed system RAM. Bandwidth is 1008 GB/s versus 1792 GB/s.

Should I buy this for local AI?

Buy VRAM first when a model you want is in Offload or No. Buy bandwidth when the model already fits and you want more tokens per second.