Hugging Face · 3.1B · Q4_K_M · 1.78 GiB

SmolLM3 3B at Q4_K_M needs about 2.99 GiB at 8K context, and the smallest listed machine that fits it is GeForce RTX 3060 Ti.

Dense 3B model from Hugging Face. All 36 layers are full attention. No Ollama library tag was found for SmolLM3.

Weights: HuggingFaceTB_SmolLM3-3B-Q4_K_M.gguf. Architecture: config source. Hugging Face repo created 2025-07-08. License field: apache-2.0.

Quants with a measured file size

QuantFileSize
Q4_K_M · recommendedHuggingFaceTB_SmolLM3-3B-Q4_K_M.gguf1.78 GiB
Q5_K_MHuggingFaceTB_SmolLM3-3B-Q5_K_M.gguf2.06 GiB
Q6_KHuggingFaceTB_SmolLM3-3B-Q6_K.gguf2.36 GiB
Q8_0HuggingFaceTB_SmolLM3-3B-Q8_0.gguf3.05 GiB

Machines

Same quant and 8,192 tokens on every row.
MachineMemoryTotalVerdictSpeed
Apple M1, 8 GB8 GB unified2.99 GiBFits15–28 tok/s
Apple M2, 8 GB8 GB unified2.99 GiBFits22–42 tok/s
Apple M3, 8 GB8 GB unified2.99 GiBFits22–42 tok/s
GeForce RTX 3060 Ti8 GB GDDR2.99 GiBFits98–187 tok/s
GeForce RTX 30708 GB GDDR2.99 GiBFits98–187 tok/s
GeForce RTX 40608 GB GDDR2.99 GiBFits59–114 tok/s
GeForce RTX 4060 Ti 8GB8 GB GDDR2.99 GiBFits63–120 tok/s
GeForce RTX 50608 GB GDDR2.99 GiBFits98–187 tok/s
GeForce RTX 5060 Ti 8GB8 GB GDDR2.99 GiBFits98–187 tok/s
Radeon RX 66008 GB GDDR62.99 GiBFits49–94 tok/s
Radeon RX 76008 GB GDDR62.99 GiBFits63–120 tok/s
GeForce RTX 3080 10GB10 GB GDDR2.99 GiBFits166–317 tok/s
GeForce RTX 3060 12GB12 GB GDDR2.99 GiBFits79–150 tok/s
GeForce RTX 407012 GB GDDR2.99 GiBFits110–211 tok/s
GeForce RTX 4070 Super12 GB GDDR2.99 GiBFits110–211 tok/s
GeForce RTX 4070 Ti12 GB GDDR2.99 GiBFits110–211 tok/s
GeForce RTX 507012 GB GDDR2.99 GiBFits147–281 tok/s
Intel Arc B58012 GB GDDR62.99 GiBFits100–190 tok/s
Radeon RX 6700 XT12 GB GDDR62.99 GiBFits84–160 tok/s
Radeon RX 7700 XT12 GB GDDR62.99 GiBFits94–180 tok/s
Apple M1 Pro, 16 GB16 GB unified2.99 GiBFits44–84 tok/s
Apple M1, 16 GB16 GB unified2.99 GiBFits15–28 tok/s
Apple M2 Pro, 16 GB16 GB unified2.99 GiBFits44–84 tok/s
Apple M2, 16 GB16 GB unified2.99 GiBFits22–42 tok/s
Apple M3, 16 GB16 GB unified2.99 GiBFits22–42 tok/s
Apple M4, 16 GB16 GB unified2.99 GiBFits26–50 tok/s
GeForce RTX 4060 Ti 16GB16 GB GDDR2.99 GiBFits63–120 tok/s
GeForce RTX 4070 Ti Super16 GB GDDR2.99 GiBFits147–281 tok/s
GeForce RTX 408016 GB GDDR2.99 GiBFits157–299 tok/s
GeForce RTX 4080 Super16 GB GDDR2.99 GiBFits161–307 tok/s
GeForce RTX 5060 Ti 16GB16 GB GDDR2.99 GiBFits98–187 tok/s
GeForce RTX 5070 Ti16 GB GDDR2.99 GiBFits196–374 tok/s
GeForce RTX 508016 GB GDDR2.99 GiBFits210–401 tok/s
Intel Arc A770 16GB16 GB GDDR62.99 GiBFits112–214 tok/s
Legion Go16 GB LPDDR5X-75002.99 GiBFits26–50 tok/s
Radeon RX 6800 XT16 GB GDDR62.99 GiBFits112–214 tok/s
Radeon RX 7800 XT16 GB GDDR62.99 GiBFits136–261 tok/s
Radeon RX 9060 XT 16GB16 GB GDDR62.99 GiBFits70–134 tok/s
Radeon RX 9070 XT16 GB GDDR62.99 GiBFits140–267 tok/s
ROG Ally16 GB LPDDR5-64002.99 GiBFits22–43 tok/s
Steam Deck LCD16 GB LPDDR5-55002.99 GiBFits19–37 tok/s
Steam Deck OLED16 GB LPDDR5-64002.99 GiBFits22–43 tok/s
Apple M3 Pro, 18 GB18 GB unified2.99 GiBFits33–63 tok/s
Radeon RX 7900 XT20 GB GDDR62.99 GiBFits175–334 tok/s
Apple M2, 24 GB24 GB unified2.99 GiBFits22–42 tok/s
Apple M3, 24 GB24 GB unified2.99 GiBFits22–42 tok/s
Apple M4 Pro, 24 GB24 GB unified2.99 GiBFits60–114 tok/s
Apple M4, 24 GB24 GB unified2.99 GiBFits26–50 tok/s
GeForce RTX 309024 GB GDDR2.99 GiBFits204–391 tok/s
GeForce RTX 409024 GB GDDR2.99 GiBFits220–421 tok/s
Radeon RX 7900 XTX24 GB GDDR62.99 GiBFits210–401 tok/s
ROG Ally X24 GB LPDDR5X-75002.99 GiBFits26–50 tok/s
Apple M1 Max, 32 GB32 GB unified2.99 GiBFits87–167 tok/s
Apple M1 Pro, 32 GB32 GB unified2.99 GiBFits44–84 tok/s
Apple M2 Max, 32 GB32 GB unified2.99 GiBFits87–167 tok/s
Apple M2 Pro, 32 GB32 GB unified2.99 GiBFits44–84 tok/s
Apple M4, 32 GB32 GB unified2.99 GiBFits26–50 tok/s
GeForce RTX 509032 GB GDDR2.99 GiBFits391–748 tok/s
Apple M3 Pro, 36 GB36 GB unified2.99 GiBFits33–63 tok/s
Apple M4 Max 14-core, 36 GB36 GB unified2.99 GiBFits90–171 tok/s
Apple M3 Max 16-core, 48 GB48 GB unified2.99 GiBFits87–167 tok/s
Apple M4 Max 16-core, 48 GB48 GB unified2.99 GiBFits119–228 tok/s
Apple M4 Pro, 48 GB48 GB unified2.99 GiBFits60–114 tok/s
Apple M1 Max, 64 GB64 GB unified2.99 GiBFits87–167 tok/s
Apple M2 Max, 64 GB64 GB unified2.99 GiBFits87–167 tok/s
Apple M3 Max 16-core, 64 GB64 GB unified2.99 GiBFits87–167 tok/s
Apple M4 Max 16-core, 64 GB64 GB unified2.99 GiBFits119–228 tok/s
Apple M4 Pro, 64 GB64 GB unified2.99 GiBFits60–114 tok/s
Apple M2 Max, 96 GB96 GB unified2.99 GiBFits87–167 tok/s
Apple M3 Max 16-core, 128 GB128 GB unified2.99 GiBFits87–167 tok/s
Apple M4 Max 16-core, 128 GB128 GB unified2.99 GiBFits119–228 tok/s

FAQ

Why Q4_K_M?

Q4_K_M is the quant this page recommends. The file is HuggingFaceTB_SmolLM3-3B-Q4_K_M.gguf at 1.78 GiB. Other single-file quants are in the table when a real size was available.

Where do the architecture numbers come from?

Official config.json. head_dim = 2048 / 16. Source: https://huggingface.co/HuggingFaceTB/SmolLM3-3B/raw/main/config.json

Does this include a long context?

The fit uses 8,192 tokens, or less when that is all the fast memory can hold. The model file allows up to 65,536 tokens, and KV cache grows as you use more of that.

Why is there no Ollama command?

No Ollama library tag was verified for this exact model revision. A nearby tag can be a different checkpoint, so it is left off.