Can I run Devstral Small 2 24B?

Devstral Small 2 is Mistral's 24B coding model, and the name undersells how much machine it asks for. At a 4-bit quant it lands around 14.6 GB of weights, so the 12 GB cards most people own are out: on an RTX 3060 it simply does not fit. You realistically want 24 GB of memory to run it comfortably, which in practice means a 24 GB GPU like a 4090 or an Apple Silicon Mac with at least 24 GB of unified memory. If you drop to a 2-bit quant it shrinks to about 10.1 GB, but that is a steep quality tradeoff for a model whose whole point is careful code generation.

On a 4090 you can expect around 59 tokens per second at 4-bit, which is genuinely pleasant for an agentic coding loop where it reads files, plans, and edits. On an M-series Max it settles closer to 24 tok/s, still usable for interactive work but you feel it on longer generations. The headline 256K context is real and useful for feeding it whole repositories, but be honest about the cost: even at 128K the fact sheet puts total memory near 42.1 GB, well past a single 24 GB card. Treat the big context as something you reach for deliberately, not a default you leave open.

Against its siblings, Devstral is the serious-work pick: Mistral 7B and Mistral Nemo 12B are lighter and easier to host but generally weaker on multi-step coding tasks, while Gemma 4 26B A4B is the closest peer for coding and reasoning and worth a side-by-side if you have the memory. What makes Devstral stand out is that it was built for agentic, tool-using code editing rather than one-shot chat, and that focus shows in how it follows a plan across files. The license is Apache 2.0, so you can use it commercially and in production without restrictions, which is rare for a model this capable.