Can I run Devstral 24B?

Devstral 24B is Mistral AI's coding specialist, built for people who want a capable local agent for software work rather than a general chat companion. At 24B dense parameters it is not a lightweight: the 4-bit quant lands around 14.6 GB and the model needs roughly 24 GB of RAM minimum to run comfortably. That puts it out of reach of a 12 GB card like an RTX 3060, where it simply does not fit. The natural home is a 24 GB GPU such as an RTX 4090, or an Apple Silicon Mac with plenty of unified memory. If you are short on space, the 2-bit build drops to about 10.1 GB.

In daily coding use on an RTX 4090 you can expect around 59 tokens per second at 4-bit, quick enough that generated code streams smoothly while you read it. On an Apple M-series Max it settles closer to 24 tokens per second, still perfectly usable for interactive work, and CPU-only on DDR5 falls to roughly 4 tokens per second, which is patience territory. The 128K context window is genuinely large, but it is not free: filling it pushes total memory to about 42.1 GB, well past the model's resting footprint, so keep working context modest unless you have the headroom to spare.

Against its stablemates, Devstral 24B is the heavyweight: Mistral 7B and Mistral Nemo 12B are far lighter and aimed at general chat, so they will run on hardware Devstral cannot touch but generally trail it on actual coding tasks. The closer comparison is Gemma 4 26B A4B, a similarly sized model that also handles coding alongside chat, reasoning, and vision, and tends to be the more versatile generalist if you want one model for everything. Devstral's standout trait is its narrow focus on code, and the Apache 2.0 license means you can use it freely, including in commercial production.