Can I run Mistral Small 3.1 24B?

Mistral Small 3.1 24B is what you reach for when an 8B model keeps disappointing you on harder prompts but you don't want to step up to a 70B. It is a 24B chat-and-vision model, so it reads images as well as text, and it ships under Apache 2.0, which means you can use it commercially and in production with no strings attached. The footprint is the catch: at a 4-bit quant it lands around 14.6 GB, and you want roughly 24 GB of memory to run it comfortably. That rules out a 12 GB card like an RTX 3060, where it simply does not fit, and points you at a 24 GB GPU or an Apple Silicon Mac with plenty of unified memory.

On a 24 GB card like an RTX 4090 it moves at about 59 tokens per second at 4-bit, which streams faster than you read and feels genuinely snappy for a model this size. On an M-series Max you are looking at closer to 24 tokens per second, still perfectly usable for interactive chat. CPU-only on DDR5 drops to around 4 tokens per second, fine for a batch job overnight but not for sitting and waiting. The 128K context window is real, but it is expensive: fill it all the way and total memory climbs to about 42 GB, well past what the model needs at short context, so keep working context modest unless you have headroom to spare.

Against its own family it is the heavyweight: Mistral Nemo 12B is the lighter, faster pick if you are memory-constrained, and Mistral 7B is the bare-minimum option, but neither matches Small 3.1 on harder reasoning or instruction-following. Gemma 4 26B A4B is the closer rival on raw capability and adds coding and reasoning strengths, so compare the two if those tasks matter most to you. Small 3.1's standout trait is the combination you rarely get together: native vision, a long context, and a fully permissive Apache 2.0 license, which makes it one of the easiest capable models to actually ship in a product.