Can I run Mellum 2 12B-A2.5B?

Mellum 2 is JetBrains' coding-focused model, built as a mixture-of-experts: 12B parameters total but only about 2.5B active per token. That's the whole point of the design. You get the speed of a roughly 2-3B model, but you still hold the full 12B in memory, so don't be fooled by the active count. At a 4-bit quant it lands around 7.3 GB, with a practical floor of about 12 GB RAM. That fits a 12 GB card like an RTX 3060 or unified memory on an Apple Silicon Mac, but 8 GB is too tight. If you live in JetBrains IDEs and want local code completion, it's aimed squarely at you.

In daily use the MoE design pays off: it feels much faster than its size suggests. On an RTX 3060 12GB you can expect around 202 tokens per second at 4-bit, and an RTX 4090 pushes past 565 — well into the range where completions land before you've finished typing the next line. The 128K context is genuinely large for a coding model, handy for feeding it whole files or a repo's worth of headers, but it isn't free. Fill it all the way and total memory climbs to roughly 27.4 GB, far past what a single 12 GB card holds, so keep working context modest unless you have a 24 GB GPU or generous unified memory.

It's worth being clear about scope: this is a coding specialist, not a general assistant. For chat, reasoning, or anything with images, a broader 12B like Gemma 4 12B generally serves you better, and Mistral Nemo 12B tends to be the friendlier pick for open-ended conversation. Mellum 2's standout trait is that MoE speed-to-size ratio on completion-style work, paired with first-class IDE integration from the people who make your editor. And the license is the easy part: Apache 2.0, so you can use it commercially and in production without legal worries. If your main job is code and you have a 12 GB card, it's a strong, fast local choice.