Can I run Gemma 4 31B?

Gemma 4 31B is Google's mid-large open-weight model at 30.7B parameters, built for chat, coding, reasoning, and vision in one package. This is not a laptop-and-go model. At a 4-bit quant it weighs about 18.6 GB, and you need at least 32 GB of RAM to load it at all, so a 12 GB card like the RTX 3060 simply does not fit. The realistic home for it is a 24 GB GPU such as an RTX 4090, or an Apple Silicon Mac with plenty of unified memory. If you want a capable all-rounder and have the hardware, this is the tier where local models start feeling genuinely useful.

In daily use it is comfortable rather than blazing. On an RTX 4090 you can expect around 46 tokens per second at 4-bit, fast enough to read along as it streams; on an Apple M Max it settles closer to 19 tokens per second, still fine for interactive work. Pure CPU on DDR5 drops to roughly 3 tokens per second, which is patience-only territory. The 256K context window is generous, but it is expensive: pushing toward 128K already takes about 49.3 GB total memory, so treat the full window as a ceiling and keep working context modest unless you have headroom to spare.

Against Qwen 3 30B-A3B, a near-identical 30.5B sibling, the trade is architectural: Qwen's mixture-of-experts design tends to run lighter per token, while Gemma 4 31B is a dense model that uses its full weight on every pass and generally feels steadier on vision and broad instruction-following. If you want something far smaller, Gemma 3 4B is the lighter pick. The standout here is breadth: one model covering chat, code, reasoning, and images, under a clean Apache 2.0 license you can use commercially and in production without provider-specific restrictions.