Can I run Gemma 4 12B?

Gemma 4 12B is a dense 12B vision-capable model for chat, coding, and reasoning, aimed at people who want one capable local assistant that also reads images. At a 4-bit quant it lands around 7.3 GB, so it fits on a 12 GB card like an RTX 3060 with room for context, and sits comfortably in unified memory on an Apple Silicon Mac. The model needs about 12 GB of RAM minimum to load. If you are tight, the 2-bit build drops to roughly 5 GB, though you trade quality for the squeeze.

In daily use it streams at about 42 tok/s on an RTX 3060, fast enough to read along with, and around 48 tok/s on an M-series Max. An RTX 4090 pushes it past 100. The 256K context window is generous, but treat it as headroom, not a default. Even at 128K context the model plus KV cache climbs to roughly 27.4 GB total, well beyond a single 12 GB card, so you will spill to system memory or slow down hard. Keep working context to a few thousand tokens on consumer hardware.

Against the lighter Gemma 3 4B, the 12B generally holds up better on multi-step reasoning and coding, which is the trade you make for the extra memory; Gemma 3 4B is the pick if you are constrained or only need chat and vision. The standout here is that vision and solid text reasoning ship in one model that still runs on a mid-range GPU. License is clean too: Apache 2.0 means you can use it commercially and in production without provider-specific strings.