Can I run Nemotron 3 Super 120B-A12B?

Nemotron 3 Super 120B-A12B is NVIDIA's mixture-of-experts model for people who want frontier-class reasoning and coding on their own hardware and have the memory to back it. Although it carries 120B total parameters, only 12B activate per token, so it runs much faster than its size suggests while still demanding room for the whole model. That is the catch: even at a 4-bit quant it lands around 72.8 GB, and you need at least 96 GB of RAM to load it. This is not an 8 GB GPU model. A 24 GB RTX 4090 cannot fit it, and even a 2-bit build at roughly 50 GB stays out of reach. A large unified-memory Apple Silicon machine or a server with plenty of system RAM is the realistic home for it.

In daily use the MoE design pays off. On an Apple M Max you can expect around 48 tokens per second, which feels genuinely interactive for chat, multi-step reasoning, and coding help despite the 120B label. Push it onto CPU with DDR5 and you drop to about 7 tokens per second, usable for batch work but not live conversation. The context window is enormous at 1000K tokens, but treat that as a ceiling, not a setting you leave maxed. Filling even 128K pushes total memory to roughly 129.5 GB once the KV cache is counted, so on a 96 GB machine you will run out of headroom long before you reach the advertised limit. Keep working context modest unless you have memory to spare.

Against its closest relatives, Nemotron 3 Super sits in interesting company. Qwen 3.5 122B-A10B is a similar-scale MoE that also handles vision, which this model does not, so if you need image input that one generally has the edge. Mistral Small 4 119B is comparable in size and also multimodal. If your hardware cannot stretch this far, the much smaller Nemotron 3 Nano 30B-A3B is the same family's lightweight alternative and tends to be the pragmatic pick on constrained machines. Super's standout trait is delivering 120B-class reasoning at 12B-class speed for those who can host it. One caveat: it ships under the NVIDIA Open Model license, so check those terms before relying on it commercially rather than assuming plain open-source freedom.