Can I run OLMo 2 13B?

OLMo 2 13B is Ai2's fully open chat model, and its appeal is as much about provenance as performance: the weights, training data, and recipe are all public under Apache 2.0, so you can use it commercially without second-guessing the terms. At a 4-bit quant it lands around 8.3 GB, which is a bit much for a 12 GB card once you account for overhead, but it sits fine on a 16 GB GPU or in unified memory on an Apple Silicon Mac. If you are tight, the 2-bit build drops to about 5.7 GB. Plan on roughly 12 GB of RAM as a floor.

In daily use it feels like a solid, even-tempered chat partner. On an RTX 3060 12 GB you can expect around 37 tokens per second at 4-bit, comfortably faster than you read, and a 4090 pushes that to about 103 tok/s. An M-series Max sits near 42 tok/s. The one thing to keep front of mind is the 4K context window, which is small by 2024-2025 standards. Even maxed out the model only needs about 9 GB total, but you simply cannot feed it long documents or sprawling chat histories without truncating, so treat it as a short-turn assistant.

Against its size class, OLMo 2 13B is the open-data pick rather than the raw-capability leader. Models like Qwen 3 14B and Phi-4 14B generally edge it out on reasoning and structured tasks, and both carry far larger context windows if long inputs matter to you. Ministral 3 14B adds vision, which OLMo does not. What OLMo offers that none of them do is genuine end-to-end openness: if you care about knowing exactly what went into your model, or you are doing research and reproducibility work, this is the one to reach for. Apache 2.0 means no usage strings attached.