Can I run Phi-4 Mini 3.8B?

Phi-4 Mini 3.8B is Microsoft's small chat model, the one to reach for when you want something genuinely lightweight that still holds a conversation. At a 4-bit quant it lands around 2.3 GB, dropping to roughly 1.6 GB at 2-bit, so it fits almost anywhere: a 6 GB GPU has room to spare, an entry M1 Mac runs it without thinking, and an old laptop on CPU can handle it. The MIT license is the easy part, no terms to read and no commercial restrictions, so you can ship it in a product freely.

In daily use it is quick. On an RTX 3060 you'll see about 133 tok/s, an M-series Max sits near 151 tok/s, and a 4090 pushes it past 370 tok/s, which is faster than you can read by a wide margin. The 128K context is the headline number but treat it with caution: fill it and total memory climbs to about 14.3 GB, far past the 6 GB this model needs at rest. On a small GPU keep working context to a few thousand tokens and you will stay comfortable. Even CPU on DDR5 manages around 22 tok/s for short prompts.

Where it shows its size is depth. For reasoning and multi-step instructions, Qwen 3 4B generally has the edge despite being roughly the same parameter count, and if you need real reasoning headroom Phi-4 14B is the obvious step up. Phi-4 Mini's standout trait is efficiency: it punches above its 3.8B for plain chat and summarisation while staying small enough to run almost anywhere, MIT-licensed and unrestricted. If you want a tiny local assistant that loads fast and gets out of your way, this is a strong default.