Can I run Llama 3.1 8B?

Llama 3.1 8B is the model most people actually mean when they say they want to "run a local LLM." At a 4-bit quant it lands around 4.9 GB, which means it fits comfortably on an 8 GB GPU, sits well within unified memory on any Apple Silicon Mac from an M1 onward, and will even run on CPU if you are patient. That combination of small footprint and genuinely useful output is why it became the default starting point for self-hosting.

In day-to-day use it feels fast and coherent for chat, summarisation, and light coding help. On a mid-range modern GPU you can expect well over 40 tokens per second at 4-bit, fast enough that the reply streams faster than you read. The 128K context window is real, but treat it as a ceiling rather than a cruising altitude: filling it pushes KV-cache memory up sharply, so on an 8 GB card you will want to keep working context to a few thousand tokens unless you drop to a smaller quant.

Where it shows its size is reasoning and instruction-following on harder, multi-step prompts — newer 7-8B models like Qwen 3 8B tend to edge it out on math and structured tasks, while Gemma 3 4B is the lighter pick if you are tight on memory. Llama 3.1 8B's advantage is maturity: it has the broadest tooling support, the most quantised builds on Hugging Face and Ollama, and the fewest surprises. If you want one model that just works as a local assistant, this is still the safe default.