Run Qwen 3.8 on Apple Silicon, Without Rate Limits

1. Get Ollama

Ollama is the local runtime that serves open-weight models over a clean HTTP API. Grab the macOS build from the Ollama download page.

2. Pull Qwen3.8

Qwen3.8 is the model I recommend for local agentic coding. It is strong at multi-step reasoning and tool use, it follows agent skills and markdown instruction files closely enough for agent mode to work, and thinking mode is on by default with the reasoning depth tunable per request.

Browse the model card on the Ollama library, then pull it from your terminal:

That pulls qwen3.8:latest, an 18GB 4-bit quant. If you have memory to spare and want higher fidelity, qwen3.8:27b-q8_0 is 30GB and qwen3.8:27b-bf16 is 56GB. Apple silicon users should also look at the mlx tags, which target Apple's own machine learning framework.

3. Raise the Context Window

This is the step that is easy to skip, and it is a common reason people conclude that local agents are useless. Ollama picks a default context length based on available VRAM: under 24GiB you get 4k tokens, 24GiB to 48GiB gets you 32k, and 48GiB or more gets you the full 256k.

Those thresholds are VRAM, not unified memory, and on Apple silicon the two are not the same number. macOS reserves part of unified memory for the system and hands the GPU roughly two-thirds of it on machines up to 36GB and roughly three-quarters above that. A 48GB Mac therefore presents about 36GiB to Ollama and lands in the 32k tier, not the 256k one. That is the real reason the sweet spot is 64GB rather than 48GB.

A 4k context is not an agent. It is an autocomplete that forgets the file it just opened. Ollama's own context length documentation is explicit about it: tasks that need large context, including agents and coding tools, should be set to at least 64000 tokens. Treat that as the floor. For agentic coding, give it the model's full window:

262144 is the 256k window Qwen3.8 was trained for, and it is the same value Ollama picks on its own once it sees 48GiB of VRAM. Setting it explicitly means you get that window regardless of how Ollama reads your hardware. On a 64GB Mac or larger there is room for it. On a smaller machine the setting will still apply, but the weights plus a context that size will not fit in what the GPU can address, and Ollama will spill to the CPU and slow to a crawl. Drop to 65536 there.

The Ollama desktop app exposes the same setting as a slider in its settings menu. Once the model is loaded, ollama ps confirms both the context window it was given and whether it is actually running on the GPU.

This is also the honest reason the memory tiers above matter. A longer context costs memory on top of the 18GB of weights, so context length and unified memory are the same budget spent twice.

4. Install VS Code

If you don't already have it, grab VS Code.

5. Install the Zoo Code Extension

Add the Zoo Code extension from the VS Code marketplace. It gives you a set of agent modes in the editor, including Architect for planning, Debug for diagnosis, and Code for everyday edits and file operations.

6. Connect to Ollama

In Zoo Code's settings, select ollama as the API provider, leave the base URL at the local default of http://localhost:11434, and enter qwen3.8 as the model. The Zoo Code Ollama provider documentation covers the full option list.

One detail worth knowing: Zoo Code defers to the model's num_ctx as Ollama reports it rather than setting a context length of its own. That is why step 3 comes before this one. If you raise the context after the model is already loaded, restart the Ollama server so the new value takes effect.

7. Start Coding

Put the tool in Code mode and start working. It picks up your agent skills and markdown instruction files the same way a cloud coding agent does. No rate limit. No meter. No waiting for a quota to reset.

1. Prompt Processing Is Bound by GPU Speed

The first pass over your prompt is compute-bound. The GPU has to process every token in the context window before it can start generating. Newer GPU architectures with more cores and wider memory interfaces finish this pass faster. An M3 Ultra will chew through a long prompt noticeably quicker than an M1, even if the two chips end up generating tokens at a similar clip.

This is the spec you feel most in agentic coding specifically, because an agent re-reads a large context on every turn. It is the difference between a pause you ignore and a pause you alt-tab away from.

2. Token Generation Is Bound by Memory Bandwidth

Once the model starts writing, every token it generates requires reading the model's weights back out of memory. That makes bandwidth the ceiling. Raw compute barely enters into it.

The practical upshot is that bandwidth tiers matter more than product names. An M1 Ultra at 800GB/s and an M3 Ultra at 819GB/s generate tokens at roughly the same speed despite two generations between them, because they sit in the same bandwidth class. A recent Max chip lands close behind. A base chip is a different category of experience entirely, and no amount of extra RAM changes that.

Worth stating plainly, since it causes a lot of confusion: there is no M4 Ultra. Apple's Ultra tier skipped the M4 generation, so today the top of the bandwidth chart is the M3 Ultra Mac Studio, followed by the M5 Max and then the M4 Max.

This is the whole case for the Studio, and I say that as someone who runs a rack of Mac minis. The mini tops out at the M4 Pro and 273GB/s. An M3 Ultra Studio reads the same weights at 819GB/s. Identical model, identical quant, three times the memory bandwidth feeding the GPU.

Here's the Apple silicon lineup, sorted by generation:

3. Memory Capacity Is a Floor, Not a Dial

You need enough unified memory to hold the weights and the context window at the same time. If it doesn't fit, it doesn't run, or it spills and crawls. That's the hard floor, and for Qwen3.8 the floor is 18GB of weights before a single token of your codebase is loaded.

Above the floor, returns diminish quickly. Going from 32GB to 64GB buys real context headroom and is the upgrade most people should make. Going from 64GB to 128GB buys very little for this model, because the weights and a generous context already fit. At that point the money is better spent on bandwidth, which is the spec that never stops mattering.

4. One Task at a Time, Around the Clock

An agentic coding workload saturates the GPU. You're not going to run two heavy coding sessions in parallel on a single Mac and expect both to stay fast. Plan for one task at a time.

But here's the trade-off that makes it worth it: you can run that one task continuously without a rate limit. No hourly cap. No daily ceiling. No concurrent-session limit. The machine doesn't care if it's 3 AM. That should soften the blow of single-task throughput.

It also points at the only real way to scale this, which is more machines rather than a bigger one. That's the entire argument behind the Mac cluster build documented in the rest of this series.