Is MLX faster than Ollama? It depends.

The short version

9/2/2026 update

Reid Peryam read this and asked a good question in the comments: was I using Rapid-MLX, rather than vanilla MLX? I was not. So I ran it.

What I tested

Three engines instead of two, on the same afternoon, the same Mac Studio, and the same two checkpoints from the tables below. Fifty-four more runs. Rapid-MLX was served the identical MLX repository that the vanilla MLX column loads, so this is a wrapper measured against the library inside it rather than against a different model.

One thing had to change, and it is the whole reason this took a day rather than an hour. Rapid-MLX exposes no timing counters of its own: token counts, and nothing else. Ollama and MLX both report their own nanosecond durations, which is what every number above is built from. Timing Rapid-MLX from the outside while reading the other two from the inside would have charged it for HTTP that vanilla MLX never pays, and that is the same shape of mistake as the cache artifact, a number that looks like a fair fight and is not one. So all three engines are timed from the client here, identically, and their own counters are recorded alongside for checking. The two agreed within about 3 percent throughout.

Prompt token counts were matched across all three engines on every repetition, and the run fails loudly if they ever diverge. Decode below is the median of three runs.

The verdict further down this page said to stay on Ollama for mixture-of-experts models. That survives, and it comes out stronger: Ollama beats both MLX-based engines at every prompt length. It also survives the one discrepancy worth naming. Ollama measured faster on the mixture-of-experts model this time than it did in the original run, 91.23 against 83.32 at the short prompt, and even taking the older and slower figure it still beats Rapid-MLX by 14.9 percent.

The 4.2x is worth taking apart, because my hardware is not the reason it does not hold up. The repository gives no machine, no model, and no method for that number, and its own body text says "up to 3x" a few paragraphs later. The benchmark it links to measures 46.7 tokens per second against Ollama's 32.1, and 58.5 against 39.0. That is 1.46x and 1.50x, on their hardware, by their method. I measured 1.40x to 1.53x on a 2022 M1 Ultra. Their work reproduces; the tagline contradicts their own data before mine enters the argument at all.

I also expected Rapid-MLX to lose on prefill, because its own published benchmark volunteers that llama.cpp beats it by 1.7 to 2x on long documents. On the dense model it had the fastest time to first token of all three at both long prompts, 18.27 seconds against MLX at 18.55 and 77.21 against 77.94. On the mixture-of-experts model it was last. Prefill depends on the architecture rather than being uniformly weak, which is the same lesson as everything else on this page.

The cache trap is not an Ollama quirk

This is the part I would keep if you keep nothing else. Send an identical 16,384 token prompt twice, then send a fresh one, and watch what each engine reports for time to first token.

Rapid-MLX reports a 74x speedup for doing nothing. Ollama reports 474x. Vanilla MLX rebuilds its cache on every call and has nothing to defeat, which is exactly why a careless comparison makes it look slow against both servers.

Rapid-MLX's other headline number is a 0.08 second cached time to first token, and now you can see what that measures. To be fair to them, it says cached right there in the claim, which is more than most vendor numbers manage. The point is not that anyone is cheating. It is that prompt caching is normal now, so any benchmark that sends the same prompt a few times and averages the result is timing a lookup on every engine that has one.

The code and the data

All fifty-four runs, the cache check, and the three-way harness are in a second gist. It is separate from the first one on purpose, because that gist backs the numbers already published on this page and rewriting it would strand anyone who had read them. Take the three-way code and data from this gist.

And thanks to Reid for the question. It is the second time this benchmark has been improved by someone pushing back on it in public, which is the entire argument for showing your working. Here is the thread it came from.

First, what the machine is doing

Two ideas make the rest of this page make sense. If you already know them, skip to the tables.

Reading is one job. Writing is a different one.

When you send a model a prompt, it does two things in sequence, and they are not remotely alike.

First it reads everything you sent. Your question, your system prompt, and the eight files your coding agent helpfully attached. This is prompt processing, or prefill. The useful thing about it is that the model already has the whole prompt, so it can chew through all of it at once. That makes it a raw arithmetic problem, and the limit is how many calculations per second the GPU can do.

Then it writes a reply, one token at a time. A token is roughly three quarters of a word. This is token generation, or decode, and it has a constraint prefill does not: the model cannot guess its fourth word before it has written its third. No shortcuts. And to produce each one, it has to consult its weights, which means reading gigabytes out of memory. Every single token.

So the limit on writing is not how fast the chip can calculate. It is how fast it can move data.

You can do that arithmetic on a napkin. This machine moves 800GB per second, and the model I use most is about 17GB. Divide one by the other and you get roughly 47 tokens per second as a hard ceiling, before any real-world inefficiency. Nothing can beat that number, and anything claiming to is measuring something else.

Quantization is like saving the model as a JPEG

A model is billions of numbers, called weights. Trained at full precision, each one takes 16 bits. A 27-billion parameter model at 16 bits is about 54GB, which will not fit comfortably on most Macs and would be slow if it did, for the memory reason above.

Quantization stores those same numbers in fewer bits. Eight instead of 16 halves the file. Four instead of 16 quarters it. That 27B model becomes 30GB at 8-bit and about 17GB at 4-bit, which is the difference between owning the right Mac and not.

It is the same trade as saving a photo as a JPEG instead of a RAW file. Much smaller, slightly less faithful, and for most purposes you cannot tell. Models get a little less precise as you compress them, and 4-bit is where most people land because the quality cost is small and the speed gain is not.

Here is the part almost nobody mentions, and it turns out to be the whole story on this page. There is more than one way to do the compressing, the two engines use different ones, and those formats do not cost the same amount of work to unpack when the model actually runs. Two files can both say 4-bit and behave differently.

The numbers

Three repetitions per cell, engines alternating so neither one eats the heat from the other. Every figure is the median. Both engines saw identical token counts.

Qwen3.8-27B, dense, 4-bit

MLX generates faster at every prompt length, and it is remarkably consistent about it: 30.86, 30.92, 30.89 across three runs. Ollama wandered between 20.54 and 23.89.

Qwen3.8-27B, dense, 8-bit

Same weights. Same machine. One variable changed, and a 41 percent lead became a rounding error.

Qwen3-30B-A3B, mixture of experts, 4-bit

Ollama wins this one on both counts.

But look past the comparison for a second. This model generates at 83 tokens per second where the dense 27B managed 22, in about the same memory. A mixture-of-experts model only wakes up a fraction of itself for each token, roughly 3B of its 30B parameters here, so you pay for the memory of a large model and get the speed of a small one. That is a bigger difference than anything else on this page, and it is the reason which model you pick matters more than which runtime serves it.

Why the answer keeps moving

The 4-bit gap is mostly not the engine

Generating a token means reading every active weight out of memory. So a model with twice the bytes should generate at roughly half the speed. That is the theory, and it is usually right.

Ollama decodes at 21.9 tokens per second at 4-bit, and 20.4 at 8-bit.

That is 7 percent slower while reading 76 percent more data. The theory says it should have been closer to half.

So Ollama's 4-bit decode is not waiting on memory. It is waiting on something else, and the obvious suspect is unpacking the weights. Q4_K_M is a mixed K-quant: the weights sit in super-blocks with their own scales and minimums, and every one has to be decoded before any arithmetic happens. MLX uses a plainer group-size-64 scheme that costs less to unwrap.

There is a second tell. Ollama prefills faster at 8-bit than at 4-bit, 229 against 203 tokens per second on a 4k prompt, with a model nearly twice the size. The only way a bigger model goes faster is if the smaller one was spending its time on something other than moving bytes.

MLX drops from 30.9 to 19.5 across that same change. Much closer to the halving the theory predicts, which is what you would expect from a format that is cheap to unpack.

The prefill crossover is chunk size

Ollama reads your prompt in 512-token batches. That is the default in its source. mlx-lm reads it in 2,048-token steps.

A 500-token prompt is one chunk either way, so all you are measuring is the overhead of getting started, and Go calling into C++ starts faster than a Python loop. Feed it 16,000 tokens and MLX makes a quarter as many trips to the GPU, each four times bigger. GPUs like that. That is the whole crossover.

These are not the same kind of thing

Half the confusion here comes from treating them as rival products. They are not.

Ollama is a Go server wrapped around llama.cpp. The math happens in hand-written Metal kernels. Everything around the math is the actual product: a model registry, memory sizing you never think about, an OpenAI-compatible endpoint, and a cache that remembers your last prompt. You install it and it works.

That description needs a date on it now, and the date matters for reading every Ollama number on this page. Ollama 0.19, in March 2026, put an MLX runner in the tree, and 0.33.2 ships both it and the llama.cpp one. Which runner you get is decided per model rather than per machine: Ollama picks MLX for checkpoints published in MLX format, and llama.cpp for everything else. The three checkpoints benchmarked here are GGUF k-quants, Q4_K_M and Q8_0, which the MLX runner does not implement. So every Ollama figure on this page came out of llama.cpp and Metal, on a build that also had MLX sitting right next to it.

Which is worth saying plainly, because "Ollama runs on MLX now" is true and does not mean this comparison measured MLX against itself. It also sets up the obvious next experiment, and it is not one I have run: pull the same weights in MLX format, let Ollama use its own MLX runner, and see whether the mixture-of-experts result below survives. If Ollama's advantage there is a llama.cpp advantage, that test is where it shows up.

MLX is Apple's array framework, closer to NumPy than to a server. It evaluates lazily, so it can see a few operations ahead and fuse them before touching the GPU, and it treats unified memory as the normal case instead of a copy to be optimized away. mlx-lm is the language model layer on top. No server. No registry. You write Python.

Which explains the shape of everything above. Ollama competes on everything except the arithmetic. MLX competes on nothing but. And they are converging anyway, since the arithmetic Ollama does not compete on is now shipping inside it. This is a snapshot, not a verdict.

The wrong answer I got first

Back to that. My first run had Ollama processing a 3,932-token prompt at 19,071 tokens per second. Then 25,596. Then 30,850.

Faster every time. Computers do not do that.

A back-of-the-envelope check settles it. Prefill costs roughly two operations per parameter per token, so 27 billion parameters across 3,932 tokens is about 200 trillion operations. An M1 Ultra does maybe 21 trillion a second. Call it ten seconds of work. Ollama was claiming 0.14.

It was reading its own cache. Send the same prompt twice and Ollama reuses the work it already did, then reports a duration covering the lookup instead of the labor. The fix is to change the first few tokens on every run:

This is not Ollama cheating. That cache is a genuinely useful feature, and it matters more than the benchmark does, which I will get to. It just cannot be running while you measure.

If you take one thing from this page, take that. It is almost certainly why you have seen Ollama prefill numbers that looked too good.

So what should you actually run?

Depends what you are doing with it.

You sit and watch the output arrive. Switch to MLX. On a dense model at 4-bit it is worth about 40 percent more tokens per second, and that is roughly the line between reading along and waiting for the machine. Twenty tokens a second is where people start checking their phone. Thirty is where they stop noticing.

You run a coding agent over a big codebase. Stay on Ollama, and it is not close. Your problem is not generation, it is the 78 to 85 seconds before the first token on a 16k prompt. An agent sends a nearly identical prompt every turn, and Ollama's cache means it pays that once instead of every time. The feature I had to switch off to measure fairly is the most valuable thing either engine does for this job. If you have not set this up yet, the Qwen and Ollama walkthrough covers the context window settings that decide whether it works at all.

You need something other tools can call. Ollama. MLX has no server, so that is yours to write and maintain, to recover a difference that disappears at 8-bit anyway.

You are picking a quantization. Stay at 4-bit unless you have actually measured a quality problem. Eight-bit cost me 35 percent of my generation speed and 12GB of memory, and it turns a 32GB Mac from comfortable into impossible.

You want the biggest win on this page. It is not the engine. Moving from the dense 27B to the mixture-of-experts model was worth three to four times the throughput in the same memory. Pick the architecture first. Then argue about runtimes.

Run it on your own machine

The harness is public, and honestly it is the part I would want if I were reading this. It handles the traps above and reports prefill and decode separately, straight from each engine's own counters.

So are the raw numbers. All fifty-four runs are in there as JSON, with the environment they ran in and every individual repetition, so you can recompute the medians in the tables above rather than take my word for them. Take the code and the data from this gist.

The smoke run pulls a 0.6B model and takes under a minute. One thing to check before you trust any of it: both engines should report the same prompt token count. If they do not, something is templating your prompt twice and the comparison is already dead.

What this ran on

The M1 Ultra is 2022 hardware, and I should be straight about what that means. These exact numbers belong to this chip. They will not transfer to an M5.

What does transfer is the shape: the dequantization cost, the chunk-size crossover, and architecture mattering more than runtime. Those live in the software, not the silicon. If you are shopping for hardware rather than a runtime, the M5 Ultra buying guide covers the current generation.

One limitation of every number on this page is worth naming, because I went and measured it afterwards. These are medians of three runs per cell, and three runs finish inside the first ninety seconds. On a machine with a fan that is fine. On a fanless one it is not: a MacBook Air holds only about three quarters of its cold throughput once it has been working for a few minutes, so a median of three overstates what you actually get by roughly 20 percent. I measured that separately in what a fanless Mac sustains under load.

Frequently asked questions