Ollama Now Runs Faster on Macs Thanks to Apple's MLX Framework

This is going to be some serious cash flow incoming for Apple in this year.

I think this could be a major business for Apple - it’s way cheaper for a small business to buy a powerful Mac and run qwen 3.5 than pay for an enterprise license for a frontier model - and you don’t need to worry about privacy issues.

On device is definitely gonna be the future.

I can’t help but wonder if Apple looked ahead and foresaw this when developing the M series, or if they’ve lucked into it.

This is going to be some serious cash flow incoming for Apple in this year.

M-Series chips at work😎

Just tested the new Ollama MLX runner in CLI via terminal (0.19.0 preview) on my Mac mini M2 Pro 32GB, running qwen3.5:35b-a3b-coding-nvfp4.

it works, and it's noticeably faster than standard non-MLX models! Standard Qwen3.5 35b is not usable on my Mac, this one yes! It’s incredible!

What works well:
- The model loads and runs without issues on 32GB
- with /set nothink it’s blazing fast
- Token generation speed is much higher than equivalent non-MLX models
- RAM pressure stays in the green/yellow zone during normal use

Limitations:
- 32GB is the hard ceiling — the model itself takes ~20GB, leaving ~12GB for KV cache. Short sessions are fine; long context pushes into swap
- Can't comfortably use it as a backend for agentic frameworks (like OpenClaw) where the context grows large — hits the 32GB wall quickly

Bottom line: great for interactive chat sessions via ollama run, but this way it’s fun just for a session, not useful. For production agentic use or long context, you'd want 48GB+. Looking forward to seeing this on an M5 Max.

rapid advancements do seem to be trickling into the LLMs much faster now.

Yes, but at the same time we're seeing that the hyperbole about AIs abilities is just that, hyperbole. Even the absolute top of the line stuff can't do what people thought would be easy within current generation stuff, and we're seeing limitations as far as training material.

In some contexts everything looks absolutely over the top amazing with that hockey stick curve of progress, but we're also seeing hard limits being hit way before we're even close to where things needs to be for practical everyday use.

As an example we have your positive experience of using it for spam filtering, which we need to put into context that I got way better numbers more than two decades ago with a framework that I built around a very small core based on a bayesian filter, with less computing power than probably your earbuds have today (or even just a USB-C plug).

It's similar to how things were and are with the crypto bros. If you want to see the amazing progress, opportunities, and revolution, it truly is there to be seen. But once you step out of the bubble it won't follow you into everyday life. You can escape it. You can live without it being an essential part of your life. You can get better results without it. Like with your spam filter example; it's more amazing within-bubble, than as a generic, practical, everyday solution.

There's no strict b/w hard line separating that inside and outside of the bubble, but things need to mature to a very high degree before it is a valid strictly better solution also outside the bubble. Which we can see in how most people in the world never ever would benefit from taking the plunge to making bitcoin and AI spamfilters their primary tools within their separate areas.

If you're into new tech the progress is amazing, but in everyday life where things just need to work it stops being about theoretical and occasional peaks, and it starts to become about how many times daily you can accept it failing you by not being good enough.

How long will your custom AI spamfilter feel amazing when you're relying on emails to always work, but that filter always gets one out of every twenty emails wrong?!