MLX vs GGUF (2026): Which Format Is Faster on a Mac, and Which Should You Download?

If you run local AI models on a Mac, you have probably seen the same model offered twice: once as a GGUF file and once as an MLX folder. LM Studio shows both, Ollama now picks between them quietly in the background, and Hugging Face has thousands of each. So which should you download? In this guide I compare MLX vs GGUF properly: what each format is, real published speed numbers (including the catch that makes MLX look faster than it feels), memory use, quality at the same bit size, which apps run which format, how to run and convert both, and fixes for the problems people hit most. By the end you should not need another tab open.

A quick note on where I stand: my own rig is a Windows PC with two RTX 5080s, so GGUF is my everyday format there. For Mac numbers I do not invent results. Everything below comes from the benchmarks and release notes linked in the Sources section, and anything that is my estimate is labelled as one.

What Is GGUF?

GGUF is the model file format of llama.cpp, the C/C++ inference engine that sits under a huge part of the local AI world. One .gguf file holds everything: the weights, the tokenizer, the chat template and the model settings. You download one file (or a few numbered parts for very large models) and any GGUF app can load it.

What makes GGUF popular:

• It runs almost anywhere. NVIDIA (CUDA), AMD (ROCm and Vulkan), Intel GPUs, Apple Silicon (Metal), and plain CPUs, on Windows, Linux and macOS, including old Intel Macs.
• Lots of quantization choices. Q2_K up to Q8_0, plus K-quants such as Q4_K_M and “IQ” i-quants, so you can match a file to your exact memory. My GGUF quantization guide explains every name.
• GPU + CPU splitting. llama.cpp can put some layers (or MoE experts) on the GPU and the rest in system RAM, which is how PC users run models bigger than their VRAM.
• Huge app support. Ollama, LM Studio, llama.cpp itself, Jan, KoboldCpp, GPT4All and many more read GGUF.

On a Mac, GGUF runs through llama.cpp’s Metal backend. It works well, but it was not designed around Apple’s hardware first, which is where MLX comes in.

What Is MLX?

MLX is an open source array and machine learning framework from Apple’s machine learning research team, built specifically for Apple Silicon. The part most people use is mlx-lm, a Python package for running and fine-tuning language models with MLX (version 0.32.2 was released on 16 October 2026).

An MLX model is not one file. It is a folder with config.json, tokenizer files and one or more .safetensors weight files, usually already quantized to 4-bit or 8-bit. Most ready-made ones live under the mlx-community organisation on Hugging Face, with names that end in the bit size, such as mlx-community/<model-name>-4bit or -8bit.

Why MLX can be faster on a Mac:

• Unified memory first. The CPU and GPU share the same memory, and MLX is designed around that, with no copying between them. Token generation is mostly limited by memory bandwidth, so this matters.
• M5 Neural Accelerators. The M5 family added neural accelerators inside the GPU cores. Apple’s own research reported up to about 4x faster time-to-first-token on M5 than M4 with MLX for a Qwen3 14B 4-bit model, and about 1.19x faster generation.
• Fast updates for new models. New architectures often reach mlx-lm within days, and Apple engineers contribute directly.

The trade-off: MLX models are for Apple Silicon. Intel Macs cannot run them, and on Windows and Linux GGUF is still the practical choice for home users.

MLX vs GGUF: Side-by-Side Table

Speed: What the Benchmarks Really Show

Most “MLX is 2x faster” posts only look at one number: tokens per second while the answer streams. That is half the story. Every reply has two phases:

1. Prompt processing (prefill): the model reads your whole prompt, chat history and any attached documents before it writes anything. This decides how long you wait for the first word.
2. Token generation (decode): the model writes the answer, one token at a time. This is the tok/s figure apps show.

Here are published results from the sources at the end of this article. They come from different people with different setups, so compare rows, not columns across rows.

What this tells us:

• Generation is clearly faster on MLX, especially for MoE models. Part of the big 3x gap in the first two rows came from Ollama’s old wrapper; against raw llama.cpp the estimated MLX lead is about 1.4-1.8x, with dense models nearer 1.4-1.6x.
• Prompt processing is MLX’s weak spot in some apps. In the M1 Max test, prefill was 94% of MLX’s total time on a long prompt, so the screen said 57 tok/s while the real experience was as low as 3 tok/s. GGUF won all five document classification tasks there.
• Prompt caching fixes most of it. oMLX keeps the KV cache in RAM and on SSD, so a repeated prefix (the same system prompt, chat history or document) does not need to be processed again. That turned a 49-second wait into 1.7 seconds.
• Long context can flip the result. Community tests on an M3 Ultra at 30K+ tokens found MLX generation about 50% slower than llama.cpp with Flash Attention. Results vary a lot by model.
• M1 and M2 are special cases. They have no native bf16, so bf16 weights are emulated and prefill is slower. M5 chips gain the most from MLX thanks to the neural accelerators.

The practical rule: MLX is best when the answer is long compared with the prompt (writing, summaries, coding, chat). GGUF is often better when the prompt is long and the answer is short (classification, RAG over big documents, tool calls), unless your MLX app caches prompts.

Memory Use and Quality at the Same Size

Both formats shrink models by storing weights in fewer bits. The names differ, but they map roughly like this:

On quality, one detailed MLX study measured perplexity (lower is better) on a test text: bf16 about 8.5, 8-bit essentially the same as bf16, standard 4-bit “over 0.5 ppl” worse, and 3-bit 12.6. 2-bit broke the model completely. Learned DWQ quants and mixed-bit “dynamic” quants closed much of the 4-bit gap (8.64-8.75 at about 5 bits per weight). This matches what GGUF users see with Q4_K_M versus Q8_0: 4-bit is fine for chat, and 6-8 bit is safer for code and exact facts.

On memory, published tests found MLX used a bit less for the same model: Qwen3-Coder-30B-A3B took 34.7 GB on MLX vs 40 GB on GGUF (13% less), and Qwen3-235B-A22B took 124 GB vs 133 GB (7% less). Do not forget the KV cache: long chats add several GB on top of the weights in both formats.

Which Apps Run MLX and Which Run GGUF

How Ollama chooses MLX or GGUF

Ollama 0.19 (31 March 2026) added an MLX preview for one model. Since Ollama 0.40 (25 September 2026) the release notes say model architectures supported by the MLX runtime “will automatically run on MLX” on Apple Silicon. Each model in the library can publish several variants; on a Mac, Ollama prefers the MLX variant, then llama.cpp, then the older ggml runner. On Windows and Linux it prefers llama.cpp. Models with MLX variants at 0.40 included qwen3.8, gemma4, qwen3.6 and qwen3.5, and later 0.40.x releases added more (for example LFM3 and d1 in 0.40.12, which also sped up MLX prompt processing for Gemma 4).

To check what a running model uses:

ollama ps

The RUNNER column shows mlx or llamacpp. If an older download still shows llamacpp, remove it and pull again so Ollama fetches the MLX variant:

ollama rm gemma4
ollama pull gemma4

Ollama also has a hidden --runner option (mlx, llamacpp or ggml) on ollama pull and ollama run. It is undocumented and may change, so only use it for testing. New to Ollama? Start with my Ollama install guide and the Ollama commands cheat sheet.

How to pick MLX or GGUF in LM Studio

On an Apple Silicon Mac, LM Studio’s model search lists both formats and marks each result as GGUF or MLX. Pick the MLX build of the same model and quant size to compare, then load each one and run the same prompt. If a long-prompt chat feels slow on MLX, raise the prefill chunk size in the model’s load settings (the famstack test found LM Studio’s MLX default of 512 too low; 8192 gave up to 1.5x faster prefill) or switch that model to GGUF. My Ollama vs LM Studio vs llama.cpp comparison covers the apps in more detail.

How to Run MLX Models (Step by Step)

You need an Apple Silicon Mac, Python 3.11-3.13 and, for the large-model memory setting below, macOS 15 or newer.

Option 1: Ollama (easiest)

Install Ollama for macOS from ollama.com/download, then run a model that has an MLX variant:

ollama run qwen3.8
ollama ps

Confirm mlx in the RUNNER column. Nothing else to set up.

Option 2: mlx-lm in Terminal (most control)

Create a virtual environment so mlx-lm does not clash with other Python tools:

python3 -m venv ~/mlx-env
source ~/mlx-env/bin/activate
pip install -U mlx-lm

Generate text. The first run downloads the model from Hugging Face into ~/.cache/huggingface/hub/:

mlx_lm.generate --model mlx-community/Mistral-7B-Instruct-v0.3-4bit --prompt "Explain unified memory in two sentences"

Chat interactively (the session keeps context until you exit):

mlx_lm.chat --model mlx-community/Mistral-7B-Instruct-v0.3-4bit

Start an OpenAI-compatible local server for other apps:

mlx_lm.server --model mlx-community/Mistral-7B-Instruct-v0.3-4bit --port 8080

Use it from Python:

from mlx_lm import load, generate

model, tokenizer = load("mlx-community/Mistral-7B-Instruct-v0.3-4bit")
messages = [{"role": "user", "content": "Write a haiku about RAM"}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
text = generate(model, tokenizer, prompt=prompt, verbose=True)

Swap the model name for any repo from mlx-community. Run mlx_lm.generate --help to see sampling flags such as temperature and max tokens.

Option 3: oMLX (fastest for long chats and agents)

oMLX is an Apache 2.0 MLX server with a macOS menu-bar app, continuous batching and a two-tier prompt cache (hot in RAM, cold on SSD, restored even after a restart). It needs macOS 15+ and Python 3.11-3.13. Install the .dmg from its GitHub releases, or with Homebrew:

brew tap jundot/omlx https://github.com/jundot/omlx
brew install jundot/omlx/omlx

Add --with-custom-kernel to the install line if you have full Xcode installed; some model families need those kernels for full speed. The Homebrew service reads models from ~/.omlx/models (one subfolder per MLX model). The API runs at http://localhost:8000/v1 with OpenAI routes such as /v1/chat/completions and the Anthropic route /v1/messages, and there is a chat page at http://localhost:8000/admin/chat.

Let large MLX models use more memory

macOS keeps part of unified memory away from the GPU. If mlx-lm warns that a model needs more memory than is available, you can raise the GPU “wired” limit (macOS 15+). The value is in MB and should be bigger than the model but smaller than your total RAM. For example, on a 64 GB Mac:

sudo sysctl iogpu.wired_limit_mb=56000

This resets on reboot. Leave several GB for macOS and your apps, or the system will start swapping.

How to Run GGUF Models on a Mac

GGUF is still worth having on a Mac: some models only ship as GGUF, some apps only read GGUF, and for long-prompt, short-answer jobs it can respond sooner. Three ways to run it:

• Ollama: models without an MLX variant run on llama.cpp automatically. To use a GGUF straight from Hugging Face: ollama run hf.co/<user>/<repo>-GGUF:Q4_K_M.
• LM Studio: pick the GGUF result in model search. Files go to ~/.lmstudio/models/.
• llama.cpp: install with Homebrew, then download and serve a model in one command (recent builds use the unified llama command):

brew install llama.cpp
llama serve -hf <user>/<repo>-GGUF:Q4_K_M --ctx-size 16384

Open http://localhost:8080 for the built-in chat page. On Apple Silicon, Metal is used automatically and all layers go to the GPU. If you have an older build that only has llama-server, use that name instead of llama serve.

Which GGUF size to download? Start with Q4_K_M, go to Q5_K_M or Q6_K if you have room, and read my GGUF quantization explained guide for the full list.

How to Convert a Model to MLX

If the model you want has no MLX version yet, mlx-lm can make one from the original Hugging Face weights (the safetensors release, not a GGUF). This quantizes to 4-bit by default:

mlx_lm.convert --model mistralai/Mistral-7B-Instruct-v0.3 -q

The result is saved in a local mlx_model folder, which you can pass to --model in any mlx-lm command. Useful extras:

• --q-bits 8 for an 8-bit version, and --q-group-size 32 for slightly better quality at a little more memory (64 is the default).
• --mlx-path ./my-model-4bit to choose the output folder.
• --upload-repo your-name/my-model-4bit to push the result to your Hugging Face account (log in first with hf auth login).
• Run mlx_lm.convert --help for every option your installed version supports.

Conversion loads the full model, so you need enough RAM for the unquantized weights, roughly 2 GB per billion parameters for bf16 (my estimate from the bf16 size; a 7B model needs about 14-15 GB). Gated models such as some Llama and Gemma repos need you to accept the licence on Hugging Face first.

Going the other way, from Hugging Face weights to GGUF, uses llama.cpp’s convert_hf_to_gguf.py and then llama-quantize. For popular models it is much easier to download a ready-made GGUF from unsloth, bartowski or ggml-org.

Bonus: fine-tune with MLX

One thing GGUF cannot do is training. mlx-lm includes LoRA and QLoRA fine-tuning on Apple Silicon with mlx_lm.lora. You point it at a folder with train.jsonl and valid.jsonl:

mlx_lm.lora --model mlx-community/Mistral-7B-Instruct-v0.3-4bit --train --data ./data --iters 600

The adapter is saved to ./adapters; check the LoRA guide in the mlx-lm repository for data formats and fusing the adapter into the model.

Which Format for Your Mac (by RAM)

macOS lets the GPU use most, but not all, of unified memory, and you need room for the KV cache and your apps. The model sizes below are my estimates from 4-bit file sizes (about 0.55-0.6 GB per billion parameters), so treat them as a starting point.

For model picks by size, see my best Ollama models by VRAM list and the Gemma 4 local guide. Still deciding on hardware? My Mac vs PC for local AI comparison covers M5 Max and Ultra against RTX cards.

Troubleshooting Common Problems

MLX shows high tok/s but replies feel slow

That is prompt processing. Keep long system prompts and documents stable so they can be cached, use an app with prompt caching (oMLX, recent Ollama), raise the prefill chunk size in LM Studio, or --prefill-step-size in mlx_lm.server (default 2048). For one-off long documents with short answers, try the GGUF version.

Ollama still shows llamacpp in ollama ps

Update Ollama to 0.40 or newer, then ollama rm and ollama pull the model again. If it still shows llamacpp, that model or tag simply has no MLX variant yet, and Ollama falls back to GGUF automatically. Imported GGUF files (hf.co/... or a Modelfile with FROM ./model.gguf) always run on llama.cpp.

“Model requires more memory than available” or heavy swapping

Pick a smaller quant (4-bit instead of 8-bit), lower the context length, close memory-hungry apps, or raise iogpu.wired_limit_mb as shown above. If Activity Monitor shows memory pressure in red, the model is too big for comfortable use.

mlx-lm says “Model type … not supported”

The architecture is newer than your mlx-lm. Run pip install -U mlx-lm inside your virtual environment. Very new models can take a few days to be added.

No MLX models in LM Studio or Ollama

MLX needs an Apple Silicon Mac. On an Intel Mac, Windows or Linux you will only see GGUF, which is expected. Use GGUF there; it is the right format for those machines.

Output is gibberish or the model never stops

Usually a broken or very low quant (2-bit, some 3-bit) or a missing chat template. Re-download a 4-bit or higher build from mlx-community or a trusted GGUF uploader, and update the app so it has the model’s template.

Frequently Asked Questions

Is MLX faster than GGUF?

On Apple Silicon, MLX usually generates tokens faster, about 1.4-1.8x over raw llama.cpp in published tests and more on MoE models. GGUF can still reply sooner on long prompts with short answers, unless the MLX app caches prompts.

Can I use MLX models on Windows or an NVIDIA GPU?

Not in practice. MLX is built for Apple Silicon Macs. On Windows, Linux and NVIDIA or AMD GPUs, use GGUF with Ollama, LM Studio or llama.cpp.

Does Ollama use MLX?

Yes. Since Ollama 0.40 (25 September 2026), supported models run on MLX automatically on Apple Silicon Macs. Run ollama ps and check the RUNNER column to see mlx or llamacpp.

Is MLX 4-bit the same quality as GGUF Q4_K_M?

They are close. MLX 4-bit with group size 64 is about 4.5 bits per weight and Q4_K_M is about 4.8, so Q4_K_M can be marginally better. MLX DWQ 4-bit quants narrow the gap. For code and exact facts, use 6-bit or 8-bit in either format.

Can I convert a GGUF file to MLX?

It is better not to. Convert from the original Hugging Face safetensors weights with mlx_lm.convert, because a GGUF is already quantized and converting it again loses quality.

Which uses less memory, MLX or GGUF?

MLX used 7-13% less memory for the same model in published tests, for example 34.7 GB vs 40 GB for Qwen3-Coder-30B-A3B. Context length adds KV cache memory on top in both formats.

Should I delete my GGUF files if I switch to MLX?

Only for models you now run on MLX. Keep GGUF for models without an MLX build and for apps that only read GGUF. My guide on deleting Ollama models shows how to free the space safely.

Final Thoughts

MLX vs GGUF is not a fight with one winner. On an Apple Silicon Mac, MLX is now the better default for most chat, writing and coding, and Ollama 0.40 makes it automatic for supported models. GGUF remains the universal format: it is the only option on Windows, Linux and Intel Macs, it has the widest app support, and it can still be quicker for long-prompt, short-answer jobs. My advice: on a Mac, try the MLX build first, measure the time to the first word rather than the tok/s number, and keep GGUF for everything else.

Next steps: set up Ollama on your Mac, compare apps in Ollama vs LM Studio vs llama.cpp, pick a model from the best Ollama models list, and if you need space, delete old Ollama models.

Sources

• ml-explore/mlx-lm on GitHub (commands, memory wiring, convert)
• mlx-lm on PyPI (version 0.32.2, 16 Oct 2026)
• Jacar: Ollama 0.40 and MLX by default on Apple Silicon
• gingter.org: Ollama goes MLX (0.19 preview, speed figures)
• Ollama release notes, Oct-Nov 2026 (0.40.9-0.40.15)
• yage.ai: MLX vs llama.cpp benchmarks and M5 Neural Accelerators
• famstack.dev: GGUF vs MLX on Apple Silicon (prefill and oMLX tests)
• An Examination of MLX Quantization (perplexity, DWQ)
• jundot/omlx on GitHub
• LM Studio Bionic changelog (oMLX engine in 1.2.0)
• 9to5Mac: LM Studio launches Bionic