A 145-billion-parameter model that runs on one gaming GPU sounded like a stretch a year ago. On 12 November 2026 the Edge0 team open-sourced Edge0-145B and a smaller Edge0-120B, both built to run on a single NVIDIA card with help from system RAM and your SSD.
The headline claims are big: 80+ tokens per second, about 29 GB peak memory, and agent-coding scores that beat Qwen3.8-Flash-Next, the model it is built on. All of those numbers come from Edge0 itself, so this guide separates what is measured from what is marketing.
Below you will find the real hardware needs, which file to download, how to set it up on Windows or Linux, the speed tables by GPU, the licence catch, and how it compares with the other big local models.
- What: Edge0-145B (145B total, 10.12B active per token) and Edge0-120B (8.64B active), sparse MoE models fine-tuned and rebuilt from Qwen3.8-Flash-Next. Released 12 Nov 2026.
- Runs on: one NVIDIA GPU (RTX 20-series or newer) with CUDA 13 on Windows or Linux, using the Edge0 runtime
edge0-infer. Edge0 tested a 16 GiB VRAM + 20 GiB RAM budget. - Download size: 40.81 GiB (Q3_K) or 47.79 GiB (NVFP4) for 145B; 27.98 / 32.96 GiB for 120B.
- Speed (vendor-reported): 69.9 tok/s chat on an RTX 4090, 99.7 tok/s on an RTX 5090 D, 36.8 tok/s at 16 GiB VRAM.
- Licence: free for personal, research and internal use, but running a commercial model service needs a written licence from Edge0.
- Catch: the “29 GB” figure is not the full footprint, Ollama and LM Studio support is not confirmed, and no independent benchmarks exist yet.
What Edge0-145B Is and Why It Matters
Edge0 is the on-device AI team behind the Edge0-35B phone model and the Audio8 TTS family. Its blog post of 12 November calls Edge0-145B “the world’s first open-source on-device AI model with 145 billion parameters”.
The trick is in how the weights are stored. Only about 10B parameters work on each token, so the GPU only needs the “hot” experts. The rest sit in system RAM, and a huge 51.2B n-gram embedding table is memory-mapped from your SSD and read on demand.
Edge0 says the models were designed for local use from the start, not squeezed down after cloud training. The Hugging Face card lists Qwen/Qwen3.8-Flash-Next as the base model, so think of it as a heavy rebuild of Qwen’s MoE for single-GPU machines.
Why local AI fans care
- One GPU, not four: Qwen3.8-Flash-Next at 4-bit needs about 70 GB by Edge0’s own estimate. Edge0-145B targets a 16-24 GB card plus RAM.
- Agent coding focus: the claimed Terminal-Bench 2.1 score of 86.52 is higher than Claude Opus 4.6 Max in Edge0’s table.
- OpenAI and Anthropic APIs locally: the server speaks both formats, so tools like Claude Code-style agents and Cline can point at it.
Specs: 145B vs 120B
Both models share the same 48-layer design. The 120B version has a smaller expert pool, which makes it faster and lighter on RAM.
| Item | Edge0-145B | Edge0-120B |
|---|---|---|
| Total / active parameters | 145B / 10.12B | 120B / 8.64B |
| Experts per layer | 512 (top-10 routed + 1 shared) | 512 (top-10 + 1 shared) |
| Expert pool | 84.6B | 60.4B |
| Layers | 48 (full attention every 4th layer + sparse indexer) | 48 |
| Context window | 256K (int8 KV, 32K resident) | 256K |
| Speculative decoding | 1.26B MTP draft head | 1.26B MTP draft head |
| N-gram embedding (PLE) | 51.2B, fp8, memory-mapped | 51.2B, fp8, memory-mapped |
| Q3_K weights | 40.81 GiB | 27.98 GiB |
| NVFP4 weights | 47.79 GiB | 32.96 GiB |
| Vision | Yes, needs separate strata-vision helper |
Same |
Note: the runtime defaults to a 131,072-token context on Windows and Linux. The 256K figure is the model limit, not the shipping default.
Hardware You Need (and the 29 GB Question)
The Edge0 Chat v0.2.0 release notes give the clearest minimums for the 145B and 120B models on Windows x64:
- GPU: NVIDIA with compute capability 7.5 or higher, so RTX 20, 30, 40 or 50 series.
- VRAM: at least 16 GiB.
- System RAM: at least 20 GiB free for the model.
- Software: CUDA Toolkit 13. Linux follows the same rule.
- Disk: a fast NVMe SSD with room for 41-48 GB of weights.
AMD and Intel GPUs are not supported for these two models. Windows on Arm works only with the RTX Spark N1X chip (sm_121).
What about the “29 GB peak memory” claim?
The launch blog says 29 GB peak. The model card tells a fuller story: with residency on, the 145B Q3_K build used 23.14 GiB VRAM plus 31.28 GiB RAM on an RTX 5090 D. That is about 54 GiB in total, which a Japanese blogger also pointed out after launch.
So 29 GB is roughly the active working set, not everything the process touches. Plan for 16-24 GB of VRAM and 32-64 GB of RAM if you want the speeds in the tables.
| Your machine | Can it run Edge0-145B? | Better pick |
|---|---|---|
| RTX 4090 / 5090, 64 GB RAM | Yes, full speed tier | 145B Q3_K |
| RTX 4080 / 5080 16 GB, 32-64 GB RAM | Yes, in the 16 GiB tier (about 37-45 tok/s per Edge0’s capped test) | 120B Q3_K for more speed |
| 2x RTX 5080 16 GB, 128 GB RAM | Yes; second GPU can act as an extra expert cache | 145B Q3_K |
| RTX 3060 12 GB | Below the official 16 GiB minimum | A smaller Qwen model |
| Mac (Apple Silicon) | Q3_K only via Metal, in the Edge0 Chat Mac build; model card says Mac results “do not establish” release support | Treat as experimental |
| AMD Radeon / Intel Arc | No (CUDA only) | Qwen3.8 GGUF in llama.cpp |
Tip: the 16 GiB tier is an estimate for real 16 GB cards. Edge0 tested it by capping an RTX 5090 D, not on an actual RTX 5080, so your numbers may differ. Our RTX 5090 vs RTX 5080 for AI guide explains the bandwidth gap.
Which File to Download: Q3_K or NVFP4
Edge0 publishes four model folders on Hugging Face under the Edge0 organisation. Pick by GPU generation:
| Folder | Size | Best for |
|---|---|---|
Edge0/Edge0-145B-q3k |
40.81 GiB | Most people: RTX 20-50, and the only build for Mac. Fastest decode in Edge0’s tests. |
Edge0/Edge0-145B-NVFP4 |
47.79 GiB | RTX 50 / Blackwell owners who want FP4 math. CUDA only. Needs more RAM (38.27 GiB). |
Edge0/Edge0-120B-q3k |
27.98 GiB | 16 GB cards and 32 GB RAM machines. |
Edge0/Edge0-120B-NVFP4 |
32.96 GiB | RTX 50 cards, 120B tier. |
Interestingly, Q3_K was faster than NVFP4 on the RTX 5090 D (99.7 vs 79.6 tok/s chat), so most people should start with Q3_K.
Warning: keep the downloaded folder exactly as it is. edge0-infer reads the model type from model.json inside the folder, and renamed or partial folders fail to load.
If you have never pulled a large repo, our guide to downloading models from Hugging Face covers the hf CLI, resuming and cache paths.
How to Run Edge0-145B, Step by Step
There are two routes. The Windows app is easiest; the command line gives you the API server and all memory flags.
Route 1: Edge0 Chat app on Windows
- Install CUDA Toolkit 13. Get it from NVIDIA’s CUDA downloads page and update your GeForce driver first. Without it, the app can fail to start with error
0xC0000135(missing DLL). - Download the app. From the GitHub releases page of
Edge0-AI/Edge0, grabedge0-chat-0.2.0-windows-x64.zip(about 264 MB). It includes a prebuiltedge0-infer.exe. - Check the file. The release lists SHA256
c7146b76d51c7986723b81a3643893e1ccbb09ce4f365a5090dc99e476bb31b9. In PowerShell runGet-FileHash .\edge0-chat-0.2.0-windows-x64.zipand compare. - Download the weights. For example:
hf download Edge0/Edge0-145B-q3k --local-dir D:\models\Edge0-145B-q3k. Weights are not inside the app zip. - Open the folder in Edge0 Chat. Unzip, start the app, and choose the model directory. First load is slow while experts are read from the SSD.
Route 2: command line and local API (Windows or Linux)
- Clone the repo.
git clone https://github.com/Edge0-AI/Edge0.git edge0 - Build the two engines. Follow the “Build (Linux, NVIDIA)” or “Build (Windows, NVIDIA)” section in
edge0-strata/EDGE0.md, then the getting-started guide inedge0-infer/docs. You need bothedge0-strataandedge0-infer. - Check the model folder.
edge0-infer inspect --model /path/to/Edge0-145B-q3k --json - Start the server.
build/cpu/bin/edge0-infer serve --model /path/to/Edge0-145B-q3k(on Windows:build\windows\bin\edge0-infer.exe). Wait for the “Listening” line onhttp://127.0.0.1:8000. - Chat or connect a tool. Run
edge0-infer chat --model /path/to/Edge0-145B-q3k, or point any OpenAI-compatible client athttp://127.0.0.1:8000/v1with model nameedge0_145b. Anthropic-stylePOST /v1/messagesalso works.
A quick test call from a terminal looks like this:
curl http://127.0.0.1:8000/v1/chat/completions -H "Content-Type: application/json" -d '{"model":"edge0_145b","messages":[{"role":"user","content":"Write a Python function that reverses a string."}],"reasoning_effort":"none"}'
Note: only serve, chat and inspect work with these models. The run and bench commands are for Edge0’s older native engine and refuse the 145B and 120B folders.
Windows users can also build the Linux way inside WSL2; our WSL2 local AI setup guide covers CUDA inside WSL.
Memory Flags That Matter on 16 GB Cards
The model card lists the flags that decide where experts live. These are the useful ones:
| Flag | Default | What it does |
|---|---|---|
--vram-reserve-mib N |
700 | Keeps VRAM free for Windows and other apps. Edge0’s 16 GiB test used 8192 on a 24 GB card. Not shown in --help. |
--expert-cache N|auto |
0 | Sizes the GPU expert cache; auto uses free VRAM. |
--expert-cache-device1..3 N|auto |
off | Uses a 2nd, 3rd or 4th GPU as extra expert cache. |
--mmap-experts |
off | Reads uncached experts from SSD on demand instead of RAM. |
--resident-budget-gib 20 |
off | Caps the RAM expert cache. Requires --expert-profile <file>. |
--kv-resident N |
0 | Streams KV cache from pinned host RAM. |
--mtp-q4 all|proj|head |
off | Shrinks the draft head, saving about 0.3-0.5 GiB VRAM. |
--spec N |
4 | Speculative tokens per step from the MTP head. |
--memory-limit-mib N |
0 (auto) | Host RAM limit for the process. |
For a single 16 GB card with 32 GB RAM, a sensible starting line is:
edge0-infer serve --model /path/to/Edge0-120B-q3k --expert-cache auto --vram-reserve-mib 1500 --mtp-q4 all
That is our suggested starting point built from the documented flags, not an Edge0-published recipe. Raise --vram-reserve-mib if your desktop stutters, and switch to the 145B folder once it runs cleanly.
Speed by GPU
All numbers below are vendor-reported from the Edge0 model card (tokens per second). “Agent decode” is faster because the MTP draft head guesses predictable code tokens well.
| Machine | Build | 16K prefill | Chat decode | Agent decode |
|---|---|---|---|---|
| RTX 5090 D 24 GB | 145B Q3_K | 3,554.9 | 99.7 | 121.2 |
| RTX 5090 D 24 GB | 145B NVFP4 | 3,165.7 | 79.6 | 98.3 |
| RTX 4090 24 GB | 145B Q3_K | 3,179.4 | 69.9 | 91.7 |
| RTX Spark N1X 60 GiB | 145B NVFP4 | 922.4 | 40.1 | 46.0 |
| Apple M2 Ultra (Metal) | 145B Q3_K | 800.3 | 60.9 | 57.0 |
| 16 GiB VRAM + 20 GiB RAM cap | 145B Q3_K | 335 | 36.8 | 44.9 |
| 16 GiB VRAM + 20 GiB RAM cap | 120B Q3_K | 778 | 62.4 | 73.3 |
| RTX 4090 24 GB | 120B Q3_K | 3,551.1 | 95.0 | 104.9 |
The big takeaway: on a 16 GB card, Edge0-120B is almost twice as fast as 145B (62.4 vs 36.8 tok/s chat) and prefill is over twice as quick. For long agent prompts, prefill speed matters more than decode.
Benchmarks (Vendor-Reported)
Edge0 ran these scores itself. Rows for Flint, Muse Glimmer and Gemma 4 mix in third-party numbers, and a full technical report with the test setup is still “to be released”.
| Model | DeepSWE 1.1 | Terminal-Bench 2.1 | HLE | GPQA-Diamond | IFBench |
|---|---|---|---|---|---|
| Edge0-145B | 64.6 | 86.52 | 35.4 | 90.4 | 80.6 |
| Edge0-120B | 53.6 | 81.3 | 31.0 | 87.4 | 79.3 |
| Qwen3.8-Flash-Next | 58.7 | 83.3 | 35.9 | 91.7 | 81.3 |
| DeepSeek-V4-Flash-0731 | 54.4 | 82.7 | 33.8 | 90.8 | 79.2 |
| Qwen3.8-27B | 42.2 | 73.0 | 30.8 | 89.2 | 79.5 |
| Claude Opus 4.6 (Max) | n/a | 78.2 | 40.0 | 91.3 | 62.5 |
Read it carefully. Edge0-145B wins the two agent coding tests, while its own base model, Qwen3.8-Flash-Next, still edges it on HLE, GPQA and instruction following. Claude Opus 4.6 is also an older Claude model; current Claude models are not in the table.
Edge0-145B vs Other Big Local Models
| Model | Active params | Memory to run (approx.) | Runtimes | Licence |
|---|---|---|---|---|
| Edge0-145B | 10.12B | 16-24 GB VRAM + 20-38 GB RAM (measured by Edge0) | edge0-infer, Edge0 Chat | Edge0 Community 1.0 (commercial service needs licence) |
| Edge0-120B | 8.64B | 16-23 GB VRAM + 18-23 GB RAM | edge0-infer, Edge0 Chat | Same |
| Qwen3.8-Flash-Next (4-bit) | 6B | ~70 GB total (Edge0 estimate) | llama.cpp, Ollama, LM Studio, vLLM | Qwen Community 1.0 |
| DeepSeek-V4-Flash-0731 (4-bit) | n/a | ~87 GB total (Edge0 estimate) | llama.cpp, vLLM | DeepSeek licence |
| Qwen3.8-27B (4-bit) | 27B dense | ~18 GB (Edge0 estimate) | All major runtimes | Qwen Community 1.0 |
Who should use which
- One 16-24 GB NVIDIA card, coding agents: try Edge0-145B or 120B. It is the only one in this list built for that setup.
- You want Ollama or LM Studio convenience: stay with Qwen3.8 models for now. Our Qwen3.8-Flash-Next local guide covers the 2x 16 GB + 128 GB RAM route.
- AMD, Intel or older Mac: Edge0 is not for you yet. Use llama.cpp; see our llama.cpp 1.0 guide.
- You sell an AI product: check the licence below before building on Edge0.
Licence: What You Can and Can’t Do
This is not Apache 2.0 for the weights. Three licences apply:
- Model weights (Edge0 parts): Edge0 Community Model License 1.0.
- Qwen-derived parts: Qwen Community License 1.0.
- Code: the framework is Apache-2.0 and the Strata engine is MIT.
| Allowed without asking | Needs a separate written Edge0 licence |
|---|---|
| Personal use and research | Selling the model or bundling it in a paid product |
| Modifying and fine-tuning | Running a commercial model service (paid API) |
| Non-commercial sharing | A standalone commercial coding or office assistant built on it |
| Internal use at your company | |
| Commercial use of the outputs you generate |
Warning: the Hugging Face page metadata shows “5B params” and architecture “qwen4exp”, which clashes with the card text. Trust the card body and the GitHub docs, and read the full licence file before any business use.
Known Problems and Fixes
- Empty replies: with thinking on, the model can spend the whole token budget reasoning and return nothing. Add
"reasoning_effort": "none"to the request, or raise max tokens. - “this server was started without the vision encoder”: image input needs the separate
strata-visionhelper, which is not in the main repo yet. - App won’t start, exit code 0xC0000135: the CUDA 13 runtime DLL is missing. Install CUDA Toolkit 13 and reboot.
- Error with
--resident-budget-gib: this flag needs a static--expert-profilefile. - Slow first answer: experts load from SSD on the first request. Later requests are faster once the OS page cache is warm. An NVMe drive helps a lot.
- Converter fails on Windows with WinError 1314: a community user hit this with the older asset converter, which uses symlinks. Turn on Windows Developer Mode or run as admin.
- Requests seem stuck: the server queues concurrent requests instead of rejecting them, so a second client waits for the first to finish.
- Different answers at temperature 0: reported in coverage of the runtime guide; do not expect bit-exact repeats.
- Out-of-memory crashes: raise
--vram-reserve-mib, use--mtp-q4 all, or drop to Edge0-120B. Our CUDA out of memory guide has general checks.
Ollama and LM Studio: the Hugging Face page shows generic “Use this model” snippets for llama.cpp, Ollama, vLLM and LM Studio with a :BF16 tag that does not exist in these repos. Edge0’s own docs only document edge0-infer, so treat those snippets as unconfirmed.
My Take
This is the most interesting local model release of the month for people with one good NVIDIA card. The architecture is clever: a tiny active set, experts spread across VRAM, RAM and SSD, and a draft head for speed.
The detailed per-GPU tables and the honest “test budget, not minimum” notes are better than most launches.
But I would not call it proven. Every speed and score is Edge0’s own, the “29 GB” headline hides a ~54 GiB real footprint, and macOS support is muddled between the blog and the docs. The agent wins over Qwen3.8-Flash-Next are also small on some tests.
If you have an RTX 4090, 5090 or a 16 GB card with 64 GB RAM, it is worth an evening to test on your own coding tasks. Start with Edge0-120B Q3_K on 16 GB cards. If you need Ollama-style ease or a commercial product, wait for the technical report and wider runtime support.
FAQ
Is Edge0-145B free?
Yes, the weights are free to download for personal, research and internal use. A commercial model service or paid assistant built on it needs a written licence from Edge0.
Can Edge0-145B run on a 16 GB GPU?
Edge0 lists 16 GiB VRAM and 20 GiB RAM as the minimum and reports 36.8 tok/s in a capped 16 GiB test. That test used a limited RTX 5090 D, so a real 16 GB card may be slower.
Does it work in Ollama or LM Studio?
Not officially yet. Edge0’s documentation only covers its own edge0-infer runtime and the Edge0 Chat app.
Does it run on a Mac?
The Edge0 Chat v0.2.0 Mac build supports the Q3_K folders through Metal on macOS 26 or later. The model card says its M2 Ultra results do not confirm release support, so treat it as experimental.
Should I pick 145B or 120B?
Pick 145B on 24 GB cards for the best scores. On 16 GB cards, 120B is nearly twice as fast and needs less RAM.
Is it better than Claude or GPT for coding?
Edge0’s table puts it ahead of Claude Opus 4.6 Max on Terminal-Bench 2.1, but newer cloud models are not compared and no independent tests exist yet.
Does it support AMD GPUs?
No. The 145B and 120B models need an NVIDIA GPU with CUDA 13. Only the older Edge0-35B runs on Vulkan or CPU.
What is the context length?
The model supports 256K tokens, but the runtime defaults to 131,072 tokens on Windows and Linux and 32,768 on macOS.
Sources
- Edge0 blog, “Introducing Edge0-145B: Frontier AI Capabilities on Your Own Computer” (12 Nov 2026)
- Hugging Face model cards: Edge0/Edge0-145B-q3k and Edge0/Edge0-145B-NVFP4 (checked 16 Nov 2026)
- GitHub, Edge0-AI/Edge0: docs/models/edge0-145b.md and Release 0.2.0 (Edge0 Chat, 11 Nov 2026)
- WindowsForum, “Edge0-145B on Windows: Local AI Hardware Requirements, RTX 5090 Results and Deployment Limits” (Nov 2026)
- Jan Sverre, “Edge0-145B on RTX 4090: speed and hardware requirements” (Nov 2026)
- DEV Community (qianchuan), “Everyone Is Benchmarking Edge0-145B on RTX 4090s” (Nov 2026)
- FreeAnswers, “Edge0-145B: Local Setup, Hardware, Benchmarks and License” (Nov 2026)




