Thinking Machines has shrunk its Inkling 2 family down to a size you can run at home. Inkling 2 Mini landed on Hugging Face on 20 November 2026 as a free, Apache 2.0 open-weight model with 26B parameters and only 3.4B active per token.
It reads text and images, has a 1M-token context window, and ties Qwen3.8-27B on the Artificial Analysis Intelligence Index while beating it on many coding and agent tests. The single checkpoint is about 20 GB, and GGUF builds start under 8 GB.
This guide covers what makes it different (it “loops” its middle layers), the benchmarks, how much VRAM you need, and step-by-step setup in Ollama, llama.cpp, LM Studio and vLLM.
- What: Inkling 2 Mini, an open-weight multimodal (text + image in, text out) looped MoE model from Thinking Machines Lab.
- Released: 20 Nov 2026, Apache 2.0 licence.
- Size: 26.4B total, 3.4B active (about 5.4B effective with looping), 1M context.
- Scores: AA Intelligence Index 20.1 (ties Qwen3.8-27B), SWE-bench Verified 79.2%, DeepSWE 48.2%.
- Hardware: official advice is 32 GB+ VRAM or a 32 GB+ Apple Silicon Mac. Unsloth Q4_K_M GGUF is 16.6 GB.
- Fastest start:
ollama run hf.co/unsloth/Inkling-2-Mini-GGUF:UD-Q4_K_M
What Is Inkling 2 Mini?
Inkling 2 Mini is the smallest member of Thinking Machines Lab’s Inkling 2 family. Its big sibling, Inkling 2 Small (594B), needs a server rack. Mini is built for a single workstation GPU, a DGX Spark or a Mac.
| Spec | Inkling 2 Mini |
|---|---|
| Developer | Thinking Machines Lab |
| Release date | 20 November 2026 |
| Licence | Apache 2.0 (plus an acceptable use policy) |
| Total parameters | 26.4B |
| Active per token | 3.4B (about 5.4B effective with 3 loops) |
| Architecture | 24-layer looped MoE, 128 experts, 8 routed per token, hybrid local/global attention |
| Context window | Up to 1M tokens |
| Inputs / output | Text and images in, text out |
| Official weights | thinkingmachines/Inkling-2-Mini (NVFP4 + MXFP8, about 20 GB) |
| GGUF | unsloth/Inkling-2-Mini-GGUF (7.8 GB to 22.9 GB) |
You can also try it without downloading anything in the Tinker Playground, Thinking Machines’ own web console. No third-party inference provider hosted it at launch.
How the Looped Design Works
Most models pass each token through every layer once. Inkling 2 Mini has 24 layers of weights, but the middle 12 run three times per token with the same weights. That gives 48 layer passes from 24 layers’ worth of memory.
- Upside: more “thinking depth” per token without a bigger download.
- Downside: each token costs more compute than the 3.4B active count suggests, so it runs slower than a plain 3B-active MoE.
- Local support: looped models are new to local runtimes, and Thinking Machines warns that speed varies by tool.
Note: Thinking Machines also released a research “variable-loop” variant. Stick with the standard model unless you are experimenting.
Benchmarks vs Qwen3.8-27B and Gemma 4
These are Thinking Machines’ own published numbers from the model card, so treat them as vendor-reported until independent tests arrive.
| Benchmark | Inkling 2 Mini | Qwen3.8-27B | Gemma 4 31B |
|---|---|---|---|
| AA Intelligence Index v5.0 | 20.1 | 20.1 | 4.4 |
| SWE-bench Verified | 79.2% | 77.8% | 46.1% |
| DeepSWE v1.1 | 48.2% | 42.2% | 0.3% |
| Terminal Bench v2.1 | 71.6% | 73.0% | 43.4% |
| Vibe Code Bench v1.1 | 77.9% | 64.9% | n/a |
| BrowseComp (compaction) | 79.0% | 61.6% | 37.0% |
| MCP Atlas | 82.7% | 77.3% | 54.2% |
| MMMU Pro (vision) | 74.9% | 76.3% | 73.4% |
| ScreenSpot Pro | 76.5% | 57.6% | 5.9% |
| LongBench v2 | 66.9% | 52.3% | 52.9% |
| HLE-Diamond | 15.2% | 15.3% | 7.2% |
| Destructive actions (lower is better) | 10.2% | 35.2% | 75.8% |
The pattern is clear: Mini matches Qwen3.8-27B overall, wins on agentic coding, browsing, long context and screen understanding, and trails slightly on vision Q&A and multi-turn chat.
Tip: the low “destructive actions” score makes it an interesting pick for local coding agents. Still, keep confirmations on.
How Much VRAM You Need
Thinking Machines officially supports one NVIDIA DGX Spark, Apple Silicon Macs with 32 GB+ unified memory, and other GPUs with 32 GB+ VRAM. Smaller cards can run Unsloth’s GGUF quants. Sizes below are file sizes; leave extra room for context (KV cache).
| Quant (Unsloth) | File size | Quality note | Good fit |
|---|---|---|---|
| UD-IQ1_S | 7.76 GB | Big quality loss | Testing only |
| UD-Q2_K_XL | 9.14 GB | 91.6% same top token vs Q8 | 12 GB GPUs |
| UD-IQ3_XXS | 11.7 GB | Usable | 16 GB GPUs (short context) |
| UD-Q3_K_XL | 13.6 GB | Good | 16 GB GPUs (tight) |
| UD-Q4_K_M | 16.6 GB | 97.9% same top token vs Q8 | 24 GB GPUs, 32 GB Macs |
| UD-Q4_K_XL | 17.6 GB | A little better than Q4_K_M | 24 GB+ GPUs |
| UD-Q8_K_XL | 22.9 GB | Near original | 32 GB GPUs, DGX Spark |
| Official NVFP4/MXFP8 | ~20 GB | Original weights | RTX 50 series 32 GB, DGX Spark (vLLM/SGLang) |
Not sure what your card can hold? See our VRAM guide and DGX Spark guide.
Run It in Ollama
- Install or update Ollama to the latest version (looped-model support is new).
- Open a terminal and run:
ollama run hf.co/unsloth/Inkling-2-Mini-GGUF:UD-Q4_K_M - On a 16 GB GPU, swap the tag for
UD-Q3_K_XLorUD-IQ3_XXS. - To attach an image, drag it into the terminal or pass the file path in your prompt.
- Raise the context only as far as your memory allows. Our Ollama context length guide shows how.
Run It in llama.cpp
- Update to the latest llama.cpp build (see our llama.cpp 1.0 guide).
- Start an OpenAI-compatible server straight from Hugging Face:
llama-server -hf unsloth/Inkling-2-Mini-GGUF:UD-Q4_K_M -ngl 99 -c 32768 - Open
http://localhost:8080for the web UI, or point any OpenAI client athttp://localhost:8080/v1. - Newer builds also accept the single-command form shown on the Unsloth card:
llama serve -hf unsloth/Inkling-2-Mini-GGUF:UD-Q4_K_M.
Our llama-server guide covers flags, API keys and multi-user setups.
Run It in LM Studio
- Update LM Studio and its llama.cpp runtime (Settings > Runtime).
- Open the Discover tab and search Inkling-2-Mini.
- Pick the Unsloth GGUF and a quant that fits your VRAM (green tick).
- Load it, set the context length, and start chatting. Drop an image into the chat to test vision.
Note: if the model fails to load, your runtime is probably too old for looped models. See LM Studio not loading models.
Run It in vLLM or SGLang
For the official NVFP4/MXFP8 weights on a 32 GB RTX 50-series card or DGX Spark:
- vLLM:
pip install -U vllmthenvllm serve "thinkingmachines/Inkling-2-Mini". The API runs athttp://localhost:8000/v1. - SGLang:
pip install -U sglangthenpython3 -m sglang.launch_server --model-path "thinkingmachines/Inkling-2-Mini" --host 0.0.0.0 --port 30000. - Docker Model Runner:
docker model run hf.co/thinkingmachines/Inkling-2-Mini.
Known Limitations
Thinking Machines lists these on the model card:
- Weaker factual recall than Inkling 2 Small, and more often confidently wrong.
- Can get stuck in long reasoning loops, especially when debugging code. Set output-token and agent-turn limits.
- Less reliable at tool use and following project conventions; may misreport results.
- Weaker outside English.
- Minimum reasoning effort does not switch reasoning off completely.
- Can be overly cautious on mental-health topics.
Troubleshooting
| Problem | Likely cause | Fix |
|---|---|---|
| “Unknown architecture” error | Old Ollama, llama.cpp or LM Studio runtime | Update to the latest version |
| Out of memory on load | Quant too big for your VRAM | Drop one quant level or lower context |
| Slower than expected | Looping triples middle-layer compute | Normal; use a smaller context and full GPU offload |
| Endless thinking | Known reasoning-loop issue | Set a max output token limit |
| Image input ignored | Vision projector not loaded | Use the Unsloth repo via -hf, which pulls the mmproj file |
| vLLM fails on older GPU | NVFP4 needs Blackwell-class hardware | Use a GGUF build in llama.cpp instead |
My Take
This is one of the most exciting local models of the month. Matching Qwen3.8-27B with a 20 GB checkpoint, Apache 2.0 licensing and real vision support is a strong package.
If you have a 24 GB card or a 32 GB Mac, grab the Q4_K_XL build today. On 16 GB, the Q3 quants are worth a try for coding help, but expect tight context. Wait a week for runtime updates if speed disappoints.
FAQ
When was Inkling 2 Mini released?
Thinking Machines released it on Hugging Face on 20 November 2026.
Is Inkling 2 Mini free for commercial use?
Yes. It uses the Apache 2.0 licence, plus Thinking Machines’ acceptable use policy.
How much VRAM does Inkling 2 Mini need?
Officially 32 GB+, but the Unsloth Q4_K_M GGUF is 16.6 GB and fits a 24 GB GPU. Q3 quants (11.7 to 13.6 GB) can run on 16 GB cards with short context.
Can Inkling 2 Mini see images?
Yes. It accepts text and images as input and replies in text.
Is it better than Qwen3.8-27B?
They tie on the AA Intelligence Index (20.1). Mini wins on most coding, agent and long-context tests; Qwen is slightly better at chat and some vision Q&A.
Does it run in Ollama?
Yes: ollama run hf.co/unsloth/Inkling-2-Mini-GGUF:UD-Q4_K_M on an up-to-date Ollama.
What is a looped transformer?
A model that runs some layers more than once per token with shared weights, adding depth without adding download size.
What context length does it support?
Up to 1M tokens, though local memory limits how much you can actually use.
Sources
- Hugging Face: thinkingmachines/Inkling-2-Mini model card (release 20 November 2026)
- Hugging Face: unsloth/Inkling-2-Mini-GGUF model card
- TestingCatalog: “Thinking Machines releases Inkling 2 Mini open-weight model”, 20 November 2026



