Skip to content
, ,

Inkling 2 Mini: How to Run Thinking Machines’ Open 26B Model Locally

6–9 minutes
Woman at a gaming PC holding a tablet with an INKLING card, hook Free 26B model runs at home

Thinking Machines has shrunk its Inkling 2 family down to a size you can run at home. Inkling 2 Mini landed on Hugging Face on 20 November 2026 as a free, Apache 2.0 open-weight model with 26B parameters and only 3.4B active per token.

It reads text and images, has a 1M-token context window, and ties Qwen3.8-27B on the Artificial Analysis Intelligence Index while beating it on many coding and agent tests. The single checkpoint is about 20 GB, and GGUF builds start under 8 GB.

This guide covers what makes it different (it “loops” its middle layers), the benchmarks, how much VRAM you need, and step-by-step setup in Ollama, llama.cpp, LM Studio and vLLM.

TL;DR

  • What: Inkling 2 Mini, an open-weight multimodal (text + image in, text out) looped MoE model from Thinking Machines Lab.
  • Released: 20 Nov 2026, Apache 2.0 licence.
  • Size: 26.4B total, 3.4B active (about 5.4B effective with looping), 1M context.
  • Scores: AA Intelligence Index 20.1 (ties Qwen3.8-27B), SWE-bench Verified 79.2%, DeepSWE 48.2%.
  • Hardware: official advice is 32 GB+ VRAM or a 32 GB+ Apple Silicon Mac. Unsloth Q4_K_M GGUF is 16.6 GB.
  • Fastest start: ollama run hf.co/unsloth/Inkling-2-Mini-GGUF:UD-Q4_K_M

What Is Inkling 2 Mini?

Inkling 2 Mini is the smallest member of Thinking Machines Lab’s Inkling 2 family. Its big sibling, Inkling 2 Small (594B), needs a server rack. Mini is built for a single workstation GPU, a DGX Spark or a Mac.

Spec Inkling 2 Mini
Developer Thinking Machines Lab
Release date 20 November 2026
Licence Apache 2.0 (plus an acceptable use policy)
Total parameters 26.4B
Active per token 3.4B (about 5.4B effective with 3 loops)
Architecture 24-layer looped MoE, 128 experts, 8 routed per token, hybrid local/global attention
Context window Up to 1M tokens
Inputs / output Text and images in, text out
Official weights thinkingmachines/Inkling-2-Mini (NVFP4 + MXFP8, about 20 GB)
GGUF unsloth/Inkling-2-Mini-GGUF (7.8 GB to 22.9 GB)

You can also try it without downloading anything in the Tinker Playground, Thinking Machines’ own web console. No third-party inference provider hosted it at launch.

How the Looped Design Works

Most models pass each token through every layer once. Inkling 2 Mini has 24 layers of weights, but the middle 12 run three times per token with the same weights. That gives 48 layer passes from 24 layers’ worth of memory.

  • Upside: more “thinking depth” per token without a bigger download.
  • Downside: each token costs more compute than the 3.4B active count suggests, so it runs slower than a plain 3B-active MoE.
  • Local support: looped models are new to local runtimes, and Thinking Machines warns that speed varies by tool.

Note: Thinking Machines also released a research “variable-loop” variant. Stick with the standard model unless you are experimenting.

Benchmarks vs Qwen3.8-27B and Gemma 4

These are Thinking Machines’ own published numbers from the model card, so treat them as vendor-reported until independent tests arrive.

Benchmark Inkling 2 Mini Qwen3.8-27B Gemma 4 31B
AA Intelligence Index v5.0 20.1 20.1 4.4
SWE-bench Verified 79.2% 77.8% 46.1%
DeepSWE v1.1 48.2% 42.2% 0.3%
Terminal Bench v2.1 71.6% 73.0% 43.4%
Vibe Code Bench v1.1 77.9% 64.9% n/a
BrowseComp (compaction) 79.0% 61.6% 37.0%
MCP Atlas 82.7% 77.3% 54.2%
MMMU Pro (vision) 74.9% 76.3% 73.4%
ScreenSpot Pro 76.5% 57.6% 5.9%
LongBench v2 66.9% 52.3% 52.9%
HLE-Diamond 15.2% 15.3% 7.2%
Destructive actions (lower is better) 10.2% 35.2% 75.8%

The pattern is clear: Mini matches Qwen3.8-27B overall, wins on agentic coding, browsing, long context and screen understanding, and trails slightly on vision Q&A and multi-turn chat.

Tip: the low “destructive actions” score makes it an interesting pick for local coding agents. Still, keep confirmations on.

How Much VRAM You Need

Thinking Machines officially supports one NVIDIA DGX Spark, Apple Silicon Macs with 32 GB+ unified memory, and other GPUs with 32 GB+ VRAM. Smaller cards can run Unsloth’s GGUF quants. Sizes below are file sizes; leave extra room for context (KV cache).

Quant (Unsloth) File size Quality note Good fit
UD-IQ1_S 7.76 GB Big quality loss Testing only
UD-Q2_K_XL 9.14 GB 91.6% same top token vs Q8 12 GB GPUs
UD-IQ3_XXS 11.7 GB Usable 16 GB GPUs (short context)
UD-Q3_K_XL 13.6 GB Good 16 GB GPUs (tight)
UD-Q4_K_M 16.6 GB 97.9% same top token vs Q8 24 GB GPUs, 32 GB Macs
UD-Q4_K_XL 17.6 GB A little better than Q4_K_M 24 GB+ GPUs
UD-Q8_K_XL 22.9 GB Near original 32 GB GPUs, DGX Spark
Official NVFP4/MXFP8 ~20 GB Original weights RTX 50 series 32 GB, DGX Spark (vLLM/SGLang)

Not sure what your card can hold? See our VRAM guide and DGX Spark guide.

Run It in Ollama

  1. Install or update Ollama to the latest version (looped-model support is new).
  2. Open a terminal and run:
    ollama run hf.co/unsloth/Inkling-2-Mini-GGUF:UD-Q4_K_M
  3. On a 16 GB GPU, swap the tag for UD-Q3_K_XL or UD-IQ3_XXS.
  4. To attach an image, drag it into the terminal or pass the file path in your prompt.
  5. Raise the context only as far as your memory allows. Our Ollama context length guide shows how.

Run It in llama.cpp

  1. Update to the latest llama.cpp build (see our llama.cpp 1.0 guide).
  2. Start an OpenAI-compatible server straight from Hugging Face:
    llama-server -hf unsloth/Inkling-2-Mini-GGUF:UD-Q4_K_M -ngl 99 -c 32768
  3. Open http://localhost:8080 for the web UI, or point any OpenAI client at http://localhost:8080/v1.
  4. Newer builds also accept the single-command form shown on the Unsloth card: llama serve -hf unsloth/Inkling-2-Mini-GGUF:UD-Q4_K_M.

Our llama-server guide covers flags, API keys and multi-user setups.

Run It in LM Studio

  1. Update LM Studio and its llama.cpp runtime (Settings > Runtime).
  2. Open the Discover tab and search Inkling-2-Mini.
  3. Pick the Unsloth GGUF and a quant that fits your VRAM (green tick).
  4. Load it, set the context length, and start chatting. Drop an image into the chat to test vision.

Note: if the model fails to load, your runtime is probably too old for looped models. See LM Studio not loading models.

Run It in vLLM or SGLang

For the official NVFP4/MXFP8 weights on a 32 GB RTX 50-series card or DGX Spark:

  1. vLLM: pip install -U vllm then vllm serve "thinkingmachines/Inkling-2-Mini". The API runs at http://localhost:8000/v1.
  2. SGLang: pip install -U sglang then python3 -m sglang.launch_server --model-path "thinkingmachines/Inkling-2-Mini" --host 0.0.0.0 --port 30000.
  3. Docker Model Runner: docker model run hf.co/thinkingmachines/Inkling-2-Mini.

Known Limitations

Thinking Machines lists these on the model card:

  • Weaker factual recall than Inkling 2 Small, and more often confidently wrong.
  • Can get stuck in long reasoning loops, especially when debugging code. Set output-token and agent-turn limits.
  • Less reliable at tool use and following project conventions; may misreport results.
  • Weaker outside English.
  • Minimum reasoning effort does not switch reasoning off completely.
  • Can be overly cautious on mental-health topics.

Troubleshooting

Problem Likely cause Fix
“Unknown architecture” error Old Ollama, llama.cpp or LM Studio runtime Update to the latest version
Out of memory on load Quant too big for your VRAM Drop one quant level or lower context
Slower than expected Looping triples middle-layer compute Normal; use a smaller context and full GPU offload
Endless thinking Known reasoning-loop issue Set a max output token limit
Image input ignored Vision projector not loaded Use the Unsloth repo via -hf, which pulls the mmproj file
vLLM fails on older GPU NVFP4 needs Blackwell-class hardware Use a GGUF build in llama.cpp instead

My Take

This is one of the most exciting local models of the month. Matching Qwen3.8-27B with a 20 GB checkpoint, Apache 2.0 licensing and real vision support is a strong package.

If you have a 24 GB card or a 32 GB Mac, grab the Q4_K_XL build today. On 16 GB, the Q3 quants are worth a try for coding help, but expect tight context. Wait a week for runtime updates if speed disappoints.

FAQ

When was Inkling 2 Mini released?

Thinking Machines released it on Hugging Face on 20 November 2026.

Is Inkling 2 Mini free for commercial use?

Yes. It uses the Apache 2.0 licence, plus Thinking Machines’ acceptable use policy.

How much VRAM does Inkling 2 Mini need?

Officially 32 GB+, but the Unsloth Q4_K_M GGUF is 16.6 GB and fits a 24 GB GPU. Q3 quants (11.7 to 13.6 GB) can run on 16 GB cards with short context.

Can Inkling 2 Mini see images?

Yes. It accepts text and images as input and replies in text.

Is it better than Qwen3.8-27B?

They tie on the AA Intelligence Index (20.1). Mini wins on most coding, agent and long-context tests; Qwen is slightly better at chat and some vision Q&A.

Does it run in Ollama?

Yes: ollama run hf.co/unsloth/Inkling-2-Mini-GGUF:UD-Q4_K_M on an up-to-date Ollama.

What is a looped transformer?

A model that runs some layers more than once per token with shared weights, adding depth without adding download size.

What context length does it support?

Up to 1M tokens, though local memory limits how much you can actually use.

Sources

  • Hugging Face: thinkingmachines/Inkling-2-Mini model card (release 20 November 2026)
  • Hugging Face: unsloth/Inkling-2-Mini-GGUF model card
  • TestingCatalog: “Thinking Machines releases Inkling 2 Mini open-weight model”, 20 November 2026
Free weekly email

Get new AI guides every week

One short weekly email with our newest local AI, ComfyUI and AI tool guides. No spam, no pop-ups, just the good stuff.

Free. Unsubscribe anytime in one click. We only use your email to send this newsletter. See our privacy policy.

Woman at a gaming PC holding a tablet with an INKLING card, hook Free 26B model runs at home