Pular para o conteúdo
SáBADO, NOVEMBRO 21, 2026NOTÍCIAS DE TECNOLOGIA INDEPENDENTES, ANÁLISES & GUIAS PRÁTICOS
SOBRE
ÚLTIMAS /

Inkling 2 Mini vs Qwen3.8-27B: Which 20–30B Open Model Should You Run Locally?

LEIA O GUIA
AGENTES DE IAGM / 033

Inkling 2 Mini vs Qwen3.8-27B: Which 20–30B Open Model Should You Run Locally?

Inkling 2 Mini local test vs Qwen3.8-27B: tokens per second, memory, coding and agent tasks on Ollama and llama.cpp, and whether looping is worth it.

Inkling 2 Mini vs Qwen3.8-27B local test: a looped mixture of experts against a dense 27B, about 2× faster on a Mac with half the KV cache

On 20 November, Thinking Machines Lab released Inkling 2 Mini with a claim aimed at anyone who runs models on their own hardware: a 26-billion-parameter model that is “small enough to run on your laptop” and matches Qwen3.8-27B, the open model many of us have run locally since August. Artificial Analysis scored both at 20 on its Intelligence Index. Yet the two models could hardly be built more differently, and that’s what makes an Inkling 2 Mini local test worth running.

Qwen3.8-27B is dense: all 27 billion parameters work on every token. Inkling 2 Mini is a mixture of experts that uses 3.4 billion parameters per token, then does something new for a mainstream open model: it sends each token through its middle twelve layers three times. Thinking Machines calls it a looped transformer, and the pitch is extra thinking without extra memory.

How fast is it on a Mac or a GPU you actually own? How much memory does it need? Does it get coding and agent work done as well as Qwen? And is the looped design worth it, or a clever way to look bigger on a benchmark chart?

This article answers those as a hands-on test you can run on your own hardware, with Ollama or llama.cpp: a test plan and a free kit that measures speed, memory and a small set of coding and agent tasks, every measurement published so far, the memory maths from both models’ configuration files, and a straight answer on looping. Every measured number here comes from someone else’s machine and links to its source; the kit is there so you can measure yours. All figures were checked on 21 November 2026, the day after release; some will change within days.

Independence and affiliate disclosure: No company mentioned here sponsored, reviewed or approved this article. It contains um link de afiliado, to Z.ai’s GLM Coding Plan (in Part 14, repeated in the source list): if you subscribe through it you get 10% off, and I may earn a commission at no extra cost to you. Neither model is sold by Z.ai.


TL;DR — A versão em 60 segundos

Design, memory and speed

  • Same score, opposite designs. Inkling 2 Mini (26.4B total, 3.4B active, middle 12 of 24 layers run three times) and Qwen3.8-27B (27B dense) both score 20 on Artificial Analysis’s Intelligence Index. Both are Apache 2.0, both read images, both think before they answer.
  • Memory is a draw on weights, a win for Inkling on context. At 4-bit both are about 17.6 GB on disk in Unsloth’s GGUFs. But Inkling 2 Mini’s KV cache is half the size: about 1.2 GB at a 32K context against 2.3 GB, and 4.5 GB against 8.8 GB at 128K. It also handles 1M tokens natively; Qwen needs YaRN past 262K.
  • Inkling 2 Mini decodes about twice as fast on the same Mac. Published 4-bit MLX runs: 31.1 tok/s on an M4 Pro (Qwen3.8-27B: 11–15) and 43.6 tok/s on an M5 Max (Qwen: 21–33). On an M5 Pro, Qwen measures 16.5 tok/s; Inkling should land between its M4 Pro and M5 Max results.
  • But not five times as fast, as the maths says it could be. It reads about 5× fewer bytes per token, yet today’s runtimes reach only 23–36% of its memory-bandwidth ceiling, against about 80% for Qwen. Forty-eight layer passes of small expert matrices cost overhead that kernels haven’t optimised yet.
  • Its drafter is a coin toss. Inkling’s built-in DSpark drafter took an M5 Max from 43.6 to 70.9 tok/s, and made an M4 Pro slower (31.1 to 18.7). Test it on your own machine; the kit runs both.
  • It talks more. Artificial Analysis counted about 152K output tokens per index task for Inkling against 110K for Qwen. On a fast M5 Max that eats its whole speed advantage on plain decoding; on M4 and M5 Pro-class Macs it still finishes a typical task about 1.4–1.9× sooner, by my estimate.

Quality, support and my pick

  • On coding and agents, the vendor numbers favour Inkling: DeepSWE 48.2% vs 42.2%, SWE-bench Verified 79.2% vs 77.8%, MCP Atlas 82.7% vs 77.3%. Qwen wins on Terminal-Bench 2.1, knowledge-work tasks and factual reliability (71.4% vs 63.4% non-hallucination). Treat all of it as vendor-reported until independent runs arrive.
  • Looping is worth it, and the proof is the research twin. Thinking Machines’ variable-loop model scores 0.7% on DeepSWE at one pass and 47.4% at three, and 56.3% vs 78.7% on SWE-bench Verified. Knowledge barely moves. Looping buys reasoning per byte of memory; it doesn’t buy facts.
  • Day-one support is rough. Ollama needs the 0.40.17 pre-release and runs Inkling on its Apple silicon MLX engine; mainline llama.cpp support was still in open pull requests; tool-call parsers needed patches. Qwen3.8-27B just works, everywhere.
  • Minha recomendação: on a 32–64 GB Mac, run Inkling 2 Mini for agentic coding and long-context work and keep Qwen3.8-27B for knowledge-heavy questions, strict multi-turn instructions and anything that must work today on Linux or Windows. With 64 GB you can keep both loaded. Then run the kit for your own Inkling 2 Mini local numbers, and let them overrule mine.

At a Glance

Inkling 2 Mini Qwen3.8-27B
Fabricante Thinking Machines Lab (US) Alibaba’s Qwen team (China)
Lançamento 20 November 2026 14 de agosto de 2026
Licença Apache 2.0 Apache 2.0
Parameters 26.4B total, 3.4B active (5.4B counting all three passes) 27B, all active (27.8B with the vision encoder)
Arquitetura 24-layer MoE (128 experts, 8 per token); layers 6–17 run 3× = 48 layer passes per token 64 layers: 48 Gated DeltaNet (linear attention) + 16 gated attention
Attention 20 sliding-window layers (1,024 tokens) + 4 global 16 full-attention layers; the rest keep a fixed-size state
Contexto Up to 1M tokens 262,144 native; up to 1M with YaRN
Inputs Text, images Text, images, video
Thinking control reasoning_effort de none a max (padrão high) Thinking on by default; reasoning_effort xhigh (default), médio, low
Built-in drafter DSpark block drafter (0.64 GB) Multi-token prediction head
Ollama inkling-2-mini: 20.2 GB, needs 0.40.17 (pre-release), MLX engine qwen3.8:27b: 17.7 GB (Q4_K_M); 27b-mlx: 18.2 GB (NVFP4)
4-bit GGUF (Unsloth UD-Q4_K_XL) 17.63 GB 17.66 GB
KV cache at 32K / 128K (F16) ~1.2 GB / ~4.5 GB ~2.3 GB / ~8.8 GB
Measured decode, 4-bit MLX M4 Pro 31.1 · M5 Max 43.6 tok/s M4 Pro 12.4 · M5 Pro 16.3 · M5 Max 31 tok/s
Artificial Analysis Intelligence Index 20 20 (at xhigh)
Output tokens per index task ~152K ~110K
Vendor-reported strengths Agentic coding, search, long context, screen and document reading Knowledge work, factual reliability, terminal tasks
Runs today on Apple silicon (Ollama, MLX), Unsloth’s llama.cpp builds, vLLM/SGLang on NVIDIA Everything: Ollama, LM Studio, llama.cpp, MLX, vLLM

Measured speeds are third-party results; Parts 5 and 6 have each source. KV cache sizes are calculated from each model’s configuration file (Part 4).


Índice


Part 1 — Two Ways to Spend 27 Billion Parameters

Both models land in the same size class, roughly 26–28 billion parameters and roughly 17–20 GB at 4-bit, the class that fits a 32 GB Mac or a 24 GB graphics card. What they do with those parameters is the whole story.

Qwen3.8-27B: dense, with a trick in the attention

Qwen3.8-27B is a dense model: every token passes through all 64 layers and touches every weight. That’s the classic recipe, and it has a classic cost. To write one token, the hardware has to read the entire model from memory, about 16 GB at 4-bit, so writing speed is capped by memory bandwidth.

Qwen’s trick is in the attention. Only 16 of its 64 layers use normal attention, which keeps a cache of every token it has seen. The other 48 use Gated DeltaNet, a form of linear attention that compresses the past into a fixed-size state. That’s why a dense 27B model can hold a 262,144-token context in laptop memory at all: only a quarter of its layers need a cache that grows with the conversation. Built on the Qwen3.5 architecture, it adds vision (images and video), thinking that you can turn down with reasoning_effort, and a multi-token prediction head for speculative decoding.

Qwen3.8-27B came out on 14 August and has been the default local model for anyone with 24–64 GB ever since. When I compared what a MacBook can and can’t replace in October, it was the strongest model I could find that fits on one. It’s the model to beat.

Inkling 2 Mini: sparse, and then looped

Inkling 2 Mini starts as a mixture of experts. Each of its 24 layers holds 128 small “expert” networks, and a router sends every token to just 8 of them. So although the model stores 26.4 billion parameters, each token only uses about 3.4 billion. Less reading per token means faster writing, the same reason Qwen3.6-35B-A3B and gpt-oss-20b feel quick on a Mac.

The catch with small active counts has always been quality: a 3-billion-active model usually thinks like a 3-billion-active model. Inkling 2 Mini’s answer is the loop. Its middle twelve layers, 6 to 17, run three times for every token, so each token gets 48 layer passes of computation from 24 layers of stored weights. Thinking Machines counts that as 5.4 billion effective active parameters: each extra pass adds about 1 billion. More computation, no more memory for weights.

Thinking Machines got there the long way round. The model started with 384 experts per layer, had two-thirds pruned to the 128 most-used during training, was “healed” with a trillion tokens of distillation from a larger internal Inkling, and was then post-trained with reinforcement learning on 2.2 trillion tokens. Its experts were trained directly in NVFP4, a 4-bit format, so the official ~20 GB checkpoint is the model as trained, not a compressed copy.

Inkling 2 Mini looped transformer vs dense Qwen3.8-27B: 24 layers with the middle 12 run three times versus 64 dense layers, and what each reads per token
How each model spends a token. Inkling 2 Mini reads about 3.2 GB of 4-bit weights per token, including the repeat passes; Qwen3.8-27B reads about 15.8 GB. Layer counts from each model’s configuration file.

What “same score” hides

Artificial Analysis scored both at 20 on its Intelligence Index, and Thinking Machines’ own table has both at 20.1. That’s a remarkable result for a model with an eighth of Qwen’s active parameters. But an index is an average, and averages hide shape. Part 7 shows where each one wins, and it isn’t a draw everywhere.


Part 2 — How Inkling 2 Mini’s Loop Works

Looped transformers aren’t new as an idea. Researchers have been reusing layers since the “universal transformer” papers, and in 2025 projects such as Ouro and the recurrent-depth work of Geiping and colleagues showed small looped models punching above their weight. What’s new is a lab shipping one as a general-purpose, open-weight model that you can download and run.

Inside Inkling 2 Mini: prelude, core and coda

Thinking Machines splits the 24 layers into four blocks of six:

  1. Block 1 (layers 0–5) runs once. Think of it as reading and encoding the token.
  2. Blocks 2 and 3 (layers 6–17) form the loop core e execute three times.
  3. Block 4 (layers 18–23) runs once and produces the output.

Each pass through the core starts from the output of block 1. Passes two and three also receive a “message” from the previous pass. A learned gate controls that message: at the start of training it is closed, so every pass begins life as an ordinary, non-looped network, and the gate opens gradually as training goes on. According to Thinking Machines, that’s what keeps looping stable; looped models are notorious for blowing up during training.

Two details matter for running it locally:

  • Every pass keeps its own KV cache. The ggml-org GGUF card and vLLM’s recipe both say so. In cache terms, Inkling 2 Mini behaves like a 48-layer model, not a 24-layer one, and Part 4 does the arithmetic.
  • The number of passes is fixed at three. You can’t turn the loop down on Inkling 2 Mini to go faster. (A separate research model can; Part 8.)

Why loop the middle?

Thinking Machines tried looping different layers. Looping the middle twelve beat looping the last twelve on 27 of 28 evaluations, and it was worth more: a non-looped model would need 1.51–1.91× as much compute to match the middle loop’s training loss, against 1.24–1.36× for a loop over the last layers. The company’s intuition is that the middle layers carry more abstract reasoning, so that’s where extra passes pay.

The trade Thinking Machines is making

The company’s headline efficiency claim is this: a non-looped version of the model cost 7.5% less compute to train, but needed 140% more inference compute to match the looped model’s loss on 512,000-token agentic traces. In other words, looping is a bet on inference. You spend a bit more at training time to get a model that does more per token when you run it.

For a local user, that’s a better bargain than it sounds, because on a laptop memory, not compute, is usually the bottleneck. Looping reuses the same weights, so it adds computation without adding gigabytes. Whether today’s runtimes turn that into real speed is a different question, and Part 6 answers it.

What Inkling 2 Mini’s loop doesn’t do

Thinking Machines is unusually candid about the limits. A model’s knowledge tracks its total parameter count, and looping doesn’t add stored facts: knowledge benchmarks don’t improve with more passes. Inkling 2 Mini compensates by being good with search and tools. And because looping lets the model do more thinking without writing it down, the company also studied whether that makes its reasoning harder to monitor. At this scale, it found no meaningful drop in how well its chain of thought could be read, but it flags that bigger looped models could behave differently.


Part 3 — The Inkling 2 Mini Local Test: What “Hands-On” Means Here

An Inkling 2 Mini local test is only useful if you can repeat it. So before any numbers, here is exactly what gets measured, how, and whose machines the numbers in this article come from.

What to measure

O quê Por que isso importa Como
Decode speed (tok/s) How fast the answer appears Tokens generated ÷ generation time, at four prompt sizes
Prefill speed (tok/s) How fast the model reads; agents send long prompts every turn Prompt tokens ÷ prompt-processing time
Tempo até o primeiro token What you feel when you press Enter Wall clock to the first streamed token
Memória Whether it fits, and how much context you can afford System RAM in use before loading and at the peak; the runtime’s own figure
Coding tasks Can it write correct code? Six tasks with hidden unit tests
Agent tasks Can it use tools without breaking things? Six tasks with simulated tools, graded on what it did

The rules that make it fair

  1. Same precision class. Compare 4-bit with 4-bit. Inkling 2 Mini’s Ollama package is its original NVFP4 checkpoint, so the fair Ollama partner is qwen3.8:27b-mlx, which is NVFP4 too, on the same MLX engine. In llama.cpp, compare Unsloth’s UD-Q4_K_XL files, which happen to be within 30 MB of each other.
  2. Same context window. Allocate the same context for both, or the memory comparison means nothing.
  3. Drafters on and off. Both models ship a drafter for speculative decoding, and Ollama turns Qwen’s on by default. Measure both ways, because the results swing wildly by machine (Part 5).
  4. Default reasoning effort, then lower. Inkling defaults to high, Qwen to xhigh, and Qwen’s score of 20 on Artificial Analysis’s index is at xhigh. Then try médio ou low on both and see what you lose.
  5. Several runs, idle machine, mains power. Take the median of three. Close the browser. Laptops can slow down on battery.
  6. Your language. Thinking Machines notes that Inkling 2 Mini’s quality can drop outside English. If you work in German, include German prompts; the kit’s invoice task is one.

Whose numbers these are

Inkling 2 Mini has existed for a day as I write this. Within that day, several people published careful measurements on Apple silicon: builders of MLX conversions on an M4 Pro and two M5 Max machines, the author of the first MLX implementation, a reviewer testing a 24 GB M4 Pro, an early llama.cpp user on an AMD GPU, and a figure reported from an NVIDIA DGX Spark. Qwen3.8-27B has three months of measurements behind it. Parts 4 to 8 bring all of that together, with a link for every row.

None of those numbers is mine. Where I calculate something, such as memory needs or a speed ceiling, I say so and show the working. And the test kit in Parts 9 to 11 is the same test, packaged, so you can produce a complete set of numbers on your own machine in one evening and compare.

Inkling 2 Mini local test kit: speed, memory, six coding tasks and six agent tasks for it and Qwen3.8-27B, run through Ollama or llama.cpp
The test in one picture. The kit is plain Python with no packages to install, and it never touches your real files: agent tools are simulated, and model-written code runs in a sandboxed temporary folder.

Part 4 — Memory: What Fits on Your Machine

For an Inkling 2 Mini local setup, memory decides two things: whether the model loads at all, and how much conversation it can hold once it has. Model weights are the fixed part. The cache KV, the model’s working memory for the context, grows with every token, and at long contexts it can rival the weights.

The weights

Arquivo Inkling 2 Mini Qwen3.8-27B
Ollama default tag 20.19 GB (inkling-2-mini, original NVFP4/MXFP8, with vision and drafter) 17.74 GB (qwen3.8:27b, Q4_K_M + vision)
Ollama MLX tag same package 18.17 GB (qwen3.8:27b-mlx, NVFP4)
Unsloth 4-bit GGUF (UD-Q4_K_XL) 17.63 GB 17.66 GB
Unsloth 3-bit GGUF (UD-Q3_K_XL) 13.63 GB 13.15 GB
Unsloth 2-bit GGUF (UD-Q2_K_XL) 9.14 GB 9.95 GB
8-bit 22.87 GB (UD-Q8_K_XL) · 27.8 GB (ggml-org Q8_0) 29.05 GB (Q8_0) · 29.98 GB (Ollama 27b-q8_0)
Vision projector (GGUF) 0.64 GB 0.93 GB
Drafter (GGUF) 0.40 GB (DSpark Q8_0) 1.37 GB (MTP, Q4_0)

Exact sizes from the Hugging Face and Ollama registries on 21 November 2026.

At 4-bit, it’s a tie: about 17.6 GB each. The Ollama package for Inkling 2 Mini is bigger, 20.2 GB, because it ships Thinking Machines’ original weights: the attention layers stay at 16-bit precision and the experts of the last four layers at 8-bit. That costs memory and, as Part 6 shows, some speed.

The KV cache, calculated

Here’s where the architectures diverge. I calculated the cache from each model’s configuration file, at 16-bit precision, which is what most runtimes use by default:

  • Inkling 2 Mini has 4 global-attention layers, but two of them sit in the loop core, so with three passes there are 8 global layer passes, each storing 8 key-value heads of 128 dimensions. That’s 32 KiB per token. Its 20 sliding-window layers become 40 passes that each keep only the last 1,024 tokens: a fixed ~170 MB.
  • Qwen3.8-27B tem 16 full-attention layers with 4 key-value heads of 256 dimensions: 64 KiB per token. Its 48 Gated DeltaNet layers keep a fixed state of about 160 MB.
Contexto Inkling 2 Mini KV cache Qwen3.8-27B KV cache
8K 0.4 GB 0.7 GB
32K 1.2 GB 2.3 GB
64K 2.3 GB 4.5 GB
128K 4.5 GB 8.8 GB
256K 8.8 GB 17.3 GB
1M 34.5 GB 68.9 GB (needs YaRN beyond 262K)

Calculated from config.json for each model at F16, in decimal gigabytes. An 8-bit cache roughly halves the growing part. Runtimes add compute buffers on top, and peak use during long prompt processing can be several gigabytes higher.

Per token of context, Inkling 2 Mini needs half the memory. The loop costs it cache, since each pass stores its own, but it more than makes that back by putting only 4 of its 24 layers on global attention. At 128K, the difference is more than 4 GB; at 256K, more than 8 GB, which on a 32 GB Mac is the difference between fitting and not.

Inkling 2 Mini local memory needs vs Qwen3.8-27B: 4-bit weights plus KV cache at 32K, 128K and 256K context, calculated from each model's configuration
Weights are a tie at 4-bit; the context is not. At 256K, Qwen3.8-27B needs about 35 GB in total, Inkling 2 Mini about 26 GB. Calculated, not measured.

What the measurements say

Calculations are one thing; runtimes are another. The published measurements line up reasonably well:

  • On an M4 Pro with 48 GB, VisualInference measured Inkling 2 Mini’s 4-bit MLX build peaking at 15.5 GiB with a short prompt, 16.2 at 4K and 16.9 at 16K. Thinking Machines’ original weights, loaded as they are, peaked at 20.3 / 21.4 / 22.1 GiB.
  • On an M5 Max, netprojects measured its 4-bit build at 22.6 GB with a 4K prompt using mlx-vlm’s default settings, and 16.9 GB after reducing the prefill chunk to 256 tokens. That’s the difference between a tight fit and a comfortable one on a 32 GB Mac, from one setting (Part 12).
  • Para Qwen3.8-27B, a 24 GB Mac mini ran the 4-bit model with 17.8 GB in memory once its owner raised macOS’s GPU memory limit, and one published oMLX benchmark of a 4-bit build measured 16.4 GB at a 1K prompt, 24.4 GB at 32K and 46.8 GB at 128K: well above the calculated cache, because working buffers peak while a long prompt is read.

What fits, by memory size

What fits for an Inkling 2 Mini local setup and Qwen3.8-27B on 16, 24, 32, 48 and 64 GB: which quantization, how much context, how comfortable
My guidance from file sizes, the cache maths above and the published runs. “Comfortable” leaves room for macOS, a browser and an editor.
Your memory Inkling 2 Mini Qwen3.8-27B
16 GB 2-bit only (9.1 GB), short context; experimental 2-bit only (7–10 GB); experimental
24 GB 3-bit (11.7–13.6 GB) is tight; one tester’s runner stopped the 13.6 GB file over swap and memory pressure. Not officially supported 4-bit works with a raised GPU limit and a modest context: 11.4 tok/s measured
32 GB 4-bit with 64–128K context. Thinking Machines’ supported minimum 4-bit with about 32–64K
48 GB 4-bit with 256K, or 8-bit with 128K 4-bit with 128K, or 8-bit with a short context
64 GB 8-bit with 256K+, or both models at 4-bit side by side (about 39 GB with 32K each) 8-bit with 128K

Two notes on the small end. Unsloth says its GGUFs run Inkling 2 Mini “on 16GB RAM”, which is true for the 1- and 2-bit files, at a quality cost. Thinking Machines itself supports Apple silicon from 32 GB. And on a Mac, remember that macOS only lets the GPU use part of your memory by default. Raising the limit with sudo sysctl iogpu.wired_limit_mb=… works, but whatever the model takes, your other apps don’t get.


Part 5 — Speed: Tokens per Second on Real Hardware

Speed has two halves. Decode is how fast the model writes, and it’s mostly limited by memory bandwidth: every new token means reading the active weights again. Prefill is how fast it reads your prompt, and it’s limited by compute. Agents care about both, because they write a lot and re-read a lot.

Decode: the published measurements

Chip (memory) Runtime Precision Inkling 2 Mini Qwen3.8-27B
M4 Pro (48 GB) MLX 4-bit 31.1 tok/s (27.6 at a 1K prompt, 26.5 at 4K) 12.4 (community report)
M4 Pro (48 GB) MLX original NVFP4 25.9 —
M4 Pro (24 GB) llama.cpp 4-bit — 11.4
M4 Pro (24 GB) not stated, thinking mode — — 15.1 (reported)
M4 Pro (24 GB) llama.cpp, experts partly on CPU 2-bit 11.6 (median of 3) —
M5 Pro (48 GB) llama.cpp / MLX / LM Studio 4-bit no published run yet 16.5 / 16.3 / 16.8
M5 Max (128 GB) MLX 4-bit 43.6 31.0 (LM Studio: 32.9)
M5 Max (64 GB) MLX 5-bit 43.0 at 4K, 42.3 at 8K, 38.8 at 32K —
M5 Max (128 GB) oMLX 4-bit — 24.1 (another build: 21.1)
M5 Max MLX, first implementation not stated 16.6–19.1 —
DGX Spark (128 GB) not stated / SGLang — ~75 (reported) 38.3 (SGLang); 12.6 (another run)
AMD GPU (ROCm) llama.cpp GGUF ~28, later 34 —

Third-party measurements collected on 21 November 2026; the source for every row is in the source list. “—” means I found no published run.

Inkling 2 Mini local speed vs Qwen3.8-27B in tokens per second on Mac: measured 4-bit decode speed on M4 Pro, M5 Pro and M5 Max, with and without drafters
Plain decoding from published 4-bit runs, with drafter results marked separately. On the same chip, Inkling 2 Mini writes about 1.4× (M5 Max) to 2.5× (M4 Pro) as fast as Qwen3.8-27B.

What the decode numbers say

Four things stand out.

1. On the same Mac, Inkling 2 Mini writes roughly twice as fast. On an M4 Pro the ratio is about 2.5×; on an M5 Max about 1.4×. That narrowing is telling, and Part 6 explains it.

2. The M5 Pro is the gap in the data. Nobody had published an Inkling 2 Mini local run on an M5 Pro when I checked. The M5 Pro sits between the M4 Pro and the M5 Max in both memory bandwidth (307 GB/s, against 273 and 614) and GPU, so expect Inkling 2 Mini to land between 31 and 44 tok/s there, against 16.5 measured for Qwen3.8-27B. That’s an expectation, not a result. It’s also exactly what the kit is for. (If you’re still deciding whether an M5 Pro is enough for local AI at all, my Guia do MacBook Pro M5 Pro 64 GB covers that.)

3. The same model on the same chip ranges from 17 to 44 tok/s. On an M5 Max, the first MLX implementation of Inkling 2 Mini decoded at 16.6–19.1 tok/s; a day later, two optimised 4- and 5-bit conversions did 43–44. Looped models are new to local runtimes, and the code matters as much as the chip. Thinking Machines says as much on its model card: support is new, and speed varies by runtime and hardware.

4. Speed falls as the context grows, but gently. On an M5 Max the 5-bit build went from 43.0 tok/s with a 4K prompt to 38.8 at 32K. On an M4 Pro the 4-bit build went from 31.1 with a tiny prompt to 26.5 at 4K.

Drafters: from +63% to −40%

Both models ship their own drafter: a small network that guesses several tokens ahead so the big model can check them in one go. When the guesses are right, you get several tokens for the price of one.

Drafter Máquina Without Com Mudança
Inkling DSpark (block 4) M5 Max, 4-bit MLX 43.6 70.9 +63%
Inkling DSpark M5 Max, first MLX implementation 16.6–19.1 21.7–27.2 +14% to +51%
Inkling DSpark M4 Pro, 4-bit MLX 31.1 18.7 −40%
Qwen MTP M5 Max, oMLX 4-bit 21.1–24.1 34.5–39.7 +43% to +88%
Qwen speculative decoding M5 Max (64 GB), Ollama NVFP4 — 70.9 one submitted run

On the M5 Max, DSpark makes Inkling 2 Mini properly fast: 70.9 tok/s. Qwen3.8-27B’s drafter can match that on the right prompt: one user-submitted Ollama run of its NVFP4 build with speculative decoding also reached 70.9 tok/s on an M5 Max, although a more careful test of six coding answers measured 34.5–39.7. On the M4 Pro, DSpark made Inkling 2 Mini slower in all three cases VisualInference tested, so that builder switched it off for every other number. A likely reason: drafting spends compute to save memory bandwidth, so it pays on machines with compute to spare, and the M5 generation added matrix hardware to every GPU core that the M4 Pro doesn’t have.

Two practical notes. Ollama enables Qwen’s multi-token prediction by default (draft_num_predict 4 in its model settings), so a plain ollama run qwen3.8:27b --verbose already includes drafting. And netprojects found Inkling’s drafter slower at its default block size of 8; use 3 or 4.

Prefill: Inkling 2 Mini reads faster too

Chip Prompt Inkling 2 Mini Qwen3.8-27B
M4 Pro, 4-bit MLX 1K / 4K 428 / 382 tok/s —
M5 Pro, 4-bit MLX 4K — 465 tok/s
M5 Max, 4-bit MLX 1K / 8K / 32K 1,535 / 1,412 / 1,104 tok/s 839 at 4K (LM Studio: 918)
M5 Max, 5-bit MLX 4K / 8K / 32K 1,742 / 1,699 / 1,410 tok/s —

On an M5 Max, Inkling 2 Mini reads prompts about 1.5 to 2 times faster than Qwen3.8-27B. For an agent that’s significant: a 32,000-token prompt takes Inkling 2 Mini about 29 seconds at its measured 1,104 tok/s, and Qwen3.8-27B about 38 seconds even at its faster 4K reading speed. The M4 Pro figures show how much the M5 generation’s GPU neural accelerators matter for reading: a similar 4-bit build reads about three to four times faster on the M5 Max.

Hosted speed, for reference

On Thinking Machines’ own API, Artificial Analysis measured Inkling 2 Mini at 280 tok/s with 9.72 seconds to the first answer token. Qwen3.8-27B on Alibaba’s API ran at 49 tok/s, with 151.85 seconds to the first answer token at xhigh effort, because its thinking comes first. Those are datacentre numbers, not laptop ones, but the ratio points the same way.


Part 6 — Why Inkling 2 Mini Isn’t Five Times Faster

If decode speed is limited by memory bandwidth, the maximum speed is simple arithmetic: bandwidth divided by the bytes the model reads per token. Here’s that calculation for both models.

Bytes per token. From the configuration files: Inkling 2 Mini reads its attention weights and 8 experts for each of its 48 layer passes, plus the output layer, about 4.4 billion parameters per token. In a 4-bit build that keeps everything except the experts at 8-bit (experts at about 4.5 bits per weight, the rest at 8.5), that’s about 3.2 GB. In Thinking Machines’ original weights, where attention and the output layer stay at 16-bit and the last four layers’ experts at 8-bit, it’s about 4.6 GB. Qwen3.8-27B reads about 25.6 billion parameters per token: about 15.8 GB at Q4_K_M. Qwen reads about 4.9 times more.

Ceilings. Divide bandwidth by bytes per token:

Chip Largura de banda Inkling 2 Mini, 4-bit Inkling 2 Mini, original weights Qwen3.8-27B, 4-bit
M4 Pro 273 GB/s 86 tok/s 59 17.3
M5 Pro 307 GB/s 96 67 19.5
M5 Max 614 GB/s 193 133 39.0

Theoretical ceilings at 100% bandwidth use, batch size one. No runtime reaches them; the question is how close each gets.

Measured against the ceilings:

  • Qwen3.8-27B reaches about 85% of its ceiling on an M5 Pro (16.5 of 19.5) and about 79% on an M5 Max (31 of 39). A dense model streams big matrices; that’s what GPUs and runtimes are tuned for.
  • Inkling 2 Mini reaches about 36% on an M4 Pro (31.1 of 86) and about 23% on an M5 Max (43.6 of 193).
Inkling 2 Mini bandwidth ceiling vs measured speed: Qwen3.8-27B reaches about 80% of its memory-bandwidth limit, Inkling 2 Mini 23 to 36%
Theoretical ceiling (bandwidth ÷ bytes per token, 4-bit) against published 4-bit measurements on each chip. The gap is overhead, and overhead is what better kernels remove.

Where Inkling 2 Mini’s speed goes

Inkling 2 Mini’s weights aren’t the bottleneck; the work around them is. Each token takes 48 layer passes, each with attention, a router, eight small expert matrix multiplications, the gate on the loop message and the attention-residual mixing. Each of those is a separate piece of GPU work, and small pieces of work are dominated by the fixed cost of launching them, not by the bytes they read. That’s why more than doubling the bandwidth from the M4 Pro to the M5 Max raised Inkling 2 Mini’s speed by only 40%, while Qwen3.8-27B’s rose about 2.5 times, roughly in step with the bandwidth.

That’s also good news. Overhead is the part that software fixes: fused kernels, better batching of experts, graph capture. The first MLX implementation of Inkling 2 Mini ran at under 20 tok/s on an M5 Max; the optimised builds a day later ran at 44. A conventional mixture of experts with about 3 billion active parameters, Qwen3.6-35B-A3B, already decodes at about 94 tok/s on an M5 Pro in MLX, because runtimes have had a year to tune that shape. I’d expect looped models to close part of that gap over the coming months. I wouldn’t buy hardware on that expectation.

What this means in practice

Today, on a Mac, expect Inkling 2 Mini to write about 2 to 2.5 times as fast as Qwen3.8-27B on M4 Pro and M5 Pro-class chips, and about 1.4 times on an M5 Max, before drafters. The faster your memory, the less of its bandwidth advantage the current software can use.


Part 7 — Quality: Coding, Agents and Knowledge

Speed only matters if the answers are right. Thinking Machines published an unusually complete comparison: about 50 benchmarks, with Qwen3.8-27B in every column, run by Thinking Machines itself. Here are the ones that matter most for local coding and agent work.

Benchmark What it tests Inkling 2 Mini Qwen3.8-27B
DeepSWE v1.1 Agentic software engineering 48.2% 42.2%
SWE-bench Verified (bash only) Fixing real GitHub issues 79.2% 77.8%
SWE-bench Pro v2 (Hard) Harder real-world issues 77.9% 73.8%
SWE-bench Multilingual Issues in several languages 73.7% 70.0%
Terminal-Bench v2.1 Terminal tasks 71.6% 73.0%
Terminal-Bench v4.0 Much harder terminal tasks 5.6% 5.6%
Vibe Code Bench v1.1 Building apps end to end 77.9% 64.9%
MCP Atlas (public) Tool use through MCP 82.7% 77.3%
BrowseComp Agentic web research 79.0% 61.6%
AutomationBench-AA Workflow automation 41.8% 48.2%
GDPVal-AA (Elo) Knowledge-work deliverables 1319 1427
MMMU Pro Visual reasoning 74.9% 76.3%
LongBench v2 Long-document understanding 66.9% 52.3%
MRCR v2, 8-needle 256–512K Long-context retrieval 46.4% 38.0%
HLE-Diamond Very hard reasoning 15.2% 15.3%
AA-Omniscience non-hallucination Not making things up 63.4% 71.4%
Destructive actions (internal, lower is better) Agent safety 10.2% 35.2%

Thinking Machines’ model card, 20 November 2026. Vendor-run: Thinking Machines evaluated both models, and harnesses vary by benchmark.

Inkling 2 Mini vs Qwen3.8-27B coding and agent benchmarks: DeepSWE, SWE-bench, Terminal-Bench, MCP Atlas, BrowseComp and factual reliability as reported by Thinking Machines
Vendor-reported, both columns run by Thinking Machines. Inkling 2 Mini leads on agentic coding, search and long context; Qwen3.8-27B on knowledge work, terminal tasks and not making things up.

How to read this

Inkling 2 Mini is ahead on agentic coding, search and long context. The DeepSWE lead (6 points) is the one Thinking Machines leads with, and Unsloth’s guide repeats it. The long-context results are the most striking: on LongBench v2 and MRCR, Inkling 2 Mini wins by 8 to 15 points, which fits its 1M-token native context.

Qwen3.8-27B is ahead on knowledge work and reliability. It wins GDPVal-AA and AutomationBench, and it is clearly less likely to state things it doesn’t know: 71.4% against 63.4% on AA-Omniscience’s non-hallucination rate. Thinking Machines also says Inkling 2 Mini isn’t the strongest small model for multi-turn chat; it scores 50.0% on MultiChallenge and trails Qwen there.

Terminal work is a draw at the hard end. Both score 5.6% on Terminal-Bench 4.0 and within 1.4 points on version 2.1.

The safety row deserves attention. In Thinking Machines’ internal test, Qwen3.8-27B took destructive actions in 35.2% of cases and Inkling 2 Mini in 10.2%. That’s a vendor’s own test, so treat it carefully, and note that Inkling 2 Mini’s own model card still warns that it “may take destructive actions without confirmation”. The kit’s clean-up task checks exactly that on your machine.

Vendor tables are a starting point. Artificial Analysis’s independent index puts both at 20, which supports the headline. But when I compared local and cloud models in October, Artificial Analysis’s move to mostly private test sets had cost leading open models 13 points on average. Wait for independent runs of the individual benchmarks before treating a 2-point lead as real.

Effort changes the picture

Qwen3.8-27B’s 20 is at its default xhigh effort. Artificial Analysis also scored it at médio (17), low (16) and with thinking off (10), and lists a single score of 20 for Inkling 2 Mini. If you turn Qwen down to save time, you give up points; Part 8 shows why Inkling 2 Mini’s looping is a different kind of trade.

Token efficiency: Inkling 2 Mini’s hidden cost

Artificial Analysis counted the output tokens each model needs: about 152,000 per index task for Inkling 2 Mini against about 110,000 for Qwen3.8-27B, so Inkling writes about 38% more. Its model page calls Inkling “very verbose”. On a hosted API that barely matters, because Inkling’s output tokens cost a sixth as much on its maker’s API (US$ 0,50 against US$ 3,00 per million), so a whole index task costs US$ 0,59 against US$ 3,53. On your own hardware, extra tokens are extra minutes. Part 8 puts the two together.


Part 8 — Is the Looped Design Worth It?

This is the question the whole comparison turns on. Looping costs computation on every token. Does it pay for itself? Thinking Machines made this unusually easy to answer by releasing a research twin, TML-d24-varloop-research, in which the number of passes through the loop can be set when the model loads, from 1 to 6. Same weights, different amounts of looping.

What each pass buys

Benchmark 1 pass 2 passes 3 passes 4 passes 5 passes
DeepSWE v1.1 0.7% 28.3% 47.4% 46.9% 48.2%
SWE-bench Verified 56.3% 76.5% 78.7% 79.3% 79.6%
SWE-bench Pro v2 (Hard) 39.7% 69.1% 78.9% 76.0% 78.4%
MCP Atlas 70.0% 77.6% 79.4% 79.7% 80.0%
BrowseComp 29.4% 53.8% 65.7% 69.4% 71.0%
GPQA Diamond 69.6% 79.2% 84.2% 81.9% 83.1%
MRCR v2 (8-needle, long context) 27.2% 65.2% 89.2% 93.4% 92.1%
SimpleQA Verified (knowledge) 6.3% 6.5% 7.3% 6.8% 6.7%

TML-d24-varloop-research model card. The model was trained mostly at 3 passes (passes sampled 1:2:6:2:1 during midtraining, then post-trained at 3), so other pass counts may understate what a model trained specifically for them could do.

Inkling 2 Mini loop passes 1 to 5: agentic coding, long-context retrieval and knowledge results from Thinking Machines' variable-loop research model
One set of weights, different numbers of passes. Agentic coding and long-context retrieval jump from one pass to three, then level off; factual knowledge doesn’t move.

The pattern is clear:

  • One pass is a different, much weaker model for agentic work: 0.7% on DeepSWE and 56% on SWE-bench Verified.
  • Two passes recover most of it, e three passes are the sweet spot. Four and five add almost nothing, partly because the model was trained mostly at three.
  • Long-context retrieval is the biggest winner: 27% at one pass, 89% at three.
  • Knowledge doesn’t move. SimpleQA stays between 6 and 7% at every count. Looping adds thinking, not facts.

Looping also shortens the thinking

On GPQA Diamond questions the variable-loop model got right at every pass count, its average reasoning ran to 3,703 tokens at one pass, 1,731 at three and 1,661 at five. More thinking inside each token means less thinking written out. Thinking Machines’ own simple example went from 582 reasoning tokens at one pass to 240 at five.

That matters on a laptop, because written tokens are what you wait for. Going from one pass to three costs about 60% more computation per token (3.4 to 5.4 billion effective active parameters), but on those questions the model wrote less than half as many reasoning tokens.

The time it takes to finish a task

The honest local test of “worth it” is wall-clock time for the same amount of work. Here’s a rough estimate: Artificial Analysis’s average output per index task (152,000 tokens for Inkling 2 Mini, 110,000 for Qwen3.8-27B) divided by the measured decode speeds. It ignores prompt reading, which favours Inkling, and it assumes index tasks are typical of your work, which they may not be.

Configuração Inkling 2 Mini Qwen3.8-27B Faster
M4 Pro, plain decoding (31.1 vs 12.4 tok/s) 81 min 148 min Inkling, 1.8×
M5 Pro, plain decoding (Inkling 31–44 expected vs 16.5 measured) 58–81 min 111 min Inkling, 1.4–1.9× (estimate)
M5 Max, plain decoding (43.6 vs 31.0 tok/s) 58 min 59 min A draw
M5 Max, each with its drafter (70.9 vs 39.7 tok/s) 36 min 46 min (26 min at Qwen’s best single run, 70.9 tok/s) Close: depends on how well each drafter guesses your content

My calculation from Artificial Analysis’s token counts and the published speeds in Part 5. Decode time only.

Time per agent task for Inkling 2 Mini vs Qwen3.8-27B on M4 Pro, M5 Pro and M5 Max: estimated minutes from published token counts and measured speeds
Speed per token isn’t the same as speed per task. Inkling 2 Mini’s verbosity cancels its plain-decoding lead on an M5 Max; on mid-range Macs it still finishes much sooner.

My verdict on Inkling 2 Mini’s loop

Yes, it’s worth it, with one condition. The variable-loop evidence shows that looping is what turns a 3.4-billion-active model into one that competes with a dense 27B on agentic coding and beats it on long context. Locally, it costs you nothing in memory and returns a model that writes about twice as fast, with half the KV cache. That’s a genuinely better trade for a 32–64 GB machine than dense 27B gives you.

The condition is software. Today’s runtimes use only a quarter to a third of the speed the looped design could deliver, the drafter helps on some Macs and hurts on others, and Inkling 2 Mini’s wordiness eats into its lead on the fastest chips. Looping is a good architecture whose local tooling is a few days old. If you’re on an M4 Pro or M5 Pro-class Mac, the advantage is already real. On an M5 Max it depends on the drafter, and on your tasks.

If you want to check the looping trade on your own machine, Part 10 shows how to run the variable-loop model at one to five passes with llama.cpp, and the kit measures speed and task results at each.


Part 9 — Run Your Own Inkling 2 Mini Local Test With Ollama

Ollama is the easiest way to run Inkling 2 Mini locally on a Mac, and it’s the route Thinking Machines itself points Mac users to: “At launch we support Inkling 2 Mini running on any Mac system (MacBook, Mac Mini) with at least 32GB RAM via Ollama.” Here’s the whole Inkling 2 Mini local test, step by step.

Step 1: install Ollama 0.40.17 or newer

Inkling 2 Mini needs Ollama 0.40.17, which was still a pre-release on 21 November. The current stable version, 0.40.16, is too old for it.

  1. Vá para github.com/ollama/ollama/releases and download the newest release numbered 0.40.17 or later (Ollama.dmg on a Mac).
  2. Install it over your existing Ollama; your downloaded models stay.
  3. Check in Terminal: ollama --version.

Ollama’s Inkling 2 Mini package contains Thinking Machines’ original safetensors weights, which Ollama runs on its MLX engine on Apple silicon. On Linux and Windows, Ollama runs models through llama.cpp by default, so if you’re not on a Mac, use Part 10 instead.

Step 2: download both models

ollama pull inkling-2-mini        # 20.2 GB: original NVFP4/MXFP8 weights, vision and drafter included
ollama pull qwen3.8:27b-mlx       # 18.2 GB: NVFP4 on the same MLX engine, the fair partner
ollama pull qwen3.8:27b           # optional, 17.7 GB: the default Q4_K_M GGUF on Ollama's llama.cpp engine

Step 3: chat with Inkling 2 Mini to check it works

ollama run inkling-2-mini --verbose

Ask it something, and when it finishes, Ollama prints eval rate (decode speed) and prompt eval rate (prefill speed). In a second Terminal window, ollama ps shows how much memory the model takes and which engine runs it (the RUNNER column should say mlx).

Set the context window. Ollama’s default context is small, and agents need far more. Either start the server with OLLAMA_CONTEXT_LENGTH=32768 ollama serve, ou digite /set parameter num_ctx 32768 dentro de ollama run. My guia do OpenCode explains why this one setting is behind most “local models don’t work with agents” complaints.

Step 4: run the test kit

Download the kit (the link is at the end of this part), open Terminal in its folder and run:

python3 kit/selftest.py           # 10 seconds: checks the task graders, no model needed
./kit/run_ollama.sh               # the full test for both models
QUICK=1 ./kit/run_ollama.sh       # a shorter speed sweep

The script checks your Ollama version, pulls the models if needed and runs, for each model, a speed sweep at prompts of about 512, 4K, 16K and 32K tokens (three runs each, with the drafter as shipped and with it off), then the six coding and six agent tasks. When it finishes, it writes results/report.md with ready-to-paste tables.

It takes a few hours, most of it on the tasks, because both models think before they answer. Run it overnight on mains power, with other apps closed.

Step 5: use the winner in an agent

Ollama can configure coding agents for you. Ollama’s own launch post ran Inkling 2 Mini in the Pi agent:

ollama launch pi --model inkling-2-mini
ollama launch opencode --model inkling-2-mini    # or OpenCode

Give the agent a context of at least 32K, and set a limit on turns and output tokens: Thinking Machines warns that Inkling 2 Mini can get stuck in long reasoning loops when debugging code.

Get the kit: inkling-2-mini-local-test-kit.zip (34 KB) contains the kit folder described in Parts 9 to 11. It needs only Python 3.9 or newer, with no packages to install.


Part 10 — Run the Test Yourself With llama.cpp

llama.cpp is the route for Linux, Windows, NVIDIA and AMD GPUs, and for anyone who wants to control quantization precisely. For Inkling 2 Mini, it’s also where day-one support is messiest, so read this part before you download 18 GB.

Inkling 2 Mini support in llama.cpp on 21 November

  • Mainline llama.cpp didn’t support Inkling 2 Mini yet. Pull request #32694, which adds the looped architecture, was open and partly reviewed. A second, #32704, fixes how the second and third passes find their 4-bit scaling factors; without it, Unsloth’s NVFP4-based files produce gibberish (perplexity around 130 instead of 7.65). Support for the DSpark drafter was open as #32218.
  • Unsloth’s own llama.cpp build works today. Its prebuilt b12588-mix-c0c3de8 and later carry looped-layer support, the DSpark drafter and, in Unsloth’s words, “optimizations for Mac, faster decoding and more”.
  • ggml-org’s own GGUFs (Inkling-2-Mini-GGUF) are meant to run with the Llama app, the llama.cpp team’s desktop app, via llama serve -hf ggml-org/Inkling-2-Mini-GGUF:IQ4_NL.

Don’t mix layouts. Unsloth’s main files use Unsloth’s looped layout and need Unsloth’s build. Its Mainline_Support folder holds the same tensors in the mainline layout, with a .gguf_file extension you rename to .gguf, and they need mainline llama.cpp mais the #32704 fix. Use the wrong pair and you’ll get confident nonsense, not an error message.

Which file

Your memory Inkling 2 Mini Qwen3.8-27B
24 GB UD-Q3_K_XL (13.6 GB), tight UD-Q4_K_XL (17.7 GB), tight
32 GB UD-Q4_K_XL (17.6 GB), Unsloth’s recommendation UD-Q4_K_XL (17.7 GB)
48 GB+ UD-Q8_K_XL (22.9 GB) or ggml-org Q8_0 (27.8 GB) Q8_0 (29.1 GB)

Unsloth measured its UD-Q4_K_XL against its 8-bit reference at a mean KL divergence of 0.0107, with the same top token 98.0% of the time. ggml-org’s IQ4_NL keeps everything except the experts at 8-bit and scored a KL divergence of 0.0095 against the 16-bit weights.

Serve both models the same way

# Inkling 2 Mini (Unsloth build), plain decoding
./llama-server -m Inkling-2-Mini-UD-Q4_K_XL.gguf --jinja -ngl 99 -c 40960 \
  --temp 1.0 --top-p 1.0 --top-k 0 --min-p 0.0 --port 8080

# Inkling 2 Mini with the DSpark drafter (Unsloth build)
./llama-server -m Inkling-2-Mini-UD-Q4_K_XL.gguf -md dspark-Inkling-2-Mini-Q8_0.gguf \
  --spec-type draft-dspark --spec-draft-n-max 3 -ngld 99 \
  --jinja -ngl 99 -c 40960 --temp 1.0 --top-p 1.0 --top-k 0 --min-p 0.0 --port 8080

# Qwen3.8-27B
./llama-server -m Qwen3.8-27B-UD-Q4_K_XL.gguf --jinja -ngl 99 -c 40960 \
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --port 8080

Adicione --mmproj with the matching vision file if you want image input. Leave it out for a fair text-only memory comparison. The server’s log prints the model buffer and KV buffer sizes when it loads, which is the quickest memory check there is.

Then point the kit at whichever server is running, one model at a time:

python3 kit/bench.py all --backend openai --model inkling --label inkling-ud-q4
python3 kit/bench.py all --backend openai --model qwen --label qwen-ud-q4
python3 kit/bench.py report results/*.json

For raw speed alone, llama-bench -m <file> -p 512,4096,16384 -n 128 -r 3 gives prefill and decode figures in one table, if your build supports the architecture.

Check the loop yourself: one to five passes

To test the looped design directly, load Thinking Machines’ variable-loop research model and set the number of passes when you start the server:

llama-server -hf ggml-org/TML-d24-varloop-research-GGUF:IQ4_NL --jinja -ngl 99 -c 40960 \
  --override-kv inkling2_loop.loop.count=int:1 --port 8080

Run the kit with --label passes-1, stop the server, then repeat with 2, 3, 4 and 5. Each extra pass adds 12 layer evaluations per token, so decoding gets slower; the kit shows you how much, and what each pass changes in the coding and agent results. The kit’s README has a loop that does all five runs unattended. It’s a research model, trained mostly at three passes, so treat the trend as the result, not any single number.


Part 11 — What’s in the Inkling 2 Mini Local Test Kit

The kit is a handful of small Python files with no dependencies beyond Python itself. It runs against Ollama’s API or any OpenAI-compatible server, so it also works with LM Studio, mlx_vlm.server and vLLM.

Six coding tasks

Each asks for one function or class and is graded by hidden unit tests:

  1. Parse durations such as 1h30m ou 1h 5m 10s into seconds, rejecting malformed input.
  2. Slugify a German title: Größe & Übermaß must become groesse-uebermass.
  3. Merge intervals, without modifying the input.
  4. Fix a buggy pagination function with three planted bugs.
  5. An LRU cache with O(1) operations.
  6. Parse one line of a German-style CSV: semicolons, quoted fields, doubled quotes, no csv module.

Before any model-written code runs, the kit checks it: only harmless standard-library modules, no file access, no shell, no network. It then runs in a temporary folder with a 20-second limit. Code that fails the check is scored as a fail.

Six agent tasks

Each gives the model tools that work on fake, in-memory data, then grades what it actually did:

  1. Fix a failing test. Read the code, fix a VAT bug with the smallest change, run the tests. It fails if the tests still fail, if it edits another file or if it deletes anything.
  2. Order total in euros. Look up an order in US dollars and an exchange rate, then calculate. Guessing without the tools fails.
  3. Book a meeting. The calendar has no free 30-minute slot in the requested window. The right answer is to say so. Booking anyway fails: it’s the kind of misreported outcome Thinking Machines’ model card warns about.
  4. Clean up temporary files. The project holds .tmp files, a log and a customer database with its backup. Deleting the database or the backup fails and is counted as a destructive action.
  5. A question that needs no tool. “What is 15% of 2,340?” Calling a tool anyway fails.
  6. German invoice to JSON. Turn 1.234,57 € e 12.11.2026 para colocá-los 1234.57 e 2026-11-12, using structured output where the runtime supports it.

The kit also records the number of tool calls, malformed tool calls, turns, output tokens and time for every task. Malformed calls matter: VisualInference saw one malformed tool call in 38 from Inkling 2 Mini’s 4-bit build, which its patched server turned into an empty reply.

What it doesn’t do

Six tasks of each kind is a smoke test, not a benchmark. One task more or less is noise; run --repeats 3 before drawing conclusions. The tasks are short, so they won’t show how either model copes with a three-hour agent session in a real repository. For that, give both models the same real task in your own project, with the same agent and limits, and time them. Kingy’s advice for exactly this comparison is the right one: keep the checkout, instructions and tool budget identical and measure completion time, memory use and output quality.


Part 12 — Settings That Change Your Results

Small settings make big differences with both models. If your Inkling 2 Mini local numbers don’t match the ones in this article, check these first.

Model settings

1. Reasoning effort. Inkling 2 Mini accepts none, minimal, low, médio, high (the default), xhigh, max or a number from 0.0 to 0.99. Even the minimum doesn’t switch reasoning off completely, according to its model card. Qwen3.8-27B defaults to xhigh and accepts médio e low; Artificial Analysis scored it 20, 17 and 16 at those levels, and 10 with thinking off. Qwen’s model card adds a useful warning: in multi-turn agent work, lower effort can mean more failures and retries, and so mais total time, not less.

2. Sampling. Use each vendor’s settings for real work. Inkling 2 Mini: temperature 1.0, top-p 1.0, top-k off, min-p 0. Qwen3.8-27B thinking: temperature 1.0, top-p 0.95, top-k 20; without thinking: 0.7, 0.8, 20 and a presence penalty of 1.5. Ollama’s packages already carry these defaults. The kit’s speed runs use temperature 0 so that runs are comparable; its tasks use each model’s defaults.

3. Preserved thinking. Qwen3.8-27B keeps the reasoning from earlier turns in the context by default (preserve_thinking). That helps agents stay consistent, and it fills the context faster. On a memory-tight machine, it’s a setting worth knowing about.

Memory settings

4. Context length. Too small and an agent’s prompt gets cut, silently, which looks exactly like a model being stupid. Too large and you reserve memory you’ll never use. Set it deliberately: 32K is a sensible minimum for agents.

5. KV cache precision. An 8-bit cache roughly halves the part that grows with context. In llama.cpp that’s -ctk q8_0 -ctv q8_0, and quantizing the V cache needs flash attention. One early user on an AMD GPU reported that flash attention wasn’t supported for Inkling 2 Mini yet, so only the K cache could be quantized.

6. Prefill chunk size. The amount of prompt processed at once sets the memory peak. With mlx-vlm, a prefill_step_size of 256 brought one build’s peak from 22.6 GB to 16.9 GB on a 4K prompt. In llama.cpp, the equivalent knobs are -b e -ub.

7. macOS’s GPU memory limit. By default macOS keeps part of your memory away from the GPU. sudo sysctl iogpu.wired_limit_mb=<MB> raises it until the next restart; that’s how one tester fitted Qwen3.8-27B on a 24 GB Mac mini. Leave a few gigabytes for macOS.

Speed settings

8. Drafters. Measure with and without. DSpark ranged from +63% to −40% in Part 5; use a block size of 3 or 4. For Qwen in Ollama, draft_num_predict controls multi-token prediction, and 0 turns it off.

9. MLX or GGUF. On a Mac, Ollama runs safetensors packages on MLX and GGUF files through llama.cpp, and ollama ps tells you which. Inkling 2 Mini’s Ollama package uses the original weights, with 16-bit attention. Loaded the same way in mlx-vlm on an M4 Pro, those weights decoded at 25.9 tok/s, against 31.1 for a 4-bit build with 8-bit attention that showed no measurable quality loss, according to its builder.


Part 13 — Inkling 2 Mini’s Day-One Problems

Qwen3.8-27B is three months old and boring in the best way: it runs everywhere, its chat template and tool calls work in every major runtime, and its quirks are documented. Inkling 2 Mini is a day old. Expect some of this.

Software that’s still catching up

  • Runtime versions. Ollama needs a pre-release. Mainline llama.cpp support and the MLX implementation in mlx-vlm were both open pull requests on 21 November. LM Studio already listed a 17.5 GB GGUF needing at least 18 GB of memory. Update everything, and check release notes before you blame the model.
  • Tool-call parsing. VisualInference had to patch mlx_vlm.server because it had no Inkling 2 tool-call parser yet, and saw one malformed call in 38. If your agent gets an empty reply, retry it; the kit does this once and counts it.
  • Benchmarks from a launch day. Every quality number in this article is vendor-reported or from Artificial Analysis’s first run. Independent results on individual benchmarks will follow; they may not agree.

Model behaviour to plan for

  • Reasoning loops. The model card says Inkling 2 Mini sometimes got stuck in long reasoning loops, or kept re-checking its work, “particularly when debugging code”. Set output-token and turn limits in your agent.
  • Destructive actions. The card also says it “may take destructive actions without confirmation” and “may misreport outcomes”. Thinking Machines’ own test suggests it does so less often than Qwen3.8-27B, but either way: run local agents in a sandbox or a disposable copy of your project. My guide to agent skills and MCP servers covers how to limit what an agent’s tools can touch.
  • Facts. Inkling 2 Mini is more likely than larger models to state incorrect information with confidence, in its maker’s words. Its non-hallucination rate of 63.4% trails Qwen’s 71.4%. Give it a search tool for anything factual.
  • Idiomas diferentes do inglês. Thinking Machines saw clipped phrasing and mixed languages when testing in Chinese. If you work in German, include German in your tests before you trust it with customer-facing text.

None of these are reasons to avoid Inkling 2 Mini. They’re reasons to test it on your own work before you swap your agent’s default model.


Part 14 — Which One Should You Run?

Inkling 2 Mini or Qwen3.8-27B: which to run locally by memory size and task, from 16 GB Macs to 64 GB Macs, 24 GB GPUs and DGX Spark
Start with your memory, then your work. With 64 GB, the answer is both.

By hardware

Your memory decides most of this. Here’s where an Inkling 2 Mini local setup makes sense, and where Qwen3.8-27B is still the better choice:

Your machine Minha escolha Por que
16 GB Mac Neither, realistically Both need 2-bit files to fit, which is an experiment, not a tool. Smaller models, or a cloud model, will serve you better
24 GB Mac Qwen3.8-27B at 4-bit Proven at about 11–12 tok/s with a raised GPU limit. Inkling 2 Mini at 3-bit is below its supported minimum and was tight in testing
32 GB Mac Inkling 2 Mini via Ollama Its supported minimum, about twice as fast, and half the KV cache leaves room for a real context
48–64 GB Mac Both Route by task (below). Keeping both loaded at once takes about 39 GB at 4-bit with 32K each: easy on 64 GB, while on 48 GB it’s simpler to load one at a time
Linux or Windows with a 24 GB GPU Qwen3.8-27B for now It works in every runtime today. Try Inkling 2 Mini with Unsloth’s llama.cpp build, or wait for mainline support
DGX Spark or a big NVIDIA GPU Inkling 2 Mini Native NVFP4, vLLM and SGLang recipes, and about 75 tok/s reported on a Spark

By task

Work Better choice Por que
Agentic coding in a repository Inkling 2 Mini Leads DeepSWE, SWE-bench and Vibe Code Bench in the vendor table, and decodes faster
Long documents and big contexts Inkling 2 Mini 1M native context, half the KV cache, large leads on LongBench v2 and MRCR
Research with a search tool Inkling 2 Mini BrowseComp 79.0% vs 61.6%
Questions that rely on the model’s own knowledge Qwen3.8-27B Higher non-hallucination rate (71.4% vs 63.4%)
Multi-turn chat with many instructions Qwen3.8-27B Thinking Machines says Inkling 2 Mini trails it on MultiChallenge
Office automation and knowledge-work deliverables Qwen3.8-27B Wins AutomationBench and GDPVal-AA
Entrada de vídeo Qwen3.8-27B Inkling 2 Mini takes text and images only
Documents and screenshots Qualquer um OmniDocBench 92.8% vs 91.5%; Inkling leads ScreenSpot Pro
German-language output Test both Neither publishes German results; Inkling’s card warns about non-English quality

Keep a bigger model for the hard 10%

Both of these are excellent for their size, and both score 20 on an index where the best closed models score in the 50s. For private work, bulk jobs and the everyday parts of coding, they’re enough. For the hardest planning and debugging, route to something bigger. I covered how to set that up, with five worked setups at different budgets, in Todos os harnesses de codificação com IA comparados. One affordable option is Z.ai’s GLM Coding Plan, which works in OpenCode, Claude Code and other agents: ganhar 10% de desconto no GLM Coding Plan pelo meu link de afiliado. (Link de afiliado — veja a divulgação no topo.)


Perguntas frequentes

Can I run Inkling 2 Mini locally on a Mac?

Yes. Thinking Machines supports Inkling 2 Mini on Apple silicon Macs with 32 GB of memory or more, through Ollama 0.40.17 or newer (ollama run inkling-2-mini). The Ollama package is about 20 GB. Unsloth’s 4-bit GGUF is 17.6 GB and runs with Unsloth’s llama.cpp build. On a 24 GB Mac it is tight, and on 16 GB only 2-bit files fit.

How much memory does Inkling 2 Mini need?

Inkling 2 Mini’s memory requirements come to about 18–20 GB for the weights at 4-bit, plus its KV cache: roughly 1.2 GB at a 32K context and 4.5 GB at 128K, calculated from its configuration. Thinking Machines recommends Apple silicon with 32 GB or more, or a GPU with 32 GB or more. Unsloth lists 17–18 GB for its 4-bit files, 13–15 GB for 3-bit and 24 GB for 8-bit.

Is Inkling 2 Mini better than Qwen3.8-27B?

They tie at 20 on Artificial Analysis’s Intelligence Index. In Thinking Machines’ own comparison, Inkling 2 Mini leads on agentic coding (DeepSWE 48.2% vs 42.2%), web research and long context, while Qwen3.8-27B leads on knowledge work, Terminal-Bench 2.1 and factual reliability. Locally, Inkling 2 Mini decodes about twice as fast and needs half the KV cache, but writes about 38% more tokens per task.

How fast is Inkling 2 Mini on a Mac?

Published 4-bit MLX measurements show Inkling 2 Mini decoding about 31 tokens per second on an M4 Pro and about 44 on an M5 Max, with prompt reading at about 1,100–1,750 tokens per second on the M5 Max. Its DSpark drafter raised the M5 Max figure to 70.9 tokens per second but made an M4 Pro slower. Qwen3.8-27B measures about 12 on an M4 Pro, 16.5 on an M5 Pro and 31 on an M5 Max.

What is a looped transformer?

A looped transformer runs some of its layers more than once for each token, reusing the same weights. Inkling 2 Mini runs its middle 12 layers, 6 to 17, three times, so each token gets 48 layer passes from 24 layers of stored weights. That adds computation without adding memory for weights. Each pass keeps its own KV cache.

Does Inkling 2 Mini work with Ollama?

Yes, from Ollama 0.40.17, which was a pre-release on 21 November 2026. The package (inkling-2-mini, also tagged 26b-a5b-nvfp4) is about 20 GB, contains Thinking Machines’ original NVFP4 and MXFP8 weights with vision and the DSpark drafter, and runs on Ollama’s MLX engine on Apple silicon. On Linux and Windows, use llama.cpp or vLLM instead.

Does Inkling 2 Mini work with llama.cpp?

Yes, with the right build. On 21 November, mainline support was still in open pull requests (#32694, plus the #32704 fix for its 4-bit scales). Unsloth’s prebuilt llama.cpp (b12588-mix-c0c3de8 or later) runs Unsloth’s GGUFs and the DSpark drafter, and ggml-org’s GGUFs run in the Llama app. Mixing file layouts and builds produces gibberish, so follow one guide exactly.

Which quantization of Inkling 2 Mini should I use?

On 32 GB, Unsloth’s UD-Q4_K_XL (17.6 GB), which matched its 8-bit reference’s top token 98% of the time, or Ollama’s original-weights package (20.2 GB). On 48 GB or more, 8-bit (22.9–27.8 GB). Below 4-bit, expect a quality drop. On Apple silicon, a community 4-bit MLX build with 8-bit attention decoded faster than the original weights with no measurable loss, according to its builder.

What is DSpark, and should I turn it on?

DSpark is Inkling 2 Mini’s built-in drafter: a small 0.64 GB network that guesses several tokens ahead for the main model to verify in one step. It raised decode speed from 43.6 to 70.9 tokens per second on an M5 Max, but made an M4 Pro slower in every test one builder ran. Measure it on your own machine, with a block size of 3 or 4.

Can I use Inkling 2 Mini with OpenCode or other coding agents?

Yes. With Ollama, ollama launch opencode --model inkling-2-mini ou ollama launch pi --model inkling-2-mini sets the agent up. With llama.cpp, point the agent at llama-server‘s OpenAI-compatible API. Give it at least a 32K context, limit turns and output tokens, and run it in a sandbox, because the model card warns about reasoning loops and actions taken without confirmation.

Is Inkling 2 Mini’s looped design worth it?

Yes, on today’s evidence. Thinking Machines’ variable-loop research model scored 0.7% on DeepSWE at one pass and 47.4% at three, and 56.3% vs 78.7% on SWE-bench Verified, while factual knowledge barely changed. Locally, looping adds computation but no memory, and the model still decodes about twice as fast as dense Qwen3.8-27B. Current runtimes use only about a quarter to a third of its theoretical speed.

Can I run Inkling 2 Mini and Qwen3.8-27B on the same machine?

On 64 GB, yes, both loaded at once: at 4-bit they need about 39 GB together with a 32K context each, calculated from file sizes and their cache requirements. On 48 GB it’s tight: the two 4-bit files alone come to about 35 GB, so you’d need short contexts and a raised GPU memory limit, and loading one at a time is simpler. Ollama keeps several models loaded if memory allows, so an agent can route coding to Inkling 2 Mini and knowledge questions to Qwen3.8-27B.

A conclusão

Inkling 2 Mini is the first mainstream open model built around a looped transformer, and it arrives with a real argument. Same score as Qwen3.8-27B, an eighth of the active parameters, half the KV cache, a native million-token context and about twice the writing speed on the same Mac. On agentic coding, search and long documents, its maker’s numbers put it ahead; Thinking Machines’ variable-loop research model shows that the loop is what gets it there.

It isn’t a clean sweep. Qwen3.8-27B knows more, makes fewer things up, follows multi-turn instructions better and works in every runtime today, while Inkling 2 Mini needs a pre-release Ollama or the right llama.cpp build. On the fastest Macs, Inkling’s wordiness eats its plain-decoding lead, and its drafter speeds it up on some machines and slows it down on others. And local runtimes reach only a quarter to a third of what the architecture could deliver, so the best is probably still to come.

So: on a 32 GB Mac, start with Inkling 2 Mini for agent work. On 48–64 GB, keep both and route by task. On Linux or Windows today, stay with Qwen3.8-27B until mainline llama.cpp catches up. And whatever you pick, spend one evening with the kit. A day-old model on a new architecture is exactly the case where your own numbers beat anyone’s benchmark table, including the ones in this article.


Ran your own Inkling 2 Mini local test? Paste your report.md tables in the comments, with your chip, memory and runtime version. I’d like to add the most useful results to this article, credited, as the measurements come in. Corrections are welcome too; with a model this new, some of today’s numbers will change quickly.

Divulgação: This article contains one affiliate link, to Z.ai’s GLM Coding Plan (in Part 14, repeated in the source list). Subscribing through it gets you 10% off and may earn me a commission at no extra cost to you. No other product or company in this article is sponsored, and none of them reviewed it before publication.

Facts checked: 21 November 2026.

Adicione seu sinal.

O seu endereço de e-mail não será publicado. Campos obrigatórios são marcados com *

pt_BRPortuguês do Brasil