Inkling 2 Mini vs Gemma 4 31B vs Qwen3.8-27B: A Local Model Buyer Guide for Late 2026
We compare memory footprints, benchmark claims, and practical deployment trade-offs for developers choosing an on-device foundation model.
10 min read
Why local model shopping changed in late 2026
For years, the local Large Language Model buyer's decision tree was brutally simple: pick the largest model your GPU could hold, accept sluggish tokens per second, and hope quantization did not lobotomize reasoning. Releases in November 2026 shattered that script. Thinking Machines Lab's Inkling 2 Mini arrived as an open-weights checkpoint roughly 20GB on disk, with 26.4B total parameters but only 5.4B effectively active per token thanks to a looped transformer that reuses middle layers three times. On paper it competes in the same mental bucket as Google's Gemma 4 31B and Alibaba's Qwen3.8-27B—models you might run on a single high-end consumer rig with 32GB of unified memory or VRAM. In practice, the benchmarks tell a story of specialization: Inkling 2 Mini is not the best at everything, but it is disproportionately strong at agentic coding, tool use, and long-horizon tasks that dominate how developers actually spend local inference hours.
This guide compares the three models for local dev setups: hardware, memory math, software stacks, benchmark cliffs, and the buyer personas each model serves honestly.
Spec sheet at a glance
Understanding the comparison requires separating total parameters, active parameters, precision, and runtime loops.
| Dimension | Inkling 2 Mini | Gemma 4 31B | Qwen3.8-27B |
|---|---|---|---|
| Total params | 26.4B (MoE + looping) | 30.7B dense | 27.0B dense |
| Active params / token | ~5.4B (3 passes through looped layers) | 30.7B | 27.0B |
| Native precisions | NVFP4 + MXFP8 last layers | Typically INT4/FP16 in local ports | INT4/FP16/GGUF variants |
| Approx. checkpoint size | ~20GB | Often 18–22GB quantized; higher if FP16 | Similar band depending on quant |
| Context (vendor claim) | Up to 1M tokens (modalities mixed) | Model-card dependent | Strong long-context in Qwen line |
| Modalities | Text + image (ViT encoder) | Varies by Gemma 4 SKU | Multimodal variants in family |
| Looped compute | Yes—middle layers run 3× | No | No |
Inkling's loop is not a gimmick; it buys depth without storing another twenty billion weights. Each extra pass adds roughly 1B effective active FLOPs per token in Thinking Machines' accounting. That matters on laptops: you trade higher per-token latency for better reasoning traces on tasks that reward deliberation—software engineering agents, kernel tinkering, multi-step search.
Gemma 4 31B and Qwen3.8-27B are more traditional single-pass decoders at full width. When they fit in memory, they can be straightforward and predictable on tokens/sec. When they do not, you quantize and watch hard benchmarks sag.
Memory and hardware: who can run what
Thinking Machines publishes explicit local targets for Inkling 2 Mini: 32GB or more on Apple silicon (MacBook, Mac mini, Studio), NVIDIA DGX Spark, and generally ≥32GB VRAM GPUs for comfortable inference. The ~20GB checkpoint is a deliberate choice—it is the first "frontier-ish" agent model many pros can colocate with an IDE without renting an A100.
Inkling 2 Mini on 32GB unified memory (Apple): Feasible with MLX, Ollama, or Exo-style runners; expect to close browser tabs. Looping increases per-token work—thermals and battery matter on laptops. Strength is overnight agent jobs, not twitchy autocomplete.
Gemma 4 31B on 32GB: Tight. Q4 quants or aggressive offload make it possible but you are living at the edge. On 48–64GB workstations, Gemma becomes a sane daily driver for general chat and mid coding.
Qwen3.8-27B on 32GB: Similar story to Gemma; community GGUF ecosystems are mature. Many teams already standardized on Qwen for bilingual (Chinese/English) workflows—an underrated buying factor.
48GB+ VRAM (4090, 5090, dual-GPU hobbyists): All three are comfortable; choose by task, not survival.
24GB cards (3090/4090 consumer): Qwen and Gemma quants often win on usability; Inkling's native NVFP4 stack needs framework support. Check your runner before buying a GPU for one model.
Rule of thumb: if your budget caps at one machine under $3k, Inkling 2 Mini's footprint is the most plausible path to SWE-agent-class behavior without cloud rent. If you already own 64GB RAM, Qwen3.8-27B is the balanced generalist with fewer moving parts.
Software ecosystem friction
A model is only local if your stack loads it.
Inkling 2 Mini expects modern runners: SGLang, vLLM, Unsloth, NVIDIA Dynamo, Tokenspeed, plus GGUF paths via llama.cpp/Unsloth for experimentation. Apple side: Ollama, Exo. RL fine-tuning hooks mention Miles, SkyRL, NeMo AutoModel—a signal that the vendor imagines you continuing training, not just chatting. That is great for research shops, mildly annoying for beginners who want one ollama run and silence.
Gemma 4 31B benefits from Google's distribution gravity: Ollama, Kaggle, Vertex, and vast tutorial surface. Integration with Google tooling can matter for teams already on GCP—even when inference is local.
Qwen3.8-27B remains the community's default wrench: llama.cpp, vLLM, LM Studio, MLX ports, endless quant recipes. If your org standardizes on Qwen tool-calling JSON, switching costs are real.
For agent frameworks (OpenHands, mini-swe-agent, Cursor-adjacent CLI agents), benchmark tables in Thinking Machines' model card show Inkling leading DeepSWE v1.1 and SWE-bench Pro v2 (Hard) against Gemma and often edging Qwen on the hardest splits—though Qwen wins some terminal benchmarks depending on harness version. That split is exactly what a buyer should inspect: your agent harness version may differ from the leaderboard.
Benchmarks: where each model wins
No single score sells a local model. Use clusters aligned to your work.
Coding and software engineering agents
Thinking Machines reports (November 2026 snapshots):
- DeepSWE v1.1: Inkling 2 Mini 48.2% vs Gemma 4 31B 0.3% vs Qwen3.8-27B 42.2%. This is not a typo band—Gemma is uncompetitive on this agentic coding suite in published numbers.
- SWE-bench Pro v2 (Hard): Inkling 77.9% vs Gemma 14.2% vs Qwen 73.8%. Inkling and Qwen are peers; Gemma lags hard on the "Hard" split.
- SWE-bench Verified (bash only): Inkling 79.2%, Qwen 77.8%, Gemma 46.1%.
- Terminal Bench v2.1 (pi harness): Qwen 73.0% noses Inkling 71.6%; Gemma 43.4%.
Interpretation: if you run coding agents locally—letting the model edit repos, run tests, iterate—Inkling vs Qwen is a genuine horse race; Gemma is for lighter automation, not hard repo surgery.
Kernel and autoresearch flavor
PMPP-Hard (AUC, long horizon): Inkling 455.82 vs Qwen 262.50 in published tables—Inkling shines on performance engineering-shaped tasks. If you touch GPU kernels or care about autoresearch loops, Inkling's looped depth appears to pay rent.
Search, tools, and knowledge work
Inkling leads several BrowseComp / DeepSearchQA / MCP Atlas style benches at comparable sizes, reflecting design for tool invocation and compaction over static parametric memory. Thinking Machines is explicit: looping improves reasoning-ish contexts but does not stuff more facts into weights. Inkling compensates by calling search well.
Gemma's strength is not these tables—it is Google-aligned general assistant quality, safety tuning familiarity, and predictable instruction following on everyday tasks when you are not spinning an autonomous agent for six hours.
Qwen sits in the middle on knowledge indices (AA Index v5.0 ties Inkling at 20.1 in vendor comparisons) while dominating if you need Chinese-English fluency or huge community forks.
Multimodal and chat
Inkling ships native image input with a ViT encoder; Gemma 4 family's multimodal SKUs vary; Qwen's VL variants compete. For screenshot-to-code hobbyists, validate your exact runner build—multimodal is where "works on my machine" fractures.
Energy and latency (the hidden benchmark)
Stanford's intelligence per joule framing appeared in Thinking Machines' Inkling 2 Mini launch comms for a reason: looped models burn more per token even when they are smaller on disk. Local buyers should measure watt-hours per completed agent task, not vanity tokens/sec. A fast dumb model that fails three times can cost more than a slow smart one that finishes once.
Risk factors and fine print
Numerics: Inkling's native NVFP4/MXFP8 stack is cutting edge. If your toolchain only supports older quants, you may leave performance on the table—or not run at all until converters catch up.
Hallucination and destructive actions: Community guides note Inkling's agent scores can pair with higher willingness to act in shells. Local agent setups need guardrails: confirmations, sandbox git, branch policies.
Licensing and updates: Compare licenses (Gemma, Qwen, and vendor-specific Inkling terms) if you ship products on top of weights.
Benchmark hygiene: Vendor tables cherry-pick harnesses. Qwen winning Terminal Bench v2.1 on one configuration while Inkling wins v4.0 on another is a lesson: replicate with your prompt templates.
Buyer personas: who should buy which
Buy Inkling 2 Mini if you are…
- A solo developer or small team running autonomous coding agents on a 32–64GB Mac or workstation.
- Chasing SWE-bench-class outcomes without API spend, and willing to tune runners.
- Doing performance engineering, long-horizon refactors, or tool-heavy research loops where search and MCP matter more than static trivia.
- Comfortable being an early adopter on NVFP4 stacks.
Skip if you need lowest latency autocomplete, refuse to babysit inference servers, or only want polite general chat.
Buy Qwen3.8-27B if you are…
- A generalist local user wanting one model for coding, writing, translation, and multilingual support.
- Already embedded in Qwen tool-calling conventions and community quants.
- On 24–32GB hardware where mature GGUF is safer than bleeding numerics.
- Needing near-parity with Inkling on many coding benches without loop-induced latency spikes.
Skip if your workload is dominated by hardest agentic coding splits where Inkling's margin matters more than convenience.
Buy Gemma 4 31B if you are…
- Invested in Google Cloud / Gemma tooling and want weight parity with Google's safety and tuning story.
- Running lighter codegen, education, or assistant tasks—not overnight agent marathons on nasty repos.
- Prioritizing tutorial surface and integration breadth over leaderboard peaks on DeepSWE.
Skip if you are explicitly buying for software engineering agents; published hard-bench gaps vs Inkling/Qwen are enormous.
Sample build recipes (November 2026)
The Apple studio agent rig: Mac Studio M2/M3 Ultra 64GB, Inkling via MLX or Ollama experimental builds, OpenHands or pi-style harness, repo sandboxes on external SSD. Expect fan noise; schedule long jobs.
The pragmatic Linux tower: RTX 4090 24GB—Qwen3.8-27B Q4_K_M via llama.cpp for daily driving; rent cloud Inkling bursts for nasty merges if needed.
The dual-use home lab: 48GB GPU, Inkling primary for agents, Gemma sidecar for family-safe chat on a different port—separates policy and risk.
The budget laptop: Forget 31B dense at full quality; use smaller Qwen instruct models or cloud API for heavy lifts.
Comparison philosophy: density vs loops vs ecosystem
Gemma 4 31B sells familiar dense transformer economics: what you load is what you burn. Qwen3.8-27B sells community scale and multilingual breadth. Inkling 2 Mini sells compute recycling—loops that fake depth—and agent-first tuning, backed by Thinking Machines' research on variable passes improving long-context graph walks without fixing parametric knowledge limits.
None replaces cloud frontier models on every task. All three can erase cloud bills for the right workflows if you accept ops toil.
Verdict matrix
| Priority | First pick | Runner-up |
|---|---|---|
| Hardest local agentic coding | Inkling 2 Mini | Qwen3.8-27B |
| General local daily driver | Qwen3.8-27B | Gemma 4 31B |
| Google ecosystem alignment | Gemma 4 31B | Qwen3.8-27B |
| 32GB-first hardware | Inkling 2 Mini (if runners supported) | Qwen quantized |
| Kernel / perf engineering | Inkling 2 Mini | Qwen3.8-27B |
| Beginner friendliness | Qwen3.8-27B | Gemma 4 31B |
Final word
The Salt Index moment for local technology in late 2026 is not "which 70B fits on four GPUs." It is whether your local stack can close an agent loop—edit, test, search, recover—without sending proprietary code to someone else's datacenter. Inkling 2 Mini, Gemma 4 31B, and Qwen3.8-27B all answer that question differently. Inkling bets loops and tools beat brute parameter width. Qwen bets ecosystem and versatility beat specialization. Gemma bets integration and accessibility beat peak agent scores.
Run your own harness on your repo before you trust any buyer's guide—including this one. The right model is the one that finishes your tasks at acceptable joules per hour, not the one with the prettiest row in a launch blog. If that winner is Inkling, you are buying a deliberate agent machine. If it is Qwen, you are buying the community default. If it is Gemma, you are buying Google's on-ramp. Know which story is yours, then buy silicon accordingly.


Comments
Loading comments…