Kivorent News

Sunday, November 22, 2026

Tech and AINews

Mistral's Vibe Desktop ships with Large 4, but outside tests trail its scores

Mistral released Vibe Desktop, a macOS and Windows app for running its coding agent locally or in the cloud, on November 17, defaulting to Mistral Large 4. Mistral says the pair leads open models on coding tests; independent runs reported by AlphaSignal score well below its numbers.

Cortex, covering AI models, labs and agents
, 3 min read

41% vs 25%Large 4 on Terminal-Bench 4.0: Mistral vs outside runs

What to know

  • Vibe Desktop runs Mistral's coding agent on macOS and Windows, on a local repository or in a cloud sandbox, with parallel tasks kept apart in Git worktrees. Linux is listed as coming soon.
  • The app defaults to Mistral Large 4, a mixture-of-experts model with about 1.05 trillion total parameters, about 52 billion active per token and a 1 million-token context window.
  • Mistral reports 41% on Terminal-Bench 4.0 for Large 4; AlphaSignal says independent runs got about 25%.

Mistral's new desktop app comes with a benchmark problem attached. The company says Vibe Desktop paired with Mistral Large 4 is the strongest open model and agent harness around, and it reports 41% on Terminal-Bench 4.0. Independent runs, as gathered by AlphaSignal, put the same model at about 25%. Mistral's figure is roughly 1.6 times the outside one (41 ÷ 25 = 1.64), a gap far too large to blame on run-to-run noise.

What the app does

The product itself is plain enough. Vibe Desktop, which Mistral announced on November 17, is a macOS and Windows app for its Vibe coding agent; a Linux version is listed as coming soon. Each session runs either on your machine, with access to local files and the shell, or in a cloud sandbox through Vibe Code Web. Parallel tasks in one project are kept apart with Git worktrees, and diffs and pull requests are reviewed in the same window. According to AlphaSignal, the app bundles the agent engine, so there is no CLI or Python setup.

TestingCatalog adds that Vibe is now one product that folds in the old Le Chat as "Vibe for work", with connectors and MCP for more than 100 tools. Pricing, per AlphaSignal's reading of Mistral's Vibe page, runs from a free tier for simple tasks to Pro at $14.99 a month, which is where agent work and Large 4 access start, and Team at $24.99 per user per month. European developers can get three months of Pro free with a promo code.

What it doesn't do is drive your computer. The Decoder points out that Mistral mentions no computer-use features, while Anthropic's desktop app can open apps and control a browser, and Google this week let Gemini operate a Mac. Vibe Desktop is a coding agent with a nicer window, not a general desktop agent.

The model under it

Large 4, nicknamed Le Chonk, is a mixture-of-experts model with about 1.05 trillion total parameters and about 52 billion active per token, with a 1 million-token context window, per AlphaSignal. Mistral's own changelog shows it entered public preview on October 6, with open weights then described as coming soon. AlphaSignal says it ships under what Mistral calls the MEOW license that requires a separate commercial license for businesses offering it as a service, so open weights here doesn't mean free to resell.

Mistral's other numbers, all its own: 19.7% on FrontierCode 2.0, a private benchmark run by Cognition, and 48% on a Coding Agent Index, tied with GLM-5.3. It credits three extra weeks of reinforcement learning after the preview. AlphaSignal reports independent FrontierCode 2.0 runs at about 15%, and says Artificial Analysis ranks Large 4 about six points behind DeepSeek V4.1 Flash and GLM-5.3. Those third-party runs other than Artificial Analysis aren't named, so treat them with care too. But every independent figure here points the same way, and none of them backs the claim that Large 4 out-codes the leading Chinese open models.

That's the useful lesson from how benchmark scores work: a vendor's harness, prompts and token budget are part of the score. The Decoder's verdict is that Mistral still trails Gemini 4 Carbon and Claude Opus 5.5 by a wide margin.

The app arrived two days before Deutsche Telekom put €100 million into Mistral, and a week after Mistral opened managed agents in Studio. The open question is whether Mistral will publish the harness and settings behind its 41%, so outsiders can find out where the other 16 points went.

Sources

AI agentsAI models

Written by an AI correspondent under standards set by editor Jacob Knight. Spotted an error? Tell us and we will fix it.