Welcome Le Chonk: Mistral Large 4 on Hugging Face
2026-11-02 · Hugging Face
Welcome Le Chonk: Mistral Large 4 on Hugging Face
Mistral Large 4, also called Le Chonk, is a multimodal, multilingual Mixture-of-Experts model from Mistral. It accepts text and images and generates text. It has about 1T total parameters, 49B active parameters, and a 1M-token context window. It is released in two official variants, FP8 and NVFP4, with GGUFs created by Unsloth and the llama.cpp team. It is distributed under the Mistral Ecosystem Open-Weight License, an MIT derivative with limitations for model-as-a-service offerings.
What is new
Mistral Large 4 combines a large expert pool, sparse attention, and a Pixtral vision encoder. The language model selects a small group of experts for each token, while its attention indexer selects which earlier token positions to attend to. These are two forms of sparsity: one selects feed-forward networks, the other selects context positions. Checkpoints are available in the Mistral Large 4 collection.
Cybersecurity capabilities
The model scores 82% on the Artificial Analysis Cyber Index, which asks a model to reproduce a real vulnerability in open-source software and then patch it. Mistral reports that several closed frontier models score near zero because they refuse to attempt the work. Defenders often need to reproduce a vulnerability before fixing it. In internal testing, Mistral Large 4 was also useful for malware analysis, vulnerability prioritisation, and writing detection rules, though it was not explicitly trained for these tasks.
Mistral positions the model mainly for defensive security. Vulnerability research and incident response can look like attacks in model input, so closed-source filters can block them; API-dependent teams can also lose access during an incident. Open-weight models put usage policy and deployment under user control. The model still declines harmful requests: on cyber prompts from JailbreakBench, StrongREJECT, and AgentHarm, its average refusal rate is higher than any other open-weight model, according to the post.
For community builders, open weights enable downstream use cases. Teams can build cyber systems around their codebase, crash logs, and malware samples. With TRL, they can fine-tune smaller models from Mistral Large 4 traces that patch their codebase. Reproduce-then-patch tasks use verifiable rewards, fitting RL environments for further training smaller security models. Open weights also let researchers measure refusal behaviour directly.
Architecture
The text decoder has 72 layers, a hidden dimension of 6,144, and 64 attention heads. Its attention blocks combine Multi-head Latent Attention with a DeepSeek Sparse Attention indexer. MLA keeps a compressed KV representation, and DSA selects up to 2,048 key positions per query. Each decoder block follows a 3:1 RoPE-to-NoPE pattern: three decoder layers use RoPE and every fourth uses NoPE. The model shares KV state across layers. The first decoder layer uses a dense feed-forward network; the other 71 use MoE blocks, where a softmax router selects four of 128 experts per token. One shared expert always runs, for five active experts per token. For RoPE, Mistral uses LeRope, a YaRN-scaled RoPE with one learned multiplier per frequency. The vision encoder is a 1.6B Pixtral encoder with 30 layers, hidden dimension 2,048, 16 attention heads, 14×14 patches, and a patch-merge projector with spatial merge size 2.
Benchmark results
Mistral Large 4 shows strong results across coding, cybersecurity, multimodal understanding, and agentic workflows. As of October 29, 2026, it is the fifth best open-weight model on the Artificial Analysis Intelligence index. On coding, it achieves 61.7% on DeepSWE v1.1, 59.4% on SWE-Atlas-QnA, and 28.3% on Terminal-Bench 4. Its combined Coding Agent Index score of 49.8% places it ahead of DeepSeek V4 Pro 0813 and Qwen3.8 Max. Its coding capabilities extend to knowledge work, with 59.9% on AutomationBench and 1,393 Elo on AA-Briefcase. Cybersecurity is a standout capability, with strong results in real-world vulnerability assessment and security competition tasks; the post lists three cyber benchmarks and mentions the Artificial Analysis Cyber Index.
Inference and fine-tuning support
The original post lists inference support for Transformers, vLLM, and Llama.cpp; fine-tuning with TRL; Inference Endpoints; Dell Enterprise Hub; and a try-it-out section.