Last Thursday Mistral put the full weights of a 1-trillion-parameter frontier model on Hugging Face, and it fits on a single 8-GPU node. The same company says the model shows "a potential for uplift to conduct real attacks", and the guardrail fixes it ships from now on will not reach the copy you download.
AG Tech Consulting's advice for clients this week: if you want a self-hosted frontier model, put Mistral Large 4 on your evaluation list, but run it as a controlled component, not a free agent. Size it on your own hardware (FP8 needs eight GPUs, NVFP4 needs four Blackwell GPUs), get legal to read the Model-as-a-Service clause before anything is exposed to customers, and treat guardrails, monitoring and red-teaming as your team's job from day one, because the vendor's safety updates now stop at its API.
What Mistral released
On 29 October Mistral released the open weights of Mistral Large 4, nicknamed "Le Chonk". It is a natively multimodal Mixture-of-Experts model: about 1 trillion parameters in total, 49 billion active per token (52 billion including embeddings and the output layer), text and image in, text out, and a 1M-token context window.
Per the model card, it was trained from scratch on 3,800 NVIDIA Grace Blackwell GPUs in Mistral's own datacenters in Europe. The model was in public preview earlier in October; Mistral says "just three weeks of additional reinforcement learning" went into the weights it released.
The claims are strong, and all of them are Mistral's own. It calls Large 4 the best open-weight model for coding: 19.7% on Cognition's private FrontierCode 2.0, 41% on Terminal-Bench 4.0, and 48% on the Artificial Analysis Coding Agent Index, effectively tied with GLM-5.3 while using 47% fewer tokens per task. On the Artificial Analysis Cyber Index it scores 51%.
Let us be very clear about what is missing. The technical report has not been published yet. Forrester's Charlie Dai told CSO Online that without it, "the best open-weight model claim remains difficult to validate across workloads."
Inside the architecture
Large 4 uses two different kinds of sparsity at once. The MoE router picks which feed-forward experts run for each token, and a sparse attention indexer picks which earlier tokens each query looks at.
Attention. The decoder has 72 layers, a hidden size of 6,144 and 64 heads. It builds on the DeepSeek-V3.2 recipe: Multi-head Latent Attention (MLA) keeps a compressed 512-dimensional KV latent, and a DeepSeek Sparse Attention (DSA) indexer selects the top 2,048 key positions for each query. Of the 72 layers, 44 run their own indexer and 28 reuse the selection of the nearest earlier indexer layer.
Cross-layer KV sharing. From layer 39 onward, every fourth layer (nine layers in all) does not write its own KV latent and reads layer 35's instead. Only 63 of 72 layers own a cache. vLLM measures 47,136 bytes of KV per token, 11.1% less than if every layer kept its own.
Position handling. Every fourth layer uses no rotary embedding (NoPE). The rest use YaRN-scaled RoPE with LeRoPE, a learned multiplier per frequency, extending a 512-token base by a factor of 2,048 to 1,048,576 positions. SSMax scales each query by a learned, length-dependent factor.
Experts. Layer 0 is dense. Layers 1 to 71 route each token to 4 of 128 experts, plus one shared expert that always runs. Final logits are soft-capped at 30, and a 1.6B Pixtral encoder handles images.
Speculative decoding in the box. The checkpoint ships its own DSpark draft head: three small MLA layers that propose a block of three tokens at once, with a recurrent Markov head so each drafted token depends on the one before. The target model verifies every token, so outputs do not change. vLLM reports about 2.9 tokens accepted per step and roughly 2x output throughput at TP8.
One practical warning from the Transformers docs: in the eager and SDPA backends, the DSA indexer is applied as a mask over dense attention. You get the same outputs, but not the long-context speed-up. The serving engine you choose decides whether the sparsity pays.
What it takes to run
The point is simple: a 1T model is now a rack-level decision, not a supercomputer decision. These are the published paths.
NVIDIA, FP8: about 1 TB of GPU memory, served with tensor parallelism across eight GPUs. Hugging Face's Inference Endpoints recipe uses 8x H200, with about 0.6 seconds to first token on a 1K prompt. NVIDIA, NVFP4: four Blackwell GPUs. vLLM reports scores within 2.1 points of FP8 on every evaluated task (mean difference -0.3 points), 1.08-1.19x faster decode per token, and about 1.5x the KV cache. AMD Instinct MI350 Series: AMD's Quark MXFP4 checkpoint re-quantizes only the routed experts, shrinking the model from about 1,061 GB to 578 GB, enough for four 288 GB GPUs. Google TPU v7x: day-0 support via vllm-torchtpu, validated on 16 chips. Apple Silicon: 2-bit GGUF quants run on a single Ultra machine with 512 GB of unified memory. Hugging Face measured about 15 tokens per second of generation with Q2_K on an M3 Ultra. Good for a lab, not for production.
Long context has its own trade-offs. Decode context parallelism (DCP) shards the latent KV cache by token and lifts NVFP4 KV capacity on TP8 from 3.60M to 28.76M tokens, but decodes 16-28% slower per token than plain TP8. vLLM's HiSparse keeps the full KV in host memory: on one GB300 node with a 440 GiB host pool, capacity rose from 1.97M tokens on GPU alone to 8.77M at startup.
There is one quieter detail worth copying. vLLM's AMD latency-specialised decode path is fail-stop: if any in-kernel wait exceeds 10 seconds, the worker aborts so that "no token from a failed step reaches a client." That is the kind of engineering discipline a self-hosted stack needs everywhere.
Read the licence before the GPUs
Large 4 ships under the Mistral Ecosystem Open-Weight License, a modified MIT licence. Use, modification, fine-tuning and commercial use are allowed, with one condition.
If you or any affiliate operate a "Model as a Service" business, you need a separate commercial licence from Mistral before commercial use. The licence defines MaaS as giving a third party access for inference or post-training "in a manner that allows such third party to exercise meaningful control over the inputs, parameters, or training data."
The exclusions matter just as much:
Internal use, where the model is not made available to third parties. End-user products where model capabilities are "solely embedded within specific features or harnesses". Merely relaying requests to models hosted by others. Use through Mistral's official products.
A customer-facing feature with a fixed harness probably sits in the exclusion. An API or platform that lets customers send their own prompts, parameters or fine-tuning data probably does not. That line must be drawn by your legal team, in writing, before launch.
The future security risk: guardrails that never update
Here is the part every CISO should read twice. Mistral's own release post says: "Though we observed the model refusing to comply with black-box attacks against live targets, we see a potential for uplift to conduct real attacks. We are modifying our guardrails accordingly, and will continue updating them in the months to come."
Pierre Stock, Mistral's VP of science, told CSO Online where those updates go: "Subsequent safeguard updates are applied to our managed API and future model releases, rather than automatically updating downloaded checkpoints."
So the safety behaviour of every downloaded copy is frozen at the 29 October release. With an API model, a vendor fix lands for everyone overnight. With open weights, the fix lands only when an operator chooses to download and redeploy a new release, and any fine-tuned derivative carries whatever behaviour its builder left in it.
Four risks follow:
1. Frozen refusals. Mistral reports a 96.6% refusal rate on malicious cyber prompts from JailbreakBench, StrongREJECT and AgentHarm. Gartner's Niyati Daftary cautions that the figure "tells you what their model does in their test scenario, not how it will behave with your data."
2. Irreversible release. The UK AI Security Institute said in July that open-weight models trailed the closed frontier on cyber tasks by four to seven months, and that for highly cyber-capable models, open release creates "a persistent and irreversible risk of misuse." Your own defenders and outside attackers now have the same weights.
3. Shadow deployments. A 2-bit quant runs on one workstation. Expect copies to appear outside your platform team's view, each with its own unmanaged guardrails.
4. Ownership shift. Dai's summary is exact: open weights and self-hosting move responsibility for guardrails, access controls, monitoring and misuse prevention to the operator. "Sovereignty and security therefore need to be evaluated together, not traded against each other."
Mistral is handling its specialised cyber variant differently: preview access only for vetted design partners such as Orange Cyberdefense, under contractual safeguards, on owned or authorised systems. That contrast tells you how the vendor itself rates the risk.
Checklist for this week
Decide which workloads need a self-hosted frontier model (data residency, offline incident response, fine-tuning) and which are fine on an API. Run your own evaluations on your own tasks before trusting any self-reported benchmark; wait for the technical report before a final decision. Size hardware from the published recipes: FP8 on eight GPUs, NVFP4 on four Blackwell GPUs, MXFP4 on four MI350 Series GPUs, and plan KV capacity for your real context lengths. Pick the serving engine deliberately (vLLM, SGLang or TensorRT LLM); eager or SDPA paths lose the sparse-attention speed-up. Get a written legal opinion on the Model-as-a-Service clause for every planned external use, including affiliates. Put your own guardrail layer in front of the model: input and output filters, tool allow-lists, and a short tool list as the model card itself advises. Run security use cases (code audit, vulnerability triage) as a controlled component: isolated environment, least-privilege access to code and credentials, sandboxed generated code, human sign-off before any fix ships. Track Mistral's releases and schedule redeploys; record the checkpoint hash in production so you know which safety behaviour is live. Inventory unofficial copies and quantised GGUFs on engineer workstations and lab GPUs. Red-team the deployment yourself before go-live and after every model or fine-tune change.
Bottom line
Mistral Large 4 is a genuine shift: frontier-class, multimodal, permissively licensed weights that one rack can serve. For regulated and sovereignty-minded enterprises, this is the need of the hour.
But the moment you download it, you become the safety vendor. Budget for that role as seriously as you budget for the GPUs.
Sources
https://mistral.ai/news/mistral-large-4-oss/ https://huggingface.co/mistralai/Mistral-Large-4-1T-A52B https://huggingface.co/mistralai/Mistral-Large-4-1T-A52B/blob/main/LICENSE https://huggingface.co/mistralai/Mistral-Large-4-1T-A52B-NVFP4 https://vllm.ai/blog/2026-10-29-mistral-large-4 https://huggingface.co/blog/mistral-large4 https://huggingface.co/docs/transformers/model_doc/mistral_large4 https://www.csoonline.com/article/4242648/mistral-releases-large-4-weights-despite-warning-of-cyberattack-uplift.html https://www.neoteo.com/en/mistral-large-4-weights-wont-get-automatic-guardrail-updates
Also on our website: https://agtechconsulting.ai/insights/frontier-model-in-your-racks-guardrail-updates-will-not.html
Want help deciding where a self-hosted frontier model fits, sizing the hardware, reading the licence against your product plans, or building the guardrail and monitoring layer that the vendor no longer updates for you? Email avinash@agtechconsulting.ai, visit agtechconsulting.ai, or book a 30-minute call: https://calendly.com/avinash-dubey-30-min-catchup/meeting-with-ag-tech
Thanks to Mistral , Hugging Face , vLLM , NVIDIA , AMD , Red Hat , Microsoft , Orange Cyberdefense , Gartner , Forrester , CSO Online , for the work this builds on. Also thanks to Arthur Mensch , Jeff Boudier , Kari Ann Briski , Cisco , Cloudflare , Pierre Stock
Originally published on LinkedIn in the Your CTO Advisor newsletter: read and comment on LinkedIn →
