Mistral Large 4: what self-hosting takes for NZ businesses

What Mistral released

Mistral, the French AI company, published the weights of Mistral Large 4 on 29 October, and its technical report followed on 3 November. Weights are the trained model itself, so a business can download the files and run Large 4 on hardware it controls, using vLLM or SGLang as the serving software. The model has about a trillion parameters and a one-million-token context window, and it accepts text and images.

Mistral is selling the release on control. Pierre Stock, its vice president of science, told TechCrunch that access to open weights can't be cut off. The licence lets organisations use, modify, fine-tune and commercialise the model free of charge, with one condition for model-as-a-service businesses. Artificial Analysis scores Large 4 at 26 on its Intelligence Index, tenth of 24 models against a median of 23, which puts the model above average and short of the leaders. Mistral's coding claims, including 47% fewer tokens per task than China's GLM-5.3, are company-reported and should be read that way.

Why the release matters here

Large 4 adds a big model from outside the US and China to the options a NZ business can run under its own control. DataNorth reports that Large 4 has the highest Artificial Analysis score of any Western open-weight model, while Chinese open-weight models score higher. A business whose procurement rules exclude Chinese-built models now has a stronger candidate to test.

Where a model was built tells you little about where your data goes, because data location depends on where inference runs. Through Mistral's own API, the default endpoint makes no promise on processing location and the EU endpoint costs 10% more, so neither route promises to keep prompts in NZ. Microsoft Foundry also offers the model. We couldn't find a published region list for Large 4 on Foundry, so ask Microsoft directly.

The Privacy Commissioner's guidance frames the provider question clearly. A provider that only stores or processes data on your behalf isn't deemed to hold the information, while use for its own purposes, such as AI training, means both parties hold it. The same page says IPP 12 may also apply to a provider based overseas, and that your organisation stays responsible for what it sends. Running Large 4 on servers you own takes the model vendor out of that chain. A NZ hosting company would still be a third-party provider under the same test, and the Privacy Act duties stay with your organisation either way.

The hardware bill decides most cases

Large 4 is server-room equipment. The full model needs about 1TB of GPU memory. The compressed NVFP4 version is 606GB and runs on four Nvidia GB300 GPUs, and Hugging Face's deployment examples use eight GPUs, or four B300s on Dell hardware. Cheaper routes exist, but they're slow. Hugging Face reports about 15 tokens a second on a 512GB Apple M3 Ultra with a 2-bit build, and about 10 tokens a second on four RTX 6000 Pro cards. Those speeds suit one person testing, and a busy team would queue.

A Telnyx pricing guide reviewed in October lists 8-GPU B300 systems at US$300,000 to US$650,000 on reseller quotes. It puts on-demand rental at US$7.10 to US$17.80 per GPU hour, with about 14kW per system before cooling and direct liquid cooling required. Those are US figures, so NZ hosting prices will differ.

At the lowest listed rate, four GPUs running for a 730-hour month cost about US$20,700. Mistral's API charges US$0.70 per million input tokens and US$2.10 per million output tokens. If you assume three input tokens for every output token, the blended rate is US$1.05 per million, so US$20,700 buys roughly 20 billion tokens a month. We'd expect few mid-sized NZ firms to get near that volume, so the case for self-hosting Large 4 has to rest on control of data and access, because savings won't carry it.

NZ telco 2degrees chose an HPE private AI platform in January to keep critical data onshore, which shows the approach is already in use here. The reporting describes an internal system for a large network operator, and it doesn't mention capacity for other businesses.

Most businesses should look at smaller models. Nvidia's Nemotron 3.5 Bolt has a 24GB checkpoint with about 22GB of weights under a licence that permits commercial use, according to a guide from GPU cloud vendor Spheron, which also says one request at a time is the practical limit on a 24GB card. Mistral told The Next Web that distilled versions are coming, including one for laptops, without giving dates. Workstation-class systems such as a private on-premise AI server belong to that smaller class, and the model choice should follow the task.

Licence and security duties move to you

Internal use is exempt from the commercial licence clause, and so are end-user products where the model sits inside particular features. The same analysis quotes the licence as treating APIs and hosted endpoints as model-as-a-service where third parties have meaningful control over inputs or parameters. DataNorth says it's unclear where a managed setup for one client crosses that line. NZ software vendors and managed service providers should get legal advice before offering Large 4 to customers.

Security deserves the same attention. Mistral's release post says "we see a potential for uplift to conduct real attacks" and that the company is changing its guardrails. Those changes reach the managed API and future releases, and copies already downloaded stay as they are. Gartner's Niyati Daftary told InfoWorld that Mistral's 96.6% refusal rate reflects the company's own test scenarios, and recommended isolated environments, least-privilege access, sandboxed generated code and human sign-off before remediation. A model connected to your ERP or document store can reach whatever its tools can reach, so a malicious instruction hidden in a supplier email becomes your problem to contain.

Next steps

• Record where each AI workload runs today, including any cross-region routing, and rank the workloads by data sensitivity.
• Measure your monthly token volume and input-to-output mix, then rerun the 20 billion token calculation with your own numbers and local quotes.
• Trial Large 4 through the API with non-sensitive data, then compare the results with a smaller model on your own documents, since vendor benchmarks won't reflect your work.
• Ask any NZ or Australian host to confirm in writing where the GPUs sit, who has admin access, what gets logged and whether the host uses your data for its own purposes.
• Name an owner for access controls, logging and model updates before go-live, because guardrail changes only arrive when you download a new version.
• Read the licence before any client-facing use, especially if you plan to resell access.