On April 5, 2025, Meta released Llama 4 Scout and Llama 4 Maverick, the first Llama models built on a mixture-of-experts (MoE) architecture and trained to be natively multimodal from the start. Scout advertises a 10-million-token context window and fits on a single Nvidia H100 with 4-bit quantization, while Maverick targets higher quality at a similar per-token cost. For developers, the Llama 4 release is a lesson in sizing MoE models, reading a custom license carefully and testing vendor benchmark claims before you commit.

Key Facts#

  • Release: Saturday, April 5, 2025, with weights on llama.com and Hugging Face. Meta AI in WhatsApp, Messenger, Instagram Direct and on the web moved to Llama 4 the same day.
  • Llama 4 Scout: 17B active parameters out of about 109B total, 16 experts, and a 10M-token context window for the instruction-tuned model.
  • Llama 4 Maverick: 17B active parameters out of about 400B total, 128 experts, a 1M-token context window, and weights in both BF16 and FP8.
  • Training data: about 40 trillion tokens for Scout and 22 trillion for Maverick, including public Instagram and Facebook posts and interactions with Meta AI, with an August 2024 cutoff. Pretraining covered 200 languages, and 12 are officially supported.
  • Teacher model: Llama 4 Behemoth, which Meta describes as having 288B active parameters and 16 experts, was previewed as still in training and not released.
  • License: the custom Llama 4 Community License, with a 700-million-monthly-active-user threshold, "Built with Llama" attribution and naming rules for derivative models.
  • Compute: Meta's model card reports 7.38 million H100 GPU hours for pretraining.

What Happened#

Meta called Scout and Maverick "the first models in the Llama 4 herd." Both use early fusion, which means text and image tokens go through the same backbone during pretraining instead of a vision encoder being attached later. To reach very long contexts, Meta combined standard rotary position embeddings (RoPE) with layers that use no positional encoding at all. According to Hugging Face's technical breakdown, every fourth layer drops RoPE and attends over the full context, while the other layers use chunked attention to save memory. Meta calls this design iRoPE. Maverick alternates MoE and dense layers and was co-distilled from Behemoth.

Meta's benchmark claims were ambitious. The company said Scout beats Gemma 3, Gemini 2.0 Flash-Lite and Mistral 3.1 across widely reported benchmarks. It said Maverick beats GPT-4o and Gemini 2.0 Flash, and matches DeepSeek V3 on reasoning and coding with less than half the active parameters. All of these are vendor claims.

The ecosystem was ready on day one. Hugging Face shipped support in Transformers 4.51.0 and Text Generation Inference (TGI), along with code for on-the-fly int4 quantization of Scout. Maverick's FP8 weights fit on a single H100 DGX host.

One benchmark detail deserves attention. Meta's announcement cited an Elo score of 1417 on the LMArena leaderboard for an "experimental chat version" of Maverick, not for the checkpoint it released. LMArena's model list includes the experimental variant and the released Maverick model as separate entries, so a leaderboard number for one does not describe the other.

Background#

Meta's Llama family defined open-weight AI in 2023 and 2024. Llama 2 allowed commercial use, Llama 3.1 scaled to 405B dense parameters, and Llama 3.2 added vision models. In January 2025, DeepSeek-R1 showed what an MIT-licensed MoE model could do. Llama 4 brought MoE efficiency to the Llama family, together with native multimodality and extreme context lengths.

The Llama 4 license stayed restrictive compared with Apache 2.0 or MIT. Companies with more than 700 million monthly active users must request a separate license. Anyone who distributes a product built on the models must display "Built with Llama," and fine-tuned models that are shared must start their names with "Llama." The Acceptable Use Policy also withholds the license rights for Llama 4's multimodal models from individuals and companies based in the European Union, although end users of products that incorporate the models are not affected.

Why It Matters for Developers#

First, size MoE models by total parameters, not active ones. Active parameters set the compute cost per token, but every expert must sit in memory. Scout's 17B active parameters make it fast, yet its roughly 109B total parameters still call for a data-center GPU. It is not a laptop model. Plan quantization and hardware budgets on that basis.

Second, read the license before you build. The EU restriction on the multimodal models matters for European companies that want Llama 4's image understanding. The attribution and naming rules affect how you ship fine-tuned derivatives.

Third, treat the 10M-token context as a ceiling, not a design strategy. Very long prompts cost memory and latency, and retrieval quality across millions of tokens should be measured rather than assumed. For most enterprise workloads, a well-built retrieval-augmented generation pipeline is still cheaper and easier to debug.

When you self-host Llama 4, vLLM and TGI both expose OpenAI-compatible endpoints, so .NET code can reach them through the OpenAI library and Microsoft.Extensions.AI, including image input:

C#
using System.ClientModel;
using Microsoft.Extensions.AI;
using OpenAI;

// vLLM and Hugging Face TGI both serve an OpenAI-compatible /v1 API.
IChatClient client = new OpenAI.Chat.ChatClient(
        model: "meta-llama/Llama-4-Scout-17B-16E-Instruct",
        credential: new ApiKeyCredential("local-no-key"),
        options: new OpenAIClientOptions { Endpoint = new Uri("http://gpu-host:8000/v1") })
    .AsIChatClient();

byte[] image = await File.ReadAllBytesAsync("invoice.png");
ChatMessage message = new(ChatRole.User,
[
    new TextContent("Return the invoice number and total as JSON."),
    new DataContent(image, "image/png"),
]);

var response = await client.GetResponseAsync([message]);
Console.WriteLine(response.Text);

Finally, the LMArena detail is a reminder to evaluate models on your own data. A leaderboard score says little about how a model handles your documents, prompts and edge cases. Build a small evaluation set early, as described in our AI evaluation guide, and rerun it whenever you switch models. For more on image and document inputs, see multimodal AI in .NET.

What's Next#

At launch, Meta said it would share more about Behemoth while it was still training. Behemoth was not released alongside Scout and Maverick. In 2026, Hugging Face's summer report described Muse Glimmer, a 30B open model, as Meta reigniting its open roots, while press reports suggested that Meta's frontier model might stay closed.

Adoption data shows how much the market shifted. According to the same Hugging Face report, Qwen models had 2.6 times as many derivatives on the Hub as Meta's entire footprint by mid-2026, and Qwen GGUF builds were downloaded about 39.6 million times a month, compared with 7.5 million for Llama. Llama 4 remains a capable, well-supported option, but it is now one choice among several strong open-weight families rather than the default.

Sources#