Llama 4: Meta releases new AI models

According to Meta, a whole herd of Llamas has trotted off. There will be four new AI models, two of which have already been released: Llama 4 Scout and Maverick are available as open models, Behemoth is still a preview for now, and Llama 4 Reasoning is also due to be launched soon. Mark Zuckerberg presented the new models in a video on Instagram. There he also talks again about how he believes that open source models will prevail and represent a benefit for everyone. Llama 4 has already moved into Meta AI. And is said to have given the AI assistant a "major upgrade".

Llama 4 Scout and Maverick are natively multimodal models with a mixture-of-experts architecture and open-weight. Multimodal means that text and images are processed in the same model. Although the various experts are located in one model, only the appropriate experts respond depending on the type of question. This makes such a large model much more efficient. Open-Weight means that these are partially open AI models – The knowledge of a model can be freely downloaded, but not the entire code and training data. Nevertheless, the models can be further developed.

Scout comes with 16 experts, each with 17 billion parameters – a total of 109 billion parameters. Meta describes the context length of 10 million tokens as "leading". Llama 4 Maverick comprises 128 experts with 17 billion parameters each, i.e. a total of 400 billion parameters. The context length is one million tokens. Scout fits on a single H100 graphics processor, writes Meta, Maverick on an H100 host.

Giant Behemoth as a model trainer

According to Zuckerberg, Behemoth, which is not yet available, will be "huge". The announced figure is 288 billion parameters with 16 experts. Meta calls the model the "ideal teacher for model distillation". This is a method for transferring the knowledge of a large model to a smaller, more cost-effective one. Scout and Maverick have already benefited from this. Behemoth is said to beat GPT-4.5, Claude Sonnet 3.7 and Gemini 2.0 Pro in some common MINT benchmarks. As always, statements about benchmarks should be taken with a grain of salt, for example, models sometimes learn the answers, and correct results do not necessarily mean correct conclusions.

For Behemoth, the pre- and post-training methods have been improved. Meta writes of better synthetic data, a new vision encoder and improved fine-tuning and reinforcement learning, in which only particularly difficult tasks had to be completed. Ultimately, Llama 4 Maverick can be operated more cost-effectively than Llama 3.3 70B. Meta has also placed particular emphasis on processing several images at the same time. This allows the models to reason better visually, even with regard to time sequences.

Meta writes in a blog post that it has of course complied with many safety standards and carried out tests. Regarding bias, it says: "It is well known that all leading LLMs have problems with bias – in particular, they have historically leaned to the left when it comes to debated political and social issues. This is due to the nature of the training data available on the internet." It remains to be seen how well known or confirmed this is as a fact. In any case, measures have been taken with Llama 4 to get more balanced answers. "Llama 4 performs significantly better than Llama 3 and is comparable to Grok."

(emw)