Meta releases Llama 3.1 405B - a "smash hit" as Zuckerberg says

Contents

Meta remains true to its policy of publishing AI models as open source. This also applies to Llama 3.1 405B. The new version of the large language model comprises 405 billion parameters. However, there are also smaller versions with 70B and 8B each. According to Meta, Llama 3.1 can compete with GPT-4o in numerous tests, but is significantly cheaper. Mark Zuckerberg says it is a "smash hit". The Meta boss has also written a detailed blog post explaining why Meta relies on open source and sees it as the future of AI. He also criticizes Apple once again.

As previously stated, Llama 3.1 will not initially be released in the EU- even though it is open source. Meta cites regulatory uncertainties as the reason - but this is not really concrete. Meta AI is also not yet available, which includes the AI applications in Meta's services, such as a chatbot. However, Llama 3.1 now supports German and several other languages. The context length is limited to 128,000 tokens. In comparison, Google's Gemini is available for cloud customers with two million tokens- but closed. Google's open model Gemma 2 is available with 9 and 27 billion parameters.

Meta: Agents are the future of AI

Meta expects Llama 3.1 to enable new workflows, including the creation of synthetic data and model distillation. Usable data is becoming scarce, not only because of the question of the legal use of data, but also because ever larger quantities are required to scale the models. Model distillation is the transfer of knowledge from a large model to a smaller one.

Agents are the next big thing after large language models at Meta, as they are at other major AI providers. They are designed to perform individual tasks for users automatically. Meta writes in the blog post that it wants to provide developers with the tools they need to build their own agents. This also includes new tools for security - Llama Guard 3 and Prompt Guard. There will be a Request for Comments (RFC) for the Llama Stack API.

Zuckerberg as an open source advocate

Meta and Mark Zuckerberg themselves believe that publishing the AI models as open source also offers the best opportunities for further development and security. Zuckerberg compares the developments with those of Linux. He wants to make his models the "industry standard", i.e. the most widely used and most advanced AI models. He sees open source as an opportunity for everyone to develop AI models according to their needs. In addition, many companies and organizations would not want to be dependent on one provider, let alone send their data to the cloud via an API. According to Zuckerberg's blog post, another advantage is that developers can run Llama 3.1 405B on their own infrastructure - at around 50 percent of the cost of using GPT-4o.

Open source also makes it possible to be more progressive, says Zuckerberg. An entire ecosystem will emerge around the models. And because it is not Meta's main source of income, they can afford to make the models freely available.

Zuckerberg clearly - and not for the first time - criticizes Apple: "With the way they tax developers, the arbitrary rules they apply, and all the product innovation they block, it's clear that Meta and many other companies could develop much better services for people if they could develop the best versions of their products and competitors weren't able to limit those opportunities."

Risks and safety of AI

Zuckerberg agrees that AI models harbor risks. He mentions two types of dangers in the blog post. Firstly, the unintentional ones, for example when an AI model gives bad or dangerous advice. However, this also includes the fact that AI could effectively become independent. Open source is an opportunity to retain the upper hand - thanks to transparency and a large number of people who keep an eye on the systems.

Intentional risks arise from "bad actors", i.e. people who have bad intentions and use AI for this purpose. These can be countered with better systems of their own, says Zuckerberg. Large organizations with more power at their disposal can control smaller actors. Zuckerberg also believes that this already works well with social media.

(emw)