Top researchers on the AI Act: "Overregulation poses security risk for the EU"

Inhaltsverzeichnis

(Diesen Artikel gibt es auch auf Deutsch.)

In an open letter to the EU Parliament, leading research institutes, scientists and business representatives call for the draft AI Regulation (AI Act, or AIA) to be reworked with a view to foreign policy security and economic competitiveness. If AI is regulated too heavily in the European Union, freedom of research and digital resilience would be at stake. China, Russia and the U.S. would not restrict themselves to the same extent. Heise spoke with two LAION founders about the background to the petition and with Tübingen-based Professor Bernhard Schölkopf, director of the Max Planck Institute for Intelligent Systems and chairman of the pan-European AI excellence network ELLIS (European Laboratory for Learning and Intelligent Systems).

The open letter suggests the creation of an AI oversight body and renews the call for publicly owned European AI research facilities equipped with sufficiently large supercomputers accessible to European AI researchers. One example would be a "multicenter AI lighthouse" similar to the EMBL (European Molecular Biology Laboratory) in Heidelberg, which the ELLIS research network is advocating. Creating an AI system the size of GPT-4 would require a powerful machine in one place, given the current state of the art (SOTA), and could not be spread across multiple sites, explained Dr. Jenia Jitsev, scientific director of LAION and a staff member at the Research Center Jülich:

The bottleneck here is speed and latency in the connections between nodes. Any supercomputing center site would need a "critical mass," Professor Schölkopf pointed out. He said it was unlikely that EU countries would quickly agree on a common site. To prevent a single project from occupying the only available facility for several months, he said, several computing clusters of Leonardo's scale would be needed.

AI supercomputers: centralized or decentralized?

Only a machine running locally with a particularly fast network (such as InfiniBand with at least 200 gigabytes per second and very low latency, 100 nanoseconds per hop) can guarantee that. As soon as it comes to communication to the outside – outside a machine – the training would slow down many times over and could thus no longer be completed in a reasonable time. In particular, latency would slow down the process by several orders of magnitude.

Estimates of the training of GPT-4

According to estimates, GPT-4 was trained for several weeks on about 25,000 A100 GPUs (Graphic Processing Units) from NVIDIA, which were housed in a central machine. If the same were to be run distributed across 5x 5 machines of 5,000 A100 GPUs each, the training would be at least an order of magnitude slower and accordingly take at least several months or even over a year. It would be interesting to know how OpenAI performed the training: According to insiders, Microsoft is said to have provided OpenAI with "a substantial part of the Azure cloud". However, the data centers of that cloud don't sit in just one location, either.

Foreseeably, according to Dr. Jenia Jitsev, this "cloud" hardware is something like compute nodes that are housed in one cluster and have local InfiniBand connections between them. "There simply wouldn't be any other way to train this in the time," Jitsev added to Heise. For Stability AI, he said, the supercomputer also consists of the compute nodes in the AWS cloud – but these also have InfiniBand connections to each other with very high bandwidth and low latency. In concrete terms, this means that all these compute nodes are located in the same center. Amazon Web Services merely uses a slightly different technology, Elastic Fabric Adapter (EFA), which is their version of InfiniBand. InfiniBand is distributed by Mellanox for HPC data centers, he said.

Acquiring and operating expensive special hardware

Anyone who wants to carry out such experiments would have to set up a data center to house such a machine and maintain it accordingly. It can be seen as a piece of necessary hardware. The analogy to CERN comes to mind when it comes to initially procuring and operating such expensive specialized hardware.

Subsequently, however, it would become flexible: once it is ensured that such a machine exists (ideally containing up to 100,000 GPUs so that several groups can run their large experiments in parallel), access and joint work on it can be managed in a decentralized manner. On the one hand, research groups should apply for computing time in a straightforward manner, as with an ordinary computing center, and on the other hand, large-scale projects should be prioritized within which people work together on giant models.

BigScience as a role model

A model could be the BigScience initiative, which put the Jean Zay machine in Paris into operation for its purposes (to train BLOOM). The machine is located in one place, but the work was done in a division of labor by many different research groups, all of which had access to it.

Cybersecurity and AI Safety

All signatories are fundamentally committed to regulation, but ask for proportionality and appropriate differentiation. Open-source AI research is fundamental for security and competitiveness: Only in this way can independent security researchers quickly find vulnerabilities and minimize risks. But security here goes beyond security in the sense of exploits; it is also about AI safety in the sense of secure AI models (for example, against negative use, hate speech, and unwanted output).

Open source brings transparency and participation of the scientific community, which is important for security. In the field of AI itself, public research enables to control and avoid negative use of AI, Bernhard Schölkopf explained. One-size-fits-all regulation that hinders open-source approaches in the field of so-called general-purpose AI stands in the way of transparency and security of AI systems, he said.

Freedom of Research and Security

European research and development would then be left behind in the long term, with significant consequences for business and research. It would also pose a political security risk. Companies would also become massively dependent as end users of US APIs, for example. Data and added value would flow out of the EU – and even if the servers were located within the EU, the local industry would be dependent on the goodwill of some foreign companies in other jurisdictions. The open letter speaks of a "point of no return" – that is, a point at which Europe would be reduced to a mere consumer of technologies if its own development did not keep pace. Europe cannot afford to relinquish sovereignty in the basic technology of AI.

While the first draft of the AI Act initially provided for an exemption for research and open source, in a revised version it introduced the unclear category of "general purpose AI" without distinguishing between application areas and providers. In particular, the researchers and entrepreneurs supporting the open letter are noticeably dissatisfied with the definition of General Purpose AI (general purpose AI). The letter to MEPs was penned by the non-profit association LAION e.V. (Large-Scale Artificial Intelligence Open Network). According to Jenia Jitsev, the scientific director of LAION and a staff member at Forschungszentrum Jülich, it is unclear what is meant by general purpose AI. General Purpose AI is not a clarified term in the research community: A large part of today's AI applications, such as existing systems for image analysis and language modeling, are already considered general according to the definition of the EU AI Act.

General Purpose AI – a Fuzzy Category

It is interesting to note how the category of General Purpose AI came about in the first place. According to computer science teacher and LAION co-founder Christoph Schuhmann, the well-known MIT physicist Max Tegmark had advocated strict regulation. Tegmark is considered charismatic, intelligent – and a representative of a particularly pessimistic view of the future of humanity and AI. The attitude is known as AI doomerism, and here the worldview of "longtermism" plays a role – a movement designed for the long-term preservation of humanity, which assumes that to achieve long-term goals in about 100,000 years, disadvantages for the people of the present must also be accepted.

According to this pessimistic worldview, AI development that gets out of hand could plunge the world into chaos. Among the better-known supporters of Longtermism and AI Doomerism are super-rich people such as Elon Musk and the investor Jaan Tallinn. If you want to know more, you can find a detailed essay on Longtermism on the Internet portal Aeon.

Tegmark is among the founders of the Future of Life Institute, which in a controversial open letter suggested a moratorium on AI development of large systems like GPT-4. In Europe, he is said to have lobbied and made specific proposals to the EU, some of which found their way into the legislative process: On March 21, 2022, Tegmark gave an AI presentation at a hearing on the AI Act in the EU Parliament; the slides are publicly available.

For example, the classification of AI into general purpose AI and narrower task categories originated with him, as did the discussed proposal to place general purpose AI in the high-risk category. At least the vocabulary coined by Tegmark and the Future of Life Institute made its way into a draft stage of the AI Act.

Agitated rhetoric: making the AI debate more objective

According to Bernhard Schölkopf, the current discussion about AI is too heated. In the interview, he suggested that it would be better to objectify the debate. Terms such as AGI (Artificial General Intelligence) or god-like AI have been disproportionately amplified by the media, but are not helpful in understanding what is actually at stake. Most large models, he said, continue to be Transformers, an architecture that has been around for five years. The essential question, he said, is how large and diverse is the training set. On the scientific side, the concern of research is to objectify, clarify and reassure the discourse.

According to Jenia Jitsev, the "Evil Adversaries" such as aggressively acting states are not necessarily needed as a scenario to understand the meaning and purpose of transparent AI research for security. Open source often makes it possible to conduct scientific studies on AI systems in this way: from open data to open-source code for training and inference and the openly available pre-trained models. It is important that large amounts of computing power are made available from the public sector for such studies – especially for basic research and the training of "foundation models". Due to a lack of resources, large-scale AI research hardly takes place in Europe at present. In this context, foundation models trained specifically for this purpose could also become a fundamental tool for AI safety and prevention that can be used to study other systems.

AI in the Service of AI Safety

When working on open source foundation models, one of the outcomes will be a foundation model for AI Safety. This refers to a model that is capable of analyzing other models and checking them against safety criteria. Often there is talk about threats to safety from AI model, hardly about the fact that there can be AI models that help us to evaluate safety of other models, detect threats in time and prevent them.

So this is how AI models could help solve security problems as a tool - by solving security problems through their powerful pattern recognition capabilities, just as they now do with image recognition, for example.

According to him, the existing publicly funded high-performance computing centers are "not enough front and center," even if in principle every project is allowed to submit an application. Nor is it enough to aim for 25,000 GPUs to implement an order of magnitude with which GPT-4 is estimated to have been created. Such a supercomputer alone would already cost around 2 billion euros, added Christoph Schuhmann. To be at the forefront, orders of magnitude in the direction of 100,000 GPUs should already be the goal, as planning for the coming years.

Open-source Business: Profitable? Perhaps, but no Panacea

Hardware resources on such a scale are enormously expensive, so the question of value creation and profitability arises. When asked who should shoulder the necessary expenses and how the purely open-source approach should result in an advantage for the economy, the LAION interlocutors pointed out that key technologies such as chatGPT and GPT 4 will have many effects on the European economy that cannot even be foreseen today. They are convinced that there will be similar effects as with Linux, where nowadays many companies profit from the open-source software and finance its further development. Initially, taxpayers in the EU and in the nation states would have to pay for research and stimulation of the economy. This would result in "advantageous moments for entire industries," according to Jitsev.

According to Schuhmann, it is illusory to train foundation models and believe that the money put into training could be recouped through, say, a closed-source API. The costs for computing power and employee salaries are simply so high overall that it is unrealistic to ever recoup the money, regardless of closed or open source – he left open whether the statement is also valid for the US hyperscalers with globally established platform offerings.

The advantage of open source, he said, is faster adoption in the community, a better reputation, and it simply works better because more people are involved. The counter to this is that there have already been serious failures in the open-source world (Log4J) or isolated projects collapse when they are attached to a few maintainers who can no longer afford to run them or who drop out for other reasons. License problems can also cause trouble (e.g. "Ruby on Rails: Library lost due to license issues causes a Domino effect"). According to the editors, open source alone is not a panacea.

Long-term Services, Inference and Fine-tuning

When asked about possible business models, Schuhmann mentioned long-term services, inference services for companies, end users and nation states, and fine-tuning. For LAION, open source itself is "free", but the LAION position is "not militant". Open source is seen as "fertile ground for industrial projects", and according to Jitsev permissive licences are possible, which do not exclude the creation of a closed source variant, as long as the foundation remains open source. The success of Linux-based ecosystems could serve as a model for the successful symbiosis of open source and industry, says Jitsev. What companies would then do to make money would be to adapt models to difficult special cases and train on customers' own data.

Customers could test extensively in advance through the generic open-source variant, and trust in the basic open model through extensive use by a broad community would be an advantage. Christoph Schuhmann envisions the business model of the future as AI startups producing open-source models. In return, they should receive public support from politicians or other large companies. They could then earn their own money through services and know-how related to the use and adaptation of the models. Ultimately, it is the quality that is decisive for customers and companies, not whether something is closed or open source.

Currently, there is a lot of duplication, and in many places "energy is wasted with similar training data because the trained models are not available," Schölkopf pointed out. The new supercomputing center recently opened in Hesse, he said, is small for AI, but a good step forward. With OpenAI, he said, it is not known exactly where the models will be trained. When asked where the desired supercomputing cluster should be located, the researchers replied that cheap cooling, available energy and rather cool temperatures were important. DeepL, for example, has a large data center in Iceland, but other countries are also possible.

Increase Security and Transparency of AI Applications

In summary, the petition is about digital resilience and security in the EU, according to the letter signatories. According to LAION and the researchers, open-source AI and models, as well as open-source datasets and training data, are "the most realistic way to increase the security and transparency of AI applications." Research and business leaders are behind the petition, which was sent to about nine members of the European Parliament on April 28, 2023, he said.

In general, the petition's authors and supporters welcome the classification of social scoring and facial recognition, for example, as high risk; they also say that dealing with biases and how algorithms work are important concerns for legislative clarification. Chatbots and media apps for producing texts, images and music or AI-assisted search functions, on the other hand, all of which fall into the new category of general purpose AI, could be too severely restricted here. Current reports from Brussels indicate that a rethink is probably taking place here in the final meters.

In addition to Bernhard Schölkopf and ELLIS, well-known researchers such as Jürgen Schmidhuber and Sepp Hochreiter (pioneers of early AI research) are among the supporters, and professors and researchers from European universities and research institutions as well as a Canadian university have signed the petition (including, for example, Irina Rish from Montréal, Kristian Kersting from TU Darmstadt and hessian.AI, and Robin Rombach, who co-developed Stable Diffusion). Business leaders also support the cause: for example, the KI-Bundesverband, which organizes about 400 German commercial enterprises, is among the signatories. A blog entry with the link to the open letter to the EU Parliament can be found on the LAION website.

(sih)