Opinion on AI regulation: Secret agents in the black box

No tool is perfect. That's why professionals need to know the strengths and weaknesses of their tools to be able to work productively with them. This applies to cordless screwdrivers as well as AI models. Before I use one at work, I need to know what it can do, where it can help me and which questions it tells me nonsense about. You have to evaluate the AI – and that can take a few weeks, just like the trial period for a new employee.

An opinion by Hartmut Gieselmann

Editor Hartmut Gieselmann, born in 1971, has been with c't since 2001. He is head of the Applications, Data Protection & Internet department and works on current topics relating to medical IT, network policy and data protection.

But precisely such tests are hardly possible with AI models at the moment. Although there are tens of thousands of benchmarks, they only test individual knowledge questions, mostly from the Anglo-American world. Their results are hardly relevant in practice. And the manufacturers reveal little about how and with what information they have trained their model. In gold-digger times, everyone prefers to keep their vein of gold and the modifications to their pickaxe to themselves. Otherwise, a competitor could quickly copy the model or an author could sue because you have trained illegally with their material.

The problem will become even more acute in the coming months, as providers are now starting to equip their AI services with different models, known as agents. These are then happily exchanged without the user noticing; there are no version numbers to track them. However, if an AI answers the same questions differently tomorrow than it does today, it is impossible to evaluate it because you are constantly chasing it.

Upcoming EU regulation

The EU has now put its headlights in position and wants to force manufacturers to provide insight into their black boxes next year. Because the AI providers don't want this, they and their lobbyists are publicly stirring things up: if you regulate too much, you will miss out on the future, they say.

But if we want more than just a handful of investors to benefit from the AI boom, we need more transparency. We need to be able to assess what the expensive machines can and cannot really do. In the interests of all end users who actually work with AI, the EU regulators should not allow themselves to be soft-pedaled and insist on reliable information and insights: Otherwise, I see a bleak future.

(hag)