These are the acute dangers of open source AI systems
Technology always gives you both sides of the coin. Therefore, it is important to protect AI against deepfakes and generated malware. An interview.
In the closing keynote of Sphere24, Mikko Hyppönen warns that positive technical developments also have their downsides. These should not be ignored, he says, because you can't just take the positives.
(Image: Jani Telatie / WithSecure)
While the capabilities of generative AI systems are usually hyped in the media, it is becoming increasingly clear that the systems also pose risks. The most acute seem to be deepfakes, attempted fraud and misinformation. Security experts are looking into these and other dangers that AI can pose to IT systems and society - including Mikko Hyppönen, who is chief scientist at Finnish IT security provider WithSecure. Hyppönen is also an author and speaker. At his company's Sphere24 conference, Hyppönen spoke about the benefits and dangers of systems - from the internet to encryption and AI - in his closing keynote. All this technology does good, but it can also be used for harmful and criminal purposes. The key point is that it is not possible to get only the good aspects of a technology. You have to weigh up the harmful aspects and protect yourself against them. iX spoke to the security expert during Sphere24.
iX: In your presentations last year, for example at itsa in Nuremberg, you were very pessimistic about the impact that freely available AI models would have on society. Today, your tone has changed somewhat. What has changed in recent months?
Mikko Hyppönen (MH): Unlike a year ago, I see fewer discussions and concerns about artificial intelligence going crazy and taking on a life of its own. Of course, there are people who worry about it and write about it, but I'm more concerned about other safety aspects of artificial intelligence than about containing the systems. Containment and targeting models are important, but I don't think they are short-term issues. We obviously have to take that into account when we build these systems. But it's not really something I'm very concerned about.
iX: What are you worried about?
MH: I'm much more worried about the practical things: deepfakes, deepscams, creating malware with large language models, automating malware campaigns and that AI can find zero days. When preparing my talk for Sphere, I looked back at my slides from the last one and so many things have changed in 11 months. The technology has gotten so much better. Even a year ago it was breathtaking. Last year I had a slide where I tried to emphasize the fact that things are evolving so fast that it's hard to keep up even if you try. Since then, development has become even faster and continues to gather pace.
Open source a trouble spot
iX: With the large, closed AI models, usage guidelines and technical guard rails are designed to prevent deepfakes, scams and the creation of malware. However, there are a large number of open models that users can configure themselves. How big is this problem?
MH: The topic of open source and closed source is a really tough nut to crack when it comes to AI. I love open source, but I see limits to how far this is practicable with artificial intelligence. I don't see a solution to the fact that you can remove the security restrictions if you have access to the code itself. Perhaps we could try to find a technical solution for this. A hybrid solution where some of the code is open source, but then some kind of guardrail application that is always open source and can be accessed through an online system. So you could customize the things you really need to change, but you can't change the security restrictions. However, I'm not sure if that would work. I'm not sure we'd be happy with that.
iX: There are a lot of people who are very much in favor of open source and are almost religious about it.
MH: Exactly, the license has to be right. If there's any part of it that's like a black box, that's not okay. So open source AI is a difficult problem. I don't know if regulation will really touch that in the future, because right now it doesn't. Are there regulations for open and closed sourcing with AI? I can't remember any. Maybe we need some.
iX: What do you think of semi-open licenses, such as Meta's Llama?
MH: I think Meta is simply reacting to the fact that they got into the game too late and OpenAI took the wind out of their sails. I think that's the real reason why they went open source. Originally, Meta only gave access to researchers and academics, some of whom shared the data immediately. Meta then turned that around and said that they actually wanted to keep the models open all the time. So they ended up releasing it with this weird license.
The whole thing has been a bit of a mess from the start in my eyes. A lot of the language models that are used for criminal purposes are based on Llama. Llama is the best of the open source models. Criminals don't care about the license. If they have the source code, that's all they need.
(Image: Jani Telatie / WithSecure)
AI tools in cyber security
iX: In addition to attackers, defenders also use AI systems, of course. One long-established application is the evaluation of alerts by AI. AI is much faster at evaluating and annotating than humans. Is there a risk of attention fatigue due to the flood of messages that humans still have to check?
MH: I think that AI is one of the biggest helpers against fatigue. Of course, if such systems are not configured correctly, they can generate an unlimited number of notifications and alarms. But you can teach them to focus on the ones that are really important to you. And that's exactly what we want to achieve in our company. We try to make systems available to other companies that they might not otherwise be able to use because of the scale of the systems. Expertise and money are the biggest hurdles here. AI plays a big role in that because it tries to eliminate the noise or the things that make you tired. Big language models and generative AI systems are really good at recognizing weird things. Much better than humans. These systems can digest a lot more data, a lot more source material, and then figure out that certain events have never happened before.
Technical revolutions make people unhappy for the time being
iX: You say in your presentations that you now use AI on a daily basis for various tasks. Would you entrust the systems with internal documents or source code?
MH: No, that would contradict our guidelines. It's not a question of whether I would believe or trust the systems. We have guidelines on the use of internal or personal identification data or our closed source applications.
For the things I do with text, I mainly use Claude. Where AI really saves me a lot of time is when reading long documents or books. The system reads 100-page white papers, even in languages I don't speak, for example in Italian, and then I can discuss the content of the white paper in Finnish, my native language.
Is this research mentioned anywhere in the paper? Yes, it will be. And if I then go on to use some of this information, such as writing my own summary, of course I check the facts, but usually they are correct. I tested this with many of my older papers that I wrote myself before I started using AI on a larger scale.
AI is very useful and I like the technology. Of course, it will make many people unhappy, as we have seen with every technological revolution. The industrial revolution also made a lot of people unhappy, but I think we're all glad it happened. We like living in a society where we don't have to use muscles for everything. We have machines and motors.
Does the result justify the cost of AI?
iX: Your company now also offers a product that uses the Claude Haiku language model. What problems have you encountered when fine-tuning and setting up the model?
MH: There are a lot of performance issues at the moment. It takes a lot of computing power to teach these machines. And we try to take that into account as well. Training these systems comes at a significant cost and the environmental costs are not negligible either. A single one of the NVIDIA GPUs used to train these devices consumes 700 watts of power. That's a vacuum cleaner. These cards run 24 hours a day in large data centers. So the training comes at a cost, it's time-consuming and expensive. But the end result seems to be worth it.
Mr. Hyppönen, thank you very much for the interview.
The interview with Mikko Hyppönen took place during the Sphere24 conference in Helsinki and was conducted in English. The editor was invited to the event by the company WithSecure, which paid for his stay in Finland.
(pst)
