Not just at Euro 2024: the role of virtual advertising in sport

Contents

"This program contains virtual advertising" – many a viewer has noticed this unexplained insert during the European Football Championship in recent weeks. But where exactly this virtual advertising appears usually remains hidden considering the perfect illusion. Unless, of course, a player suddenly dashes into the advertising hoarding and then partially disappears from the screen. Such amusing finds can be seen on TikTok and other social networks.

However, virtual advertising is no longer as new as the ad for the current European Championship suggests. According to the organizing UEFA, it was used for the first time at a European Football Championship. In general, however, it has been used in soccer for a long time. However, the fact that it is so inconspicuous shows how well the technology works most of the time. There are different approaches to how the whole thing is put into practice. There are various companies around the world that provide solutions. Heise online spoke to one provider that uses a software-based approach.

What methods are available

"In soccer, virtual advertising is currently limited to the main camera. In other sports, such as skiing or motorsports, where the event is spread across many cameras, there are more," explains Ralf Dragon, founder of the company uniqFEED. If you look closely, you can also see the "real" advertising boards here and there that spectators get to see in the sports venue. This is because "direct replays and slow motion are usually without virtual advertising for technical reasons. Match summaries, on the other hand, are edited together from different material and can therefore also contain virtual advertising."

Two methods are currently being used to implement virtual advertising in sports broadcasts: in hardware-based systems, a camera head is used to precisely track the camera movement. To differentiate between the foreground and background, the bands emit an infrared signal, known as keying. This is recorded in the camera as an additional color channel to analyze occlusions later. "It's like green screening, except that the color is not visible and is actively emitted by the background," explains Dragon.

This contrasts with software-based systems such as those offered by uniqFEED: "Our software-based system was developed with the premise that the production conditions on site remain unchanged. Instead of sensors, we use software to analyze camera movements and occlusions. Virtual objects are inserted into the already edited signal." The challenge here is keying in software. The advances made recently in the hardware computing power required to do all this in real time have played into the provider's hands.

The role of AI in virtual advertising

Artificial intelligence is playing a growing, but not all-determining, role in virtual advertising. Dragon says: "Virtual advertising is far from being solved by AI. Many problems are also solved by engineering. The analysis of occlusions is a classic AI topic, as the problem can be trained, for example, with a hardware-based system. Analyzing camera movement is more of an engineering topic, as the problem can be described using formulas."

The use of virtual advertising is not limited to perimeter advertising. Dragon explains: "You can place virtual objects anywhere, but physical interaction with virtual objects is not possible. That's why you're usually limited to virtual boards or content on the ground, re-texturing existing surfaces. If interaction is ruled out, you can also place virtual 3D objects."

A lot is possible, not everything is allowed

Theoretically, the pitch in the stadium could also be used as a huge advertising space. However, the technical possibilities are restricted by legal requirements: in Germany, the law stipulates that "virtual advertising must be referred to in the program and only existing advertising may be replaced," says Dragon. "In other countries, such as Switzerland, virtual advertising may not be displayed on free, unaltered spaces. So there must already be something artificial, even if it's just a white band."

Dragon, who co-founded the company uniqFEED in 2015 as a spin-off from ETH Zurich and now employs around 50 people, has set himself a goal for the next few years: "Our vision is that virtual advertising will be used across the board in professional sport by 2030." While different advertising is currently played out in different countries, an even higher degree of personalization would be conceivable in the future if the respective regulations allow it.

Training is important - not just for athletes

Despite occasional mistakes, such as the clip of a partially "invisible" player circulating on social media, the technology is advancing rapidly. Dragon is relaxed about such errors: "This is probably a typical AI artifact, as too little training data was used for such a case. In the future, these problems will most likely be reduced as much as video coding artifacts with additional data."

Virtual advertising opens up new opportunities for marketers and broadcasters, but it also raises questions about the authenticity of the sports experience and, like developments in AI, puts the concept of reality to the test. One thing is certain: "This program contains virtual advertising" will probably be seen more and more frequently in the future – and may soon be as commonplace as advertising itself.

(mki)