RSL 1.0: Standard to regulate content usage

Really Simple Licensing 1.0 – the name says it all. RSL 1.0 is intended to regulate how AI companies can use content from publishers and other content creators. This also includes licensing agreements. It is an open web standard that is also supported by organizations such as Cloudflare, Akamai, Creative Commons, and the IAB, the association for the online advertising industry.

More than 1500 media companies are said to already support the standard. The website specially set up for this purpose states that RSL is used by billions of websites. This would correspond to most of the high-quality content on the internet that is relevant for the AI training of large providers. It is "the first licensing standard that protects content in the age of AI," it says.

The standard is intended to provide machine-readable and transparent licensing specifications; it is based on the well-known RSS standard (Really Simple Syndication). The robots.txt file, used until now, only allows website operators simple yes/no rules. RSL is intended to expand this with a "universal language for content rights and licensing terms." Website operators can thus specify and communicate that, for example, search engines may use content, but AI search applications are excluded. Furthermore, there is also the possibility of demanding monetary or in-kind payments when content is used – for example, for training AI models.

RSL itself cannot technically exclude crawlers. However, supporters of the standard such as Cloudflare and Akamai can.

Until now, content has been fair game for AI providers

Currently, for example, Google does not offer any differentiation of its crawlers. Anyone who wants to appear in search results must also allow those crawlers that scrape content for AI products and training. The EU Commission has just initiated proceedings this week to investigate precisely this behavior. It is being considered whether Google is acting in violation of competition law and disadvantaging both content creators and other AI companies.

Individual licensing agreements negotiated by Google, OpenAI, and Perplexity with selected publishers are also not a comprehensive option for creating fair conditions. They always only affect a few content creators.

Furthermore, the robots.txt file has been ignored by some crawlers. Even with it, a technical block cannot be set up; it is the machine-readable form of a request, a standard that everyone adhered to for a long time. This makes it all the more important that CDN providers like Cloudflare are now involved.

The IAB is likely also interested in a solution due to the changing advertising market. The well-known problem of disappearing clicks and thus missing ad placements on websites when people only read AI summaries and AI answers also affects them. Even if Google, OpenAI, and co. integrate advertising into their AI products, they will then also be responsible for the associated business, including all terms and conditions.

(emw)