AI systems prefer their own texts: study warns of "anti-human bias"

Contents

When artificial intelligence is given the choice between texts written by a human and another AI, it prefers its peers. This is the result of a study by Charles University in Prague. Researchers tested well-known models such as GPT-3.5, GPT-4 and open-weight models from Meta, Mistral and Alibaba in three different scenarios. The AI systems were asked to choose between product descriptions from e-commerce sites, scientific texts and movie summaries – without knowing whether the text was written by humans or machines.

According to the researchers' publication, the result was clear: in all three categories, the LLMs preferred the AI-generated content significantly more often than human test subjects in comparative studies. This "AI-for-AI bias" was particularly pronounced for product descriptions, where GPT-4 chose the AI-generated texts in 89 percent of cases, while humans only showed this preference 36 percent of the time.

LLMs apparently with their own evaluation criteria

To rule out the possibility that the AI texts were simply better, the scientists conducted parallel experiments with human raters. These showed significantly weaker or no preference for AI-generated content. "This indicates that LLMs use specific evaluation criteria that are not based on objective quality signals," explain the study authors.

In addition, the researchers systematically controlled for the so-called "first item bias" – the tendency to choose the item presented first. To do this, they presented each text pair twice in a different order.

Where this could become a problem

The researchers see this development as problematic, particularly considering the increasing use of AI in decision-making systems. If, for example, job applications are increasingly pre-sorted by AI tools, those who have written their application with the help of AI would have an advantage. As AI tools are chargeable above a certain threshold, people would have to be able to afford this AI help. There is a threat of a kind of digital class society.

Regarding agentic systems that perform complex tasks themselves, even more far-reaching discrimination against humans is conceivable. For example, AI systems could advise in favor of other AI systems and against the use of humans when making economic decisions.

Still many unanswered questions

Legislation such as the European Union's AI Act sets out requirements for the use of AI systems in critical areas. However, the study suggests that even seemingly neutral applications such as product recommendations or job application pre-selection could contain systematic biases.

The researchers emphasize that further research is needed to understand the exact causes of the phenomenon. LLMs might react to stylistic markers in the texts. Concrete solutions have yet to be found. This would first require a more precise understanding of why AI systems show this tendency. Activation steering, a technique for specifically influencing model behavior, could perhaps be used to exert an influence.

Researchers advise caution

Until then, companies and institutions should take this systematic bias into account when using LLM-based decision-making systems, the researchers advise – particularly in areas such as personnel selection, research funding or marketplace algorithms. It should always be borne in mind that AI tends to give preference to its own kind.

(mki)