Study: AI model "Centaur" predicts human behavior

Contents

An international research team has presented “Centaur”, an AI model that can predict human behavior across numerous cognitive tasks with deceptive accuracy. This is the result of the study “A foundation model to predict and capture human cognition”, which was published in Nature.

Centaur is based on Meta AI's Llama 3.1 (70B) language model and was fine-tuned using a specially curated dataset called “Psych-101”. This comprises over 10 million decisions from more than 60,000 participants from 160 psychological experiments in areas such as object classification and gambling. The researchers point out that although the data set is very large, it is not perfect: for example, information on the age or socio-economic status of the participants is missing, and the test subjects are predominantly from Western countries.

The team trained Centaur with 90 percent of the data and then tested how well the model could predict decisions made by people who were not included in the training data set. Even with slightly different test setups, the predictive power remained high. In both cases, Centaur performed better than the original Llama model or other models.

Human information processing mechanisms?

The researchers also show that the model's internal representations correlate closely with measured brain activity (fMRI data) – although it was not trained to do so. This suggests that Centaur uses similar information processing mechanisms to the human brain.

In the future, Centaur could help to develop psychological theories, simulate experiments or better understand individual decision-making strategies. According to the researchers, both the Psych-101 data set, and the Centaur model could be widely used in behavioral research in the future, for example to prepare new experiments or to better understand cognitive processes. The model also offers new possibilities for the simulation of subject behavior in research.

The authors speak of a “turning point for cognitive science”. The Llama-3.1-Centaur-70B model and the “Psych-101” data set are available on Hugging Face. The team plans to expand the data set further – to include psycholinguistic, social and intercultural facets of human behavior, for example.

“Not yet transferable to complex everyday situations”

Experts such as Prof. Andreas Glöckner from the University of Cologne, Prof. Markus Langer from the University of Freiburg and Prof. Clemens Stachl from the University of St. Gallen see Centaur as an important step forward. However, they also emphasize that the predictive power has so far been limited to laboratory and decision-making experiments and cannot yet be transferred to complex everyday situations. They also warn against over-interpreting the results and urge caution when it comes to ethical issues and potential misuse. Langer questions whether the “accuracy in the prediction of human behavior – which in some cases is 64 percent – can really be rated as 'good'”.

“The model is state of the art and the methodology is extremely solid. The authors took great methodological care and took numerous precautions to ensure the robustness of the results and to assess their generalizability to new data,” says Stachl. Glöckner also found it interesting to compare the performance of Centaur with previously developed, specific neural networks “that were trained to predict individual task areas”. According to Stachl, the data set is also very impressive: “One of the key obstacles currently preventing major advances in the social and behavioral sciences is the lack of large, curated data sets on human behavior – Psych-101 is a prime example of this”.

Big tech may already be using such models

Stachl is also certain that “large technology companies are already using similar models to predict our decision-making behavior and preferences – for example when shopping online or on social media –. These models are now very good – think of ChatGPT, for example, or how well TikTok suggests videos to keep users in the app for as long as possible. We don't know exactly how good they are because these models are now the most closely guarded trade secrets.” He also warns of the danger of digital dependency, or even “digital slavery”. The question of how to deal with technology must be answered by society as a whole. Science, lawyers and political decision-makers “will be more challenged in the future”.

(mack)