AI as boss: better at key figures, but not good at market shocks

Contents

If you leave strategic business decisions to the generative AI ChatGPT-4o, it performs better than human decision-makers in many key figures, but does not cope well with unexpected events. This is the result of an experiment conducted by researchers at the University of Cambridge's Business School. The researchers emphasized that AI cannot be a substitute for the human CEO.

For their experiment, which the researchers present in an article in the Harvard Business Review, they had 344 students from Asian universities and managers from a South Asian bank play a simulation. The game was a "coarse-grained digital twin of the US automotive industry" that used mathematical models to depict real data on vehicle sales, market shifts and the impact of events such as Covid-19. Players were asked to make strategic decisions for each fiscal year of their virtual company like a CEO. In parallel, OpenAI's LLM ChatGPT-4o played the game.

Don't get fired from the virtual board

The game had over 500,000 possible decision combinations per round and no fixed winning formula. The aim of the players was to survive as long as possible without being fired by a virtual board of directors, and to maximize the market capitalization of the virtual company. A number of performance indicators set by the board of directors was the measure of success. The performance of the best two students and best two managers was then compared with ChatGPT-4o.

The result was ambivalent: On the one hand, the LLM outperformed the human participants on almost every metric. "It designed products with surgical precision, maximizing appeal while maintaining tight cost control," the researchers describe. It also responded well to market signals, built strong momentum and kept its human competitors on their toes.

AI acted quickly and aggressively

At the same time, however, the AI was fired from the company's virtual board of directors faster than any of the student participants. This was since ChatGPT-4o had problems adapting to unexpected events such as a market slump during the coronavirus pandemic, which were programmed into the game. The best-performing students would have been better able to take such events into account with long-term strategies. With carefully managed growth, they would have focused more on adaptability than on aggressive short-term gains.

GPT-4o, on the other hand, had completely focused on short-term optimization after initial successes and then relentlessly maximized growth and profitability – until the market shock ended the winning streak. AI can learn and iterate quickly in a controlled environment, the researchers conclude. However, human intuition and foresight are lacking when dealing with highly disruptive events.

Help, but no substitute

Incidentally, the human managers who took part in the game also fell into the same trap as the AI and relied on short-term and rather aggressive behavior. The virtual board fired them more quickly than the students.

The researchers conclude from their experiment that AI is not a complete CEO replacement, but can be a valuable decision-making aid for the boardroom. In the experiment, even a model that had not been specially trained for this purpose was able to offer unique and creative approaches to strategy. This could help bosses to make better decisions. However, without human supervision and appropriate ethical guidelines within the company, it would not work.

(axk)