Artificial intelligence: Pope warns against "deadly autonomous weapons"

Pope Francis sees killer robots, autonomous drones and similar weapons systems as a major threat. "No machine should ever be able to choose to take the life of a human being", he emphasized in his eagerly awaited keynote speech on artificial intelligence (AI) on Friday evening at the G7 summit in Borgo Egnazia in Apulia, southern Italy. "We must give human control over the selection process of artificial intelligence programs a significant space, guarantee and protect it," demanded the 87-year-old. Because: "Human dignity itself is at stake."

Even "in a drama such as an armed conflict", it is "urgently necessary" to reconsider the development and use of devices such as "lethal autonomous weapons", the head of the Roman Catholic Church warned the assembled heads of state and government of the major industrialized nations. He made a clear plea for a ban on the use of such technology. Until this is ready to be decided, he said, there must be "a proactive and concrete commitment to the introduction of ever greater and significant human control".

It was the first time in the history of the G7 that a Pope was a guest. Francis had no doubt that the emergence of AI "represents a truly cognitive-industrial revolution". This will "contribute to the creation of a new social system" that is "characterized by complex epochal changes". For example, the key technology could drive "a democratization of access to knowledge, the exponential progress of scientific research and the possibility of handing over tedious work to machines". However, he devoted the greater part of his presentation to the risks of AI. According to him, this could also lead to "greater injustice between advanced and developing nations, between ruling and oppressed social classes". The "culture of encounter" threatens to be replaced by one of "disposability".

Criticism of AI as a judicial decision-making aid

For the Pope, the rapid pace of technological progress makes artificial intelligence "a fascinating and at the same time uncanny instrument. This requires "a reflection that does justice to the situation". Francis himself has already become part of the AI meme culture thanks to an artificially created photo in which he is wearing a white designer padded jacket in rapper style.

In his presentation, he pointed out a massive difference: "It should always be borne in mind that the machine can make algorithmic choices in some forms and with these new tools." This procedure is based "either on precisely defined criteria or on statistical inferences". Humans, on the other hand, not only make choices, "but are also capable of deciding in their hearts".

AI is therefore designed to "solve specific problems", explained the Pope. For users, however, the temptation is often "irresistible to draw general, even anthropological conclusions from the specific solutions it proposes". One example is the use of programs to help judges decide whether to grant house arrest to convicted criminals. Here, AI is used to predict the likelihood of reoffending based on predefined categories. Relevant systems would have access to private data categories such as ethnic origin, education level or credit line. There is not only a great risk here of "de facto giving a machine the last word on a person's fate". Rather, there could also be "implicit reference to the preconceived assessments", meaning that the AI could adopt prejudices or stereotypes from the training data.

"Generative" AI is not creative, but "reinforcing"

Franziskus thinks little of knowledge systems and bots such as ChatGPT from OpenAI or Gemini from Google. "Many of us are impressed by the applications that are easily available online to write a text or create an image on any topic," he stated. "Students who make disproportionate use of it when they have to write papers are particularly taken with it," he said, unable to resist a sideswipe. However, the students forget that, strictly speaking, so-called generative AI is not really "creative". It actually searches big data for information and packages it in the style it wants. The more often such a system repeatedly finds a term or hypothesis, the more it considers it to be legitimate and valid.

Such large language models are "reinforcing" in the sense that they reorganize existing content and cheat to consolidate it, Pope explained. However, they do not check whether errors or prejudices are included. Not only does this run the risk of "legitimizing fake news and strengthening the advantage of a dominant culture", but it also undermines the educational process at its core. There is no longer any room for the authentic reflection that is actually needed.

Francis identified a "lack of an ethos linked to the perception of the value and dignity of the human person" as the greatest weakness in the implementation and development of these systems. In order for AI to become an instrument "for building the good and a better future", it must "always be oriented towards the good of each individual person". The Vatican and tech companies, together with the Italian government, signed a call for an ethical approach to algorithms back in 2020. AI must not reinforce the "technocratic paradigm", the Pope added, according to which "the view of the world is limited to realities expressed in numbers and prefabricated categories". (des)