Doctors discuss artificial intelligence: how much science fiction is acceptable?

Contents

A key topic at the 129th Medical Assembly in Leipzig is "Artificial intelligence". More than 20, partly overlapping, motions for resolutions were submitted on this topic. Particularly important: AI must not jeopardize the doctor-patient relationship. The increasing use of AI systems for automatic speech transcription and processing could undermine trust if sensitive information is disclosed to third parties. As a result, it was decided to set up an interdisciplinary AI expert committee comprising representatives from the fields of medicine, IT, law and ethics to develop guidelines, among other things.

Co-determination instead of heteronomy

Doctors want to actively support and shape the development and introduction of AI instead of being at the mercy of market-driven, "disruptive" solutions. "We don't have much time left," said Dr Peter Bobbert, Co-Chair of the "Digitalization in Healthcare" committee of the German Medical Association (BÄK). "We now need sovereignty, independence and a real European 'Airbus moment' in AI development." AI systems should not be introduced without experts. Problems in the digital infrastructure, in particular the lack of interfaces between different IT systems in hospitals, are cited as a major obstacle to the sensible use of AI.

Sovereign AI systems

In one of the motions, the Medical Assembly calls for regulatory measures to "limit the monopolization of AI systems in the healthcare sector by a few large providers". Accordingly, "the development and implementation of AI applications by universities, independent research centers and professional societies [...] must be promoted accordingly". In addition to liability issues, doctors also see risks such as overdiagnosis and possible discrimination due to faulty or biased algorithms. It is emphasized that AI systems cannot do everything and that the quality of the data is crucial.

AI could even soon take over some of the medical tasks. "A few years ago, AI sounded like science fiction, today it is medical practice," said Dr. Klaus Reinhardt, President of the German Medical Association (BÄK), at the start of the debate. "But its use also raises questions about data security, transparency and medical responsibility." What is important to many doctors is that, despite all the technical advances, human care, empathy and the ability to empathize with patients are still of central importance. Doctors do not see AI as a replacement for the doctor, but as a supporting tool.

Not a "doctor" from Star Trek

"I don't want a 'doctor' from the spaceship Voyager," explained Dr. Tilmann Kaethner, a paediatrician from Lower Saxony. Like many of his colleagues, he fears a complete takeover of medical treatment by AI. In the Star Trek series, the Emergency Medical Hologram (EMH) embodies the ship's doctor as a holographic, AI-based character. Doctors are concerned that this could make the human component and medical judgment redundant.

Doctor should have the last word

Studies have used AI to identify high-risk patients and thus prevent deaths. "Data saves lives," emphasizes Prof. Aldo Faisal, digital health researcher at the University of Bayreuth and member of the German Ethics Council. In several motions, the medical profession is calling for clear guidelines and regulations to ensure that people remain the final decision-making authority. Although the manufacturer of the medical device is primarily liable for product defects, there are still uncertainties, which is why doctors are pushing for legal safeguards and clear regulations.

Doctors also discussed the "black box of AI". The representatives of the German Medical Association called for transparency in AI-supported decision-making processes and clear labeling of AI-generated suggestions in the documentation. The Executive Board of the German Medical Association will vote on a corresponding motion. The promotion of digital and AI expertise within the medical profession is considered essential so that doctors understand how AI works and can use this technology sensibly and safely. "We are still at a very early stage of implementation, at most in the pilot phase," explains Prof. Ulrike Attenberger, Head of the Department of Radiology and Nuclear Medicine at the Medical University of Vienna.

Independent and secure servers

The doctors also voted positively on a motion on the obligation to use secure and independent servers for medical AI systems. The explanatory statement states that this is the only way to ensure the long-term protection of sensitive health data and that "server locations outside the European Union pose considerable risks to data security".

AI is considered an already present reality and a great opportunity for medicine that will change everyday medical practice. For example, there are already digital health applications that use artificial intelligence. The medical profession believes it has a responsibility to ensure the ethical and responsible use of AI and is calling for clear rules and guidelines that go beyond the current European requirements. According to the medical profession, AI must not discriminate and put people disadvantaged.

Who benefits from the data?

There is also concern that control over data and the economic value created by AI systems could lie outside the medical profession. Motions have therefore been submitted to safeguard the medical profession's legitimate interest and economic participation in the data economy.

It was also criticized that big tech companies such as Google, Microsoft and others will in future be able to access data via the Health Research Data Center (FDZ Gesundheit), which will then mainly be entered by doctors. The data will then be used to develop and train AI systems, which will then be sold back to doctors. Instead, the medical profession spoke out in favor of supporting European solutions.

There was also a positive vote in favor of a motion on "Protection against re-identification of health data from the electronic patient file via calendar patterns [--] especially through the use of artificial intelligence". One concern is that it is currently only possible to draw conclusions about individuals from the health data held by the FDZ using only a small amount of information and with the help of calendar patterns from metadata. This raises questions about confidentiality and could enable discrimination

Erik Bodendieck, Co-Chair of the Digitalization Committee, emphasized the differentiated approach of doctors to the topic of AI. In the thesis paper "Artificial intelligence in healthcare" and in the statement "Artificial intelligence in medicine", the BÄK had formulated requirements in advance "that should be fulfilled in a digitalized future from a medical perspective".

(mack)