Study: Generative AI writes good doctor's letters

Generative AI should be able to write usable doctor's letters and thus potentially speed up medical documentation, according to a study by the University Medical Center Freiburg. Around 93 percent of the AI-generated reports could have been used with only minimal adaptations, the researchers found.

"Our results show that models specially trained for the German language can provide valuable support in the creation of medical reports. This could significantly simplify workflows in everyday clinical practice," said study leader Dr. Christian Haverkamp according to the press release. The AI software is already being used to some extent in regular operations at the University Medical Center Freiburg.

95 out of 102 AI doctor's letters usable

Specifically, the researchers trained four smaller, non-proprietary models based on LLaMA, LLaMA-2-Chat and BLOOM-CLP-German (two variants), each with around seven billion parameters for the task. This also involved the ability to run in an on-premises environment with limited computer resources. Around 90,000 real clinical documents from the Department of Ophthalmology at the Freiburg University Medical Center formed the training material. The four then competed against each other in writing medical letters.

The evaluation took place in two stages. The first evaluation was also in the hands of the AI, namely the Claude v2 model. In this stage, BLOOM-CLP-German was ahead of LLaMA models, although the researchers describe it as under-trained for its size. In the second stage, the winner BLOOM-CLP-German generated medical letters for 102 cases, which were reviewed by medical professionals. Both evaluators rated 95 of the letters as suitable for direct use or for use with minor modifications. Only seven of the reports were classified as unusable by at least one of the evaluators, three of them due to errors in content. Further details of the study can be found in the paper published in the Journal of Medical Internet Research.

Three hours a day for bureaucracy

However, the researchers caveat that it remains to be seen whether AI writing assistance can also make everyday clinical practice easier. If the output still has to be checked in a more complex case, this could be perceived as a burden rather than a relief.

Work is already underway elsewhere on the topic of doctor's letters using AI: The University Medical Center Hamburg-Eppendorf, for example, has taken the AI application "ARGO" into live operation to support the creation of doctor's letters. And the Fraunhofer Institute for Intelligent Analysis and Information Systems has been developing a "doctor's letter generator" since 2023, which is due to be launched on the market by the end of 2024.

There is definitely a need for relief: doctors and nursing staff in German hospitals spend an average of almost three hours a day dealing with bureaucracy, according to a recent survey conducted by the German Hospital Institute (DKI) on behalf of the German Hospital Federation (DKG). The DKG sees bureaucracy as a burden on patient care and is calling for a reduction in the amount of documentation required. A reduction in documentation tasks of just one hour per day and worker could ensure that 21,000 more doctors and 47,000 more nursing staff are available for medical and nursing care. Digitalization has not yet provided any real relief in this respect.

(axk)