Seite 2: Documentation Support by Generative AI

Inhaltsverzeichnis

iX: Tools based on artificial intelligence (AI)—especially generative AI—can now support developers in many areas of their everyday work, such as in the form of AI-powered coding assistants. How does AI help you with documentation today and how may it help in the future?

Charlotte: We once had the idea of letting AI generate the documentation. I tried it out a bit, at the time with GPT-3.5 from OpenAI, because that was one of the few AI models that we considered due to its terms of use. Unfortunately, the model was not powerful enough for our requirements, which is why the project was initially put on hold. However, there are already plans to revive the topic and try out whether newer AI models are a viable option. At the moment, though, I can't give a clear recommendation.

Alexander: First of all, documentation should be consistent as well as grammatically and syntactically correct. In addition, documentation should, for example, favor lists and tables over continuous text and use active language for good readability. In the past, support tools were usually limited to nagging: This sentence is too long, the verb is in passive voice, and the grammar and syntax are wrong. Gradually, grammar checking has improved, but now generative AI can really help. It can rewrite entire sections, adjust passive to active language and correct grammatical errors. This makes it much easier for authors who have little practice or who write in a language other than their first language to produce high-quality documentation. By interacting with the AI model, they can improve their writing style over time.

Generative AI can also fill in missing image descriptions, which improves accessibility. Sometimes it can also create diagrams for an existing text, which the authors can then modify. It can also create a rough version of the documentation. However, there is a risk that the AI may hallucinate things that the software cannot yet do. Humans have to find and remove such sections during proofreading. If the idea is good, it should be implemented as a change in the next software release.

Once the documentation has been written, it is the perfect input for a generative AI model. If the AI is given access to the documentation, it can then answer questions about the software and help users. It should always provide a link to the documentation alongside the answer so that people can continue reading there.

Falk: I think we should be cautious for now. AI is only as good as our input. And we must always check and question the results. However, tools like ChatGPT can support us in creating documentation. Be it as a patient sparring partner to discuss content and suggest alternative formulations, or to create textual diagrams according to the "as code" principle with the documentation tools such as PlantUML. ChatGPT, for example, provides plugins that can create architectural diagrams based on our input.

Gernot: I'm sure we've all tried to have generative AI explain code fragments to us—assuming the AI is compliant with data privacy regulations. My own experiments in this regard have been quite successful, and the AI has even untangled heavily nested passages for me. So far, however, this has primarily worked on low-level constructs, such as individual functions or methods. AI has not yet been able to give me an understanding from a bird's-eye view, at least not yet (as of April 2024). AI is still unable to provide an overview on a large scale. In terms of arc42—an open-source, pragmatic approach for documenting and communicating software systems—, this means neither a context delimitation nor a top-level building block view. AI also fails miserably when it comes to the reasoning that led to certain decisions. For my practical work, this means that I should concentrate even more than before on the high-level overview and the more strategic topics.

But it's worth taking a look at requirements engineering: AI has proven to be amazingly good at phrasing specific quality requirements in a measurable and testable way. These types of requirements—such as usability, performance, security, and maintainability—are among the most important parts of technical documentation because they link requirements, architecture, and development. There are almost one hundred examples of concrete quality requirements at quality.arc42.org, of which presumably around a dozen were created with the help of ChatGPT on the basis of GPT-4.