Practical use of artificial intelligence in software architecture
Artificial intelligence makes software architecture and can support architects in a variety of ways during design, testing or diagramming.
(Image: erzeugt mit Midjourney durch iX)
- Ralf D. Müller
- Artificial intelligence can support architectural work in design, concepts and diagrams in many ways – even without disclosing internal data to the cloud.
- Various settings and functions in the front end help to better control and vary the context of dialogues and optimize the result.
- The LLM has no memory, but users can preset recurring preferences with special system prompts. There is a special system prompt for the arc42 documentation templates, for example.
- The models can also validate their results themselves and run a proof of concept with executable code.
Pandora's box has been opened: Generative AI in the form of Large Language Models (LLMs) such as ChatGPT, Gemini, Claude etc. has developed rapidly and is conquering more and more areas in the work of software developers. It can also support the activities of software architects, as a few examples will show below.
One of the most important characteristics of today's LLMs is that the size of the underlying neural network, measured in parameters (weightings of the neurons), also determines the performance. A large model is advantageous for the flexible support of architecture work, which often cannot be operated locally, which is why cloud services are indispensable. This raises questions about trust in data processing in the cloud, with each provider having its own guidelines here. However, the article goes on to show that LLMs provide benefits even without the disclosure of internal data.
Front end and context
There are various components behind a technology like ChatGPT: The LLMs are only one aspect, because the frontend that controls the LLM is also very important. A look at the API(documentation and reference) shows the stateless nature of all requests to the LLM. It receives a request with every request and delivers a response. To create a dialog, the front end builds a context from the questions and answers, which grows over time and is repeatedly passed to the model.
Users can take advantage of certain features of the front end. Figure 1, for example, shows some of the smart controls that can be found under each part of the dialog in the ChatGPT frontend. Custom prompts can be edited here at any time (edit icon) to specify requests more precisely. A response from the bot can be regenerated at any time (reload icon) to obtain a better result. In this way, users replace the actual linear chat process with a dialog tree, which they navigate through using the <> icons (see Figure 2).
Even though the size of the context that the newer models can handle continues to grow, several factors play an important role in the quality of the answers. Just as people only work with a limited short-term memory in conversation, the same applies to LLMs. If users overload the model with information in context, at some point answers run out of context and the model suddenly no longer knows what the actual task of the dialog was. A dialog tree reduces the context, as only one branch of the tree is used for this and not the complete dialog with all corrections.
Regenerating a response usually only works if the model allows variations in the response. This is controlled via the temperature parameter in the API. If it is greater than zero, the model becomes more creative by not always generating the most likely answer. For architectural work, it can be useful to vary this parameter depending on the task.
The costs of the API also play a role in this context: requests are billed in input and output tokens. One token corresponds to approximately one to four characters, meaning that the requests become more expensive with each dialog step as the context grows. These costs quickly add up. An optimized, shorter context, which is created by editing a prompt instead of reformulating it, can quickly pay for itself. This is particularly noticeable when the model generates a long response to a query. If the user reformulates the prompt instead of editing it, the long response ends up multiple times in the context and generates costs.
Users should also keep an eye on the aspect of session poisoning, i.e. poisoning the dialog. If it has gone in the wrong direction because users have stated the wrong conditions or the model has made a mistake in the answer, this misinformation remains in a linear chat process. In a human dialog, the conversation partners would overlook this – not so the machine. The dialog is poisoned with the wrong information, which subsequently affects the quality of the answers. This is why navigation within the dialog is so important, as the user goes back up the dialog tree as soon as they recognize an error and branches off before the error.
(Image: erzeugt mit Midjourney durch iX)
This article can also be found in the special issue of iX/Developer, which will be published in the fall and is aimed at software architects. In addition to the classic architecture content on methods and patterns, there are articles on socio-technical systems, quality assurance and architecture and society. Domain Driven Design is also a topic, as are Team Topologies and Green Scrum.
We have been able to attract well-known experts as authors, who pass on their knowledge in many exciting articles – such as this one – for both architecture beginners and specialists.
