From Legacy Monoliths to Self-contained Systems
Modern architectural systems divide work among independent teams. An existing monolithic application must be decomposed for this purpose.
(Image: iX)
- Johannes Seitz
Legacy system – hardly any other term stirs more aversion and fear in the hearts of software developers. What once began as a flexible and simple application has grown over the years into an unmanageable monolith that can only be further developed, tested, and deployed with the greatest effort.
Such an application is a typical example of a poorly or not at all modularized system, where an adjustment in one place entails further adjustments in others. Architects speak of strong coupling. If all functions are poured into a large, unstructured block of code, the coupling effect can also be observed within it. Adjustments can then lead to misbehavior in an unexpected, different place.
If the monolith is broken down into loosely coupled modules, local changes no longer result in adjustments in other systems. If these modules are implemented as autonomous but integrated systems, you also gain the advantage that they can be deployed largely independently of each other. This approach is the well-known microservices architectural style [1], which many architects, however, now view with skepticism, as complexity grows rapidly.
In addition, there is a simpler approach that promises similar and even further advantages: Self-contained Systems (SCS).
Videos by heise
What are Self-contained Systems?
The Self-contained Systems architecture places great emphasis on the high independence of different modules. This loosely coupled system architecture (see text box below "Small building blocks, loosely coupled") promises that teams can develop and deploy individual parts largely independently. While the relatively vague definition of a microservice as an "independently deployable service, modeled around a business domain" [1] leaves a lot of room for choice in the architectural style, the criteria for Self-contained Systems are more clearly defined.
Strong coupling can take many forms, from rather technical ones like "I need to know exactly on which IP this service is running" or "Every time a field is added, I have to update the deserialization logic everywhere" to a very subtle form of coupling: "If I change something in system A from a business perspective, system B must also be adapted."
To build largely independent systems, the last-mentioned form is particularly hindering. It is therefore not only necessary to abstract the technical implementation details to share as few infrastructure artifacts (database, frontend, and others) as possible between systems. Rather, interfaces must be created between systems that hide as many of the business implementation details as possible. This recommendation is not new, but goes back to David Parnas' work in the 70s [2].
To find the functionally abstracted interfaces, architects often draw from the repertoire of strategic Domain-Driven Design [3]. A rule of thumb for Self-contained Systems is that each system ideally implements a Bounded Context, i.e., a self-contained area with a uniform language originating from the business domain.
Like microservices, they are provided with a strictly drawn business boundary, but often this is a complete Bounded Context – i.e., a self-contained area with a uniform language originating from the business domain – in the sense of strategic Domain-Driven Design (DDD). This means they tend to be larger than microservices, resulting in less effort for integration and infrastructure.
In addition, there are a number of best practices for system integration for SCS. SCS should preferably use messaging according to the "fire and forget" credo and not wait for a response after sending. Synchronous request-response communication (e.g., via RESTful services) should be avoided. This creates a system that reacts very resiliently to partial failures.
The user interface is always treated as part of the self-contained system (an idea that has also gained a foothold in the microservices world within the framework of micro frontends). This primarily keeps more work within a team and avoids frontend monoliths becoming bottlenecks. Another, often surprising, effect is that often, data structure exchange between UIs and the resulting business coupling through shared data formats can be completely avoided. Developers can instead use parts of the user interfaces of one system in another.
To ensure that the independent development of the different SCS does not end in a different look and feel or a wild growth of integration approaches, the SCS architecture requires all systems to adhere to a common, coordinated macro-architecture. This includes, for example, the use of specific integration techniques, UI components, or style guides. They enable largely seamless interaction in the system of systems.
The best practices for system integration, in particular, open up migration paths to refactor a grown architecture into an SCS architecture in small, incremental steps. A conversion according to the big bang pattern is, in the author's experience, only recommended or economically viable in rare cases.
Planning the Migration
The first step from monolith to SCS is to identify the parts that can be transferred into their own SCS. Since the target architecture consists of functionally segmented systems, the business logic contained in the monolith must be analyzed more closely. Developers should not take this step lightly, as the division of the application into its various building blocks determines how strongly or loosely they are coupled (see text box above "Small building blocks, loosely coupled").
One way to perform an analysis of the business domains is Event Storming to identify the Bounded Contexts. Figure 1 shows the result of such an analysis for a fictional car-sharing application. The business domains "User Registration," "Fleet Management," "Trips," "Billing," and "Customer Service" were identified. During the analysis, initial ideas emerge about how the different business functions interact and what messages they exchange.
Once a division has been found in dialogue with the business departments that reflects the desired business, loose coupling, architects begin to gradually replace parts of the old system with new SCS. These work together with the old system without users noticing a difference. This is an approach also known as Strangler Fig Application, as a new application, like the namesake strangler fig, slowly but surely covers larger and larger parts of the old host system and ultimately kills it (see Figure 2).
A good criterion for choosing the first business function to be extracted from the legacy system is the resulting business value [4]. For example, the migration adds additional functions to the legacy system, such as a check-in process for both desktop and smartphone. Migration can also provide business value by eliminating an acute pain point with the legacy system: particularly unreliable, error-prone functions.
The effort of migration is another criterion. If the business function plays a role in many parts of the monolith, this strong interweaving can significantly complicate re-integration. Both criteria can be combined in a 2x2 decision matrix (see Figure 3). Business functions located in the upper right quadrant are good candidates for an initial SCS. Another analysis technique that is gaining increasing popularity in the DDD community is Core Domain Charts.
Integrating the First SCS
Once a good candidate has been chosen, numerous technical questions arise. It is precisely here that the strengths of very loose coupling through the SCS architecture have a positive effect. The following options are available:
Link Integration: In the simplest case, users jump from the old system to the new one via a link. A small amount of data and a return address for success or error cases can be passed as request parameters:
<a href="https://fuhrparkmanagement/me-iq-007&success=http%3A%2F%2Fkundensupport%2Ffall%2F4711">To the vehicle</a>
UI Inclusion: Embedded UI elements also allow for very loose coupling between two systems. Since only formats like pure HTML and CSS are exchanged, the consuming system needs to know very little about data structures or their interpretation from the source system. If, for example, the customer service system is to display vehicle master data, it does not require the complete data structures from fleet management; a brief description of the vehicle is sufficient (see Figure 4).
The advantages are obvious: If architects decide, for example, to name the data of a vehicle differently or to include additional characteristics such as license plates, the consuming system can remain as it is. However, this very loose form of coupling is paid for with runtime dependency. If the fleet management SCS is not available, only master data from the cache will be available.
Messaging for Data Exchange: If runtime dependency is not desired for UI integration or if the system needs to not only display but also evaluate the data, integration is required that leads to stronger coupling between the two systems. One of the principles of SCS architecture is that each system should have its own database containing all the data it needs. If multiple systems use certain data, it becomes necessary to exchange them between the SCS.
Transferring in the form of Domain Events has proven effective. In this process, a system generates a Domain Event with all relevant information (but no more!) upon certain events (e.g., completion of a business process) and publishes it. The consuming system can react to the publication with its own logic and create or adapt its own copy of the corresponding data record. When storing redundantly, it is important to precisely define which system defines the truth. Only from this source should subsequent systems consume or change data.
The Event Storming described above has already identified possible events for integration (see the orange circles in Figure 1). These must be published from the legacy system at relevant times so that the new SCS can consume them.
Occasionally a system wants to change or create data located elsewhere. This is best solved by UI integration: If, for example, customer service needs to correct an entry in the fleet, jumping into the fleet management UI with a jump back to the customer service SCS is the best choice. A counterexample where this tactic does not work would be the automatic blocking of a user who does not pay their bills. Here, a command (green in Figure 1) can instruct the source system to make a data change. However, the source system can reject this request if it violates its own rules.
Integration via Synchronous API Calls: Another system can also be addressed directly via a RESTful API. Since this method strongly binds systems at runtime and can lead to a failure-prone, difficult-to-intercept cascade of blocking calls, it is considered a last resort. It is used, for example, when the data from the source system is far too extensive to replicate via events, or when it can only be generated ad-hoc by applying business logic.
Conclusion
Even after choosing and planning an initial SCS candidate, questions remain: How will a common look and feel be established between the self-contained systems and the monolith in the future? How can a common user and rights management be established? These and many other questions do not need to be answered immediately or at least not correctly immediately by architects.
A significant advantage of the incremental migration strategy is that the consequence of a wrong decision is not catastrophic. As long as only one or two systems are running, there is time to learn which decisions are truly considered made for all systems (keyword macro-architecture) and which are better made locally (micro-architecture). Many approaches to integration, ensuring UI consistency, or for single sign-on depend on the specific circumstances.
The macro-architecture grows with each additional self-contained system, while the monolithic system continues to shrink. Support in the form of best practices for the macro-architecture of independent systems can also be found in the principles of Independent Systems Architecture, which harmonize very well with SCS.
With persistence and a focus on business value, architects will, sooner rather than later, learn to enjoy a landscape of well-modularized, loosely coupled SCS and the associated benefits. And they may find that some parts of their system do not require migration at all.
Sources
[1] Sam Newman; Building Microservices: Designing Fine-Grade Systems; 2021
[2] David Parnas; On the Criteria To Be Used in Decomposing Systems into Modules; 1972
[3] Eric Evans; Domain-Driven Design: Tackling Complexity in the Heart of Software, 2003
[4] Sam Newman; Monolith to Microservices: Evolutionary Patterns to Transform Your Monolith; 2019
(mack)
