Why microservices work only with well-defined processes and how to avoid rising costs when moving to a service model.
17.4.2023 Why does microservice architecture not always work as intended? This article explains where to start a business transformation and what to pay attention to.
Accounting as an Architectural Example: How Service Architecture Turns into a Monolith
Why Microservices Cannot Work Properly
How to Make Microservice Architecture Work Article published on GlobalCIO Reading time: 7 min.
Choosing microservice architecture looks reasonable for business and attractive for IT. A service responsible for its own slice of the business is easier to control and develop separately from all other company services and systems: it is clear, limited, isolated from others, and has a clearly defined purpose.
In reality, businesses are often unhappy with the result of such an implementation: instead of new features, a flexible system, and fast-executing processes, they get rising IT costs and endless follow-up changes in a chain reaction.
With every new change, the parts of the system become more complex and the connections between them more tangled.
KT.Team CEO Andrey Putin explains why this happens and what to do about it.
In a monolithic organization, it is tempting to blame a process's cost or lack of transparency on the 'wrong' technology or architecture. In this paradigm, it seems that replacing the system or introducing microservice architecture should solve the problem. I am sure that not only KT.Team but other IT companies are often approached with exactly this request. Even if previous iterations were unsuccessful, refining a microservice can still make sense, as it may improve the situation.
In reality, implementing microservice architecture is often like trying to split a frozen lump of minced meat into separate pieces of beef, fat, and pepper. No matter how much effort you put into cutting up a monolith with the IT knife, you will not get separate ingredients, only the same monolith in smaller pieces. IT architecture works the same way: if you roughly divide a monolithic organization or a set of functions into smaller chunks, they will only look like microservices on the surface.
Imagine that you lead a large organization with thousands of employees. The organization has a large accounting department divided into separate groups of accountants to handle different kinds of tasks: issuing certificates, employee onboarding, salary reviews, and reporting.
In IT terms, you could say that you have a service (accounting) and a set of microservices within it (the target microdepartments).
All groups work independently, but while carrying out tasks they exchange messages, for example by passing documents or requests.
The chief accountant has an obvious need for strict process compliance and clear division of responsibilities - that is how they understand what each group does, how departments connect with one another, and the status of every process and document.
Moreover, even if, say, the payroll department is idle right now, its staff should not be loaded with tasks from other micro-departments - that would disrupt established, well-tuned processes. From the accounting department's perspective, one day of downtime for three people is cheaper than switching them to processes outside their area of expertise.
This way of organizing accounting work is conceptually very similar to microservice architecture.
Each isolated part of the system performs one isolated task, has inputs and outputs, a clear internal process, and protection against misuse.
Yes, accounting still has the same set of tasks, but given the scale of the business, there is no need to "reserve" staff for every separate task. For example, a team of seven can handle everything. The person who worked on personal income tax today will handle invoice payments tomorrow, update reports the day after, and then return to personal income tax.
Since the accounting team here is much smaller, the classic interdepartmental 'rituals' are inevitably broken.
According to the instructions, for example, the certificate issuance department must write a letter, record it in the outgoing documents register, then collect the letter from the employee registration department and note the receipt in the register.
But if the same person performs the functions of both microdepartments, the process looks absurd.
An attempt to get rid of this absurdity leads to process inputs and outputs being merged or disappearing, while the boundaries between microdepartments are blurred. In the end, accountants may seem to be working in a microservice mode at any given moment, but in reality it turns into a monolith.
It performs all the necessary functions, but each function is opaque.
The situation can be even worse: accounting is a mess, and no one really knows how the processes are set up.
Different employees prepare certificates according to different standards; Masha calculates reports in Excel, and Petr
Semenovich uses a calculator; new employees are entered into the system in three different ways...
When the algorithms for solving routine tasks are not documented at all, every new person only makes the resulting chaos worse.
Splitting such accounting into micro-departments is no easier than separating a frozen block of minced meat into its individual ingredients.
Accounting staff from different microdepartments communicate freely with one another because they share a common context.
For example, "product" means the pair "SKU + price" both for purchasing and for inventory accounting.
If this context is missing, internal accounting documents become a source of chaos: different microdepartments understand and record the same term differently.
As a result, reports contain duplicates and gaps, and accounting staff spend all their working time constantly figuring out where certain accounting objects came from and where they went.
If the pairing "one department (one service) = one context"
everything is clear, then what should be done with processes and documents that move between departments (services)? In the sales department, for example, the word "product" means not a SKU, but a model in all its color variations. So for an accountant, a black monitor and a white monitor are two different products, while for the sales department they are two variants of the same product.
Each department can justify why its understanding of the term is the correct one. And both would be right in their own way.
To avoid misunderstandings without disrupting the processes of each department, you need a translator that can correctly map one context to another. In microservice architecture, the equivalent of such a translator is middleware (an ESB).
It is a separate service dedicated entirely to establishing understanding between other systems - even if they speak completely different languages.
There may be no bus - in that case, microservices will call each other directly whenever the relevant algorithm built into their logic requires it.
But this approach can be used only when a small number of microservices need to be connected through a small number of integrations.
For complex systems with many senders and recipients, this approach is impractical because it turns the architecture back into a tangled monolith: each microservice spreads into dozens of others and has to take their processes and contexts into account. In that case, you cannot change one part of the system without affecting another. With every subsequent change, maintaining and testing such a system becomes increasingly difficult.
Developers shift their focus from delivering new features to worrying about making sure nothing breaks.
Documentation for context transfer between departments and services becomes more complex and grows every day. Improvements slow down, top specialists leave, tired of working on the edge of failure.
In short, start with the company structure: form departments and micro-departments and determine whether such micro-departments can be identified in every unit; distribute responsibilities among the micro-departments; create business processes with clear algorithms for each micro-department; define clear inputs and outputs for each micro-department (requests, documents, messages); build clear relationships between the micro-departments and their business processes; agree on how the micro-department contexts relate to one another.
After completing all the steps above, transfer the finished organizational structure into the IT environment, where each microdepartment should be reflected as a microservice: each microservice should mirror the processes and interactions of its own microdepartment in its processes and interactions.
Microservice architecture is not suitable for every company or every department.
If a company or department is monolithic by structure, it is impossible to split its functions into separate microservices, no matter how appealing that may seem.
As Melvin Conway's law, formulated back in 1967, states, "any organization that designs a system (in the broad sense) is forced to create projects whose structure is a copy of its own internal communication structure."
At KT.Team, we rely on this law when developing IT products for clients: we start by mapping the business processes the product will be tied to, and only then do we move down to the code level. This is how an IT product is seamlessly integrated into the company's operations and proves effective for solving its tasks.
