Subscribe to our newsletter!
Get the latest development and management news and the updates on our cases, events and experience!

A Tool for Keeping the Business Young

    11/08/2021
    Business is often compared to a living creature: it is born, grows, ages, and dies. However, business has unique advantages compared to a human being, for example – a business is much easier to rejuvenate, whereas for people the eternal youth is still a point of concern (and multiple explorations).

    Business can be rejuvenated on several levels. This time, let's focus on the question of whether – and how exactly — it is possible to keep the company's IT infrastructure young so that the business is not at risk for bottlenecks.

    Traits of a young
    IT infrastructure

    In one of the previous articles, we've compared the company's IT infrastructure with the nervous system. We believe the metaphor is also good in the context of young / old businesses. Moreover, all the properties of a young nervous system are characteristic of a young IT infrastructure.
    • 1
      The capacity for renewal – in the context of IT infrastructures, it means the ability to scale or replace the existing systems with those that are more up-to-date or effective for current business goals relatively quickly and without degradation in business processes.
    • 2
      The ability to quickly transmit and process information is just as important for the IT as for the nervous system.
    • 3
      Rapid development of new neural pathways – i.e. the ability to create new integrations and re-establish the existing ones.
    • 4
      Rapid response to external changes  the market is constantly evolving, the business is faced with new rules of work, along with new restrictions and challenges, to which the IT infrastructure should be able to adapt as quickly as possible. Otherwise, the business becomes outdated and can even close down.
    • 5
      Good memory. The closest, although not quite exact, analogy in IT is event logging, i.e. "memorizing" everything that happens with files and messages. The best option for the IT infrastructure is when event logging is uniform, all events are recorded and are easy to track.
    It seems as if there is nothing special about it – this is what the business expects from its IT infrastructure. However, the reality is not always consistent with the youth standards.

    Age-Related Changes

    No IT infrastructure is born clumsy, heavy, and poorly scalable. This is what it becomes gradually – as new systems and integrations are being added to the IT contour. Any expansion of the IT infrastructure automatically increases the system's entropy – the question is to what extent. To answer this question (and crack the secret of youth) we should take a look at the IT architecture.

    Let's consider a couple of examples of an aged e-Commerce infrastructure – as a rule, these companies are very indicative, because they work at a change-sensitive market and have to adapt.

    In one of the cases, we came across a structure where the product information was moving as described below.
    en: Project architecture without ESB
    Before it's loaded to a third-party website, the information passes through two additional systems and two additional integrations. Meanwhile, data collection, processing, and transforming is performed by systems that were not originally designed for this.

    Such an allocation of responsibilities to different systems was quite justified as long as there used to be only one website. But in its current form, the mobility, flexibility, fast and error-free transmission of information are hardly optimal.

    Or, another classic example – an ecosystem with point-to-point "star" integrations.
    en: Point-to-point integrations in a project architecture
    In this case, the quick formation of new connections is also problematic. The integration of each new system becomes a longer and more time-consuming; the slightest changes in systems or business processes require large-scale and expensive improvements.

    IT Infrastructure Aging Process

    It's not rapid or immediate. Just like a human, it is not born old and at the beginning it only has preconditions for age-related changes. It becomes old – in other words, difficult to update, clumsy, inflexible  over time.
    • When the number of systems and integrations in the IT infrastructure exceeds a critical threshold. For example, a company with about 500 employees will normally have dozens of systems and hundreds or even thousands of point-to-point integrations connecting them in its contour. Adding each new system and integration makes the system less flexible.

    • When the information path gets too long and complicated for no reason. The longer it becomes, the higher the chances of having errors, losses, and distortions in the data and the less opportunities you have to track where and for what reason the error occurred.

    • When the key business processes in a company are fully or partially linked to outdated systems. It seems that a system cannot be replased smoothly.
    Eventually, the infrastructure dies – some data blocks become immutable, despite the ever-changing business requirements. When deciding on a change, the business takes into account all the limitations due to its data systems.

    All this is described by the complexity curves of changes in the monolithic architecture.
    en: Monolithic architecture vs service oriented architecture (SOA)
    As long as there are not so many systems and integrations between them, the difference in the integration support costs in service-oriented and monolithic architectures is not critical. But as the number of integrations is increasing, and their connections in a monolithic architecture evolve and get more complex, the difference becomes more noticeable. And over time, things are getting worse.
    en: Monolithic architecture vs service oriented architecture (SOA)
    As you can see from the graph, in a service-oriented architecture, the time for integrations support increases in direct proportion to their number, whereas in a monolithic one it grows parabolically. And if you think that your architecture is not monolithic and that's not your case, try to remember if you've ever dealt with the following situations:
    • when changing one system affected another system;

    • when the interaction between systems was becoming the object of knowledge of the finite systems themselves (i.e. updating such systems equals checking the operation of systems related to it, which brings us back to the previous point).
    If at least one of these cases is familiar to you, most likely, you do have a monolithic architecture, though it might be very similar to SOA/MSA.
    SOA (abbr. service-oriented architecture) is IT architecture, which relies on distributed, loosely connected, easily replaceable components (services) that perform certain functionality. MSA (abbr. microservice architecture) is a specific instance of SOA.
    What's the outcome of having a monolithic architecture? It becomes difficult to change systems along with the business within the existing infrastructure. The growing amount of time and effort is spent on supporting systems and integrations, leaving less for the development. To improve the situation, the company implements new systems or tools that are aimed at particular needs. But in fact, these are not systematic solutions – such implementations only provide a tactical solution for a specific department, making the problem itself even worse.
    As a result, the aged infrastructure becomes one of the main obstacles for digital transformation, business scaling, business model changes... and for any more or less large-scale change in the company's life.

    Possible Reasons for Infrastructure's Aging

    • At the time of the creation of an "age-prone" architecture, the company didn't care about the IT infrastructure development and wanted to focus on 3-5 essential systems. But over time, requirements of the business forced it to revise the number of systems and their functions, which was not provided for in the original architecture.

    • At that time, an alternative architecture looked pointlessly expensive and its implementation was postponed "to the future" – but, "in the future" due to the growth of infrastructure, the refactoring was considered expensive and time-consuming, and the scope of necessary improvements turned out to be too stiff to handle.

    Is It Possible to Keep IT Infrastructure Young?

    Without preamble – yes, it is possible. But, you will have to go back to the level of architecture and change the very approach to the information movement.

    To ensure the IT ecosystem's capacity for change, each system must have capacities for change and independent development. If you want to replace one system with another, ensure that it can be implemented without problems, and the scale of changes will only apply to a specific service in your company.

    How to ensure it in theory?
    • 1
      There should be no gargantuan all-in-one systems. If you already have a system of this kind, think, which functions can be reassigned to other "function-specific" systems.
    • 2
      There should be no tight connections between the systems.

      What does it mean? Everyone understands what a service does, but at the same time the messages used in data exchange are abstract instead of exact. For example, the service needs to send an order for picking, but other systems "have no idea" about the anticipated format for the end service: a file, via the API, or a common table entry. They also do not "know" the measures necessary for proper attribute conversion. Perhaps the service expects the address in a specific form, and it comes as a text string instead. That's a problem at the integration layer, not in the end systems.
    So, let's proceed with the problem of weak connections. In service-oriented architectures (MSA included) a message exchange location is of critical importance. We believe that the creation of ESB layer (i.e. enterprise service bus layer) is the most effective way to rejuvenate the company's IT infrastructure.

    Previously, we've talked about the ESB layer and advantages it gives to a company in the related articles: "Service bus as an evolutionary advantage for company's development" and "Comparison of Popular ESB solutions". Therefore, here we'll only describe how the ESB use helps keep the IT infrastructure young.
    • 1
      Updating ability There is no need to redesign the existing architecture if you need to add or update a specific system, connect an additional sales channel or a new supplier. You should connect them to the ESB layer and improve the jobs (or write some new ones). That's all – no need to create dozens of integrations between systems. A great option to minimize the time and costs.
    • 2
      The ability to transmit and process information quickly. Let's recall the diagram with a long chain of data transfer between Bitrix and three websites. That's how the diagram looks with the ESB implemented.
    Джоба, или job — is a service that ensures a specific functionality and allows customizing its logic and work procedure.
    en: Project architecture with ESB
    The speed of processing and transmitting information in this diagram is tied to the ESB layer, which is originally designed for these actions and is aimed at working with high loads. Changes in prices, product descriptions, and stock balances are simultaneously displayed in all sales channels without "getting stuck" in intermediary systems.
    3
    Quick formation of new connections. Developing the logic and integrating the system using ESB is much faster than developing and writing dozens of integrations for all the connections.

    Point-to-point integrations are usually handled by development teams that focus on a separate system and implement integration logic through the lens of the selected system. The situation with the ESB-based integrations is different: the development team is focused on business logic and is only responsible for transferring information from their system to the bus and backwards.
    4
    Quick response. This point is closely related to the previous one. The faster new integrations are formed, the quicker the company responds to external impacts using IT tools and the less time it takes to adjust its IT structure to new business processes.
    5
    Memory. Logging and storage of information until the actual delivery are embedded in the very concept of the ESB layer and are implemented with the help of internal and external bus tools.

    The ESB layer easily organizes the recording of everything that happens with information, whether it is product data, messages from marketplaces or customer information. In this case, the bus takes responsibility for the message delivery and stores messages until the successful completion of a delivery. If the process was not completed for some reason, logging allows you to see what went wrong and eliminate the cause, and the message itself won't disappear.

    Decision in Favour of ESB

    One of our clients, a large European brand of household appliances, approached us with the task of integrating a PIM system for centralized storage and processing of information about goods into the company's IT contour. Here's the simplified diagram of client's original IT infrastructure, which processed data about goods, orders and balances.
    en: E-Commerce project architecture with point-to-point integrations
    Adding a PIM system to this structure could help the customer to remove extra and untypical functions from some of the systems and reduce the load on the online store. The local task would be solved, and the online sales channels would receive the most complete and reliable information about the brand's products, including technical parameters, photos, descriptions, information about the product availability, related products, etc.

    At the level of business goals, the customer was not only thinking about adding a new system to company's IT contour. First of all, he focused on expanding his online presence and increasing online sales. This requires the provision of correct and sufficient information about the products. But in the long term, the customer wanted to be able to quickly add new sales channels and independently configure data transmission in accordance with the requirements of a new channel (attributes, format, etc.).

    Therefore, kt.team specialists offered the client to improve the IT architecture and integrate an ESB layer based on WSO2 into it.

    The tasks for ESB:
    • collect all data about products, orders, prices, stock balances, customers, etc. from systems that are part of the company's IT contour;

    • combine and / or convert the collected data;

    • transfer data both to internal systems and to online sales channels.
    en: E-Commerce project architecture with ESB
    Instead of writing dozens of integrations to connect a new sales channel, all you need is several jobs.

    WSO2 is a low-code platform with a graphical interface. Therefore, the customer will not necessarily need to contact an IT company for new integrations – integrations can be developed by business analysts or managers with enough competence.

    Integration of a new marketplace will become significantly less time-consuming.

    The architecture, which is centered around the online store, is contrary to the task of the IT infrastructure youth preservation. After all, first off, such an architecture implies that the online store would be assigned some untypical functions: data bus, PIM, etc. And secondly, the inevitable replacement of the central element – in our example, the online store itself – for the sake of rejuvenation would become extremely complex and expensive.

    We would like to emphasize that the ESB bus and the message broker are not the same thing.

    We often find that Kafka, RabbitMQ or code-first buses are equated with low-code ESB. This is fundamentally wrong. Only the systems that allow the IT department to check any integrations in a growing IT contour can be considered modern buses. All these are considered an integral part of a contemporary integration bus the same as apart from having standard engine, transmission and seats, a contemporary car would be equipped with air conditioning, speaker system, ABS and much more.

    Share
    Do you want to get more useful materials?
    Subscribe to our newsletter!

    News of the world of design and development, articles, and our latest cases.We write about complex things in simple words — without ads or spam.

    Also see:

    We use Talend, Mule, WSO2 ESB systems and Apache Kafka message broker in our work.

    Check our website to learn more about the advantages and capacities of each system and read about the actual cases of integration.
    ESB Integration
    Sergey Vlaznev
    ANY QUESTIONS LEFT?
    You can discuss any questions you have about your IT project with our expert.
    © 2021 KT Group LLC
    KOMPLIZIERTE TECH LLC
    Komplizierte Technologien, GmbH
    Office
    Togliatti, Ofitserskaya Street, 12A, bldg. 1

    Cost
    Development
    Other services
    APMinvest
    MirrAR
    Sharing our experience