Simplify Enterprise Infrastructure Integrations Using ESB

Enterprise service bus (abbr. ESB) is a software with a set of rules and principles providing the message exchange between different systems and applications with a single point routing configuration.
rus: Интеграция ESB-систем | kt.team

Possible Problems in the Absence of ESB

If your project has a complex architecture, for example, the star architecture, there is no clear understanding of which of the systems has failed to transmit data. Even developers won't always be able to help you solve the problem. As the result, the change implementation slows down, because each change requires a verification and relevant adjustments to the integrations. If one of the systems fails, message repetition at a certain point becomes problematic, and restoring the full chain of events in complex or microservice architectures becomes more complicated as the system complexity increases.
The point–to-point integration might seem relatively simple. But even at the stage of debugging integrations, the process becomes more complicated due to the abundance of field formats. As the project grows, the number of integrations increases, and connections management becomes more and more complicated. Each system ends up having lots of connectors for the same type of messages (for example, when the "order" entity has to be represented in different ways in the ERP and in the CRM).
In case you don't have an ESB, you need a developer to make even slight modifications – for example, to add an attribute to the entity. If changes are made to one of the systems, you will need to change and test lots of integrations instead of changing one common message, that sometimes can only be performed by the developers of a specific product (for example, SAP) and leads to the integration time increase. Moreover, as time goes by, project developers themselves might get lost in its architecture, and the lack of self-documented systems only complicates understanding.

Project Architecture Complexity

Modification Complexity

Support Complexity

rus: ESB Централизованное и децентрализованное управление (по требованию) | kt.team
rus: ESB Централизованное и децентрализованное управление (по требованию) | kt.team
rus: ESB Централизованное и децентрализованное управление (по требованию) | kt.team
en: Project Architecture without ESB
Project Architecture without ESB

ESB Layer Capacities

en: Routing ESB

Routing

ETL solutions in the ESB layer allow you to create microservices for sending messages depending on their content, with transformations, and in accordance with the specified rules. You can filter and combine messages, as well as independently receive data from one system via any API and transmit it to other systems.

Data Conversion

You can configure the internal service bus rules to convert messages from a certain system to be sent to other systems. At the connector (microservice) level, a data flow can be verified, parallelized, transformed in many ways, and sent to the corresponding systems.
en: Data Conversion ESB
en: Scalability ESB

Scalability

The enterprise service bus has its own scheduling and load distribution mechanisms, that you can use to configure individual logic for getting and sending data from and to different systems and applications and to monitor their workload. For example, if your CRM can accept 100 transactions per minute in the daytime and 500 transactions per minute at night, the ESB will take this logic into account and will not overload your systems.

Low-code Platform

The ESB service bus includes the means of rapid development, consisting of a large spectrum of pre-programmed actions, for example:
  • to get / send a message via JSON or as XML to FTP;
  • to get / send product information from one system, and in the absence of the required attribute to "turn to" another system to get it and then put it in a queue to add it to yet another system.
As a rule, if a change of logic is required, it is made with several lines of code, and sometimes with a single line.
en: Talend ESB. Complete microservice
en: Self-documentability ESB

Self-documentability

All actions with messages are visualized and clear: if you comply with the basic naming rules, any business analyst would be able to understand how the integration works and what is to be changed. Also, the integration code can be packaged and deployed in a standalone service (JAR or Docker image).
Talend ESB. Complete microservice

Speeding up integration with marketplaces

The ESB layer presence in the enterprise's IT circuit simplifies the logic of integrations, making them fast and predictable even without extra developments to the end systems where information about goods, prices and warehouse balance is stored, and, at the same time ensures full control of the integration microservice source code.
en: Speeding up integration with marketplaces

ESB Advantages

ETL are low-code systems with an opportunity to quickly develop the microservices with a desirable flexibility, which do not require a separate development team and lots of work hours — a major part of work can be performed by a business analyst in a RAD environment (with a computer mouse), whereas the developers would only be responsible for the most difficult parts. The advantage is that developers do not have to reinvent the wheel or come up with exclusive customized solutions.
ESB provides a single center for events (including sending logs to other services), setup, configuration, and viewing queues and flows. If necessary, you can make some integrations completely independent — in the form of a microservice.
rus: ESB Централизованное и децентрализованное управление (по требованию) | kt.team
rus: ESB Простота разработки | kt.team

Centralized and Decentralized Management (on demand)

Ease of Development

The service data bus provides early detection of problems by means of monitoring and allows hiding some of the integrations from the outside world. Many ESBs support EIPs (external identity provider), providing an opportunity of centralized access and Permissions management actions, taking into account all the features of a particular integration.
Instead of having integration modules with duplicate logic, your ERP, CRM, WMS, OMS, PIM systems just need to have a common universal API. In most cases, this allows you to avoid the individual integration logic in the end systems.
rus: ESB Уменьшение кодовой базы | kt.team
rus: ESB Мониторинг аномалий | kt.team

Code Base Reduction

Anomaly Monitoring

Project Architecture with ESB

en: Project architecture with ESB

What Does an ESB Consist of?

Ensures the centralized management of functions, including the logging components and a queue interface support.
They are used for connecting ESB to various systems. Identify where to take the data and where to send it. They run on your servers and have a comprehensible source code (Java, as a rule).

Auditor

Connectors (ETL)

Message Broker

Defines and monitors the order of sending and receiving messages across the applications in real time.
Allows a business analyst to work in the ESB service bus independently, without any help from a development team, ensuring the low-code of all integrations and reducing the number of potential problems in the code. At the same time, you control the executed files within the framework of the integration itself.

Integration Constructor (ETL)

Industry-Specific Solutions

ASICS is a Japanese manufacturer of sports shoes and clothing.

Solution
ASICS used ESB to create API interfaces that allowed a new e-Commerce platform to access the information about customers, order statuses, stock units, prices — long story short, to all information that was previously stored in different repositories. Moreover, the ESB integration simplified one of the most complicated parts of the project — the networking, which finally required just a few hours to develop.

E-Commerce

British Petroleum (abbr. BP) is a transnational oil and gas company with a chain of filling stations all around the world.

Solution
BP used ESB to create a network of applications supported with API. This allowed to accelerate the applications development and scaling and ensured a secure data transmission. Now BP can process more than 290 thousand transactions daily.

Power industry

Siemens is the engineering company in Europe.

Solution
ESB allowed Siemens to increase the project development speed by 50% and reduced the time of submitting new MVP projects twofold. The ESB integration helped to lay the foundations for further integrations as the business is expanding.

Production

Our Cases

FM Logistic is an international logistics company that provides warehousing, cargo handling and packaging solutions and supply chain services.
Objective
The number of client's projects is constantly growing. It was necessary to find a solution that would speed up the development and integration of new systems, as well as allow managers to perform basic tasks independently, without developers' help.
Solution
The implementation of the Talend ESB service bus in the project infrastructure allowed us to speed up integration process by 4 times. Now, instead of writing a code, the developer just configures a job responsible for a specific functionality. Using the Talend ESB, the managers can work independently, without developers' help to perform simple tasks, such as file convertion from one format to another.

The Talend bus allowed saving time on routine management tasks, giving developers extra time for more complex tasks.


A major retailer, one of the leading Russia's suppliers of medicinal products and medical devices.
Objective
The client was aiming at the retail segment, which required a large number of integrations. The existing servers were insufficient for the increasing load, and it was necessary to optimize the website and increase its fault tolerance.
Solution
To remove extra load from the site and simplify integration, we decided to implement the ESB service bus. At the development start, we were choosing between the two platforms — Talend ESB and Red Hat Fuse. In accordance with the client's requirements and objectives, our final solution was the Talend. With the help of Talend ESB, we managed to reduce the time for Pimcore and marketplaces integration with the website, 1C and the medications directory.

Also, to expand the functionality and control the Talend ESB operation, we installed:

  • Apache Kafka Message Broker;
  • Checkmk monitoring system;
  • ELK logging system.
A major manufacturer and retailer of seating and cabinet furniture, mattresses and sleep products.
Objective
For such a big company with multiple information systems, a point–to-point integration is not the best choice. The customer wanted a light data migration mechanism with a simple logic with a sole function of transmitting data from one system to another. At the same time, it was necessary to take into account the company's need for systems' weak interconnection, so that it was possible to remove or replace one of the systems in such a way that other systems were not affected.
Solution
To cope with the customer's task, we've integrated Kafka into the enterprise infrastructure of the project. Kafka is a distributed system that allows users to exchange messages and provides permanent data storage capacities.
Job — a service, responsible for a particular functionality that allows configuring its logic and the operating procedure.

The Systems We Implement

OUR WORK PRINCIPLES

The manager's task is to develop the client's project by improving its metrics. Therefore, we pay close attention not only to the recruitment process (more than 100 interviews per vacancy), but also to the constant improvement of our current team.
We support processes that allow analyzing all the required metrics and provide simple communication with us for big Russian and international businesses.
Deep expertise and a wide range of technologies applied to enable project implementation on any stack and the use of a low-code system or a ready-made solution.

Process approach

Focus on business goals

Solutions instead of technologies

PRICING
Each project is evaluated separately.
To calculate the approximate cost of your project, use our commercial offer.
We follow the
time and materials
model with a fixed cost
72 €
en: kt.team PRICING
per hour

YOUR PERSONAL MANAGER

rus: Сергей Влазнев Персональный менеджер
Sergey Vlaznev
Sales Manager
Phone:
WhatsApp:
Telegram:
Email:
You can ask Sergey any questions related to your project and he will guide you through the work scope, terms and costs.
© 2023 KT Group LLC
KOMPLIZIERTE TECH LLC
Komplizierte Technologien, GmbH
Office
Togliatti, Ofitserskaya Street, 12A, bldg. 1

Cost
Development
Other services
Sharing our experience