Global experience in implementing Talend ESB - KT.team

Global experience in implementing Talend

Managed integrations with WSO2 ESB

Talend ESB is software with a set of rules and principles that allows messaging between information systems and applications with routing settings through a single point.

Industry solutions at Talend

Implementation of Talend ESB in the banking sector - KT.team

Banking sector

Modernization of the banking system by integrating the ESB service bus

Landesbank Berlin is a German international universal bank that provides services in four areas: private banking (VIP personal banking), corporate banking, commercial real estate financing and selective capital market activities. The bank annually serves about two million retail customers and about 69,000 corporate clients from more than 200 branches.

The problem

Obsolescence of the bank's IT system

To cope with the ever-increasing volume of account data, Landesbank Berlin has decided to modernize its core banking system. However, the introduction of the new system required thorough preliminary preparation, and if the transition attempt failed, there was a risk of closing the bank.
Landesbank is a member of the German group of national savings banks, and its business processes are closely linked to the central information technology provider Finanz Informatik (abbreviated. FI), which uses a modern banking system — OSplus. The difficulty of updating the system was also due to the fact that in order to integrate our own banking applications with data on Landesbank Berlin's customers, addresses, income and accounts, it was necessary to create a new interface in parallel with the deployment of OSplus.
It was also necessary to take into account that incoming data comes in different formats, and for their further transfer to specialists from different departments, each accepting documents in its own format, it was necessary to convert this data into the requested formats. The previous integration platform used by the bank could not process a large data stream, and the transition to the new system required several stages, which only increased the time.

Solution

Optimization of information transfer processes

Talend ESB integration has helped optimize information transfer processes. Now, when a business process is launched in the OSplus banking system, data is transferred to other applications via the Talend ESB service bus. Most interfaces are programmed to work according to events, and only a few are “scheduled” in advance.
To ensure the transparency of processes within the company, a special TMF portal was created on the basis of Talend ESB (short from English. Tools & Methods Framework), which monitors error management.

Implementing Talend ESB into an e-commerce company - KT.team

E-commerce

Simplifying integrations in the online store infrastructure using ESB

Otto Group is a German online store, Europe's largest mail order store with more than 2.8 million items and 6,800 clothing brands for the whole family. It processes about 7 million online orders annually and up to ten new orders per second on peak days.

The problem

The complexity of the project's IT landscape

The existing IT architecture of the online store consisted of a Teradata database and a large number of data islands. The complexity of the architecture was also that some data islands used SQL, while others used PL/SQL, Oracle, and Perl scripts. Thus, to determine which advertising offer would be most suitable for customers, the company had to transform huge amounts of data, and the information system needed a single integration tool.
In addition, the project had a lot of integration scripts that were very difficult to maintain, as the developers who originally wrote the code were no longer on staff.

Solution

Data Lake linked to the company's systems through Talend

Otto decided to create a new data lake from scratch and use Talend as a bridge between a new data lake, which stores data from an online store and partner websites, and a 250 TB Exasol database. Talend also made it possible to use its own CI/CD process in development. The use of Talend made it possible to continuously integrate data sources, develop and deliver new updates and applications, and create test environments for launching new software. All this helped to increase the speed of development and the speed of fixing problems on the project and, in general, helped reduce the risks associated with new development and integrations. For example, if the system receives a request from a client to delete their data from the Otto system, this no longer takes much time, since there is no need for lengthy searches through all databases. Talend made it possible to quickly exchange data with their sources, making it easier to delete data from Exasol and the Cloudera data lake.

Introduction of Talend ESB into a pharmaceutical company - KT.team

Pharmaceutical industry

Process automation, processing and storage of all data in a single system using the ESB enterprise service bus

Bayer AG is a German chemical and pharmaceutical multinational corporation with more than 150 years of history and key competence in the field of healthcare in the world.

The problem

Outdated information system with manual data processing

Bayer is constantly researching and testing new drugs to determine their efficacy and safety for further market launch. This process requires the accumulation, analysis and storage of a huge amount of clinical data from patients and volunteers. Data is also collected from laboratories and electronic devices. All information is automatically anonymized at the point of collection.
The old information system was outdated and could not cope with the load and the constantly growing data flow. Moreover, some processes were not automated, operations were processed manually, which often led to errors caused by human factors.

Solution

Microservice IT architecture

Instead of supporting the old system, Bayer decided to completely change the environment and infrastructure architecture. Bayer held a hackathon, during which it thoroughly tested Talend and came to the conclusion that it was best suited for continuous improvement of the system, given its principles of flexibility and the ability to support a microservice architecture.
The Talend ESB bus has become a middleware that connects repositories while being hidden from users' eyes. The data enters Oracle's local data lake, connects to the SAS system for advanced analytics and uses Oracle Life Sciences Data Hub to store data on clinical trials of drugs and drugs.
The Talend ESB service bus made it possible to quickly and in accordance with requirements add new functions, make the received data better and increase the speed of its processing. Another advantage of Talend for the company is the ability to track everything that happens to data, including where it comes from and where it is sent.

YouTube

We have collected all the mistakes in integrations,
for you not to
make them

Watch all

ESB system implementation cases

Watch all

Watch all

We write articles for our blog and for specialized publications

Read more

Talend implementation cases in enterprise projects: global experience

Learn more

Why are techies against data buses: middleware, ESB, message brokers?

Learn more

How to choose the right type of integration for your IT architecture

Learn more

Watch all

IT consulting for medium and large businesses | KT.team

System integration calculator (ESB)

System Integration Project (ESB) Calculator

How many streams will the systems send
Example: The “Product Management System” will send product data. “Order Management System” — about orders. “Warehouse management system” — about the status of the shipment. This is 3 streams.
0
Example: The “Product Management System” will send product data. “Order Management System” — about orders. “Warehouse management system” — about the status of the shipment. This is 3 streams.
0
100
How many streams will the system receive
Example: The “Warehouse Management System” will receive data on goods and orders. “Order Management System” — about goods and shipment status. This is 4 streams.
0
Example: The “Warehouse Management System” will receive data on goods and orders. “Order Management System” — about goods and shipment status. This is 4 streams.
0
100
The calculator calculates using an accurate but simplified formula. The scope of work for your project and the final cost may vary. The final calculation will be made by your personal manager.

1

Calculation example

Learn more about Mule ESB features — KT.team
Creating and hosting services on Mule ESB — KT.team

To transfer data between systems, we create a “stream”. Some streams are needed to send data, while others are needed to receive data. Orders, goods, or other entities may be transferred in a separate stream.

For example, on the diagram:
1. The “Merchandise Management System” sends goods. “Warehouse management system” is the fact that an order has been shipped. “Order Management System” — orders. In total, the systems will send 3 streams;

2. The Warehouse Management System accepts goods and orders. “Order management system” — goods and the fact that the order has been shipped. In total, the systems will receive 4 streams.

2

Scope of work in the calculator

Learn more about Mule ESB features — KT.team

Included in the calculation

Additionally

Preparing a map of systems and data flows (SOA scheme)

Preparing the infrastructure for connectors to operate

Development of object logic (connector business process diagram)

Setting up a monitoring and logging loop

Creating connectors for exchanging data for each stream on 3 stands (test, preprod, prod)

Creating connectors (storage - receiver) for exchanging data on each high-load stream (>100 messages per minute) on 3 stands (test, preprod, prod)

Set up to three dashboards per connector within a ready-made monitoring circuit

Over 15 attributes per stream

Documentation on copying integration, reusing, and maintaining

Demonstration of the implemented functionality

Included into account

Preparing a map of systems and data flows (SOA scheme)

Development of object logic (connector business process diagram)

Creating connectors (source - storage, storage - receiver) for exchanging data on each object on 3 stands (test, preprod, prod)

Set up to three dashboards per connector within a ready-made monitoring circuit

Over 15 attributes per object

Additionally

Preparing the infrastructure for connectors to operate

Setting up a monitoring and logging loop

Creating connectors (storage - receiver) for exchanging data on each high-load object (>100 messages per minute) on 3 stands (test, preprod, prod)

Over 15 attributes per object

Your application has been sent successfully

Submit again

Let's discuss your project

Personal managers will contact you

Contact form

Something went wrong! Please try again.
Your application has been sent successfully!

Submit again

Something went wrong! Please try again.

Contact us

Make an appointment

Book a meeting with Google Calendar

We use cookies to provide the best site experience

Ok