Managing 1C Integrations Through ESB: Simple and Reliable

What this article is about

26.9.2023 Reading time: 13 min. One and a half million CIS companies use one or more 1C products for management and accounting, project and product management, enterprise management, procurement, and sales. In most cases, 1C is not the only product in the IT architecture, so for most of these companies, data exchange between applications in the IT environment is an urgent issue.

In this article, we will explain how to properly structure integrations between 1C applications and third-party vendor apps to avoid data conflicts and information loss.

How Data Is Structured in 1C: A Task with Many Variables

The storage system in any 1C application is built as a complex, multi-layered database. It defines data relationships and dependencies, and their links to objects at a high level of abstraction. However, in our experience, companies very rarely use 1C in the same form and configuration that is initially delivered to the user out of the box.
Each company customizes its systems to fit its own business processes, organizational structure, products, and so on.
As a result, the data structure becomes even more specific and unique.
As long as all of this runs inside a single system, there are no problems.
But if you need to integrate 1C with other systems, the specifics of how information is stored make the task more complex.
This raises the need to clarify the following nuance. Which system will the integration itself belong to: 1C or the other participant in the process?
This determines which development team will build and modify the integration and which system will carry the load. In what format should the data be sent and received?
Different systems consume data in different formats. 1C:ERP
Enterprise Management” in EnterpriseData (XML-based), while Axapta, for example, also uses XML, but in a “plain” form.
The differences may seem minor, but they can affect data quality. And either in the receiving system or in the source system, conversion to the required format will need to be configured.
Which exact data needs to be pulled and sent?
As a rule, the receiving system does not need everything, everywhere, all at once, only specific fields or records for a certain period of time.
You need to identify which fields they are, separate them from the overall data set, and route them into the appropriate receiver slots.
Which exact entity attributes need to be transferred to and from 1C? And what should you do if the target system does not have the required value? For example, item master attributes in 1C may be stored as several reference catalogs that are updated independently. For example, you send 1C:Retail data about the item master "desk lamp," whose reference attribute "color" has the value "red."
If the color reference list in 1C:Retail does not yet contain the value "red," it will be impossible to save such data.
All of this must be taken into account at the initial integration development stage. 1C regularly releases updates and refreshes its product code.
The data structure you built your integration for may become outdated after any update.
Field names will change, some fields will split into two or merge into one. And all integrations related to 1C will have to be reworked to match the new conditions.
Changes can also happen on the side of the second system involved in data exchange, and the integration will have to be changed again.

Integration options for 1C and other applications

In essence, there are only two ways to integrate 1C with applications in the IT environment: direct integrations and integrations through an enterprise service bus, or ESB (short for enterprise service bus). Each approach has its own characteristics.

Direct integrations: OData protocol and API

The first, and in practice the more preferable, option for direct integrations is using the universal OData protocol (short for open data protocol). 1C products, in particular 1C:Enterprise version 8.3.5 and later, include built-in support for OData - an open web protocol for querying, adding, deleting, and updating data. OData makes it possible to perform operations on resources via HTTP requests and receive responses in XML and JSON formats.

This is a universal protocol used not only by 1C but also by other systems.

As a result, the systems in the environment will more easily understand the received data.

To use the OData protocol, you need to enable support for it in the 1C settings.

After that, the system will automatically create a REST interface for data exchange between 1C and other systems. The integration can then be connected to it, and data export and import can be configured.

If the system is older than version 8.3.5 (for example, 8.2), you will not be able to use the universal OData protocol.

You will have to build an API and integration for each connected system.

API development is the standard way to set up communication between two systems.

But it is worth remembering that APIs are written by people.

And the people involved were different, with different experience and different ways of thinking.

Even the same task can be solved differently. As a result, if you have a dozen systems integrated with 1C, plus several 1C developers, you will very likely end up with a zoo of ten or more APIs.

Each of them is written with its own logic, operates according to its own logic, and in the future must be further developed, yes, again according to its own logic.

After each 1C update, your team will have to adjust each API separately to reflect the latest changes and the API's internal logic. In the best case, this will be the same specialist who developed the API - it will be easier for them to reconstruct their own reasoning when writing the code. In the worst case, in terms of time spent, it will be a completely new specialist who uses different approaches to API development.

The “wild zoo” problem can be addressed by imposing strict API design standards, though in practice that does not always work, or by artificially reducing the number of integrations.

We saw an example of such a reduction in one of our projects.

To reduce the “wild zoo” in the 1C source system, the system where supplier product data was stored, the company's developers implemented a chained integration. Bitrix collected stock, pricing, product, and similar data from other systems and sent it to the 1C source system. The 1C source system then passed all product data to the 1C system responsible for offline retail.

Part of the initial information from it was sent to 1C, which was responsible for the online storefront. On the one hand, this reduced the number of integrations as such. On the other hand, each additional link made the overall environment more vulnerable and created more room for errors and losses.

Any failure in the integration or in an intermediate system caused information to get stuck and not reach the next stages.

Direct integration summary

1C can be integrated directly with other systems via the OData protocol or via API.
The OData protocol is universal and makes it easy to exchange data between 1C and other systems. By default, OData is available only in relatively recent 1C versions.
API development in 1C is not standardized.
Each developer implements it based on their own experience, logic, and approach.
After updates to 1C and other systems, all APIs will need to be changed.
If they are built on different logic, the task becomes more difficult.
Reducing the number of integrations by making system interaction logic more complex actually makes the entire environment more error-prone.

We'll curate materials for your task

clients@kt.team Telegram @kt_team_blog

We'll reply within 30 minutes and send relevant cases, diagrams, or analyses tailored to your context.

1C integration through a data bus (ESB)

If the company's environment has only two applications (for example, 1C:ERP
Enterprise Management” and WMS), you can choose a point-to-point integration format.
You know exactly what data is stored in 1C and in what volume, what is stored in the warehouse accounting system, what each system must send, and what it should keep for itself.
No major growth is planned in the coming years, nor is the implementation of other IT systems.
But if the IT architecture includes more systems, more entities move between them, and growth is planned in the near future, it is better to look for fundamentally different solutions. Ones that will be easy to maintain and will not require building something unique every time the company adds new systems to the IT environment.
The solution is to integrate 1C and other systems through an enterprise service bus. An ESB is a software solution that acts as a single hub for message exchange between information systems and applications.
The service bus takes information from the source system in the form that is easiest to extract, stores it in repositories or routes it through a message broker, and transforms it into the target system's format when needed.
Logging and monitoring tools that can be integrated with an ESB, and in some systems are already included out of the box, make it possible to track data delivery between systems and quickly detect and fix errors.
A very simplified and schematic view of system integration through ESB looks like this.
Each system sends and receives data not directly from other systems, but through the ESB.

Integration options through ESB and protocol selection

An ESB cannot be called a magic bullet for integrating 1C with anything. First, the integrations still need to be designed properly by separating domains and information flows. Second, to create a connection between 1C and the ESB, you will have to develop a separate API for each entity or use the same OData protocol.

Most often, 1C exchanges not one or two, but five or even ten data flows with each system. And the real data exchange scheme is a bit more complex than a simple web.

We do not recommend funneling the entire data set through a single connector and a single stream when integrating via ESB.

Moreover, we strongly do not recommend doing this, because it increases the integration load by tens of times and proportionally raises the risk of failures.

Based on our integration development experience, the most rational approach is to separate entities and send each one through its own data flow.

Depending on how complex the entity data storage is, you can choose a transfer protocol: if the entity data is stored in one or two tables, it is better to use JDBC with a direct database connection, without an API; for a more complex data storage structure (for example, if the entity is described through several additional reference catalogs at once), you will need to build an API and use OData, RESTful, SOAP, or GraphQL data transfer protocols.

Benefits of Integrating 1C Through ESB

Reducing the load on 1C and on the end systems In a point-to-point scheme, one of the systems must handle data transfer and conversion between the formats in which the data is stored.

These tasks can be distributed among the exchange participants or assigned to one of the systems. In any case, implementing them requires expanding the system's codebase and increasing its load. In an ESB scheme, the only additional element is the API.

All data splitting, filtering, and conversion actions happen within the ESB layer. 2.

Reducing the volume of data transferred In a direct integration scheme, systems are forced to transfer the same information repeatedly. For example, 1C sends order information to WMS, logistics software, CRM, a sales commission system, and so on.

The same data has to be transmitted five, ten, or twenty times. With ESB, instead of sending customer or order information ten times to different systems at different frequencies, 1C sends the information once at a defined interval. The receiving systems then pull this information from the storage in the ESB layer through separate integrations at whatever frequency they need. In KT.Team's practice, there was a case where a client had integrations set up with three systems.

One was responsible for warehouse stock (WMS), the second for pricing, and the third for product information (PIM).

Warehouse stock levels change most often: to keep an online store up to date, 1C had to check stock data every 15 seconds, about 40,000 times per week.

But the integration logic was configured so that 1C queried all three systems every 15 seconds and received 120,000 data packages per week in response.

Both 1C and the other systems in the environment were overloaded.

When the KT.Team team implemented integrations through ESB, each system began receiving and sending information at the required frequency. WMS still sends inventory stock data every 15 seconds. The pricing system transmits data once an hour (168 times a week). PIM, however, reduced scheduled data transfers to once a week - by 40,000 times. As a result, 1C began pulling just over 40,000 data packages instead of 120,000! 3.

Accounting for system performance The ESB logic can take into account data export and import speeds for target systems based on their performance and daily load patterns. For example, if your CRM can handle 100 transactions per minute during the day and 500 at night, the ESB will account for that logic and will not overload your systems. 4.

Direct integrations

do not necessarily include logging and monitoring.
The company only learns about emerging issues when they start affecting the business.
A properly designed ESB layer must include logging, monitoring, and alerting systems for emerging issues.
As soon as a message delivery problem occurs in any of the systems, engineers or support receive an incident alert and its location.

Example of Building Integrations Between 1C and Other Systems

In one of our recent projects for an international company in e-commerce and retail, the KT.Team team set up integrations for several 1C systems: ERP, OMS, UMP, CMS.

We implemented an integration bus and built 13 connectors for seven data streams: one connector receives orders from OMS; one connector passes order data to ERP; two connectors handle order status exchange between OMS and ERP; one connector sends order data to the rewards calculation service; two connectors send user data to the rewards calculation service; four connectors handle user currency and country data; two connectors are needed for warehouse filtering.

A separate connector is configured for each entity, or several if the logic requires it.

This makes it possible to transfer data between systems in a more controlled way.

Summary

: 1C can be integrated with ESB via JDBC (direct connection to simple databases) or through APIs using OData, RESTful, SOAP, JDBC, or GraphQL protocols (if the entity is defined by a complex database with many tables).
Each entity needs a separate connector and a separate API.
Integration through ESB allows 1C to transfer each entity once and at a defined frequency, regardless of the number of receiving systems.
Integration through ESB reduces system load, shrinks the codebase, lowers the total volume of transferred data, takes system performance into account, and loosens system coupling.

Which ESB to Choose for 1C Integration

If you have already looked into the topic, the first and most logical answer may be 1C:Shina.

A product from the same family does integrate easily with 1C:ERP

Enterprise Management, 1C:Accounting, 1C:Payroll and HR, etc.

Another CIS data bus, DATAREON, is fairly well adapted for working with 1C products.

But if your environment includes applications from other vendors, it makes sense to choose a data bus based on a broader set of criteria than compatibility with 1C alone.

Moreover, experience shows that any ESB can be easily integrated with 1C. For example, we have worked with Mule on projects with hundreds of 1C connectors and heavy bus traffic, and we did not observe any problems even during peak load periods.

When choosing a bus, our clients usually pay attention to the following parameters: the availability of an OData connector for 1C (this is extremely rare, and a ready-made connector is unlikely to exist in non-CIS ESBs); the presence of low-code implementation issues; the way related message information is stored; the ability to build a microservice-based exchange architecture with flexible scaling; the ability to keep the full scope of one connector's responsibility entirely within a single job (i.e.

small program for working with the data stream) or a set of jobs; version control; stream encryption; availability of a visual development interface; support for asynchronous mode; availability of a message transformation mechanism and the formats it supports; transformation method; licensing cost.

If you need to integrate 1C products with other systems and are not sure which ESB can be used to implement it, send us a message through the form.

Contacts

Let's Discuss Your Project

Leave your current contact details and describe your task. We will come back with clarifying questions and a proposal for the next step.

Proper Integration of 1C with Other Systems