#ai#podcast#data

You Already Have a Distributed Database—No Guarantees

1C, PIM, and your website already form a distributed system without guarantees. Learn which CockroachDB and Raft principles prevent lost orders.

  • aanleiding: a podcast about CockroachDB
  • What problem a distributed database solves
  • Guarantees are limited by physics
  • Your distributed system operates without a quorum

aanleiding: a podcast about CockroachDB

  1. Gergely Orosz spoke with on The Pragmatic Engineer podcast

  2. Peter Mattis, co-founder of Cockroach Labs and one of CockroachDB’s authors (episode).

  3. This edition explores how to make dozens of servers work as one database. For an IT leader, the discussion offers a practical takeaway.

  4. Most companies already operate a distributed system: 1C, PIM, the website, and the warehouse exchange data.

  5. But it has none of the guarantees on which CockroachDB is built.

What problem a distributed database solves

Before Cockroach Labs, Mattis worked at Google on the Colossus file storage system. In 2012, Google described Spanner—a global database in which transactions in New York and in

Tokyo sees the same data

CockroachDB emerged in 2015 as an open implementation of this idea, without the atomic clocks used by Spanner. The mechanism works as follows.

The database splits data into ranges and stores each in three replicas.

A write is considered complete when two of the three replicas confirm it.

The Raft protocol defines the consensus process: each range has a leader that maintains an operation log, while the other replicas replay it.

If a server or an entire rack fails, the database keeps running without losing confirmed writes.

Guarantees are limited by physics

Consistency costs money and milliseconds. From Moscow to

Novosibirsk is about 2,800 km away in a straight line

Light travels through fiber at about 200,000 km/s, so a round-trip signal takes at least 28 ms—and longer over real routes and through equipment.

If replicas are located in different cities, every write waits for a quorum, and the user experiences that delay each time an order is saved.

The CAP theorem defines this trade-off precisely.

When the network between nodes fails, the system must choose between continuing to respond and guaranteeing data consistency. CockroachDB chooses consistency. Nodes without a quorum stop accepting writes, so the data does not diverge.

Map out your integration landscape

clients@kt.team Telegram @kt_team_it

Your distributed system operates without a quorum

  1. A typical retailer or distributor landscape: 1C:ERP manages stock and prices, Akeneo or Pimcore stores item cards, Magento or 1C-Bitrix sells, and the WMS ships orders.

  2. Data moves between them through exports every 15 minutes and HTTP scripts once written by a contractor.

  3. Essentially, this is a four-replica database without a quorum, a log, retry on failure, or a single data owner.

  4. The cost of this becomes visible in sales reports.

  5. The website sells an item that went out of stock an hour ago.

  6. The price on the storefront differs from the price in 1C.

  7. The script times out and the order disappears.

  8. There is no divergence monitoring, so customers are the first to learn about a failure when they contact support.

How to apply CockroachDB principles to integration

It is enough to apply CockroachDB’s engineering principles to data exchange between systems; there is no need to install the database itself.

This is how we build integrations at KT.Team: - Event log.

Systems publish events to Apache Kafka

Kafka stores an ordered log similar to a Raft journal: after a failure, a consumer resumes reading missed events from where it stopped.

With a modest event volume, a Datareon or Talend ESB integration bus serves the same purpose. - One owner for every fact.

Stock belongs to 1C, content to PIM, and order status to OMS. In CockroachDB, exactly one leader accepts writes for each range.

Here, the same rule applies at the system level. - Idempotency.

Every message carries a unique key.

Delivering the same order again does not create a duplicate order. - Transactional outbox.

The system records the change and the outgoing event in a single transaction in its own database.

A separate process delivers the event even if the broker was unavailable when the write occurred. This prevents the scenario “it was saved in 1C but never reached the website.”

- Divergence metric. Stock levels between 1C and the website are reconciled on a schedule, and the discrepancy rate is displayed on a dashboard.

As it grows, the engineer receives an alert before the customer encounters an error. Each item looks simple.

In practice, outbox and idempotency require discipline in every system in the landscape, including 1C customizations that companies postpone for years.

This is precisely the integrator’s main job.

When a distributed database is really needed

  1. Rule of thumb: a distributed SQL database pays off when the business needs to write data from multiple regions simultaneously and withstand the loss of an entire data center without downtime.

  2. This rule applies to banks, large-scale e-commerce, and logistics companies with nationwide networks.

  3. For everyone else, PostgreSQL with synchronous replication is enough: it is cheaper, easier to operate, and has a larger pool of available specialists.

  4. Consider the license when choosing a solution. In 2024, Cockroach Labs discontinued the free Core edition.

  5. The free Enterprise edition is available to companies with annual revenue of up to $10 million; everyone else pays.

  6. CIS companies can use Yandex's YDB, a distributed SQL database released under Apache 2.0 in 2022 and tested under Yandex's own workloads.

Conclusion

  1. Since 2015, Mattis’s team has tackled a problem most companies never even articulate: how to keep data replicas from contradicting one another.

  2. It is usually too early to buy a distributed database for this.

  3. It is enough to acknowledge that 1C, PIM, and the website already form a distributed system, then give it an event log, data owners, and a divergence metric.

  4. The result is measured by the number of orders that were not lost in transit between systems.

Discuss the article: You already have a distributed database…

Enter your email or phone number so we can get back to you.

Send via:

By clicking the button, you accept the offer and grant consent to the processing of personal data.

Related Cases

All cases

Related solutions

All solutions
PIM System: Implementation, Integrations, MarketplacesIndustry Solutions on AkeneoAkeneo for Supplier Onboarding in DistributionAkeneo for Manufacturing: technical data from PLM and ERP into specs

Articles on the topic

Collection on PIM and master data

Related videos

All videos
3:10 PIM integration with CRM / WHO needs it and WHEN? PIMCRMIntegrations
2:08 Implementing a PIM system / Who needs it and when? PIM
Shorts Why MDM systems are not a cure but temporary pain relief, in 55 sec | Andrey Putin MDM

News on the topic

All news