Reactive maintenance is expensive: unplanned line downtime means lost shifts, penalties for missed shipments, and emergency spare parts purchases. Predictive maintenance flips the logic: equipment is taken out for service based on wear signals, not after it breaks. The technical foundation of this shift is continuous collection of sensor telemetry into an event log and automatic threshold-based triggers. Below is an open breakdown of how this is handled with Apache Kafka. This is an overview of industry practice based on public sources, not a description of a KT.Team project.
What business result comes from moving to an event log
The main result is that equipment is serviced when it is truly needed, not on a calendar and not after a shutdown. Key effects cited by industry practice include no unplanned downtime, higher OEE (overall equipment effectiveness), and resource savings from condition-based rather than schedule-based maintenance (Kai Waehner). The second architectural result is this: telemetry is collected once and consumed many times - monitoring, ML models, ERP, and dashboards all read the same stream independently, without rewriting the data collection layer.
How telemetry gets into Kafka
Two industrial standards coexist on the shop floor, and Kafka does not compete with them - it complements them. OPC-UA an automation standard that is natively supported by almost all modern machines, PLCs, and IoT gateways. MQTT a lightweight publish/subscribe protocol for unstable links and tens of thousands of devices. Apache Kafka acts as a central hub that collects streams from both protocols and provides what they lack: true decoupling with backpressure handling and data replayability (Kai Waehner, OPC-UA + MQTT + Kafka).
A typical flow looks like this: machines (OPC-UA / MQTT) -> edge gateway -> Kafka cluster -> analytics / ERP. In a public BMW example, OPC-UA connects factory equipment locally, while Kafka replicates the data to the public cloud in real time, enabling global coordination of smart factories without tight OT-IT coupling (Kai Waehner). The last-mile connection to devices is done through Kafka Connect or specialized connectors.
Threshold triggers: from a simple filter to a window
The logic of when to raise an alert is built in layers, and the key point here is simple: the simpler the threshold, the cheaper it is to operate.
Simple threshold (stateless). The stream is filtered without state: only threshold-exceeding events continue down the pipeline - for example, temperature spikes above 100 degrees (Kai Waehner). This filters out noise at the input.
Cumulative threshold (stateful, sliding window). A single spike does not always mean a problem. ksqlDB or Kafka Streams calculate aggregates in a sliding window: if within one hour there are more than ten peaks with an average temperature above 100 degrees, the risk of failure rises sharply, and the operator is sent a real-time maintenance alert (Kai Waehner).
The windowing mechanics are well illustrated by a Confluent Kafka Streams example: events are grouped, aggregated in a `TimeWindows.of(Duration.ofMinutes(1))` window, and filtered by a numeric threshold - there it is an error rate above 5%, but the same pattern applies to temperature, vibration, or pressure (Confluent):
```java
apiEvents
.groupByKey()
.windowedBy(TimeWindows.of(Duration.ofMinutes(1)))
.aggregate(/* running count */)
.filter((windowedKey, metric) ->
metric.value > THRESHOLD); // numeric threshold
```
Triggered thresholds are written to a separate alerts topic, where downstream consumers process them: the repair dispatch system, the technician's mobile app, and the service center.
ML on top of the same log - without changing collection
When static thresholds are not enough, a model is layered on top of the same stream. A TensorFlow model is embedded as a user-defined function (UDF) in ksqlDB - for example, an autoencoder for unlabeled anomaly detection (Kai Waehner). A public example of an end-to-end pipeline: an LSTM network trained to predict failure from historical features (temperature, pressure, compression ratio) performs inference on the incoming Kafka stream (Nick Alonso, Medium).
The value of the architecture lies in layer decoupling: thresholds can be tightened, models retrained, and new edge cases added without touching the data collection and transport layer. Telemetry collection remains a stable contract, while business logic lives alongside it and evolves separately. This matches the principle of minimal intervention in the core and loose coupling.
Data flow diagram
Left to right: machine sensors and PLCs publish signals via OPC-UA and MQTT -> edge gateway aggregates and normalizes -> raw telemetry topic in Kafka (collect once) -> three independent consumers branch off from it: (1) stateless filter simple thresholds, (2) stateful processor ksqlDB with a sliding window, (3) ML inference (autoencoder / LSTM). Triggered thresholds and anomalies converge in alerts topic -> it is read by repair dispatch, supervisor mobile app, ERP, and OEE dashboard. An archived copy of the telemetry topic is sent to storage for model retraining (replay).
Business process takeaway
An event log in Kafka turns maintenance from reactive to controlled: a sensor detects an anomaly -> a threshold or model evaluates the risk in real time -> a repair work order reaches the technician before the line stops. The process of "wait for failure -> emergency downtime -> unplanned repair" is replaced by "continuous wear signal -> planned maintenance window -> predictable workload." Because data collection is separated from threshold and model logic, the business can tighten rules and retrain models without stopping production or rewriting equipment integration.
