DigitalThreads
// Solutions · Private Cloud

An on-premises Kubernetes cluster for the plant.

The edge devices contextualize your plant data and stream it out through a secure tunnel. This is the other end of that tunnel: three Dell PowerEdge servers in a rack you own, running Red Hat OpenShift. Kafka takes the ingest, MSSQL keeps the transactions, ClickHouse handles the billion-row queries, and every service runs as replicas spread across all three machines, so losing a server doesn't take anything down. The same cluster hosts your VMs, your private VPN, and your zero-trust access.

EDGE TUNNEL KAFKA CLUSTER ANSWERS
// The cluster01

The three-node cluster.

A compact Red Hat OpenShift cluster on three Dell PowerEdge rack servers: the control plane is spread across all three for quorum, and every workload runs as multiple replicas with anti-affinity rules that keep them on separate machines. Everything the cluster hosts gets the same high availability, including the Windows VM nobody wants to touch.

// one service, three replicas, three servers: lose a node and the workload keeps running THE BUSINESS dashboards · shift reports · ERP · quality · ML BI · SQL · REST · Grafana reached over private VPN / zero-trust access PRIVATE CLOUD · RED HAT OPENSHIFT · 3× DELL POWEREDGE NODE-01 control plane + worker kafka-0 mssql-ag-1 clickhouse-1 ingest-a grafana cloudflared-a NODE-02 control plane + worker kafka-1 mssql-ag-2 clickhouse-2 ingest-b api-svc vm · historian NODE-03 control plane + worker kafka-2 clickhouse-3 prometheus dashboards cloudflared-b vm · eng-tools BLOCK databases · kafka · low-latency IOPS NFS shared files · recipes · many readers S3 OBJECT data lake · archive · backups // storage replicated across all three nodes' drives · a StorageClass per tier, apps claim what fits IT every plant firewall: outbound only zero inbound ports · mTLS per site Cloudflare Tunnel · one per site Kafka payloads · outbound-only 443 PLANT 01 edge device · SCADA on site PLANT 02 edge device · SCADA on site PLANT NN edge device · SCADA on site
// edge → tunnel → kafka → storage & databases → the business · kafka-0/1/2 highlighted: one broker per node, by rule
tap a component for the technical detail
compute 3× Dell PowerEdge platform Red Hat OpenShift design N+1 · any node can fail storage block · NFS · S3 footprint one rack · on premises
// Data services02

Two database engines.

"Which operator signed off order 550123?" is a row question, and it needs a transactional engine with locks, constraints, and an audit trail. "What was the p95 cycle time on every filler across three years?" is a column question, and it needs to scan billions of rows in seconds. The cluster runs an engine for each, with Kafka feeding both from the same edge stream. The questions themselves are covered on the Analytics page; this one is about keeping the engines fast and running.

Transactional: MSSQL

Orders, genealogy, e-signatures, and confirmations go in SQL Server, where a write either commits or doesn't happen at all. An Always On availability group keeps synchronized replicas on separate nodes, so if a server fails, the database fails over in seconds with zero lost transactions.

Always On AG · RPO 0

Analytics: ClickHouse

Every telemetry point from every plant lands in a replicated columnstore built for exactly this: billions of rows, 10:1 compression, and aggregate queries that come back in seconds. Year-over-year OEE across all sites is one query, and it runs while ingest continues at full rate.

columnstore · billions of rows

Kafka in between

Edge payloads land on Kafka first: a durable, replayable buffer with three brokers, one per node, and every partition replicated three ways. MSSQL, ClickHouse, dashboards, and any future consumer subscribe independently. Add a system next year and you can replay the full history into it.

3 brokers · RF=3 · replay

Storage matched to the workload

Databases and brokers claim replicated block volumes for low-latency IOPS. Shared recipes, reports, and file drops mount NFS. The data lake, backups, and cold archive go to S3-compatible object storage. Each application asks for a tier by name and the cluster provisions it.

block · NFS · S3 object
see the analytics these engines power → /analytics.html
// Observability03

Monitoring the cluster.

The cluster is instrumented like a plant process. CPU, RAM, disk throughput, IOPS, and network utilization are scraped per pod, per volume, and per node, so "the platform feels slow" turns into a specific workload on a specific server in about a minute. Capacity planning is based on trends, not guesses.

CPU · cores in use ■ n01 ■ n02 ■ n03 peak 58% · fits on 2 nodes MEMORY · GiB resident ■ n01 ■ n02 ■ n03 412 / 1,536 GiB · zero OOM kills DISK · IOPS & MB/s ■ n01 ■ n02 ■ n03 1.2 GB/s read · 38k IOPS · p99 1.9 ms NETWORK · Gb/s ■ n01 ■ n02 ■ n03 east-west 8.4 Gb/s · tunnels 240 Mb/s
// scraped every 15 s from every pod, volume, and NIC · retained in the cluster's own time-series store

Metrics per workload

CPU throttling, memory pressure, restarts, and network throughput are tracked per workload, so the ClickHouse merge that eats six cores, the VM that leaked eight gigabytes, or the ingest pod saturating a NIC shows up by tenant, pod, and node, with history.

cpu · ram · net · per pod

Disk performance

Databases feel storage latency before anything else does, so every persistent volume reports IOPS, MB/s, and p99 latency continuously. A noisy neighbor shows up on a chart, storage growth turns into a forecast, and the next drives get ordered months before they're urgent.

IOPS · MB/s · p99 latency

Paged before you notice

Alert rules watch the same metrics: consumer lag rising, a volume trending toward full, a certificate near expiry, a replica count below spec. We carry the pager for the platform, and most incidents are fixed before the morning shift would have run into them.

alert → page → fixed
// Beyond containers04

Virtual machines and other plant IT.

Every plant has workloads that will never be containers: a Windows historian, a license server, an engineering workstation image. OpenShift Virtualization runs them as VMs on the same three servers, with the same replication, storage, and monitoring. The cluster also hosts the network services: the tunnels, the VPN, and the zero-trust policy that decides who can reach any of it.

VMs beside containers

VM workloads run under OpenShift Virtualization with live migration, so when a node drains for patching, the Windows historian moves to another server without stopping. You operate one platform, however old the workload is.

KubeVirt · live migration

Cloudflare Tunnel, hosted here

The cluster runs redundant tunnel connectors, so every edge site's stream terminates on hardware you own and every published app reaches its users through outbound-only connections. Like the plants that feed it, the rack has no internet exposure.

cloudflared ×2 · zero inbound

Private VPN for engineering

Controls engineers get a WireGuard-based VPN, served from the cluster, for the work that needs a real network path: online PLC edits, SCADA development, drive commissioning. Short-lived access, per-user keys, every session logged.

WireGuard · per-user keys

Zero-trust access

Grafana, dashboards, and admin consoles are published per app, behind SSO and MFA, with policy by identity: quality sees quality, maintenance sees maintenance, and the auditor's access expires Friday. Every request is authenticated, wherever on the network it comes from.

per-app policy · SSO · MFA
// The failure drill05

Testing failover during commissioning.

High availability is only a claim until a server actually dies. So during commissioning we kill node-02 at the power strip, on purpose, with your team watching the dashboards. Here's what happens:

t + 0 snode-02 loses both power feedsthat takes out a third of the cluster's compute, one Kafka broker, one MSSQL replica, and three ClickHouse merges in progresspulled
~5 sthe cluster noticeshealth probes fail, node-02 is marked unreachable, and traffic stops routing to every pod it was hostingautomatic
~10 sKafka re-elects partition leadersbrokers 0 and 2 take over the partitions broker 1 led; edge producers keep publishing and never see an errorRF=3
~15 sthe database fails overthe availability group replica on node-01 becomes primary with zero lost transactions; in-flight queries retry and completeRPO 0
~2 minworkloads resettle on two nodesreplacement pods schedule onto nodes 01 and 03, block volumes reattach, and the VMs restart from shared storage; capacity is sized so two nodes carry the full loadN+1
laternode-02 returnsit rejoins the cluster, replication catches up, and new pods can schedule on it again. Nobody edited a config fileautomatic
// dashboards stayed up, edge data kept landing, and the morning report went out on time
// what the customer gets

The rack is just the hardware. What you're paying for is a proper home for plant IT workloads:

  • Ready before the projects are. MES, historians, dashboards, and the next integration deploy onto shared, highly available infrastructure with storage and monitoring already in place.
  • Built to lose a server. Replicas spread across three servers, quorum-based control, and N+1 capacity mean a hardware failure is a log entry, and the weekend stays quiet.
  • The right engine for each question. MSSQL holds the transactional record, ClickHouse serves the billion-row analytics, and Kafka feeds both from the same contextualized edge stream.
  • Numbers on everything. CPU, RAM, IOPS, disk throughput, and network utilization per workload, with alerting and capacity forecasts, watched by the people who built it.
  • Data stays on site. Cloud-style architecture in your own rack, under your own compliance rules, reachable only through tunnels, VPN, and zero-trust policy you control.
// Contact06

Talk to us about a private cloud.

Tell us what should land on it: how many edge sites, which databases, which VMs refuse to die. We'll spec the servers, the cluster, and the storage, and stand it up beside your plants.

Talk to an engineer