High-Availability Clustering

Keep messaging available when a node fails

A single broker is a single point of failure. High-availability clustering runs several Coreflux MQTT Broker nodes as one system: if one node stops, the others keep serving clients. Clients can connect to any healthy node. Important messages (QoS 1 and 2) are copied across nodes so a crash of a minority of the cluster cannot erase a message the broker already accepted.

Like a team with a captain and shared playbook. One node leads changes; the others stay ready. If the captain drops out, the team elects a new one and keeps playing — without rewriting every client’s connection plan.

When to use clustering

Use clustering when you need:

continuous MQTT service if one broker stops or is restarted;
important publishes (QoS 1/2) to survive if some of the nodes crash after the broker has accepted them; or
a fixed or elastic set of broker nodes that should act as one cluster.

Prefer odd node counts (3 or 5) so a clear majority can decide who leads. Clustering requires a license that grants the cluster feature. If you enable clustering without that license, the broker logs a warning and runs standalone.

Clustering is broker configuration, not a LoT System Route. For topic sync with another MQTT broker (including non-Coreflux), use an MQTT Bridge instead.

Configure each node with the same cluster identity and discovery settings, then deploy the same LoT project to every node through your normal workflow.

Quick Start

Add a Cluster section to the broker configuration (or matching environment variables — env vars override the file). On every node, create a dedicated MQTT user for inter-node traffic and set DataPlaneUsername to that username so nodes can exchange topic data.

Seeds (typical production)
Multicast (same LAN)
Kubernetes

Use fixed peer addresses when automatic discovery is unavailable (cloud, most Kubernetes networks, multi-subnet LANs):

{
  "Cluster": {
    "Enabled": true,
    "ClusterId": "factory-floor",
    "NodeName": "broker-a",
    "Discovery": "seeds",
    "Seeds": ["broker-a:9200", "broker-b:9200", "broker-c:9200"],
    "SharedSecret": "a-strong-secret",
    "DataPlaneUsername": "cluster"
  }
}

Auto-discover peers on a shared local network (default discovery mode):

{
  "Cluster": {
    "Enabled": true,
    "ClusterId": "lab-cluster",
    "NodeName": "broker-1",
    "Discovery": "multicast",
    "SharedSecret": "a-strong-secret",
    "DataPlaneUsername": "cluster"
  }
}

Discover peers via a headless Service DNS name; better for elastic pods than a static address list:

{
  "Cluster": {
    "Enabled": true,
    "ClusterId": "k8s-brokers",
    "NodeName": "broker-0",
    "Discovery": "kubernetes",
    "KubernetesServiceDns": "coreflux-broker-headless.default.svc.cluster.local",
    "KubernetesPeerPort": 9200,
    "SharedSecret": "a-strong-secret",
    "DataPlaneUsername": "cluster"
  }
}

Concepts operators need

Plane	What it does for you
Control plane	How nodes find each other, who leads, and which node may accept LoT writes
Data plane	Moves MQTT topic traffic between nodes so subscribers on any node can receive publishes

One leader. Runtime LoT write commands run only on the current leader. Followers reject writes and tell you which node is the leader. Read-only and node-local commands (queries, diagnostics, license/instance key) still run locally.
LoT is not copied between nodes. Each node loads its own LoT from disk / GitOps. A write applied on the leader changes only that node until you distribute the same project files to the others. Point runtime tooling at the leader, and treat your project repository as the source of truth for all nodes.
Only matching nodes join. Nodes must share the same cluster identity and join secret so unrelated brokers do not join by accident. (See ClusterId and SharedSecret in the configuration reference.)
Session failover needs durability. Enable MQTT state durability so a client that reconnects to another node can get its subscriptions back. Messages queued for offline clients stay on the node that hosted them (see Limitations).

Configuration reference

Core options

Config field	Env var	Default	Description
`Enabled`	`COREFLUX_CLUSTER_ENABLED`	`false`	Master switch
`ClusterId`	`COREFLUX_CLUSTER_ID`	`coreflux-cluster`	Logical cluster identity
`NodeName`	`COREFLUX_CLUSTER_NODE_NAME`	hostname	This node’s identity
`BindIP`	`COREFLUX_CLUSTER_BIND_IP`	`0.0.0.0`	Control-plane listener bind address
`Port`	`COREFLUX_CLUSTER_PORT`	`9200`	Control-plane TCP port
`AdvertisedHost`	`COREFLUX_CLUSTER_ADVERTISED_HOST`	`NodeName`	Address peers use to reach this node
`AdvertisedPort`	`COREFLUX_CLUSTER_ADVERTISED_PORT`	`Port`	Port peers use to reach this node
`Discovery`	`COREFLUX_CLUSTER_DISCOVERY`	`multicast`	`multicast`, `seeds`, or `kubernetes`
`Seeds`	`COREFLUX_CLUSTER_SEEDS`	`[]`	Comma/semicolon list of `host:port`
`MulticastGroup`	`COREFLUX_CLUSTER_MULTICAST_GROUP`	`239.63.114.71`	Multicast group
`MulticastPort`	`COREFLUX_CLUSTER_MULTICAST_PORT`	`9201`	Multicast announce port
`KubernetesServiceDns`	`COREFLUX_CLUSTER_K8S_SERVICE_DNS`	—	Headless Service DNS name
`KubernetesPeerPort`	`COREFLUX_CLUSTER_K8S_PEER_PORT`	—	Peer port for Kubernetes discovery
`SharedSecret`	`COREFLUX_CLUSTER_SHARED_SECRET`	—	Join secret (discovery filtering and data-plane integrity)
`DataPlaneEnabled`	`COREFLUX_CLUSTER_DATAPLANE_ENABLED`	`true`	Exchange MQTT traffic between nodes
`DataPlaneMqttPort`	`COREFLUX_CLUSTER_DATAPLANE_MQTT_PORT`	broker MQTT port	MQTT port used between nodes
`DataPlaneUsername`	`COREFLUX_CLUSTER_DATAPLANE_USERNAME`	—	MQTT username for inter-node links (password from the user store)
`UseTls`	`COREFLUX_CLUSTER_TLS`	`false`	TLS on inter-node MQTT links
`Topology`	`COREFLUX_CLUSTER_TOPOLOGY`	`mesh`	`mesh` (recommended) or `ring` (complete only for ≤ 3 nodes)
`DataDir`	`COREFLUX_CLUSTER_DATA_DIR`	`{CFBasePath}/cluster`	Cluster state directory

The data plane starts only when DataPlaneEnabled is true and DataPlaneUsername names a user that exists in the broker MQTT user store on that node.

Replication options

Config field	Env var	Default	Description
`ReplicationEnabled`	`COREFLUX_CLUSTER_REPLICATION_ENABLED`	`true`	Require quorum durability before acknowledging QoS ≥ 1 publishes
`ReplicationFactor`	`COREFLUX_CLUSTER_REPLICATION_FACTOR`	`0` (= auto min(3, N))	Replica-set size per partition
`PartitionCount`	`COREFLUX_CLUSTER_PARTITION_COUNT`	`32`	Logical partitions (pick once; changing re-hashes keys)
`ReplicationTimeoutMs`	`COREFLUX_CLUSTER_REPLICATION_TIMEOUT_MS`	`5000`	Quorum wait before rejecting a publish
`ReplicationReplaySeconds`	`COREFLUX_CLUSTER_REPLICATION_REPLAY_SECONDS`	`300`	Failover replay window (0 = no replay)

Ports

Port	Protocol	Purpose
`9200`	TCP	Control plane (leader election and membership)
`9201`	UDP	Multicast discovery (multicast mode only)
Broker MQTT port	TCP	Inter-node MQTT data plane

Open these between cluster members in your firewall or network policies.

Delivery guarantees

QoS	What clustering adds
0	Best-effort delivery to peers; may be lost if a target node is down
1	The broker accepts the publish only after a majority of the replica set has stored it; other nodes receive it reliably; after failover it may be delivered again (at-least-once)
2	Same majority check before the broker accepts the publish in normal operation; across failover, treat delivery as at-least-once and design consumers to ignore duplicates

Summary: once the broker has accepted a QoS 1 or 2 publish, that message is stored on a majority of nodes and can be replayed if ownership moves after a failure. If the cluster is split and this side no longer has a majority, it rejects those publishes instead of accepting messages it cannot keep safe.

Operations

Bootstrap and failover

A node that sees an existing leader joins as a follower.
If no leader is visible and peers exist, the cluster elects a leader and adds members as they are discovered.
When the leader is lost, a new one is elected quickly (typically within a fraction of a second). Survivors stop contacting confirmed-dead peers and bring returning nodes back into the mesh automatically.
With seeds discovery, a stopped node can remain a configured member (marked unavailable) while majority is computed over the configured list. Prefer kubernetes discovery when instances come and go often.

Observability

Signal	What it shows
`GET /cluster`	Full status: role, leadership, members, data-plane health, replication, and related fields
`GET /info`	Lightweight `cluster` summary
Prometheus `/metrics`	Cluster, partition, replication, and related series (`coreflux_cluster_`, `coreflux_replication_`, and related)
Retained `$SYS` topics	`$SYS/Coreflux/Cluster/{Role,IsLeader,Leader,Term,Members,Status}` (refreshed periodically)

Use MQTT Explorer or any MQTT client to subscribe to the $SYS/Coreflux/Cluster/# topics, or call GET /cluster on each node’s operations HTTP port.

Limitations

LoT is not replicated between nodes. Distribute the same project/configuration to every node. Runtime edits on the leader do not appear on followers until you provision them.

Retained messages are node-local. They survive a restart on that node but are not a cluster-wide retained store. A subscriber on another node may not see a retain set elsewhere.

Subscriptions can move; offline queues do not. A client reconnecting with a persistent session to another node can regain its subscriptions when durability is enabled. Queued QoS 1/2 messages for offline clients stay on the node that hosted them until that node returns.

QoS 2 across failover is at-least-once. Design consumers to tolerate duplicates after a node failure or replay.

Next Steps

MQTT State Durability

Persist retained messages, sessions, and queued QoS across restarts — required for session failover.

Broker Configuration

Ports, TLS, listeners, and other broker settings for each node.

Quick Start

Coreflux MQTT Broker

LoT Language

Coreflux HUB

VS Code - LoT Notebooks

Using AI with Coreflux

Product Updates

Keep messaging available when a node fails

When to use clustering

Quick Start

Concepts operators need

Configuration reference

Core options

Replication options

Ports

Delivery guarantees

Operations

Bootstrap and failover

Observability

Limitations

Next Steps

MQTT State Durability

Broker Configuration

​Keep messaging available when a node fails

​When to use clustering

​Quick Start

​Concepts operators need

​Configuration reference

​Core options

​Replication options

​Ports

​Delivery guarantees

​Operations

​Bootstrap and failover

​Observability

​Limitations

​Next Steps

MQTT State Durability

Broker Configuration

Keep messaging available when a node fails

When to use clustering

Quick Start

Concepts operators need

Configuration reference

Core options

Replication options

Ports

Delivery guarantees

Operations

Bootstrap and failover

Observability

Limitations

Next Steps