> ## Documentation Index
> Fetch the complete documentation index at: https://docs.coreflux.org/llms.txt
> Use this file to discover all available pages before exploring further.

# System Routes

> Configure broker clustering and high-availability setups with MQTT_CLUSTER routes

## MQTT Cluster Routes

System routes configure broker-level infrastructure, primarily for clustering and high availability. The `MQTT_CLUSTER` route type enables multiple Coreflux brokers to work together as a unified system.

<Note>
  System Routes configure broker infrastructure rather than external data connections. For data integration routes, see [Data Pipeline Routes](/lot-language/routes/data-pipeline-routes) or [Data Storage Routes](/lot-language/routes/data-storage).
</Note>

<Tip>
  Clustering allows you to scale horizontally and provides redundancy - if one broker fails, others continue operating.
</Tip>

## MQTT\_CLUSTER Syntax

```lot theme={null}
DEFINE ROUTE <route_name> WITH TYPE MQTT_CLUSTER
    ADD CLUSTER_CONFIG
        WITH BROKER_ADDRESS "<primary_broker_address>"
        WITH BROKER_PORT '<port>'
        WITH CLIENT_ID "<cluster_node_id>"
        WITH USERNAME "<username>"
        WITH PASSWORD "<password>"
        WITH MESSAGE_VALIDITY_SEC '<seconds>'
```

### Configuration Parameters

<ParamField path="BROKER_ADDRESS" type="string" required>
  IP address or hostname of the primary/peer broker in the cluster.
</ParamField>

<ParamField path="BROKER_PORT" type="integer" required>
  MQTT port of the cluster peer (typically 1883 or 8883 for TLS).
</ParamField>

<ParamField path="CLIENT_ID" type="string" required>
  Unique identifier for this node within the cluster.
</ParamField>

<ParamField path="USERNAME" type="string">
  Authentication username for cluster communication.
</ParamField>

<ParamField path="PASSWORD" type="string">
  Authentication password for cluster communication.
</ParamField>

<ParamField path="MESSAGE_VALIDITY_SEC" type="integer">
  How long messages remain valid for synchronization (in seconds). Default varies by configuration.
</ParamField>

## Basic Cluster Configuration

```lot theme={null}
DEFINE ROUTE ClusterNode1 WITH TYPE MQTT_CLUSTER
    ADD CLUSTER_CONFIG
        WITH BROKER_ADDRESS "192.168.1.100"
        WITH BROKER_PORT '1884'
        WITH CLIENT_ID "ClusterNode1"
        WITH USERNAME "cluster_user"
        WITH PASSWORD "cluster_password"
        WITH MESSAGE_VALIDITY_SEC '30'
```

## Cluster Architecture

A typical cluster setup involves configuring each node to connect to its peers:

```mermaid theme={null}
graph LR
    subgraph cluster [Coreflux Cluster]
        Node1[Node 1<br/>192.168.1.100]
        Node2[Node 2<br/>192.168.1.101]
        Node3[Node 3<br/>192.168.1.102]
    end
    
    Node1 <--> Node2
    Node2 <--> Node3
    Node1 <--> Node3
    
    Client1[Client A] --> Node1
    Client2[Client B] --> Node2
    Client3[Client C] --> Node3
```

## Complete 3-Node Cluster Setup

The following example demonstrates a complete 3-node cluster setup. Each node needs route definitions pointing to its peer nodes. We'll configure Node 1 first, then show the pattern for Nodes 2 and 3.

<Info>
  **Before You Begin:** Ensure all three brokers are installed and accessible on your network. You'll need the IP addresses and credentials for each node.
</Info>

### Node 1 Configuration

```lot theme={null}
DEFINE ROUTE ClusterPeer2 WITH TYPE MQTT_CLUSTER
    ADD CLUSTER_CONFIG
        WITH BROKER_ADDRESS "192.168.1.101"
        WITH BROKER_PORT '1883'
        WITH CLIENT_ID "Node1-to-Node2"
        WITH USERNAME "cluster"
        WITH PASSWORD "secure_cluster_pass"
        WITH MESSAGE_VALIDITY_SEC '30'

DEFINE ROUTE ClusterPeer3 WITH TYPE MQTT_CLUSTER
    ADD CLUSTER_CONFIG
        WITH BROKER_ADDRESS "192.168.1.102"
        WITH BROKER_PORT '1883'
        WITH CLIENT_ID "Node1-to-Node3"
        WITH USERNAME "cluster"
        WITH PASSWORD "secure_cluster_pass"
        WITH MESSAGE_VALIDITY_SEC '30'
```

## Use Cases

### High Availability

Ensure continuous operation even if individual brokers fail:

* Clients can connect to any cluster node
* Messages are synchronized across nodes
* Failover is automatic

### Geographic Distribution

Deploy brokers in different locations with cluster synchronization:

```lot theme={null}
DEFINE ROUTE EuropeCluster WITH TYPE MQTT_CLUSTER
    ADD CLUSTER_CONFIG
        WITH BROKER_ADDRESS "eu-broker.example.com"
        WITH BROKER_PORT '8883'
        WITH CLIENT_ID "US-to-EU"
        WITH USE_TLS true
        WITH MESSAGE_VALIDITY_SEC '60'
```

### Load Balancing

Distribute client connections across multiple nodes:

* Each node handles a portion of clients
* Messages reach all subscribers regardless of which node they're connected to
* Horizontal scaling as load increases

## Configuration Best Practices

<AccordionGroup>
  <Accordion title="Use Unique Client IDs">
    Each cluster connection should have a unique CLIENT\_ID:

    ```lot theme={null}
    WITH CLIENT_ID "Node1-to-Node2"  // Unique per connection
    ```

    Duplicate IDs can cause connection conflicts.
  </Accordion>

  <Accordion title="Set Appropriate Message Validity">
    Balance between message reliability and memory usage:

    * **Short (10-30s)**: Fast-changing data, high throughput
    * **Medium (30-60s)**: Standard operational data
    * **Long (60-300s)**: Critical messages, unreliable networks

    ```lot theme={null}
    WITH MESSAGE_VALIDITY_SEC '30'
    ```
  </Accordion>

  <Accordion title="Use TLS for Production">
    Always enable TLS for cluster communication in production:

    ```lot theme={null}
    ADD CLUSTER_CONFIG
        WITH BROKER_ADDRESS "peer-broker.example.com"
        WITH BROKER_PORT '8883'
        WITH USE_TLS true
    ```
  </Accordion>

  <Accordion title="Consistent Authentication">
    Use the same credentials across all cluster nodes:

    ```lot theme={null}
    WITH USERNAME "cluster_service"
    WITH PASSWORD "shared_secure_password"
    ```
  </Accordion>
</AccordionGroup>

## Monitoring Cluster Health

Monitor cluster status through system topics:

* `$SYS/broker/cluster/status` - Overall cluster health
* `$SYS/broker/cluster/nodes` - Connected peer nodes
* `$SYS/broker/cluster/sync` - Synchronization status

## Limitations

<Warning>
  * Cluster routes require network connectivity between nodes
  * High latency between nodes can affect synchronization
  * Very large message volumes may require tuning MESSAGE\_VALIDITY\_SEC
</Warning>

## Next Steps

<CardGroup cols={2}>
  <Card title="Data Pipeline Routes" icon="arrow-right-arrow-left" href="./data-pipeline-routes">
    Configure MQTT bridges for edge-to-cloud connectivity.
  </Card>

  <Card title="Route Examples" icon="code" href="./examples">
    See complete route configuration examples.
  </Card>
</CardGroup>