> ## Documentation Index
> Fetch the complete documentation index at: https://docs.coreflux.org/llms.txt
> Use this file to discover all available pages before exploring further.
# MODEL Protobuf
> Decode and encode binary protobuf on MQTT with LoT models, GET PROTO, and PUBLISH MODEL
## Speak the same binary schema as your gateway
Industrial gateways and PLCs often publish **binary protobuf** on MQTT while the broker maps plant tags (Modbus, S7, OPC UA) to plain topics. LoT models let you **decode** those commands, run normal tag logic in the middle, and **encode** telemetry back — without a separate transformation service.
Dynamic protobuf encoding and decoding in LoT is **beta**. Prefer **`PUBLISH MODEL`** for binary MQTT output. Match your vendor `.proto` with `PROTO_TAG` and report issues with that layout and your broker version.
**Wire numbers are mailbox slots, not labels.** Protobuf finds a field by its number (`PROTO_TAG`), not by the LoT or `.proto` field name. If the slot numbers drift, the message still arrives — it just lands in the wrong boxes, or looks empty.
### When to use this
Use protobuf models when:
* a gateway or SCADA system publishes **raw protobuf bytes** on MQTT;
* you must **emit protobuf** that a vendor decoder already understands;
* field numbers in the `.proto` are **sparse** (for example `101`) or nested.
Use [JSON models](./overview) and [`GET JSON`](/v2.1/lot-language/actions/operations#get-json) when payloads are JSON.
***
Start from the direction you need: decode inbound bytes, publish outbound bytes, or both.
## Quick Start
Read top-level fields from a binary command payload with `GET PROTO`. Declare every command arm so unused submessages keep their wire numbers:
```lot wrap theme={"theme":"css-variables","languages":{"custom":["/languages/lot.json"]}}
DEFINE MODEL DeviceCommand WITH FORMAT PROTOBUF COLLAPSED
ADD INT "command_type" PROTO_TAG 1
ADD OBJECT "header" PROTO_TAG 2
ADD OBJECT "write_setpoint" PROTO_TAG 3
ADD OBJECT "ack_alarm" PROTO_TAG 4
ADD OBJECT "reserved_slot" PROTO_TAG 5
ADD OBJECT "read_snapshot" PROTO_TAG 6
```
The Action reads `command_type` from the triggering payload:
```lot wrap theme={"theme":"css-variables","languages":{"custom":["/languages/lot.json"]}}
DEFINE ACTION OnPlcCommand
ON TOPIC "plant/+/+/command" DO
SET "cmd" WITH (GET PROTO "command_type" IN PAYLOAD AS INT USING MODEL "DeviceCommand")
KEEP TOPIC "plant/pending_command" WITH {cmd}
```
Nested messages you fill on publish use `ADD MODEL`:
```lot wrap theme={"theme":"css-variables","languages":{"custom":["/languages/lot.json"]}}
DEFINE MODEL MessageHeader WITH FORMAT PROTOBUF COLLAPSED
ADD STRING "device_id" PROTO_TAG 1
ADD STRING "timestamp" PROTO_TAG 2
DEFINE MODEL ProcessValues WITH FORMAT PROTOBUF COLLAPSED
ADD DOUBLE "temperature" PROTO_TAG 1
ADD DOUBLE "pressure" PROTO_TAG 2
DEFINE MODEL TelemetryMessage WITH FORMAT PROTOBUF COLLAPSED
ADD INT "message_type" PROTO_TAG 1
ADD MODEL MessageHeader "header" PROTO_TAG 2
ADD MODEL ProcessValues "process_values" PROTO_TAG 3
```
The Action packs JSON tag values into that envelope and publishes **bytes** on the gateway topic:
```lot wrap theme={"theme":"css-variables","languages":{"custom":["/languages/lot.json"]}}
DEFINE ACTION PublishTelemetry
ON TOPIC "plant/+/+/tags" DO
SET "line" WITH TOPIC POSITION 2
SET "device_id" WITH TOPIC POSITION 3
SET "temp" WITH GET JSON "Temperature" IN PAYLOAD AS DOUBLE
SET "pressure" WITH GET JSON "Pressure" IN PAYLOAD AS DOUBLE
SET "ts" WITH TIMESTAMP "UNIX"
PUBLISH MODEL TelemetryMessage TO "plant/" + {line} + "/" + {device_id} + "/telemetry" WITH
message_type = 2
header.timestamp = {ts}
process_values.temperature = {temp}
process_values.pressure = {pressure}
```
***
## Where protobuf applies
| Mechanism | Protobuf on MQTT? |
| ------------------------------------------------------------ | ------------------------------------------------------------------------- |
| **`PUBLISH MODEL … TO "topic"`** in an Action | **Yes** — binary (or JSON plus a `/protobuf` sidecar topic for `BOTH`) |
| **Trigger-based `DEFINE MODEL … WITH TOPIC` + `AS TRIGGER`** | **No** — still JSON on the base topic; use **`PUBLISH MODEL`** for binary |
| **OT route tag topics** (Modbus, S7, OPC UA mappings) | Usually **JSON or plain values** — not protobuf unless you add it |
Define protobuf models for **edge** topics. Use **`GET PROTO`** to read and **`PUBLISH MODEL`** to write. Tag reads and writes stay on your existing OT routes.
```mermaid actions={false} theme={"theme":"css-variables","languages":{"custom":["/languages/lot.json"]}}
sequenceDiagram
participant GW as Gateway or SCADA
participant Broker
participant PLC as PLC via OT route
GW->>Broker: DeviceCommand proto bytes
Broker->>Broker: GET PROTO command_type
Broker->>PLC: TRIGGER route or tag write
PLC->>Broker: tag values on MQTT topics
Broker->>GW: PUBLISH MODEL TelemetryMessage proto bytes
```
***
## Match your `.proto`
To stay aligned with a hand-written schema (including **non-contiguous** field numbers and nested messages):
1. **`PROTO_TAG`** — wire numbers must match the `.proto` (not field names).
2. **Nested messages you fill on publish** — **`ADD MODEL ChildName "field"`** where `ChildName` is a `DEFINE MODEL`.
3. **Reserved wire slots** — **`ADD OBJECT "field" PROTO_TAG n`** when the parent must keep empty submessage fields so later tags stay aligned.
`GET PROTO` and `GET PROTOBUF` are the same verb. `PROTO` and `PROTOBUF` are the same format keyword.
### Type mapping
A field is only decoded when the **wire type** the sender used matches the type your model implies. Get this wrong and that field is skipped — there is no error on the wire.
| Proto type | Declare in LoT | Read with `GET PROTO … AS` |
| -------------------------------------------------------------------- | ------------------------- | ---------------------------- |
| `double` | `ADD DOUBLE` | `DOUBLE` |
| `int32` · `int64` · `uint32` · `uint64` · `sint32` · `sint64` · enum | `ADD INT` | `INT` |
| `bool` | `ADD BOOL` | `BOOL` |
| `string` | `ADD STRING` | `STRING` |
| `bytes` | `ADD OBJECT` | `BYTES` |
| nested `message` (publishing a filled body) | `ADD MODEL Child "field"` | — (fill via `PUBLISH MODEL`) |
| nested `message` (holding a wire slot open) | `ADD OBJECT "field"` | `BYTES` |
`ADD INT` is a **varint**. Proto `fixed32`, `sfixed32`, `fixed64`, and `sfixed64` use a different wire encoding and have no LoT keyword (`ADD FIXED32` does not exist). If those types appear in the vendor schema, declaring them as `ADD INT` will skip or misread the field.
### What happens on a mismatch
The decoder follows the protobuf unknown-field rule rather than failing the whole message:
| Situation | Result |
| -------------------------------------------- | ------------------------------------------------------------------ |
| Declared type matches the incoming wire type | Field decodes |
| Field number not in your model | Skipped |
| Truncated or malformed field | Decoding of that message stops there; fields already read are kept |
A skipped field is **absent**, and `GET PROTO` returns `null` for it. There is no error log — an unexpected `null` almost always means a wire-type mismatch.
### How field numbers are assigned
The broker does not ship a plant schema. Numbers come from your `DEFINE MODEL`:
| Your model definition | Wire field numbers |
| --------------------- | ----------------------------------------------- |
| No `PROTO_TAG` | **1, 2, 3, …** in LoT source order |
| Every field tagged | Exactly those tags |
| Mixed | Tagged = explicit; untagged = next free integer |
If the `.proto` uses field **101** but LoT omits `PROTO_TAG 101`, the next sequential number is used and the gateway decoder fails.
***
## Inbound: `GET PROTO`
Prefer **`IN PAYLOAD`** on the triggering topic. Use the other sources when the bytes are already cached or wrapped as Base64:
| Source | Bytes come from |
| ----------------------- | ------------------------------------------------------- |
| **`IN PAYLOAD`** | The MQTT payload that fired the Action |
| **`IN GET TOPIC "…"`** | Last payload stored for that topic |
| **`IN` a LoT variable** | A Base64 string in that variable (decoded before parse) |
Pass **`USING MODEL "Name"`** unless a `DEFINE MODEL … WITH TOPIC` already registered that schema on the topic. **`AS INT` / `DOUBLE` / `STRING` / `BOOL` / `BYTES`** selects the LoT type after decode.
Read from the last payload stored on another topic:
```lot wrap focus={3} theme={"theme":"css-variables","languages":{"custom":["/languages/lot.json"]}}
DEFINE ACTION RetryCachedCommand
ON TOPIC "plant/+/+/command/retry" DO
SET "cmd" WITH (GET PROTO "command_type" IN GET TOPIC "plant/line1/plc01/command" AS INT USING MODEL "DeviceCommand")
```
Or decode Base64 that arrived as JSON:
```lot wrap focus={4} theme={"theme":"css-variables","languages":{"custom":["/languages/lot.json"]}}
DEFINE ACTION DecodeBase64Command
ON TOPIC "plant/+/encoded" DO
SET "b64" WITH GET JSON "payload_b64" IN PAYLOAD AS STRING
SET "cmd" WITH (GET PROTO "command_type" IN {b64} AS INT USING MODEL "DeviceCommand")
```
Notes that matter in practice:
* **Each `GET PROTO` decodes the payload from scratch.** Reading two fields means two independent passes.
Do not use `IF PAYLOAD IS PROTO` to detect binary MQTT. Call **`GET PROTO … IN PAYLOAD`** instead.
### Parent command with reserved field numbers
Vendor `.proto` files often declare several command arms but send only one. List **every** submessage slot on the parent so unused fields do not shift wire numbers. Use **`ADD OBJECT`** for unused slots. When **publishing** a filled nested message, use **`ADD MODEL`** on the parent instead.
```lot wrap theme={"theme":"css-variables","languages":{"custom":["/languages/lot.json"]}}
DEFINE MODEL DeviceCommand WITH FORMAT PROTOBUF COLLAPSED
ADD INT "command_type" PROTO_TAG 1
ADD OBJECT "header" PROTO_TAG 2
ADD OBJECT "write_setpoint" PROTO_TAG 3
ADD OBJECT "ack_alarm" PROTO_TAG 4
ADD OBJECT "reserved_slot" PROTO_TAG 5
ADD OBJECT "read_snapshot" PROTO_TAG 6
```
***
## Middle: tags stay JSON
Between command and telemetry, most OT setups already have **tag values** on topics — from a `DEFINE ROUTE` mapping, a collapsed JSON model, or single-value publishes. That step stays JSON or plain text unless your plant standard requires protobuf on those topics too.
```lot wrap theme={"theme":"css-variables","languages":{"custom":["/languages/lot.json"]}}
DEFINE ACTION ReadPlcTags
ON TOPIC "plant/line1/plc01/tags" DO
SET "temp" WITH GET JSON "Temperature" IN PAYLOAD AS DOUBLE
SET "pressure" WITH GET JSON "Pressure" IN PAYLOAD AS DOUBLE
SET "speed" WITH GET JSON "MotorSpeed" IN PAYLOAD AS DOUBLE
PUBLISH TOPIC "plant/line1/plc01/snapshot/temperature" WITH {temp}
```
***
## Outbound: `PUBLISH MODEL`
### Topic vs format
| Model format | Payload on **`TO "…"`** topic |
| ---------------------------------------- | -------------------------------------------------------------- |
| **`WITH FORMAT JSON`** (default) | JSON string |
| **`WITH FORMAT PROTOBUF`** / **`PROTO`** | Raw protobuf **bytes** |
| **`WITH FORMAT BOTH`** | JSON on the main topic + binary on a `/protobuf` sidecar topic |
The next `PUBLISH MODEL` to the same topic **merges**: binary for `PROTO` only; JSON from the main topic for `JSON` / `BOTH`. **`KEEP` + protobuf** updates the byte cache without per-field JSON fan-out.
Trigger-based models (`WITH TOPIC` + `AS TRIGGER`) still auto-publish **JSON** on the base topic. Use `PUBLISH MODEL` when the edge must see binary.
### Response envelope with sparse tags
Alarms and events often use high field numbers (**101**, **102**). Tag them explicitly; use **`ADD MODEL`** for nested bodies you fill on publish. `MessageHeader` and `ProcessValues` match Quick Start — the new pieces are `AlarmEvent` and field **101**:
```lot wrap focus={10-18} theme={"theme":"css-variables","languages":{"custom":["/languages/lot.json"]}}
DEFINE MODEL MessageHeader WITH FORMAT PROTOBUF COLLAPSED
ADD STRING "device_id" PROTO_TAG 1
ADD STRING "timestamp" PROTO_TAG 2
DEFINE MODEL ProcessValues WITH FORMAT PROTOBUF COLLAPSED
ADD DOUBLE "temperature" PROTO_TAG 1
ADD DOUBLE "pressure" PROTO_TAG 2
ADD INT "motor_speed" PROTO_TAG 3
DEFINE MODEL AlarmEvent WITH FORMAT PROTOBUF COLLAPSED
ADD INT "alarm_code" PROTO_TAG 1
ADD STRING "alarm_text" PROTO_TAG 2
DEFINE MODEL TelemetryMessage WITH FORMAT PROTOBUF COLLAPSED
ADD INT "message_type" PROTO_TAG 1
ADD MODEL MessageHeader "header" PROTO_TAG 2
ADD MODEL ProcessValues "process_values" PROTO_TAG 3
ADD MODEL AlarmEvent "alarm_event" PROTO_TAG 101
```
LoT names are for **`PUBLISH MODEL`** / **`GET PROTO`**. Wire numbers come only from **`PROTO_TAG`**. Set `message_type` to the enum integer your `.proto` expects.
### Raw `bytes` fields
Some messages carry opaque **`bytes`** (recipe blob, certificate, file chunk). Assign them with **`BYTES FROM BASE64`**:
```lot wrap theme={"theme":"css-variables","languages":{"custom":["/languages/lot.json"]}}
DEFINE MODEL RecipeBlock WITH FORMAT PROTOBUF COLLAPSED
ADD INT "recipe_id" PROTO_TAG 1
ADD OBJECT "payload" PROTO_TAG 2
```
Then the Action fills `payload` from a Base64 JSON field:
```lot wrap focus={3-5} theme={"theme":"css-variables","languages":{"custom":["/languages/lot.json"]}}
DEFINE ACTION StoreRecipe
ON TOPIC "plant/+/recipe/upload" DO
PUBLISH MODEL RecipeBlock TO "plant/archive/recipe" WITH
recipe_id = 42
payload = BYTES FROM BASE64 (GET JSON "data_b64" IN PAYLOAD AS STRING)
```
To forward an entire protobuf payload unchanged (no field access), republish `PAYLOAD` as-is instead of decoding with `GET PROTO`.
***
## Common pitfalls
**`ADD OBJECT`** when **publishing** a populated nested **message** produces JSON-in-bytes. Use **`ADD MODEL`**.
Events at **101+** need an explicit tag. Omitting unused command slots on the parent also shifts later wire indexes.
`device` is a LoT keyword, so `SET "device"` may parse but interpolating that name does not. Use `device_id`. If an Action fails with `Unexpected '' where IDENTIFIER was expected`, rename the variable.
***
## Troubleshooting
`GET PROTO` returns `"null"` rather than raising, so a wrong schema looks like missing data.
| Symptom | Likely cause | Fix |
| ------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------------------- |
| Every field is `null` | Wrong model in `USING MODEL`, or the payload is not protobuf | Confirm the message name and that the publisher sends raw bytes, not JSON |
| Values decode but the gateway rejects what you publish | Field numbers drifted from the `.proto` | Add explicit `PROTO_TAG` to every field |
| Numbers are wildly wrong (huge or negative) | Fixed-width integer read as a varint, or signedness mismatch | Check `fixed32` vs `sfixed32` in the `.proto` |
To confirm what is on the wire, decode a captured payload with a protobuf raw decoder (`protoc --decode_raw`). It reports field numbers and wire types without needing the schema — that pair must match your `DEFINE MODEL`.
**Beta limitations**
* Trigger-based models auto-publish JSON only — use `PUBLISH MODEL` for binary.
* No native `google.protobuf.Timestamp`; carry timestamps as `STRING` or Unix `INT`.
* The sidecar topic for `BOTH` is a `/protobuf` suffix on the destination topic (not `/proto`).
* No protobuf **groups** (wire types 3 and 4) — a group stops decoding of that message.
* Packed repeated scalars are skipped so decoding can continue; they are not expanded into LoT arrays.
***
## Next Steps
Control when and where COLLAPSED models publish, including protobuf bytes.
Full GET PROTO syntax, sources, and types.