MODEL Protobuf

Speak the same binary schema as your gateway

Industrial gateways and PLCs often publish binary protobuf on MQTT while the broker maps plant tags (Modbus, S7, OPC UA) to plain topics. LoT models let you decode those commands, run normal tag logic in the middle, and encode telemetry back — without a separate transformation service.

Dynamic protobuf encoding and decoding in LoT is beta. Prefer PUBLISH MODEL for binary MQTT output. Match your vendor .proto with PROTO_TAG and report issues with that layout and your broker version.

Wire numbers are mailbox slots, not labels. Protobuf finds a field by its number (PROTO_TAG), not by the LoT or .proto field name. If the slot numbers drift, the message still arrives — it just lands in the wrong boxes, or looks empty.

When to use this

Use protobuf models when:

a gateway or SCADA system publishes raw protobuf bytes on MQTT;
you must emit protobuf that a vendor decoder already understands;
field numbers in the .proto are sparse (for example 101) or nested.

Use JSON models and GET JSON when payloads are JSON.

Start from the direction you need: decode inbound bytes, publish outbound bytes, or both.

Quick Start

Decode a command
Publish telemetry

Read top-level fields from a binary command payload with GET PROTO. Declare every command arm so unused submessages keep their wire numbers:

DEFINE MODEL DeviceCommand WITH FORMAT PROTOBUF COLLAPSED
    ADD INT "command_type" PROTO_TAG 1
    ADD OBJECT "header" PROTO_TAG 2
    ADD OBJECT "write_setpoint" PROTO_TAG 3
    ADD OBJECT "ack_alarm" PROTO_TAG 4
    ADD OBJECT "reserved_slot" PROTO_TAG 5
    ADD OBJECT "read_snapshot" PROTO_TAG 6

The Action reads command_type from the triggering payload:

DEFINE ACTION OnPlcCommand
ON TOPIC "plant/+/+/command" DO
    SET "cmd" WITH (GET PROTO "command_type" IN PAYLOAD AS INT USING MODEL "DeviceCommand")
    KEEP TOPIC "plant/pending_command" WITH {cmd}

Nested messages you fill on publish use ADD MODEL:

DEFINE MODEL MessageHeader WITH FORMAT PROTOBUF COLLAPSED
    ADD STRING "device_id" PROTO_TAG 1
    ADD STRING "timestamp" PROTO_TAG 2

DEFINE MODEL ProcessValues WITH FORMAT PROTOBUF COLLAPSED
    ADD DOUBLE "temperature" PROTO_TAG 1
    ADD DOUBLE "pressure" PROTO_TAG 2

DEFINE MODEL TelemetryMessage WITH FORMAT PROTOBUF COLLAPSED
    ADD INT "message_type" PROTO_TAG 1
    ADD MODEL MessageHeader "header" PROTO_TAG 2
    ADD MODEL ProcessValues "process_values" PROTO_TAG 3

The Action packs JSON tag values into that envelope and publishes bytes on the gateway topic:

DEFINE ACTION PublishTelemetry
ON TOPIC "plant/+/+/tags" DO
    SET "line" WITH TOPIC POSITION 2
    SET "device_id" WITH TOPIC POSITION 3
    SET "temp" WITH GET JSON "Temperature" IN PAYLOAD AS DOUBLE
    SET "pressure" WITH GET JSON "Pressure" IN PAYLOAD AS DOUBLE
    SET "ts" WITH TIMESTAMP "UNIX"
    PUBLISH MODEL TelemetryMessage TO "plant/" + {line} + "/" + {device_id} + "/telemetry" WITH
        message_type = 2
        header.timestamp = {ts}
        process_values.temperature = {temp}
        process_values.pressure = {pressure}

Where protobuf applies

Mechanism	Protobuf on MQTT?
`PUBLISH MODEL … TO "topic"` in an Action	Yes — binary (or JSON plus a `/protobuf` sidecar topic for `BOTH`)
Trigger-based `DEFINE MODEL … WITH TOPIC` + `AS TRIGGER`	No — still JSON on the base topic; use `PUBLISH MODEL` for binary
OT route tag topics (Modbus, S7, OPC UA mappings)	Usually JSON or plain values — not protobuf unless you add it

Define protobuf models for edge topics. Use GET PROTO to read and PUBLISH MODEL to write. Tag reads and writes stay on your existing OT routes.

Match your `.proto`

To stay aligned with a hand-written schema (including non-contiguous field numbers and nested messages):

PROTO_TAG — wire numbers must match the .proto (not field names).
Nested messages you fill on publish — ADD MODEL ChildName "field" where ChildName is a DEFINE MODEL.
Reserved wire slots — ADD OBJECT "field" PROTO_TAG n when the parent must keep empty submessage fields so later tags stay aligned.

GET PROTO and GET PROTOBUF are the same verb. PROTO and PROTOBUF are the same format keyword.

Type mapping

A field is only decoded when the wire type the sender used matches the type your model implies. Get this wrong and that field is skipped — there is no error on the wire.

Proto type	Declare in LoT	Read with `GET PROTO … AS`
`double`	`ADD DOUBLE`	`DOUBLE`
`int32` · `int64` · `uint32` · `uint64` · `sint32` · `sint64` · enum	`ADD INT`	`INT`
`bool`	`ADD BOOL`	`BOOL`
`string`	`ADD STRING`	`STRING`
`bytes`	`ADD OBJECT`	`BYTES`
nested `message` (publishing a filled body)	`ADD MODEL Child "field"`	— (fill via `PUBLISH MODEL`)
nested `message` (holding a wire slot open)	`ADD OBJECT "field"`	`BYTES`

ADD INT is a varint. Proto fixed32, sfixed32, fixed64, and sfixed64 use a different wire encoding and have no LoT keyword (ADD FIXED32 does not exist). If those types appear in the vendor schema, declaring them as ADD INT will skip or misread the field.

What happens on a mismatch

The decoder follows the protobuf unknown-field rule rather than failing the whole message:

Situation	Result
Declared type matches the incoming wire type	Field decodes
Field number not in your model	Skipped
Truncated or malformed field	Decoding of that message stops there; fields already read are kept

A skipped field is absent, and GET PROTO returns null for it. There is no error log — an unexpected null almost always means a wire-type mismatch.

How field numbers are assigned

The broker does not ship a plant schema. Numbers come from your DEFINE MODEL:

Your model definition	Wire field numbers
No `PROTO_TAG`	1, 2, 3, … in LoT source order
Every field tagged	Exactly those tags
Mixed	Tagged = explicit; untagged = next free integer

If the .proto uses field 101 but LoT omits PROTO_TAG 101, the next sequential number is used and the gateway decoder fails.

Inbound: `GET PROTO`

Prefer IN PAYLOAD on the triggering topic. Use the other sources when the bytes are already cached or wrapped as Base64:

Source	Bytes come from
`IN PAYLOAD`	The MQTT payload that fired the Action
`IN GET TOPIC "…"`	Last payload stored for that topic
`IN` a LoT variable	A Base64 string in that variable (decoded before parse)

Pass USING MODEL "Name" unless a DEFINE MODEL … WITH TOPIC already registered that schema on the topic. AS INT / DOUBLE / STRING / BOOL / BYTES selects the LoT type after decode. Read from the last payload stored on another topic:

DEFINE ACTION RetryCachedCommand
ON TOPIC "plant/+/+/command/retry" DO
    SET "cmd" WITH (GET PROTO "command_type" IN GET TOPIC "plant/line1/plc01/command" AS INT USING MODEL "DeviceCommand")

Or decode Base64 that arrived as JSON:

DEFINE ACTION DecodeBase64Command
ON TOPIC "plant/+/encoded" DO
    SET "b64" WITH GET JSON "payload_b64" IN PAYLOAD AS STRING
    SET "cmd" WITH (GET PROTO "command_type" IN {b64} AS INT USING MODEL "DeviceCommand")

Notes that matter in practice:

Each GET PROTO decodes the payload from scratch. Reading two fields means two independent passes.

Do not use IF PAYLOAD IS PROTO to detect binary MQTT. Call GET PROTO … IN PAYLOAD instead.

Parent command with reserved field numbers

Vendor .proto files often declare several command arms but send only one. List every submessage slot on the parent so unused fields do not shift wire numbers. Use ADD OBJECT for unused slots. When publishing a filled nested message, use ADD MODEL on the parent instead.

DEFINE MODEL DeviceCommand WITH FORMAT PROTOBUF COLLAPSED
    ADD INT "command_type" PROTO_TAG 1
    ADD OBJECT "header" PROTO_TAG 2
    ADD OBJECT "write_setpoint" PROTO_TAG 3
    ADD OBJECT "ack_alarm" PROTO_TAG 4
    ADD OBJECT "reserved_slot" PROTO_TAG 5
    ADD OBJECT "read_snapshot" PROTO_TAG 6

Middle: tags stay JSON

Between command and telemetry, most OT setups already have tag values on topics — from a DEFINE ROUTE mapping, a collapsed JSON model, or single-value publishes. That step stays JSON or plain text unless your plant standard requires protobuf on those topics too.

DEFINE ACTION ReadPlcTags
ON TOPIC "plant/line1/plc01/tags" DO
    SET "temp" WITH GET JSON "Temperature" IN PAYLOAD AS DOUBLE
    SET "pressure" WITH GET JSON "Pressure" IN PAYLOAD AS DOUBLE
    SET "speed" WITH GET JSON "MotorSpeed" IN PAYLOAD AS DOUBLE
    PUBLISH TOPIC "plant/line1/plc01/snapshot/temperature" WITH {temp}

Outbound: `PUBLISH MODEL`

Topic vs format

Model format	Payload on `TO "…"` topic
`WITH FORMAT JSON` (default)	JSON string
`WITH FORMAT PROTOBUF` / `PROTO`	Raw protobuf bytes
`WITH FORMAT BOTH`	JSON on the main topic + binary on a `/protobuf` sidecar topic

The next PUBLISH MODEL to the same topic merges: binary for PROTO only; JSON from the main topic for JSON / BOTH. KEEP + protobuf updates the byte cache without per-field JSON fan-out. Trigger-based models (WITH TOPIC + AS TRIGGER) still auto-publish JSON on the base topic. Use PUBLISH MODEL when the edge must see binary.

Response envelope with sparse tags

Alarms and events often use high field numbers (101, 102). Tag them explicitly; use ADD MODEL for nested bodies you fill on publish. MessageHeader and ProcessValues match Quick Start — the new pieces are AlarmEvent and field 101:

DEFINE MODEL MessageHeader WITH FORMAT PROTOBUF COLLAPSED
    ADD STRING "device_id" PROTO_TAG 1
    ADD STRING "timestamp" PROTO_TAG 2

DEFINE MODEL ProcessValues WITH FORMAT PROTOBUF COLLAPSED
    ADD DOUBLE "temperature" PROTO_TAG 1
    ADD DOUBLE "pressure" PROTO_TAG 2
    ADD INT "motor_speed" PROTO_TAG 3

DEFINE MODEL AlarmEvent WITH FORMAT PROTOBUF COLLAPSED
    ADD INT "alarm_code" PROTO_TAG 1
    ADD STRING "alarm_text" PROTO_TAG 2

DEFINE MODEL TelemetryMessage WITH FORMAT PROTOBUF COLLAPSED
    ADD INT "message_type" PROTO_TAG 1
    ADD MODEL MessageHeader "header" PROTO_TAG 2
    ADD MODEL ProcessValues "process_values" PROTO_TAG 3
    ADD MODEL AlarmEvent "alarm_event" PROTO_TAG 101

LoT names are for PUBLISH MODEL / GET PROTO. Wire numbers come only from PROTO_TAG. Set message_type to the enum integer your .proto expects.

Raw `bytes` fields

Some messages carry opaque bytes (recipe blob, certificate, file chunk). Assign them with BYTES FROM BASE64:

DEFINE MODEL RecipeBlock WITH FORMAT PROTOBUF COLLAPSED
    ADD INT "recipe_id" PROTO_TAG 1
    ADD OBJECT "payload" PROTO_TAG 2

Then the Action fills payload from a Base64 JSON field:

DEFINE ACTION StoreRecipe
ON TOPIC "plant/+/recipe/upload" DO
    PUBLISH MODEL RecipeBlock TO "plant/archive/recipe" WITH
        recipe_id = 42
        payload = BYTES FROM BASE64 (GET JSON "data_b64" IN PAYLOAD AS STRING)

To forward an entire protobuf payload unchanged (no field access), republish PAYLOAD as-is instead of decoding with GET PROTO.

Common pitfalls

ADD OBJECT vs ADD MODEL

ADD OBJECT when publishing a populated nested message produces JSON-in-bytes. Use ADD MODEL.

Missing PROTO_TAG on sparse fields

Events at 101+ need an explicit tag. Omitting unused command slots on the parent also shifts later wire indexes.

Reserved words as variable names

device is a LoT keyword, so SET "device" may parse but interpolating that name does not. Use device_id. If an Action fails with Unexpected '<name>' where IDENTIFIER was expected, rename the variable.

Troubleshooting

GET PROTO returns "null" rather than raising, so a wrong schema looks like missing data.

Symptom	Likely cause	Fix
Every field is `null`	Wrong model in `USING MODEL`, or the payload is not protobuf	Confirm the message name and that the publisher sends raw bytes, not JSON
Values decode but the gateway rejects what you publish	Field numbers drifted from the `.proto`	Add explicit `PROTO_TAG` to every field
Numbers are wildly wrong (huge or negative)	Fixed-width integer read as a varint, or signedness mismatch	Check `fixed32` vs `sfixed32` in the `.proto`

To confirm what is on the wire, decode a captured payload with a protobuf raw decoder (protoc --decode_raw). It reports field numbers and wire types without needing the schema — that pair must match your DEFINE MODEL.

Beta limitations

Trigger-based models auto-publish JSON only — use PUBLISH MODEL for binary.
No native google.protobuf.Timestamp; carry timestamps as STRING or Unix INT.
The sidecar topic for BOTH is a /protobuf suffix on the destination topic (not /proto).
No protobuf groups (wire types 3 and 4) — a group stops decoding of that message.
Packed repeated scalars are skipped so decoding can continue; they are not expanded into LoT arrays.

Quick Start

Developing and Managing Coreflux

Coreflux MQTT Broker

LoT Language

Product Updates

Speak the same binary schema as your gateway

When to use this

Quick Start

Where protobuf applies

Match your `.proto`

Type mapping

What happens on a mismatch

How field numbers are assigned

Inbound: `GET PROTO`

Parent command with reserved field numbers

Middle: tags stay JSON

Outbound: `PUBLISH MODEL`

Topic vs format

Response envelope with sparse tags

Raw `bytes` fields

Common pitfalls

Troubleshooting

Next Steps

Publishing Models

GET PROTO in Actions

​Speak the same binary schema as your gateway

​When to use this

​Quick Start

​Where protobuf applies

​Match your .proto

​Type mapping

​What happens on a mismatch

​How field numbers are assigned

​Inbound: GET PROTO

​Parent command with reserved field numbers

​Middle: tags stay JSON

​Outbound: PUBLISH MODEL

​Topic vs format

​Response envelope with sparse tags

​Raw bytes fields

​Common pitfalls

​Troubleshooting

​Next Steps

Publishing Models

GET PROTO in Actions

Speak the same binary schema as your gateway

When to use this

Quick Start

Where protobuf applies

Match your `.proto`

Type mapping

What happens on a mismatch

How field numbers are assigned

Inbound: `GET PROTO`

Parent command with reserved field numbers

Middle: tags stay JSON

Outbound: `PUBLISH MODEL`

Topic vs format

Response envelope with sparse tags

Raw `bytes` fields

Common pitfalls

Troubleshooting

Next Steps