LOT Language - Model Management Documentation
1. Overview
Models in Coreflux use the LOT language syntax to define how data is processed, transformed, and published. Models take input data (triggered by specific topics), process it through expressions, constants, or transformations, and output the results to new MQTT topics.
For practical examples of LOT Models, check out the Models examples in the LOT Samples Repository.
Models serve as data pipelines or virtual sensors that can enrich data from existing MQTT topics, perform computations, and publish processed results. By using LOT language models, you can easily configure and maintain logic at the broker level, enabling dynamic calculation and formatting of data without the need for external services.
1.1 Two Types of Models
Coreflux supports two distinct types of models:
- Data Transformation Models: Traditional models that process and transform data in real-time
- LOT Models: Structured data schemas used with
PUBLISH MODEL
commands in Actions
2. LOT Models (Structured Data Schemas)
LOT Models provide a powerful way to define structured data schemas within Coreflux. They are used in combination with the PUBLISH MODEL
command inside ACTION
s to create and send well-formed, predictable JSON payloads without the need for manual and error-prone string construction.
This approach promotes reusability, improves readability, and ensures data consistency across your system.
2.1 DEFINE MODEL
for Structured Data
The DEFINE MODEL
statement is used to declare a reusable data structure, or schema.
Syntax
DEFINE MODEL <ModelName> [COLLAPSED]
ADD <TYPE> "<FieldName>"
ADD COLLECTION "<CollectionName>" OF <AnotherModelName>
<ModelName>
: A unique name for your data model.[COLLAPSED]
: An optional keyword that creates a flat key-value JSON object. If omitted, the model will produce a nested JSON object with the model's name as the root key.ADD <TYPE> "<FieldName>"
: Defines a field within the model.
Supported Field Types
Type | Description |
---|---|
STRING |
A text string. |
INT |
An integer number. |
DOUBLE |
A floating-point number. |
BOOL |
A boolean value (true or false ). |
OBJECT |
A raw JSON object or array. The value provided in PUBLISH MODEL will be parsed and embedded directly, not as a string. This is extremely useful for complex, nested structures or when part of the payload is dynamic. |
COLLECTION |
Represents a JSON array containing objects of another defined model type. |
2.2 PUBLISH MODEL
Command
The PUBLISH MODEL
command is used within an ACTION
to instantiate a defined model with data and publish the resulting JSON payload to an MQTT topic.
Syntax
<ModelName>
: The name of theDEFINE MODEL
to use.- **
"<topic>"**
: The MQTT topic to publish the final JSON payload to. FieldName = <value>
: Assigns a value to a field defined in the model. Values can be literals, variables fromSET
commands, or results from functions likeGET JSON
.
2.3 LOT Model Examples
Example 1: Simple, Collapsed Model
DEFINE MODEL KafkaMessage COLLAPSED
ADD STRING "componentId"
ADD STRING "topic"
ADD OBJECT "payload"
DEFINE ACTION ProcessComponentsPayload
ON TOPIC "Raw/Components/+/+" DO
SET "systemId" WITH (GET JSON "info" IN PAYLOAD AS STRING)
SET "topicv" WITH ("Components/" + TOPIC POSITION 3 + "/" + TOPIC POSITION 4 + "/" + {systemId})
PUBLISH MODEL KafkaMessage TO ({topicv}+"/Unit1") WITH
componentId = (TOPIC POSITION 3 + "_" + TOPIC POSITION 4 + "_" + {systemId} + "_" + "Unit1")
topic = ({topicv}+"/Unit1")
payload = (GET JSON "Unit1" IN PAYLOAD AS STRING)
Example 2: Complex, Nested Model
DEFINE MODEL SmartLight COLLAPSED
ADD STRING "deviceId"
ADD BOOL "online"
ADD OBJECT "state"
ADD STRING "power"
ADD INT "brightness" // 0-100
ADD INT "temperature" // Color temperature in Kelvin
ADD TIMESTAMP "last_updated" WITH "UNIX"
DEFINE ACTION UpdateLightState
ON TOPIC "lights/office/status" DO
PUBLISH MODEL SmartLight TO "lights/office/state" WITH
deviceId = "light-office-1",
online = true,
state.power = (GET JSON "power" IN PAYLOAD AS STRING),
state.brightness = (GET JSON "brightness" IN PAYLOAD AS INT),
state.temperature = (GET JSON "temp" IN PAYLOAD AS INT)
2.4 STORE Functionality
LOT Models can be integrated with database storage using the STORE
clause and ROUTE
s.
Database Routes
DEFINE ROUTE mongo_route WITH TYPE MONGODB
ADD MONGODB_CONFIG
WITH CONNECTION_STRING "mongodb+srv://<username>:<password>@<cluster-uri>/<database>?tls=true&authSource=admin&replicaSet=<replica-set>"
WITH DATABASE "admin"
Model with Storage
DEFINE MODEL MachineData WITH TOPIC "Simulator/Machine/+/Data"
ADD "energy" WITH TOPIC "raw_data/+" AS TRIGGER
ADD "device_name" WITH REPLACE "+" WITH TOPIC POSITION 2 IN "+"
ADD "energy_wh" WITH (energy * 1000)
ADD "production_status" WITH (IF energy > 5 THEN "active" ELSE "inactive")
ADD "production_count" WITH (IF production_status EQUALS "active" THEN (production_count + 1) ELSE 0)
ADD "stoppage" WITH (IF production_status EQUALS "inactive" THEN 1 ELSE 0)
ADD "maintenance_alert" WITH (IF energy > 50 THEN TRUE ELSE FALSE)
ADD "timestamp" WITH TIMESTAMP "UTC"
STORE IN "mongo_route"
WITH TABLE "MachineProductionData"
3. Data Transformation Models (Traditional)
3.1 Model Processing Overview
Models operate as functions or pipelines triggered whenever their input topics receive new messages. When a trigger topic updates:
- Input Acquisition: The model reads the defined input properties. Some properties may be constants, others may reference topics.
- Computation and Expressions: Each property's definition can involve direct values, references to other properties, or arithmetic/conditional expressions.
- Output Publishing: After processing, the model publishes each property's current value to the model's output namespace, essentially creating a set of derived topics.
Models can thus aggregate or transform data from multiple inputs and produce consolidated or computed outputs.
3.2 Properties in Models
Each model consists of one or more properties defined with the ADD
statement. A property can be:
- Directly from a Topic (with
AS TRIGGER
or without): IfAS TRIGGER
is used, changes in that topic will cause the model to recompute and update outputs. - A Fixed Constant: A property can be assigned a static number or string.
- An Expression: A property can be computed from other properties using arithmetic operations, conditional expressions (IF ... THEN ... ELSE), or timestamps.
3.3 Topics and Unified Namespace
Models can use wildcards (+
) in their topics to dynamically handle multiple sources in a Unified Namespace (UNS). For example, if a model's output topic is Coreflux/+/+/+/+/energy
, and it references an input topic shellies/+/+/+/+/device/energy
as a trigger, then any shellies/x/y/z/w/device/energy
incoming value will map to Coreflux/x/y/z/w/energy/...
outputs.
3.4 Constants and Expressions
Properties can hold constant values or can be computed using arithmetic operations, conditional logic, or timestamps. For example:
- Constant:
ADD "energy_price" WITH 3
- Expression:
ADD "cost" WITH (total_energy * energy_price)
These capabilities allow building complex logic directly in the broker without external code.
4. Default Behavior
By default, models do not apply any transformations unless defined. They do not alter the original incoming data but produce new topics for processed results. If a model references a topic that never updates, the model will not publish new outputs.
5. Model Syntax
Models in the LOT language follow this general syntax:
DEFINE MODEL <model_name> WITH TOPIC "<output_base_topic>"
ADD "<property_name>" WITH TOPIC "<input_topic>" [AS TRIGGER]
ADD "<property_name>" WITH <constant_value>
ADD "<property_name>" WITH (expression)
5.1 Components Explained
- DEFINE MODEL
<model_name>
: Defines a new model with a unique<model_name>
. - WITH TOPIC
<output_base_topic>
: Specifies the base output topic for the model's computed properties. - ADD
<property_name>
...: Defines a property of the model.- WITH TOPIC "
" [AS TRIGGER] : Indicates this property's value comes from another MQTT topic. IfAS TRIGGER
is present, updates on this topic cause the model to recompute. - WITH
: Assigns a fixed constant value to the property. - WITH (expression): Defines a property's value as an expression based on other properties, arithmetic, conditions, or timestamps.
- WITH TOPIC "
5.2 Timestamps and Conditions
-
IF Conditions:
ADD "message" WITH IF alarm == "true" THEN "all ok!" ELSE "message has an alarm 33"
-
Timestamp:
ADD "timestamp" WITH TIMESTAMP "UTC"
This can add a timestamp to your computed data in various formats (
"UTC"
,"ISO"
,"UNIX-MS"
, or"UNIX"
).
6. Model Management Guidelines
Like rules, models can be added or removed dynamically using commands published to the $SYS/Coreflux/Command
topic.
6.1 Add a Model: addModel <DEFINE MODEL code>
This command allows you to add a new model to the system. Example:
-addModel DEFINE MODEL LampEnergyCost WITH TOPIC "Coreflux/+/+/+/+/energy"
ADD "total_energy" WITH TOPIC "shellies/+/+/+/+/device/energy" AS TRIGGER
ADD "energy_price" WITH 3
ADD "cost" WITH (total_energy * energy_price)
In this example:
- The model is named
LampEnergyCost
. - The output base topic is
Coreflux/+/+/+/+/energy
. total_energy
comes fromshellies/+/+/+/+/device/energy
and triggers computations.energy_price
is a constant3
.cost
is computed as(total_energy * energy_price)
.
Whenever a shellies/.../device/energy
message arrives, the model computes outputs and publishes to Coreflux/.../energy/total_energy
, Coreflux/.../energy/energy_price
, and Coreflux/.../energy/cost
.
Another example:
-addModel DEFINE MODEL LampEnergyCostFixed WITH TOPIC "shellies/Coreflux/Students/Lamp/Lamp1/costs"
ADD "total_energy" WITH TOPIC "shellies/Coreflux/Students/Lamp/Lamp1/device/energy" AS TRIGGER
ADD "energy_price" WITH 2
ADD "cost" WITH (total_energy * energy_price)
Here, the model is fixed to a specific device path without wildcards, outputting computed values to shellies/Coreflux/Students/Lamp/Lamp1/costs/...
.
6.2 Remove a Model: removeModel <model_name>
This command removes an existing model by specifying its name. Example:
In this example, the model LampEnergyCost
is removed from the system. This stops the model from processing future messages and publishing outputs.
7. Additional Examples
7.1 UNS Style Models
DEFINE MODEL GenericEnergyCost WITH TOPIC "Coreflux/+/+/+/+/energy"
ADD "total_energy" WITH TOPIC "shellies/+/+/+/+/device/energy" AS TRIGGER
ADD "energy_price" WITH 3
ADD "cost" WITH (total_energy * energy_price)
For any device matching the shellies/.../device/energy
pattern, this model will process data and publish to Coreflux/.../energy/...
.
7.2 Direct, Singleton Model
If a JSON payload like:
arrives at company/factory/department/machine/stopOrder
, the model will break it down into:
company/factory/department/machine/stopOrder/reason
-> 42company/factory/department/machine/stopOrder/stop
-> truecompany/factory/department/machine/stopOrder/time
-> "2024-12-11T15:12:54Z"
(Note: If you want to define additional properties or triggers, you can ADD
them similarly.)
7.3 Conditional Message Models
DEFINE MODEL AlertMessage2 WITH TOPIC "Company2/+/+/+/AlarmMessage"
ADD "alarm" WITH TOPIC "shellies2/+/+/+/device/alarm" AS TRIGGER
ADD "message" WITH IF alarm == "true" THEN "all ok!" ELSE "message has an alarm 33"
ADD "timestamp" WITH TIMESTAMP "UTC"
When alarm
triggers, the model publishes a message to Company2/.../AlarmMessage/...
including a processed message
and a timestamp.
7.4 Counters and Running Totals
DEFINE MODEL ProductionCounterModel WITH TOPIC "machine/productionCounter"
ADD "Value" WITH TOPIC "machine/signal/countPart" AS TRIGGER
ADD "Counter" WITH (Counter + Value)
Each time machine/signal/countPart
receives a new number, the model adds it to the Counter
property, creating a running total published under machine/productionCounter/Counter
.
7.5 Advanced Data Transformation Models with Storage
Sensor Data Aggregation Model
DEFINE MODEL SensorAggregation WITH TOPIC "Sensors/+/Aggregated"
ADD "temperature" WITH TOPIC "sensor/+/temperature" AS TRIGGER
ADD "humidity" WITH TOPIC "sensor/+/humidity" AS TRIGGER
ADD "pressure" WITH TOPIC "sensor/+/pressure" AS TRIGGER
ADD "sensor_id" WITH REPLACE "+" WITH TOPIC POSITION 2 IN "+"
ADD "temp_fahrenheit" WITH ((temperature * 9/5) + 32)
ADD "comfort_index" WITH ((temp_fahrenheit + humidity) / 2)
ADD "alert_level" WITH (IF comfort_index > 80 THEN "high" ELSE IF comfort_index > 60 THEN "medium" ELSE "low")
ADD "timestamp" WITH TIMESTAMP "UTC"
STORE IN "postgres_route"
WITH TABLE "SensorReadings"
IoT Device Status Model
DEFINE MODEL DeviceStatus WITH TOPIC "IoT/+/Status"
ADD "voltage" WITH TOPIC "device/+/voltage" AS TRIGGER
ADD "current" WITH TOPIC "device/+/current" AS TRIGGER
ADD "device_id" WITH REPLACE "+" WITH TOPIC POSITION 2 IN "+"
ADD "power_watts" WITH (voltage * current)
ADD "status" WITH (IF power_watts > 0 THEN "online" ELSE "offline")
ADD "health_score" WITH (IF voltage > 10 AND current > 0.1 THEN 100 ELSE 50)
ADD "last_seen" WITH TIMESTAMP "UTC"
STORE IN "mysql_route"
WITH TABLE "DeviceHealth"
8. Best Practices
8.1 LOT Models vs Data Transformation Models
- Use LOT Models when you need structured, predictable JSON output for external APIs or systems
- Use Data Transformation Models when you need real-time data processing and transformation
- Combine both for complex scenarios requiring both structured output and real-time processing
8.2 Database Integration
- Choose the appropriate database type for your use case:
- MongoDB: Document-based storage for flexible data structures
- PostgreSQL: Relational database for structured sensor data
- MySQL: Traditional relational database for device management
- InfluxDB: Time-series database for environmental metrics
- TimescaleDB: Time-series extension of PostgreSQL for energy data
8.3 Performance Considerations
- Use
COLLAPSED
models for flat JSON structures when possible - Leverage wildcards (
+
) for scalable multi-device processing - Use
OBJECT
type for complex, dynamic JSON structures - Implement proper error handling in your models