Skip to content

Data Format Entities (CSV, JSON)

These entities are used implicitly when processing structured data within LOT, primarily when accessing specific fields from a payload assumed to be in CSV or JSON format.

CSV

Feature Since Version Notes
Implicit CSV access ({value.csv.fieldName}) 1.x (?) Used in SQL connector assets

When integrating with systems that expect CSV data (like certain SQL connector tags), LOT allows accessing fields using the {value.csv.fieldName} syntax within the query string.

Usage: This syntax is typically used within the Query parameter of database connector tags.

Example (Illustrative - Used in Connector Config, not direct LOT):

Consider a payload sent to sql/insert topic: name,age\nJohn,30

In the corresponding SQL connector tag configuration: 

{
  "Query": "INSERT INTO users(name, age) VALUES ('{value.csv.name}', {value.csv.age});"
}
Note: The LOT engine itself doesn't directly parse CSV, but this syntax is recognized by specific connectors.

JSON

Feature Since Version Notes
Implicit JSON access ({value.json.fieldName}) 1.x (?) Used in SQL connector assets
Automatic JSON breakout in Models 1.x (?) If model has only rawData trigger

Similar to CSV, LOT facilitates access to JSON fields using {value.json.fieldName...} syntax, primarily within connector configurations.

Additionally, DEFINE MODEL has a feature for automatic JSON breakout.

Usage:

  1. Connector Configuration: Accessing fields in queries.
  2. Models: Automatic property creation from JSON fields.

Examples:

1. Connector Configuration (Illustrative):

Payload to sql/insert: {"user": {"name": "Jane", "id": 123}, "status": "active"}

In the SQL connector tag configuration: 

{
  "Query": "UPDATE user_status SET status = '{value.json.status}' WHERE id = {value.json.user.id};"
}

2. Automatic Model Breakout:

If a model is defined with only one property, which is a topic trigger named rawData, the broker automatically attempts to parse the incoming payload as JSON and creates outputs for each top-level field.

DEFINE MODEL SensorJsonBreakout WITH TOPIC "sensors/parsed"
    // Input topic receives JSON like: {"temp": 25.5, "humidity": 60}
    ADD "rawData" WITH TOPIC "sensors/json_input" AS TRIGGER

// This will automatically publish:
// -> sensors/parsed/temp WITH 25.5
// -> sensors/parsed/humidity WITH 60

PROTOBUF

(Coming Soon) - Support for Protocol Buffers is planned.

Related Documentation