Azure Blob Storage and Data Lake Gen2

Technical Preview

Overview

A client-mode connector for a curated IIoT object-storage slice of the Azure Blob service REST API under https://{account}.blob.core.windows.net:

Catalog connector event	Entity / path	Notes
(smoke)	`GET /{container}?restype=container&comp=list`	List Blobs
PUT blob	`PUT /{container}/{blob}`	`x-ms-blob-type: BlockBlob`
(optional) Create Append Blob	`PUT /{container}/{blob}`	`x-ms-blob-type: AppendBlob`, empty body
PUT append block (continuous log)	`PUT /{container}/{blob}?comp=appendblock`	Existing AppendBlob only
GET blob	`GET /{container}/{blob}`	Download body
(optional) HEAD properties	`HEAD /{container}/{blob}`	Metadata without body
PUT / GET via SAS	`PUT/GET {sas_path}`	No Authorization; switch BASE to URL origin
Event Grid blob webhook	—	Docs-only; future `ENABLE_SERVER`

BASE_ADDRESS defaults to the placeholder https://ACCOUNT.blob.core.windows.net (no trailing slash). Replace ACCOUNT with your storage account. Topics use storage/azure-blob-storage-and-data-lake-gen2/<object>/<verb>.

Data Lake Gen2: hierarchical namespace filesystem REST uses https://{account}.dfs.core.windows.net. Keep primary events on the Blob endpoint; swap BASE only when calling DFS path APIs.

SAS PUT/GET: set BASE_ADDRESS to the SAS URL origin only, put path+query in sas_path, and omit Shared Key / Bearer — the query string carries sv / sig. Ideal for edge devices that must not hold account keys.

Event Grid blob created/deleted subscriptions are not inbound HTTP on this client connector. Receiving Event Grid validation and event POSTs needs a separate ENABLE_SERVER REST_API template (not packaged).

Prefer Hub secrets for account keys or Entra client credentials used by an upstream signer. This template has not been verified against a live Azure storage account. Status remains blocked until an authenticated live smoke (or working SAS URL smoke) succeeds.

Basic Syntax

# Coreflux REST_API template — Azure Blob curated slice
# Auth: SharedKey or Bearer via {value.json.authorization},
#       {value.json.x_ms_date}, {value.json.x_ms_version}
#       — or PutViaSas / GetViaSas (no Authorization)
# BASE_ADDRESS: https://ACCOUNT.blob.core.windows.net

DEFINE ROUTE AzureBlobStorageAndDataLakeGen2RestApi WITH TYPE REST_API
    ADD REST_API_CONFIG
        WITH BASE_ADDRESS "https://ACCOUNT.blob.core.windows.net"
        WITH ENABLE_CLIENT "true"
        WITH USE_SSL "true"
        WITH REQUEST_TIMEOUT '60'
        WITH DEFAULT_CONTENT_TYPE "application/octet-stream"
    ADD EVENT Listblobs
        WITH SOURCE_TOPIC "storage/azure-blob-storage-and-data-lake-gen2/blobs/list"
        WITH DESTINATION_TOPIC "storage/azure-blob-storage-and-data-lake-gen2/blobs/list/response"
        WITH METHOD "GET"
        WITH ENDPOINT "/{value.json.container}?restype=container&comp=list"
        WITH HEADERS
            ADD "Accept" "application/xml"
            ADD "Authorization" "{value.json.authorization}"
            ADD "x-ms-date" "{value.json.x_ms_date}"
            ADD "x-ms-version" "{value.json.x_ms_version}"
    ADD EVENT Putblob
        WITH SOURCE_TOPIC "storage/azure-blob-storage-and-data-lake-gen2/blob/put"
        WITH DESTINATION_TOPIC "storage/azure-blob-storage-and-data-lake-gen2/blob/put/response"
        WITH METHOD "PUT"
        WITH ENDPOINT "/{value.json.container}/{value.json.blob}"
        WITH HEADERS
            ADD "Accept" "application/xml"
            ADD "Content-Type" "{value.json.content_type}"
            ADD "x-ms-blob-type" "BlockBlob"
            ADD "Authorization" "{value.json.authorization}"
            ADD "x-ms-date" "{value.json.x_ms_date}"
            ADD "x-ms-version" "{value.json.x_ms_version}"
        WITH BODY "{payload}"
    ADD EVENT Createappendblob
        WITH SOURCE_TOPIC "storage/azure-blob-storage-and-data-lake-gen2/blob/create-append"
        WITH DESTINATION_TOPIC "storage/azure-blob-storage-and-data-lake-gen2/blob/create-append/response"
        WITH METHOD "PUT"
        WITH ENDPOINT "/{value.json.container}/{value.json.blob}"
        WITH HEADERS
            ADD "Accept" "application/xml"
            ADD "Content-Type" "application/octet-stream"
            ADD "Content-Length" "0"
            ADD "x-ms-blob-type" "AppendBlob"
            ADD "Authorization" "{value.json.authorization}"
            ADD "x-ms-date" "{value.json.x_ms_date}"
            ADD "x-ms-version" "{value.json.x_ms_version}"
    ADD EVENT Appendblock
        WITH SOURCE_TOPIC "storage/azure-blob-storage-and-data-lake-gen2/blob/append-block"
        WITH DESTINATION_TOPIC "storage/azure-blob-storage-and-data-lake-gen2/blob/append-block/response"
        WITH METHOD "PUT"
        WITH ENDPOINT "/{value.json.container}/{value.json.blob}?comp=appendblock"
        WITH HEADERS
            ADD "Accept" "application/xml"
            ADD "Content-Type" "{value.json.content_type}"
            ADD "Authorization" "{value.json.authorization}"
            ADD "x-ms-date" "{value.json.x_ms_date}"
            ADD "x-ms-version" "{value.json.x_ms_version}"
        WITH BODY "{payload}"
    ADD EVENT Getblob
        WITH SOURCE_TOPIC "storage/azure-blob-storage-and-data-lake-gen2/blob/get"
        WITH DESTINATION_TOPIC "storage/azure-blob-storage-and-data-lake-gen2/blob/get/response"
        WITH METHOD "GET"
        WITH ENDPOINT "/{value.json.container}/{value.json.blob}"
        WITH HEADERS
            ADD "Accept" "*/*"
            ADD "Authorization" "{value.json.authorization}"
            ADD "x-ms-date" "{value.json.x_ms_date}"
            ADD "x-ms-version" "{value.json.x_ms_version}"
    ADD EVENT Getblobproperties
        WITH SOURCE_TOPIC "storage/azure-blob-storage-and-data-lake-gen2/blob/head"
        WITH DESTINATION_TOPIC "storage/azure-blob-storage-and-data-lake-gen2/blob/head/response"
        WITH METHOD "HEAD"
        WITH ENDPOINT "/{value.json.container}/{value.json.blob}"
        WITH HEADERS
            ADD "Authorization" "{value.json.authorization}"
            ADD "x-ms-date" "{value.json.x_ms_date}"
            ADD "x-ms-version" "{value.json.x_ms_version}"
    ADD EVENT Putviasas
        WITH SOURCE_TOPIC "storage/azure-blob-storage-and-data-lake-gen2/blob/put-via-sas"
        WITH DESTINATION_TOPIC "storage/azure-blob-storage-and-data-lake-gen2/blob/put-via-sas/response"
        WITH METHOD "PUT"
        WITH ENDPOINT "{value.json.sas_path}"
        WITH HEADERS
            ADD "Content-Type" "{value.json.content_type}"
            ADD "x-ms-blob-type" "BlockBlob"
            ADD "x-ms-version" "{value.json.x_ms_version}"
        WITH BODY "{payload}"
    ADD EVENT Getviasas
        WITH SOURCE_TOPIC "storage/azure-blob-storage-and-data-lake-gen2/blob/get-via-sas"
        WITH DESTINATION_TOPIC "storage/azure-blob-storage-and-data-lake-gen2/blob/get-via-sas/response"
        WITH METHOD "GET"
        WITH ENDPOINT "{value.json.sas_path}"
        WITH HEADERS
            ADD "Accept" "*/*"
            ADD "x-ms-version" "{value.json.x_ms_version}"

Configuration

Setting	Value
Connector name	`AzureBlobStorageAndDataLakeGen2RestApi`
Type	`REST_API` (client)
BASE_ADDRESS	`https://ACCOUNT.blob.core.windows.net` (replace ACCOUNT)
Topic prefix	`storage/azure-blob-storage-and-data-lake-gen2/...`
Default Content-Type	`application/octet-stream`
API version header	`x-ms-version: 2023-11-03` (payload-driven)
Hub status	Technical preview (`WITH STATUS "beta"`)

Authentication

Shared Key / Entra Bearer

Blob does not use a simple static Bearer for Shared Key. Sign each request or obtain an Entra token and pass:

MQTT JSON field	HTTP header
`authorization`	`Authorization` (`SharedKey account:…` or `Bearer …`)
`x_ms_date`	`x-ms-date` (RFC 1123 UTC)
`x_ms_version`	`x-ms-version` (e.g. `2023-11-03`)

Use Hub secrets for account keys / Entra client credentials in an upstream signing helper. Never commit real keys.

SAS URLs (recommended for edge)

Generate a time-limited service or user-delegation SAS with Azure CLI/SDK/portal. For PutViaSas / GetViaSas:

Point BASE_ADDRESS at the URL origin.
Put path + query into sas_path.
Omit Authorization — do not re-sign on the broker.

MQTT topics

Event	Request topic	Response topic
ListBlobs	`storage/azure-blob-storage-and-data-lake-gen2/blobs/list`	`…/response`
PutBlob	`storage/azure-blob-storage-and-data-lake-gen2/blob/put`	`…/response`
CreateAppendBlob	`storage/azure-blob-storage-and-data-lake-gen2/blob/create-append`	`…/response`
AppendBlock	`storage/azure-blob-storage-and-data-lake-gen2/blob/append-block`	`…/response`
GetBlob	`storage/azure-blob-storage-and-data-lake-gen2/blob/get`	`…/response`
GetBlobProperties	`storage/azure-blob-storage-and-data-lake-gen2/blob/head`	`…/response`
PutViaSas	`storage/azure-blob-storage-and-data-lake-gen2/blob/put-via-sas`	`…/response`
GetViaSas	`storage/azure-blob-storage-and-data-lake-gen2/blob/get-via-sas`	`…/response`

Example payloads

ListBlobs smoke (after signing):

{
  "container": "iiot-rollups",
  "authorization": "SharedKey myaccount:…",
  "x_ms_date": "Sun, 04 Oct 2026 04:00:00 GMT",
  "x_ms_version": "2023-11-03"
}

PutBlob CSV rollup:

{
  "container": "iiot-rollups",
  "blob": "line-a/2026-10-04/rollup.csv",
  "content_type": "text/csv",
  "authorization": "SharedKey myaccount:…",
  "x_ms_date": "Sun, 04 Oct 2026 04:01:00 GMT",
  "x_ms_version": "2023-11-03",
  "payload": "ts,device,temp\n2026-10-04T04:01:00Z,sensor-01,72.5\n"
}

PutViaSas:

{
  "sas_path": "/iiot-rollups/line-a/snapshot.json?sv=2023-11-03&ss=b&srt=o&sp=cw&se=2026-10-04T12:00:00Z&sig=…",
  "content_type": "application/json",
  "x_ms_version": "2023-11-03",
  "payload": "{\"device\":\"sensor-01\",\"temp\":72.5}"
}

References

Package status

Blocked on the factory until a storage account + SAS (or Shared Key / Entra token) + container enables a live authenticated smoke recorded in TEST.md.

Quick Start

Coreflux MQTT Broker

LoT Language

Coreflux HUB

VS Code - LoT Notebooks

Coreflux Portal

Using AI with Coreflux

Product Updates

Overview

Basic Syntax

Configuration

Authentication

Shared Key / Entra Bearer

SAS URLs (recommended for edge)

MQTT topics

Example payloads

References

Package status

​Overview

​Basic Syntax

​Configuration

​Authentication

​Shared Key / Entra Bearer

​SAS URLs (recommended for edge)

​MQTT topics

​Example payloads

​References

​Package status

Overview

Basic Syntax

Configuration

Authentication

Shared Key / Entra Bearer

SAS URLs (recommended for edge)

MQTT topics

Example payloads

References

Package status