> ## Documentation Index
> Fetch the complete documentation index at: https://docs.coreflux.org/llms.txt
> Use this file to discover all available pages before exploring further.

# Azure Blob Storage and Data Lake Gen2

> Client-mode REST API connector for a curated Azure Blob slice: Put Blob, Append Block, Get Blob, List Blobs, and PUT/GET via SAS URL

Technical Preview

## Overview

A client-mode connector for a **curated IIoT object-storage** slice of the [Azure Blob service REST API](https://learn.microsoft.com/en-us/rest/api/storageservices/blob-service-rest-api) under `https://{account}.blob.core.windows.net`:

| Catalog connector event | Entity / path | Notes |
| - | - | - |
| (smoke) | `GET /{container}?restype=container&comp=list` | List Blobs |
| PUT blob | `PUT /{container}/{blob}` | `x-ms-blob-type: BlockBlob` |
| (optional) Create Append Blob | `PUT /{container}/{blob}` | `x-ms-blob-type: AppendBlob`, empty body |
| PUT append block (continuous log) | `PUT /{container}/{blob}?comp=appendblock` | Existing AppendBlob only |
| GET blob | `GET /{container}/{blob}` | Download body |
| (optional) HEAD properties | `HEAD /{container}/{blob}` | Metadata without body |
| PUT / GET via SAS | `PUT/GET {sas_path}` | No Authorization; switch BASE to URL origin |
| Event Grid blob webhook | — | Docs-only; future `ENABLE_SERVER` |

`BASE_ADDRESS` defaults to the placeholder `https://ACCOUNT.blob.core.windows.net` (no trailing slash). Replace `ACCOUNT` with your storage account. Topics use `storage/azure-blob-storage-and-data-lake-gen2/<object>/<verb>`.

> [!NOTE]
>
> **Data Lake Gen2:** hierarchical namespace filesystem REST uses `https://{account}.dfs.core.windows.net`. Keep primary events on the Blob endpoint; swap BASE only when calling DFS path APIs.

> [!NOTE]
>
> **SAS PUT/GET:** set `BASE_ADDRESS` to the SAS URL origin only, put path+query in `sas_path`, and omit Shared Key / Bearer — the query string carries `sv` / `sig`. Ideal for edge devices that must not hold account keys.

> [!NOTE]
>
> **Event Grid** blob created/deleted subscriptions are not inbound HTTP on this client connector. Receiving Event Grid validation and event POSTs needs a separate `ENABLE_SERVER` REST\_API template (not packaged).

> [!NOTE]
>
> Prefer Hub secrets for account keys or Entra client credentials used by an upstream signer. This template has not been verified against a live Azure storage account. Status remains blocked until an authenticated live smoke (or working SAS URL smoke) succeeds.

## Basic Syntax

```lot wrap theme={"theme":"css-variables","languages":{"custom":["/languages/lot.json"]}}
# Coreflux REST_API template — Azure Blob curated slice
# Auth: SharedKey or Bearer via {value.json.authorization},
#       {value.json.x_ms_date}, {value.json.x_ms_version}
#       — or PutViaSas / GetViaSas (no Authorization)
# BASE_ADDRESS: https://ACCOUNT.blob.core.windows.net

DEFINE ROUTE AzureBlobStorageAndDataLakeGen2RestApi WITH TYPE REST_API
    ADD REST_API_CONFIG
        WITH BASE_ADDRESS "https://ACCOUNT.blob.core.windows.net"
        WITH ENABLE_CLIENT "true"
        WITH USE_SSL "true"
        WITH REQUEST_TIMEOUT '60'
        WITH DEFAULT_CONTENT_TYPE "application/octet-stream"
    ADD EVENT Listblobs
        WITH SOURCE_TOPIC "storage/azure-blob-storage-and-data-lake-gen2/blobs/list"
        WITH DESTINATION_TOPIC "storage/azure-blob-storage-and-data-lake-gen2/blobs/list/response"
        WITH METHOD "GET"
        WITH ENDPOINT "/{value.json.container}?restype=container&comp=list"
        WITH HEADERS
            ADD "Accept" "application/xml"
            ADD "Authorization" "{value.json.authorization}"
            ADD "x-ms-date" "{value.json.x_ms_date}"
            ADD "x-ms-version" "{value.json.x_ms_version}"
    ADD EVENT Putblob
        WITH SOURCE_TOPIC "storage/azure-blob-storage-and-data-lake-gen2/blob/put"
        WITH DESTINATION_TOPIC "storage/azure-blob-storage-and-data-lake-gen2/blob/put/response"
        WITH METHOD "PUT"
        WITH ENDPOINT "/{value.json.container}/{value.json.blob}"
        WITH HEADERS
            ADD "Accept" "application/xml"
            ADD "Content-Type" "{value.json.content_type}"
            ADD "x-ms-blob-type" "BlockBlob"
            ADD "Authorization" "{value.json.authorization}"
            ADD "x-ms-date" "{value.json.x_ms_date}"
            ADD "x-ms-version" "{value.json.x_ms_version}"
        WITH BODY "{payload}"
    ADD EVENT Createappendblob
        WITH SOURCE_TOPIC "storage/azure-blob-storage-and-data-lake-gen2/blob/create-append"
        WITH DESTINATION_TOPIC "storage/azure-blob-storage-and-data-lake-gen2/blob/create-append/response"
        WITH METHOD "PUT"
        WITH ENDPOINT "/{value.json.container}/{value.json.blob}"
        WITH HEADERS
            ADD "Accept" "application/xml"
            ADD "Content-Type" "application/octet-stream"
            ADD "Content-Length" "0"
            ADD "x-ms-blob-type" "AppendBlob"
            ADD "Authorization" "{value.json.authorization}"
            ADD "x-ms-date" "{value.json.x_ms_date}"
            ADD "x-ms-version" "{value.json.x_ms_version}"
    ADD EVENT Appendblock
        WITH SOURCE_TOPIC "storage/azure-blob-storage-and-data-lake-gen2/blob/append-block"
        WITH DESTINATION_TOPIC "storage/azure-blob-storage-and-data-lake-gen2/blob/append-block/response"
        WITH METHOD "PUT"
        WITH ENDPOINT "/{value.json.container}/{value.json.blob}?comp=appendblock"
        WITH HEADERS
            ADD "Accept" "application/xml"
            ADD "Content-Type" "{value.json.content_type}"
            ADD "Authorization" "{value.json.authorization}"
            ADD "x-ms-date" "{value.json.x_ms_date}"
            ADD "x-ms-version" "{value.json.x_ms_version}"
        WITH BODY "{payload}"
    ADD EVENT Getblob
        WITH SOURCE_TOPIC "storage/azure-blob-storage-and-data-lake-gen2/blob/get"
        WITH DESTINATION_TOPIC "storage/azure-blob-storage-and-data-lake-gen2/blob/get/response"
        WITH METHOD "GET"
        WITH ENDPOINT "/{value.json.container}/{value.json.blob}"
        WITH HEADERS
            ADD "Accept" "*/*"
            ADD "Authorization" "{value.json.authorization}"
            ADD "x-ms-date" "{value.json.x_ms_date}"
            ADD "x-ms-version" "{value.json.x_ms_version}"
    ADD EVENT Getblobproperties
        WITH SOURCE_TOPIC "storage/azure-blob-storage-and-data-lake-gen2/blob/head"
        WITH DESTINATION_TOPIC "storage/azure-blob-storage-and-data-lake-gen2/blob/head/response"
        WITH METHOD "HEAD"
        WITH ENDPOINT "/{value.json.container}/{value.json.blob}"
        WITH HEADERS
            ADD "Authorization" "{value.json.authorization}"
            ADD "x-ms-date" "{value.json.x_ms_date}"
            ADD "x-ms-version" "{value.json.x_ms_version}"
    ADD EVENT Putviasas
        WITH SOURCE_TOPIC "storage/azure-blob-storage-and-data-lake-gen2/blob/put-via-sas"
        WITH DESTINATION_TOPIC "storage/azure-blob-storage-and-data-lake-gen2/blob/put-via-sas/response"
        WITH METHOD "PUT"
        WITH ENDPOINT "{value.json.sas_path}"
        WITH HEADERS
            ADD "Content-Type" "{value.json.content_type}"
            ADD "x-ms-blob-type" "BlockBlob"
            ADD "x-ms-version" "{value.json.x_ms_version}"
        WITH BODY "{payload}"
    ADD EVENT Getviasas
        WITH SOURCE_TOPIC "storage/azure-blob-storage-and-data-lake-gen2/blob/get-via-sas"
        WITH DESTINATION_TOPIC "storage/azure-blob-storage-and-data-lake-gen2/blob/get-via-sas/response"
        WITH METHOD "GET"
        WITH ENDPOINT "{value.json.sas_path}"
        WITH HEADERS
            ADD "Accept" "*/*"
            ADD "x-ms-version" "{value.json.x_ms_version}"
```

## Configuration

| Setting | Value |
| - | - |
| Connector name | `AzureBlobStorageAndDataLakeGen2RestApi` |
| Type | `REST_API` (client) |
| BASE\_ADDRESS | `https://ACCOUNT.blob.core.windows.net` (replace ACCOUNT) |
| Topic prefix | `storage/azure-blob-storage-and-data-lake-gen2/...` |
| Default Content-Type | `application/octet-stream` |
| API version header | `x-ms-version: 2023-11-03` (payload-driven) |
| Hub status | Technical preview (`WITH STATUS "beta"`) |

## Authentication

### Shared Key / Entra Bearer

Blob does **not** use a simple static Bearer for Shared Key. Sign each request or obtain an Entra token and pass:

| MQTT JSON field | HTTP header |
| - | - |
| `authorization` | `Authorization` (`SharedKey account:…` or `Bearer …`) |
| `x_ms_date` | `x-ms-date` (RFC 1123 UTC) |
| `x_ms_version` | `x-ms-version` (e.g. `2023-11-03`) |

Use Hub secrets for account keys / Entra client credentials in an upstream signing helper. Never commit real keys.

### SAS URLs (recommended for edge)

Generate a time-limited service or user-delegation SAS with Azure CLI/SDK/portal. For PutViaSas / GetViaSas:

1. Point `BASE_ADDRESS` at the URL origin.
2. Put path + query into `sas_path`.
3. Omit Authorization — do not re-sign on the broker.

## MQTT topics

| Event | Request topic | Response topic |
| - | - | - |
| ListBlobs | `storage/azure-blob-storage-and-data-lake-gen2/blobs/list` | `…/response` |
| PutBlob | `storage/azure-blob-storage-and-data-lake-gen2/blob/put` | `…/response` |
| CreateAppendBlob | `storage/azure-blob-storage-and-data-lake-gen2/blob/create-append` | `…/response` |
| AppendBlock | `storage/azure-blob-storage-and-data-lake-gen2/blob/append-block` | `…/response` |
| GetBlob | `storage/azure-blob-storage-and-data-lake-gen2/blob/get` | `…/response` |
| GetBlobProperties | `storage/azure-blob-storage-and-data-lake-gen2/blob/head` | `…/response` |
| PutViaSas | `storage/azure-blob-storage-and-data-lake-gen2/blob/put-via-sas` | `…/response` |
| GetViaSas | `storage/azure-blob-storage-and-data-lake-gen2/blob/get-via-sas` | `…/response` |

## Example payloads

**ListBlobs smoke (after signing):**

```json theme={"theme":"css-variables","languages":{"custom":["/languages/lot.json"]}}
{
  "container": "iiot-rollups",
  "authorization": "SharedKey myaccount:…",
  "x_ms_date": "Sun, 04 Oct 2026 04:00:00 GMT",
  "x_ms_version": "2023-11-03"
}
```

**PutBlob CSV rollup:**

```json theme={"theme":"css-variables","languages":{"custom":["/languages/lot.json"]}}
{
  "container": "iiot-rollups",
  "blob": "line-a/2026-10-04/rollup.csv",
  "content_type": "text/csv",
  "authorization": "SharedKey myaccount:…",
  "x_ms_date": "Sun, 04 Oct 2026 04:01:00 GMT",
  "x_ms_version": "2023-11-03",
  "payload": "ts,device,temp\n2026-10-04T04:01:00Z,sensor-01,72.5\n"
}
```

**PutViaSas:**

```json theme={"theme":"css-variables","languages":{"custom":["/languages/lot.json"]}}
{
  "sas_path": "/iiot-rollups/line-a/snapshot.json?sv=2023-11-03&ss=b&srt=o&sp=cw&se=2026-10-04T12:00:00Z&sig=…",
  "content_type": "application/json",
  "x_ms_version": "2023-11-03",
  "payload": "{\"device\":\"sensor-01\",\"temp\":72.5}"
}
```

## References

* [Put Blob](https://learn.microsoft.com/en-us/rest/api/storageservices/put-blob)
* [Append Block](https://learn.microsoft.com/en-us/rest/api/storageservices/append-block)
* [Get Blob](https://learn.microsoft.com/en-us/rest/api/storageservices/get-blob)
* [List Blobs](https://learn.microsoft.com/en-us/rest/api/storageservices/list-blobs)
* [Authorize with Shared Key](https://learn.microsoft.com/en-us/rest/api/storageservices/authorize-with-shared-key)
* [Delegate access with SAS](https://learn.microsoft.com/en-us/rest/api/storageservices/delegate-access-with-shared-access-signature)
* [Azure Data Lake Storage Gen2](https://learn.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-introduction)
* [Event Grid blob events](https://learn.microsoft.com/en-us/azure/event-grid/event-schema-blob-storage)
* [Coreflux REST API Connectors](/latest/lot-language/routes/rest-api-routes)

## Package status

Blocked on the factory until a storage account + SAS (or Shared Key / Entra token) + container enables a live authenticated smoke recorded in `TEST.md`.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.