Google Cloud Storage

Technical Preview

Overview

A client-mode connector for a curated IIoT object-storage slice of the Google Cloud Storage JSON API under https://storage.googleapis.com:

Catalog connector event	Entity / path	Notes
(smoke)	`GET /storage/v1/b?project={project}`	ListBuckets for the OAuth identity
(alt smoke)	`GET /storage/v1/b/{bucket}`	Bucket metadata
POST upload (media)	`POST /upload/storage/v1/b/{bucket}/o?uploadType=media&name=`	Objects.insert media; body = raw bytes
GET object	`GET /storage/v1/b/{bucket}/o/{object}`	Object metadata (JSON)
GET object bytes	`GET …/o/{object}?alt=media`	Object data
PUT/POST to signed URL	`PUT	POST `	No Authorization; switch BASE to URL origin
Pub/Sub notifications	—	Docs-only; future `ENABLE_SERVER`

BASE_ADDRESS defaults to https://storage.googleapis.com (no trailing slash). JSON API paths use /storage/v1/...; media upload uses /upload/storage/v1/... on the same host. Topics use storage/google-cloud-storage/<object>/<verb>.

Object names: path segments with / must be URL-encoded as %2F (e.g. line-a%2Fsnapshot.json).

Signed URL: set BASE_ADDRESS to the signed URL origin only, put path+query in signed_path, and omit Bearer — the query string carries the signature. Typical GCS V4 upload signed URLs use PUT (PutSignedurl); use PostSignedurl only when the signer issued POST.

Pub/Sub / bucket notifications are not inbound HTTP on this client connector. Receiving notification POSTs needs a separate ENABLE_SERVER REST_API template (not packaged).

Prefer Hub secrets for GCS_ACCESS_TOKEN (or GOOGLE_CLOUD_STORAGE_ACCESS_TOKEN / GOOGLE_ACCESS_TOKEN). Service-account JWT → access-token exchange is out of band. This template has not been verified against a live GCP project. Status remains blocked until an authenticated live smoke (or working signed URL smoke) succeeds.

Basic Syntax

# Coreflux REST_API template — Google Cloud Storage JSON API curated slice
# Auth: Bearer {value.json.access_token} — or Put/PostSignedurl (no Authorization)
# BASE_ADDRESS: https://storage.googleapis.com

DEFINE ROUTE GoogleCloudStorageRestApi WITH TYPE REST_API
    ADD REST_API_CONFIG
        WITH BASE_ADDRESS "https://storage.googleapis.com"
        WITH ENABLE_CLIENT "true"
        WITH USE_SSL "true"
        WITH REQUEST_TIMEOUT '60'
        WITH DEFAULT_CONTENT_TYPE "application/octet-stream"
    ADD EVENT Listbuckets
        WITH SOURCE_TOPIC "storage/google-cloud-storage/buckets/list"
        WITH DESTINATION_TOPIC "storage/google-cloud-storage/buckets/list/response"
        WITH METHOD "GET"
        WITH ENDPOINT "/storage/v1/b?project={value.json.project}"
        WITH HEADERS
            ADD "Accept" "application/json"
            ADD "Authorization" "Bearer {value.json.access_token}"
    ADD EVENT Getbucket
        WITH SOURCE_TOPIC "storage/google-cloud-storage/bucket/get"
        WITH DESTINATION_TOPIC "storage/google-cloud-storage/bucket/get/response"
        WITH METHOD "GET"
        WITH ENDPOINT "/storage/v1/b/{value.json.bucket}"
        WITH HEADERS
            ADD "Accept" "application/json"
            ADD "Authorization" "Bearer {value.json.access_token}"
    ADD EVENT Uploadobjectmedia
        WITH SOURCE_TOPIC "storage/google-cloud-storage/object/upload"
        WITH DESTINATION_TOPIC "storage/google-cloud-storage/object/upload/response"
        WITH METHOD "POST"
        WITH ENDPOINT "/upload/storage/v1/b/{value.json.bucket}/o?uploadType=media&name={value.json.name}"
        WITH HEADERS
            ADD "Accept" "application/json"
            ADD "Content-Type" "{value.json.content_type}"
            ADD "Authorization" "Bearer {value.json.access_token}"
        WITH BODY "{payload}"
    ADD EVENT Getobject
        WITH SOURCE_TOPIC "storage/google-cloud-storage/object/get"
        WITH DESTINATION_TOPIC "storage/google-cloud-storage/object/get/response"
        WITH METHOD "GET"
        WITH ENDPOINT "/storage/v1/b/{value.json.bucket}/o/{value.json.object}"
        WITH HEADERS
            ADD "Accept" "application/json"
            ADD "Authorization" "Bearer {value.json.access_token}"
    ADD EVENT Getobjectmedia
        WITH SOURCE_TOPIC "storage/google-cloud-storage/object/get-media"
        WITH DESTINATION_TOPIC "storage/google-cloud-storage/object/get-media/response"
        WITH METHOD "GET"
        WITH ENDPOINT "/storage/v1/b/{value.json.bucket}/o/{value.json.object}?alt=media"
        WITH HEADERS
            ADD "Accept" "*/*"
            ADD "Authorization" "Bearer {value.json.access_token}"
    ADD EVENT Putsignedurl
        WITH SOURCE_TOPIC "storage/google-cloud-storage/object/put-signed"
        WITH DESTINATION_TOPIC "storage/google-cloud-storage/object/put-signed/response"
        WITH METHOD "PUT"
        WITH ENDPOINT "{value.json.signed_path}"
        WITH HEADERS
            ADD "Content-Type" "{value.json.content_type}"
        WITH BODY "{payload}"
    ADD EVENT Postsignedurl
        WITH SOURCE_TOPIC "storage/google-cloud-storage/object/post-signed"
        WITH DESTINATION_TOPIC "storage/google-cloud-storage/object/post-signed/response"
        WITH METHOD "POST"
        WITH ENDPOINT "{value.json.signed_path}"
        WITH HEADERS
            ADD "Content-Type" "{value.json.content_type}"
        WITH BODY "{payload}"

Configuration

Setting	Value
Connector name	`GoogleCloudStorageRestApi`
Type	`REST_API` (client)
BASE_ADDRESS	`https://storage.googleapis.com`
Topic prefix	`storage/google-cloud-storage/...`
Default Content-Type	`application/octet-stream`
Hub status	Technical preview (`WITH STATUS "beta"`)

Authentication

OAuth 2.0 Bearer

Pass a short-lived access token on each MQTT payload:

MQTT JSON field	HTTP header
`access_token`	`Authorization: Bearer …`

Prefer Hub secrets / GET SECRET "GCS_ACCESS_TOKEN" (aliases GOOGLE_CLOUD_STORAGE_ACCESS_TOKEN, GOOGLE_ACCESS_TOKEN). Scope: https://www.googleapis.com/auth/devstorage.read_write (or .full_control / .read_only). Service-account JWT assertion exchange is out of band — use gcloud, ADC, or your IdP, then place only the access token in the Hub secret / MQTT payload. Never commit real credentials.

Signed URLs (recommended for edge)

Generate a time-limited V4 signed URL with gsutil / gcloud / client libraries. For PutSignedurl:

Point BASE_ADDRESS at the URL origin.
Put path + query into signed_path.
Omit Authorization — do not re-sign on the broker.

MQTT topics

Event	Request topic	Response topic
ListBuckets	`storage/google-cloud-storage/buckets/list`	`…/response`
GetBucket	`storage/google-cloud-storage/bucket/get`	`…/response`
UploadObjectMedia	`storage/google-cloud-storage/object/upload`	`…/response`
GetObject	`storage/google-cloud-storage/object/get`	`…/response`
GetObjectMedia	`storage/google-cloud-storage/object/get-media`	`…/response`
PutSignedurl	`storage/google-cloud-storage/object/put-signed`	`…/response`
PostSignedurl	`storage/google-cloud-storage/object/post-signed`	`…/response`

Example payloads

ListBuckets smoke:

{
  "project": "my-gcp-project",
  "access_token": "ya29...."
}

Media upload (CSV rollup):

{
  "bucket": "my-iiot-rollups",
  "name": "line-a%2F2026-10-04%2Frollup.csv",
  "content_type": "text/csv",
  "access_token": "ya29....",
  "payload": "ts,device,temp\n2026-10-04T04:01:00Z,sensor-01,72.5\n"
}

Get object metadata:

{
  "bucket": "my-iiot-rollups",
  "object": "line-a%2F2026-10-04%2Frollup.csv",
  "access_token": "ya29...."
}

PutSignedurl:

{
  "signed_path": "/my-iiot-rollups/line-a/snapshot.json?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=...&X-Goog-Signature=...",
  "content_type": "application/json",
  "payload": "{\"device\":\"sensor-01\",\"temp\":72.5}"
}

References

Package status

Blocked on the factory until GCS_ACCESS_TOKEN (or alias) + a GCP project id (and preferably a lab bucket) or a valid signed URL enables a live authenticated smoke recorded in TEST.md.

Quick Start

Coreflux MQTT Broker

LoT Language

Coreflux HUB

VS Code - LoT Notebooks

Coreflux Portal

Using AI with Coreflux

Product Updates

Overview

Basic Syntax

Configuration

Authentication

OAuth 2.0 Bearer

Signed URLs (recommended for edge)

MQTT topics

Example payloads

References

Package status

​Overview

​Basic Syntax

​Configuration

​Authentication

​OAuth 2.0 Bearer

​Signed URLs (recommended for edge)

​MQTT topics

​Example payloads

​References

​Package status

Overview

Basic Syntax

Configuration

Authentication

OAuth 2.0 Bearer

Signed URLs (recommended for edge)

MQTT topics

Example payloads

References

Package status