Hugging Face Inference Endpoints

Technical Preview

Overview

Client-mode REST API connectors for Hugging Face: triage alarms with OpenAI-compatible chat completions through Inference Providers, create embeddings and classify camera snapshots with HF Inference, pause, resume or scale dedicated Inference Endpoints to zero to control cost, call your own anomaly model, and search Hub models, all with a Hugging Face token. This is one of the templates in the Hugging Face Inference Providers and Endpoints REST API app (hugging-face-inference-providers-and-endpoints). Deploy it alongside the other connectors of that app when they depend on each other (for example a token connector plus an API connector).

Not yet verified against the live vendor API: No Hugging Face token on the factory box. Needs a Hugging Face account with a fine-grained user access token (HF_TOKEN) with ‘Make calls to Inference Providers’ (plus credits: free monthly $0.10 or PRO/Team) for ChatCompletion / FeatureExtraction / ImageClassification, and Inference Endpoints read/write + a namespace (user or org with a payment method) for ListEndpoints / GetEndpoint / Pause / Resume / ScaleToZero, ideally one small CPU test endpoint (e.g. aws eu-west-1 intel-spr x1, scale-to-zero) for the dedicated connector. Then run HF_TOKEN=… [HF_NAMESPACE=…] python3 smoke_huggingface.py: WhoAmI GET + ListRouterModels GET + ChatCompletion POST + FeatureExtraction POST (+ ListEndpoints GET) must return 200. Public Hub/router GETs returning 200 without a token do not clear the gate.

Auth

Hugging Face user access token as Authorization: Bearer {value.json.hf_token} on every event. Fine-grained token: ‘Make calls to Inference Providers’ (router chat / task events) + Inference Endpoints read/write (management + dedicated endpoint calls). Hub secret suggested HF_TOKEN + GET SECRET. Public router / Hub GETs answer without a valid token. Never commit tokens. Auth headers used by the template: Authorization: Bearer {value.json.hf_token}.

Template parameters

Placeholder	Type	Description	Required
`RouteName`	string	Name of the Hugging Face Inference Endpoints connector	no
`BaseUrl`	string	Hugging Face Inference Endpoints base URL without trailing slash (e.g. https://api.endpoints.huggingface.cloud)	yes

BASE_ADDRESS default: https://api.endpoints.huggingface.cloud.

Events

Event	Method	Endpoint	Request topic	Response topic	Payload fields
`ListEndpoints`	GET	`/v2/endpoint/{value.json.namespace}`	`ai/huggingface/endpoint/list`	`ai/huggingface/endpoint/list/response`	`hf_token`, `namespace`
`GetEndpoint`	GET	`/v2/endpoint/{value.json.namespace}/{value.json.name}`	`ai/huggingface/endpoint/get`	`ai/huggingface/endpoint/get/response`	`hf_token`, `name`, `namespace`
`CreateEndpoint`	POST	`/v2/endpoint/{value.json.namespace}`	`ai/huggingface/endpoint/create`	`ai/huggingface/endpoint/create/response`	`body`, `hf_token`, `namespace`
`PauseEndpoint`	POST	`/v2/endpoint/{value.json.namespace}/{value.json.name}/pause`	`ai/huggingface/endpoint/pause`	`ai/huggingface/endpoint/pause/response`	`hf_token`, `name`, `namespace`
`ResumeEndpoint`	POST	`/v2/endpoint/{value.json.namespace}/{value.json.name}/resume`	`ai/huggingface/endpoint/resume`	`ai/huggingface/endpoint/resume/response`	`hf_token`, `name`, `namespace`
`ScaleToZeroEndpoint`	POST	`/v2/endpoint/{value.json.namespace}/{value.json.name}/scale-to-zero`	`ai/huggingface/endpoint/scale-to-zero`	`ai/huggingface/endpoint/scale-to-zero/response`	`hf_token`, `name`, `namespace`

Deploy

Get the template card (JSON with template and metadata) from the broker, then deploy the connector:

mosquitto_pub -t '$SYS/Coreflux/Command' -m "-routeCode HUGGING_FACE_INFERENCE_ENDPOINTS"

The deployable connector from the REST API app, with BASE_ADDRESS already filled in:

DEFINE ROUTE HuggingFaceInferenceEndpoints WITH TYPE REST_API
    ADD REST_API_CONFIG
        WITH BASE_ADDRESS "https://api.endpoints.huggingface.cloud"
        WITH ENABLE_CLIENT "true"
        WITH USE_SSL "true"
        WITH REQUEST_TIMEOUT '60'
        WITH DEFAULT_CONTENT_TYPE "application/json"
    ADD EVENT ListEndpoints
        WITH SOURCE_TOPIC "ai/huggingface/endpoint/list"
        WITH DESTINATION_TOPIC "ai/huggingface/endpoint/list/response"
        WITH METHOD "GET"
        WITH ENDPOINT "/v2/endpoint/{value.json.namespace}"
        WITH HEADERS
            ADD "Accept" "application/json"
            ADD "Authorization" "Bearer {value.json.hf_token}"
    ADD EVENT GetEndpoint
        WITH SOURCE_TOPIC "ai/huggingface/endpoint/get"
        WITH DESTINATION_TOPIC "ai/huggingface/endpoint/get/response"
        WITH METHOD "GET"
        WITH ENDPOINT "/v2/endpoint/{value.json.namespace}/{value.json.name}"
        WITH HEADERS
            ADD "Accept" "application/json"
            ADD "Authorization" "Bearer {value.json.hf_token}"
    ADD EVENT CreateEndpoint
        WITH SOURCE_TOPIC "ai/huggingface/endpoint/create"
        WITH DESTINATION_TOPIC "ai/huggingface/endpoint/create/response"
        WITH METHOD "POST"
        WITH ENDPOINT "/v2/endpoint/{value.json.namespace}"
        WITH HEADERS
            ADD "Accept" "application/json"
            ADD "Content-Type" "application/json"
            ADD "Authorization" "Bearer {value.json.hf_token}"
        WITH BODY "{value.json.body}"
    ADD EVENT PauseEndpoint
        WITH SOURCE_TOPIC "ai/huggingface/endpoint/pause"
        WITH DESTINATION_TOPIC "ai/huggingface/endpoint/pause/response"
        WITH METHOD "POST"
        WITH ENDPOINT "/v2/endpoint/{value.json.namespace}/{value.json.name}/pause"
        WITH HEADERS
            ADD "Accept" "application/json"
            ADD "Authorization" "Bearer {value.json.hf_token}"
        WITH BODY ""
    ADD EVENT ResumeEndpoint
        WITH SOURCE_TOPIC "ai/huggingface/endpoint/resume"
        WITH DESTINATION_TOPIC "ai/huggingface/endpoint/resume/response"
        WITH METHOD "POST"
        WITH ENDPOINT "/v2/endpoint/{value.json.namespace}/{value.json.name}/resume"
        WITH HEADERS
            ADD "Accept" "application/json"
            ADD "Authorization" "Bearer {value.json.hf_token}"
        WITH BODY ""
    ADD EVENT ScaleToZeroEndpoint
        WITH SOURCE_TOPIC "ai/huggingface/endpoint/scale-to-zero"
        WITH DESTINATION_TOPIC "ai/huggingface/endpoint/scale-to-zero/response"
        WITH METHOD "POST"
        WITH ENDPOINT "/v2/endpoint/{value.json.namespace}/{value.json.name}/scale-to-zero"
        WITH HEADERS
            ADD "Accept" "application/json"
            ADD "Authorization" "Bearer {value.json.hf_token}"
        WITH BODY ""

Deploy it with -addRoute (do not include ADD METADATA).

Usage example

Call ListEndpoints by publishing a JSON payload to its request topic, and read the reply on the response topic:

mosquitto_sub -t 'ai/huggingface/endpoint/list/response' &
mosquitto_pub -t 'ai/huggingface/endpoint/list' -m '{"hf_token": "<hf_token>", "namespace": "<namespace>"}'

Quick Start

Coreflux MQTT Broker

LoT Language

Coreflux HUB

VS Code - LoT Notebooks

Coreflux Portal

Using AI with Coreflux

Product Updates

Overview

Auth

Template parameters

Events

Deploy

Usage example

​Overview

​Auth

​Template parameters

​Events

​Deploy

​Usage example

​Related

Overview

Auth

Template parameters

Events

Deploy

Usage example

Related