> ## Documentation Index
> Fetch the complete documentation index at: https://docs.coreflux.org/llms.txt
> Use this file to discover all available pages before exploring further.

# Prometheus, Alertmanager and Pushgateway

> Client-mode REST API connector for a curated Prometheus / Alertmanager / Pushgateway slice: PromQL instant and range queries, scrape targets, Pushgateway metric push, and Alertmanager alert list

Technical Preview

## Overview

A client-mode connector for a **curated observability / ITSM** slice of the [Prometheus HTTP API](https://prometheus.io/docs/prometheus/latest/querying/api/), [Pushgateway](https://prometheus.io/docs/instrumenting/pushing/), and [Alertmanager API](https://prometheus.io/docs/alerting/latest/management_api/):

| Catalog connector event | Entity / path | Notes |
| - | - | - |
| GET query | `GET /api/v1/query?query=` | PromQL instant query; typically **no auth** |
| GET query\_range | `GET /api/v1/query_range` | PromQL range (`start` / `end` / `step`) |
| GET targets | `GET /api/v1/targets` | Scrape targets smoke / discovery |
| POST Pushgateway metrics | `POST /metrics/job/{job}` | text/plain OpenMetrics / Prometheus exposition; **BASE = Pushgateway :9091** |
| GET Alertmanager alerts | `GET /api/v2/alerts` | Alertmanager API v2; **BASE = Alertmanager :9093** |

Default `BASE_ADDRESS` is `http://prometheus.example.com:9090` (replace with your Prometheus host). Topics use `itsm/prometheus/<object>/<verb>`.

> [!NOTE]
>
> Prometheus, Alertmanager, and Pushgateway are usually separate hosts/ports (`:9090` / `:9093` / `:9091`). Point `BASE_ADDRESS` at the matching host for each event, or deploy separate connector copies. For HTTPS reverse proxies set `WITH USE_SSL "true"`. Do not put a trailing slash on `BASE_ADDRESS`.

> [!NOTE]
>
> Prefer Hub secrets when a reverse proxy requires Basic or Bearer. Public [demo.promlabs.com](https://demo.promlabs.com) and [prometheus.demo.prometheus.io](https://prometheus.demo.prometheus.io) allow anonymous PromQL reads, but they do not expose a writable Pushgateway — status remains blocked until a Pushgateway (and preferably owned stack) smoke succeeds.

## Basic Syntax

```lot wrap theme={"theme":"css-variables","languages":{"custom":["/languages/lot.json"]}}
# Coreflux REST_API template — Prometheus / Alertmanager / Pushgateway curated slice
# Auth: none by default; optional Bearer/Basic via reverse proxy
# BASE_ADDRESS: http://prometheus.example.com:9090
# PushMetrics → Pushgateway :9091; ListAlerts → Alertmanager :9093

DEFINE ROUTE PrometheusRestApi WITH TYPE REST_API
    ADD REST_API_CONFIG
        WITH BASE_ADDRESS "http://prometheus.example.com:9090"
        WITH ENABLE_CLIENT "true"
        WITH USE_SSL "false"
        WITH REQUEST_TIMEOUT '60'
        WITH DEFAULT_CONTENT_TYPE "application/json"
    ADD EVENT Queryinstant
        WITH SOURCE_TOPIC "itsm/prometheus/query/instant"
        WITH DESTINATION_TOPIC "itsm/prometheus/query/instant/response"
        WITH METHOD "GET"
        WITH ENDPOINT "/api/v1/query?query={value.json.query}&time={value.json.time}"
        WITH HEADERS
            ADD "Accept" "application/json"
    ADD EVENT Queryrange
        WITH SOURCE_TOPIC "itsm/prometheus/query/range"
        WITH DESTINATION_TOPIC "itsm/prometheus/query/range/response"
        WITH METHOD "GET"
        WITH ENDPOINT "/api/v1/query_range?query={value.json.query}&start={value.json.start}&end={value.json.end}&step={value.json.step}"
        WITH HEADERS
            ADD "Accept" "application/json"
    ADD EVENT Listtargets
        WITH SOURCE_TOPIC "itsm/prometheus/targets/list"
        WITH DESTINATION_TOPIC "itsm/prometheus/targets/list/response"
        WITH METHOD "GET"
        WITH ENDPOINT "/api/v1/targets"
        WITH HEADERS
            ADD "Accept" "application/json"
    ADD EVENT Pushmetrics
        WITH SOURCE_TOPIC "itsm/prometheus/pushgateway/push"
        WITH DESTINATION_TOPIC "itsm/prometheus/pushgateway/push/response"
        WITH METHOD "POST"
        WITH ENDPOINT "/metrics/job/{value.json.job}"
        WITH HEADERS
            ADD "Accept" "application/json"
            ADD "Content-Type" "text/plain"
        WITH BODY "{payload}"
    ADD EVENT Listalerts
        WITH SOURCE_TOPIC "itsm/prometheus/alertmanager/alerts/list"
        WITH DESTINATION_TOPIC "itsm/prometheus/alertmanager/alerts/list/response"
        WITH METHOD "GET"
        WITH ENDPOINT "/api/v2/alerts"
        WITH HEADERS
            ADD "Accept" "application/json"
```

## Events

| Event | Method | Endpoint | Request topic | Response topic |
| - | - | - | - | - |
| Queryinstant | GET | `/api/v1/query?query=&time=` | `itsm/prometheus/query/instant` | `itsm/prometheus/query/instant/response` |
| Queryrange | GET | `/api/v1/query_range?query=&start=&end=&step=` | `itsm/prometheus/query/range` | `itsm/prometheus/query/range/response` |
| Listtargets | GET | `/api/v1/targets` | `itsm/prometheus/targets/list` | `itsm/prometheus/targets/list/response` |
| Pushmetrics | POST | `/metrics/job/{job}` | `itsm/prometheus/pushgateway/push` | `itsm/prometheus/pushgateway/push/response` |
| Listalerts | GET | `/api/v2/alerts` | `itsm/prometheus/alertmanager/alerts/list` | `itsm/prometheus/alertmanager/alerts/list/response` |

## Auth

**None (default):**

1. Most Prometheus installs expose `/api/v1/*` without auth on private networks.
2. Leave Authorization headers off for Queryinstant / Queryrange / Listtargets.

**Bearer via reverse proxy (optional):**

1. Terminate TLS + auth in nginx / Traefik / similar in front of Prometheus / Alertmanager / Pushgateway.
2. Pass `token` on the MQTT request payload:
   * `Authorization: Bearer {value.json.token}`
3. Store the token in Hub secrets / `GET SECRET "PROMETHEUS_TOKEN"`. Never commit real credentials.

**Basic via reverse proxy (optional):**

1. Configure `WITH USERNAME` / `WITH PASSWORD GET SECRET "PROMETHEUS_PASSWORD"` on `REST_API_CONFIG`.
2. Remove any Bearer headers from events.

## Install

1. Copy `PrometheusRestApi.lot` into your broker examples / connectors folder (no `ADD METADATA` in the deployable file).
2. Set `BASE_ADDRESS` to your Prometheus URL (use `https://…` and `WITH USE_SSL "true"` when TLS terminates on the API host or proxy).
3. `-addRoute` the file (or paste into Hub).
4. For Pushgateway and Alertmanager, either change BASE and re-deploy, or keep separate connector copies with the matching BASE.
5. Subscribe to the `/response` topics; publish JSON to the request topics.

Hub template listing uses `hub-metadata.lot` (`WITH STATUS "beta"`, icon, doclink) — keep that file separate from the deployable connector.

## Example payloads

**Instant query**

```json theme={"theme":"css-variables","languages":{"custom":["/languages/lot.json"]}}
{ "query": "up", "time": "" }
```

**Range query**

```json theme={"theme":"css-variables","languages":{"custom":["/languages/lot.json"]}}
{
  "query": "rate(http_requests_total[5m])",
  "start": "2026-10-04T00:00:00Z",
  "end": "2026-10-04T01:00:00Z",
  "step": "1m"
}
```

**List targets**

```json theme={"theme":"css-variables","languages":{"custom":["/languages/lot.json"]}}
{}
```

**Pushgateway push** — MQTT JSON must supply `job`; body `{payload}` is raw text/plain exposition (example metrics body):

```text theme={"theme":"css-variables","languages":{"custom":["/languages/lot.json"]}}
# TYPE coreflux_smoke_gauge gauge
coreflux_smoke_gauge{instance="factory"} 1
```

**List Alertmanager alerts** (BASE = Alertmanager)

```json theme={"theme":"css-variables","languages":{"custom":["/languages/lot.json"]}}
{}
```

## Complementary surfaces (not client events here)

| Surface | Role |
| - | - |
| Alertmanager webhook receiver | Inbound: Alertmanager `webhook_config` POSTs alert payloads to a Coreflux REST\_API **server** connector or HTTP ingress |
| Remote write | Prometheus remote\_write (protobuf/snappy) — different protocol; not this JSON/text HTTP slice |
| OpenMetrics | Exposition format used by Pushgateway bodies and scrapes |

## References

* [Prometheus HTTP API](https://prometheus.io/docs/prometheus/latest/querying/api/)
* [Pushing metrics (Pushgateway)](https://prometheus.io/docs/instrumenting/pushing/)
* [Alertmanager HTTP API](https://prometheus.io/docs/alerting/latest/management_api/)
* [Alertmanager configuration (webhooks)](https://prometheus.io/docs/alerting/latest/configuration/)
* [Remote write spec](https://prometheus.io/docs/specs/remote_write_spec/)
* [OpenMetrics](https://github.com/OpenObservability/OpenMetrics)
* Public demos: [demo.promlabs.com](https://demo.promlabs.com), [prometheus.demo.prometheus.io](https://prometheus.demo.prometheus.io)

## Suggested branches (Hugo pushes)

* App package: `feat/rest-api-route-prometheus-alertmanager-and-pushgateway`
* Docs: `feat/docs-rest-api-prometheus-alertmanager-and-pushgateway`


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.