Audio Input Connector

Publish microphone audio to MQTT

Audio Input opens a microphone, a synthetic source, or a file on the broker host and publishes raw 16-bit little-endian PCM. Each MQTT message is one time slice of samples — bytes, not JSON and not Base64. A LoT action can wrap those bytes in protobuf when a downstream service expects a message, not a bare buffer.

Like a voice recorder that saves short tapes. Every tenth of a second (by default) a new tape of samples lands on the topic, ready for the next step in your project.

When to use this

Use TYPE AUDIO_INPUT when you need live microphone audio, a test tone, silence for an automated run, or PCM from a file, on an MQTT topic. FFmpeg must be on the broker host (or in the slim image). The connector does not open Core Audio, WASAPI, or Pulse itself.

Do not set WITH FORMAT. That keyword belongs to models. This connector always publishes s16le PCM.

Video Input is the camera sibling: JPEG data-URIs instead of PCM, with the same FFmpeg install.

Each tab is a full connector. Publish any payload to the start topic to begin. Chunks then arrive on DESTINATION_TOPIC until stop.

Quick Start

Silence for tests
Microphone
Windows microphone
Timed capture

anullsrc needs no microphone. Use it in CI and on hosts with no audio device:

DEFINE ROUTE MicCi WITH TYPE AUDIO_INPUT
    ADD AUDIO_INPUT_CONFIG
        WITH SOURCE "anullsrc"
        WITH DESTINATION_TOPIC "audio/pcm/ci"
        WITH SAMPLE_RATE 16000
        WITH CHANNELS 1
        WITH CHUNK_MS 100
    ADD EVENT Start
        WITH SOURCE_TOPIC "audio/control/ci/start"
        WITH QUERY "START"
    ADD EVENT Stop
        WITH SOURCE_TOPIC "audio/control/ci/stop"
        WITH QUERY "STOP"

mic:0 is the default input on macOS and Linux. Windows rejects mic:N — use the DirectShow tab:

DEFINE ROUTE BuiltInMic WITH TYPE AUDIO_INPUT
    ADD AUDIO_INPUT_CONFIG
        WITH SOURCE "mic:0"
        WITH DESTINATION_TOPIC "audio/pcm/builtin"
        WITH SAMPLE_RATE 16000
        WITH CHANNELS 1
        WITH CHUNK_MS 100
    ADD EVENT Start
        WITH SOURCE_TOPIC "audio/control/start"
        WITH QUERY "START"
    ADD EVENT Stop
        WITH SOURCE_TOPIC "audio/control/stop"
        WITH QUERY "STOP"

Set the environment variable to the exact DirectShow name, including the audio= prefix (audio=Microphone (Realtek Audio)):

DEFINE ROUTE WindowsMic WITH TYPE AUDIO_INPUT
    ADD AUDIO_INPUT_CONFIG
        WITH SOURCE GET ENV "AUDIO_SOURCE"
        WITH DESTINATION_TOPIC "audio/pcm/headset"
        WITH SAMPLE_RATE 16000
        WITH CHANNELS 1
        WITH CHUNK_MS 100
    ADD EVENT Start
        WITH SOURCE_TOPIC "audio/control/start"
        WITH QUERY "START"
    ADD EVENT Stop
        WITH SOURCE_TOPIC "audio/control/stop"
        WITH QUERY "STOP"

CAPTURE 5 SECONDS records a finite window, then stops on its own:

DEFINE ROUTE ClipMic WITH TYPE AUDIO_INPUT
    ADD AUDIO_INPUT_CONFIG
        WITH SOURCE "mic:0"
        WITH DESTINATION_TOPIC "audio/pcm/clip"
        WITH CHUNK_MS 100
    ADD EVENT RecordFiveSeconds
        WITH SOURCE_TOPIC "audio/control/clip"
        WITH QUERY "CAPTURE 5 SECONDS"
        WITH DESTINATION_TOPIC "audio/pcm/clip"

Configuration

string

required

Microphone, DirectShow name, synthetic source, or file path. See the source table below.

string

Topic for PCM chunks. Required on the config or on the event that starts capture. START and STOP events often omit it and inherit this topic.

integer

Output rate in Hz, from 8000 to 48000. Default: 16000.

integer

1 (mono) or 2 (stereo). Default: 1.

integer

Slice length in milliseconds, from 20 to 1000. Default: 100.

Capture sources

`SOURCE`	Broker host	What opens
`mic:0`	macOS	First input in the system audio list
`mic:0`	Linux	PulseAudio `default`
`mic:1`	Linux	ALSA `hw:1,0`
`mic:N`	Windows	Rejected. Use `audio=<name>`
`audio=Microphone (Realtek Audio)`	Windows	DirectShow device. The name must match exactly
`anullsrc` or `lavfi:anullsrc`	Any	Silence. No hardware
`tone:440` or `sine:440`	Any	A sine tone at that frequency
File path	Any	Passed through as a media file (`.wav` and other formats FFmpeg can read)

List Windows devices on the broker host:

ffmpeg -list_devices true -f dshow -i dummy

Copy the audio device name into audio=….

Events

string

required

START, STOP, or CAPTURE <N> SECONDS.

string

required

Topic that runs the event.

string

Override for this event. When omitted, chunks use the config topic.

Query	Result
`START`	Stop any capture already running, then capture until `STOP`
`STOP`	Stop FFmpeg and stop publishing
`CAPTURE <N> SECONDS`	Capture for N seconds, then stop

There is no audio callback on the connector. Control is only these queries.

What arrives on the topic

Each payload is raw s16le bytes. At the defaults — 16 kHz, mono, 100 ms — one chunk is 3200 bytes, about 32 KB/s. That is small next to Video Input JPEGs. Keep the default CHUNK_MS unless the consumer expects a different window. Do not retain the PCM topic; the next chunk replaces it immediately and retained audio is stale the moment it is stored. Subscribe with any MQTT client that can show a binary payload. Do not JSON-parse the chunk.

Wrap the bytes for gRPC

Publish PCM on your own topic (audio/pcm/…). When a service expects protobuf, a LoT action copies PAYLOAD AS BYTES into a model and publishes that message. A gRPC connector with RPC_TYPE BIDI can forward those bytes. Field layout is a protobuf model: bytes fields use ADD OBJECT, assigned from the raw payload rather than from Base64.

DEFINE ACTION ForwardPcmChunk
ON TOPIC "audio/pcm/builtin" DO
    PUBLISH TOPIC "audio/wrapped/builtin" WITH PAYLOAD AS BYTES

Use GET ENV or GET SECRET for device names and for any gRPC address. Keep the hot path in LoT. Do not shell out to a script to move each chunk.

Docker and FFmpeg

Install FFmpeg on the broker host and confirm ffmpeg -version there. This is the same install as Video Input. The slim broker image includes FFmpeg on amd64 and arm64. The minimal image does not, and arm/v7 does not. anullsrc and tone:440 need no device mount, including inside Docker. A physical microphone has to be visible to FFmpeg on the broker host. macOS and Windows Docker Desktop usually cannot pass the host microphone into the Linux VM; run the broker on the host, or use a synthetic source, when you need audio in that setup.

Best Practices

Test without hardware

anullsrc for silence and tone:440 for a sine wave. Automated runs should not depend on a plugged-in microphone.

Keep device names out of LoT

On Windows, WITH SOURCE GET ENV "AUDIO_SOURCE" and set the variable to audio=Exact Device Name.

Leave PCM as bytes

Downstream models and connectors should read the payload as bytes. Parsing it as JSON or as a string yields an empty or garbage field.

Match the chunk the consumer expects

100 ms at 16 kHz mono is the default. Change SAMPLE_RATE, CHANNELS, or CHUNK_MS only when the next service requires a different window.

Troubleshooting

The connector is added but no chunks appear

FFmpeg is missing or not on PATH. Run ffmpeg -version on the broker host, or use the slim image.

mic:0 is rejected on Windows

Indexed microphones are not supported. List DirectShow devices and set audio=<exact name>.

A protobuf audio field is empty

The chunk was read as text or JSON. Assign the MQTT payload with PAYLOAD AS BYTES into an ADD OBJECT bytes field. See protobuf models.

A REST call sends the text System.Byte[]

A body template such as {payload} stringifies the buffer. Omit BODY on the REST event so the connector posts the raw bytes as application/octet-stream.

Quick Start

Coreflux MQTT Broker

LoT Language

Coreflux HUB

VS Code - LoT Notebooks

Coreflux Portal

Using AI with Coreflux

Product Updates

Publish microphone audio to MQTT

When to use this

Quick Start

Configuration

Capture sources

Events

What arrives on the topic

Wrap the bytes for gRPC

Docker and FFmpeg

Best Practices

Troubleshooting

Next Steps

Video Input

gRPC Connector

​Publish microphone audio to MQTT

​When to use this

​Quick Start

​Configuration

​Capture sources

​Events

​What arrives on the topic

​Wrap the bytes for gRPC

​Docker and FFmpeg

​Best Practices

​Troubleshooting

​Next Steps

Video Input

gRPC Connector

Publish microphone audio to MQTT

When to use this

Quick Start

Configuration

Capture sources

Events

What arrives on the topic

Wrap the bytes for gRPC

Docker and FFmpeg

Best Practices

Troubleshooting

Next Steps