Live signing
Stream a speaker's audio and get their words back as they talk, a few at a time and in order, ready to sign. For studio feeds, conferences, sermons and lessons.
How a session goes
wss://api.usesigna.org/v1/live- Connect, then send the start message (below). The key goes in the message because browsers can't set headers on a WebSocket; from a web page or app, use a client token there instead of the key.
- Wait for
{"type": "ready"}, then send the audio as binary messages: 16 kHz, mono, signed 16-bit little-endian PCM, in chunks of 20 to 100 ms. - Words come back as
finalmessages. Send each one to /v1/sign and play the signs one after another, inidorder. - Send
{"type": "stop"}to finish. The rest comes back, then{"type": "done"}.
import WebSocket from "ws"; // npm i ws
const ws = new WebSocket("wss://api.usesigna.org/v1/live");
ws.on("open", () => {
ws.send(JSON.stringify({ type: "start", lang: "rw", token: process.env.SIGNA_KEY }));
});
ws.on("message", (data, isBinary) => {
if (isBinary) return;
const msg = JSON.parse(data.toString());
if (msg.type === "ready") {
// Start sending audio: 16 kHz, mono, 16-bit PCM, in 20-100 ms chunks.
studioAudio.on("data", (pcm) => ws.send(pcm));
}
if (msg.type === "partial") showCaption(msg.text);
if (msg.type === "final") signNext(msg.text); // POST it to /v1/sign, in order
if (msg.type === "error") console.error(msg.message);
});| Start field | Required | Details |
|---|---|---|
type | Yes | "start" |
token | Yes | Your key (sk_live_…) on a server, or a client token (ct_…) in a page or app. |
lang | No | What is spoken: rw (default), en, fr, sw, ja. |
sign | No | true: Signa signs every final itself and sends the signs down this connection (below), so you don't call /v1/sign. |
overlay | No | An overlay id: every final is also signed onto it, for the signer on air. |
to | No | With overlay: the sign language, default rsl. |
Messages from the server
| Message | Meaning |
|---|---|
{"type": "ready"} | Start sending audio. |
{"type": "speaking", "active": true} | Someone started (or stopped) speaking. |
{"type": "partial", "text": "…"} | What has been heard so far, still changing. Good for live captions. |
{"type": "final", "id": 3, "text": "…"} | Words that are settled: sign them. Ids only go up. |
{"type": "error", "message": "…"} | The session could not go on, and why. |
{"type": "done"} | Everything has been sent after a stop. |
Words are approved while the speaker is still talking: a word that comes out the same in two readings in a row is sent straight away, so signing starts within a second or two even for someone who never pauses.
Signs in the same connection
With "sign": true (and to) in the start message, each final is signed on our side and its signs come straight back: a {"type": "signing", "id": 3} message, the words as binary .sgn segments, then {"type": "signed", "id": 3, "words": 2}. It saves a round trip per final and keeps your code to one connection.
Captions never wait for signs
partial and final messages keep arriving at full speed while the signs for earlier finals are still on their way.Getting the audio in shape
Most sources need converting. ffmpeg turns almost anything, a studio feed included, into the right format:
ffmpeg -i <your input> -f s16le -ac 1 -ar 16000 -Read its output in chunks and send each chunk as one binary message.
Sessions and billing
- A session can run for up to 12 hours, or as long as your live minutes this month allow.
- If no audio arrives for 30 seconds the session closes, so keep sending through breaks (silence is fine).
- Time connected counts against your live minutes; signing the finals counts as characters, as usual.
Reconnecting