SIPSorcery.Cli 0.2.0-beta

SIPSorcery.Cli

The sipsorcery command: route and bridge live media streams between SIP, WebRTC and the major realtime fabrics (Cloudflare Realtime, LiveKit) and AI agents (OpenAI Realtime), built on the SIPSorcery library.

Where the sibling sipsorcery-diags tool is a probe/test/benchmark harness, this tool treats a stream as the primitive and lets you attach edges to it. It is human and agent friendly: every verb supports --json and meaningful exit codes.

Install

dotnet tool install -g SIPSorcery.Cli --prerelease

The installed command is sipsorcery:

sipsorcery route --from testpattern --to out.ivf -d 10

Usage

# Route (v0.1): wire a source edge to one or more sink edges over a stream graph. The stream is the
# noun; --from / --to attach edges to it. The graph repacketises, it does not transcode -- frames
# travel encoded from source to sink. See "The route verb" below.
sipsorcery route --from testpattern --to out.h264 -d 10         # generate H264, record to Annex B
sipsorcery route --from testpattern --to play                  # generate H264, watch in ffplay
sipsorcery route --from testpattern --to play --to out.h264    # tee: watch AND record at once
sipsorcery route --from whep:https://b.siobud.com/api/whep --to out.ivf --token key  # record a live WebRTC stream
sipsorcery route --from sip:music@iptel.org --to whip:http://localhost:8080/whip      # bridge a SIP call to WebRTC
sipsorcery route --from sip:music@iptel.org --to whip:http://localhost:8080/whip --scope  # ...and add an audio-scope video
sipsorcery route --from testpattern --to whip:https://b.siobud.com/api/whip --token key  # publish to Broadcast Box (H264 + opus, both now the default)
sipsorcery route --from testpattern --to web                   # watch in a browser: self-hosts a WHEP server + player on http://localhost:8080
sipsorcery route --from sip:music@iptel.org --to web:9000 --scope          # bridge a SIP call (opus if the peer supports it, else G.711 auto-transcoded) to a browser page with an audio-scope video
sipsorcery route --from testpattern --to livekit:demo-room                 # publish into a LiveKit room (creds via --url/--api-key/--api-secret or LIVEKIT_* env)
sipsorcery route --from livekit:demo-room --to web                         # ...and in a SEPARATE process, subscribe to that room and watch it in a browser
sipsorcery route --from testpattern --to cloudflare                        # publish into a Cloudflare Realtime SFU session (creds via --app-id/--token or CLOUDFLARE_APPID/CLOUDFLARE_API_TOKEN)
sipsorcery route --from cloudflare:<sessionId> --to web                    # ...and in a SEPARATE process, pull that session (id printed by the sink) and watch it in a browser

# Bridge: connect two duplex endpoints BOTH ways (full duplex). Where route is directional (--from/--to),
# bridge is symmetric and order-agnostic. v0.1 endpoints: web (a browser mic+speaker page), agent (an
# Azure speech-to-text -> LLM -> Azure text-to-speech voice agent), openai (the OpenAI Realtime API) and
# sip:<uri> (a phone call, transcoded G.711 <-> Opus). See "The bridge verb" below.
sipsorcery bridge web agent                    # open a browser and TALK to an Azure voice agent (e.g. Max Headroom)
sipsorcery bridge web openai                   # ...or talk to an OpenAI Realtime agent (OPENAI_API_KEY)
sipsorcery bridge web openai --avatar          # ...with a lip-synced video face (mouth driven by the speech envelope)
sipsorcery bridge sip:alice@example.com agent  # ...or let a PHONE caller talk to the agent (G.711 <-> Opus transcoded)

# The realtime fabrics (Cloudflare, LiveKit, OpenAI) have no standalone verbs: each is reached as a
# route/bridge edge, shown above. Probe/test commands for them live in the sibling sipsorcery-diags tool.

The route verb

route is the first cut of the stream router: rather than a one-shot diagnostic, it treats a media stream as the primitive and lets you attach edges to it. A graph is a source node, one or more sink nodes, and the directed edges between them; a routing decision is just a mutation of that graph. v0.1 implements the simplest shape — one source fanned out to N sinks (a free tee).

Two design rules carry through from the bigger picture:

• Repacketise, not transcode. Frames travel the edges still encoded (VP8 stays VP8, H264 stays H264). The tool stands in the media path only far enough to depacketise and re-emit; it never decodes to samples. Per-sample work (a real transcode) is the job of a future transform node backed by ffmpeg, not managed code.
• Edges are addressable. A --from / --to spec is either a bare keyword/path (out.ivf, play, null, -) or scheme:rest (whep:https://host/whep). Adding a transport edge is one new case in the edge factory and nothing else.

Edges:

The --scope audio-scope video

With a sip: source, --scope adds a second video track that is a live visualisation of the call audio — a moving waveform (--scope-mode waves, the default) or a scrolling spectrum (--scope-mode spectrum), sized with --scope-size WxH. The forwarded audio still travels repacketised; the scope is the one transform that decodes to samples, and that rendering plus its H264 encoding is delegated to an external ffmpeg process (the showwaves/showspectrum filters + libx264, the same way the sinks shell out to ffplay) — never a managed node. ffmpeg must be on the PATH or located with --ffmpeg-path.

sipsorcery route --from sip:music@iptel.org --to whip:http://localhost:8080/whip --scope --scope-mode spectrum

The web sink (watch in a browser)

--to web turns the CLI into its own viewer. It binds a local HttpListener that serves a small auto-connecting HTML player at http://localhost:8080/ (pick another port with web:9000) and answers the browser's WHEP request, streaming the graph's H264 video + audio over WebRTC. Unlike whip:, which dials out to a remote endpoint and blocks until connected, this listens for viewers — so it never blocks waiting for one. Open the URL whenever, in as many tabs as you like, each becoming its own peer connection fanned from the one graph tee. The browser opens for you (suppress with --no-open, or --json); the run continues until ctrl-c. The page and the WHEP signalling share one port (GET / serves the player, POST /whep does the offer/answer), and it binds localhost only.

sipsorcery route --from testpattern --to web                 # open http://localhost:8080 and watch
sipsorcery route --from sip:music@iptel.org --to web --scope

The livekit: and cloudflare: source and sink are both wired — publish into a room / SFU session, and subscribe to (pull) one — so running the two in separate processes is a full round trip (route --from testpattern --to cloudflare in one, route --from cloudflare:<id> --to web in another). sip: as a sink is recognised but reports it is not wired into route yet; it becomes a route edge in a later version, at which point route --from sip:… --to livekit:room bridges a call into a room with no bespoke command.

The bridge verb

route is directional — --from → --to. A conversation is duplex, and --from/--to reads wrong for that, so the duplex case has its own verb: bridge <a> <b> connects two endpoints both ways (a → b and b → a). The endpoints are symmetric / order-agnostic — bridge web agent and bridge agent web are the same.

v0.1 endpoints:

# Set the agent's credentials (or pass --azure-key/--azure-region/--llm), then:
export AZURE_SPEECH_KEY=…  AZURE_SPEECH_REGION=westeurope
export LLM_ENDPOINT=http://localhost:11434/v1/chat/completions  LLM_MODEL=llama3.2   # e.g. Ollama
sipsorcery bridge web agent

Open the page, grant the microphone, and talk to Max — full duplex (the browser's echo canceller keeps the agent out of its own mic, so you can interrupt). --voice, --persona, --greeting, --llm /--llm-model/--api-key tune it; without an --llm the agent just repeats what it heard.

Or talk to an OpenAI Realtime agent instead — same browser, no Azure or local LLM, OpenAI does the whole listen→think→speak in one hop:

export OPENAI_API_KEY=sk-…
sipsorcery bridge web openai                                 # talk to OpenAI Realtime
sipsorcery bridge web openai --voice verse --persona "You are a terse pirate."

Add --avatar to put a lip-synced video face (the SkiaSharp Max Headroom avatar) on the voice — the browser then shows his face alongside the audio, on either agent:

sipsorcery bridge web agent  --avatar                       # mouth driven by Azure TTS visemes (phoneme-accurate)
sipsorcery bridge web openai --avatar                       # mouth driven by the speech envelope (no visemes)

The two avatars differ in how the mouth is driven: the Azure agent has a viseme timeline, so its lip-sync is phoneme-accurate; the openai voice has none, so its mouth is audio-driven from the speech envelope (reads as generic talking that tracks the speech).

Either agent also answers the phone: a sip:<uri> peer places a SIP call and bridges the caller to the other side, so someone on a handset can speak to the agent. The endpoints are symmetric, so the agent can be on either side:

sipsorcery bridge sip:alice@example.com agent          # call alice, bridge her to the Azure agent
sipsorcery bridge sip:alice@example.com openai         # ...or to the OpenAI Realtime agent
sipsorcery bridge sip:1001@pbx.local agent --user 1000 --password secret   # authenticate the call

The SIP leg is G.711 (8 kHz); every other endpoint speaks 48 kHz Opus, so this peer transcodes G.711 ↔ Opus at its boundary (audio is light enough to transcode in managed code — only video is delegated to ffmpeg). The agent greets the moment the call is answered.

Adding --avatar turns it into a video call: the SIP leg offers a send-only H264 m-line and the agent's lip-synced face is sent down it, so a video softphone (Linphone, Zoiper, …) shows the avatar while you talk to it — the same --avatar that paints the face in the browser, no extra flag:

sipsorcery bridge sip:alice@example.com agent  --avatar    # Azure agent, viseme lip-sync, on the phone
sipsorcery bridge sip:alice@example.com openai --avatar    # OpenAI agent, envelope lip-sync, on the phone

An audio-only phone simply rejects the video m-line and the call carries on as audio. Pluggable/external avatars are planned.

Exit codes

Status

Early. The route engine carries audio and video over a single source → N sinks (a free tee), with whep:/sip:/livekit:/cloudflare: sources, whip:/web/livekit:/cloudflare sinks and an ffmpeg-backed audio-scope transform wired in. The bridge verb adds the duplex case (two endpoints both ways), with web, a local Azure voice agent, an openai Realtime agent and a sip: phone peer (transcoded G.711 ↔ Opus) as the endpoints, the agents each with an optional lip-synced --avatar. Streaming verbs default to -d 0 (run until ctrl-c); pass a positive --duration for a fixed window. Fan-in, richer transforms, the remaining transport edges, pluggable/external avatars, and the authoring layers above the graph (declarative routing, scripted policies, an external control API) are planned. Feedback and issues welcome on the SIPSorcery repo.