Rooms and lifecycle
Join, leave, presence, liveness, reconnect, cursor replay, media repair, quality-gated recovery, and stale-participant cleanup.
The participant fabric behind every live room.
People, AI agents, robots, devices, applications, and services join one room model. Identity, media, state, commands, recovery, and operations stay together.
humanOperatormedia + commands
AI agentVision workeranalysis ready
robotField unittelemetry live
deviceMQTT sensorstate synced
Mesh + SFU routingIdentity + admissionTyped room stateRelay + recoveryTelemetry + support
A production core for product-critical communication.
Adopt the stable media and signaling paths first, with explicit alternatives for room size, infrastructure policy, and deployment requirements.
Browser media
Mesh audio/video for small rooms and a production mediasoup SFU path for larger topologies, with screen share and file-backed media sources.
TURN/STUN
First-party relay infrastructure with tenant-aware metering, dynamic credentials, regional configuration, and relay validation tooling.
Data and messaging
Chat, receipts, typed signals, schema-validated data channels, chunked files, compression, and participant-aware routing.
Make live state a product primitive.
Coordinate participants and product interfaces without funding and maintaining a separate real-time state system for every workflow.
Signals and awareness
Ephemeral typed events and per-participant last-write-wins state with late-joiner snapshots for cursors, device status, UI state, and workflow events.
Shared realtime objects
Transport-neutral room objects for whiteboards, annotations, controls, lightweight application state, and agent-visible context. Server snapshots and persistence remain roadmap work; the model is per-key LWW, not a collaborative-text CRDT.
Realtime media surfaces
Turn room state, chat, telemetry, and external data into deterministic canvas surfaces that can be interactive, published as tracks, recorded, and replayed semantically. A dedicated PMG adapter remains roadmap work.
Presence surfaces and local TTS
Audio-reactive AI personas, capture-stream virtual cameras, speech lexicons, browser-native speech, and self-hosted Piper-WASM output with a publishable audio track.
Extend the participant model beyond browsers.
Workers, devices, native applications, and robots gain the same governed identity and room context as human participants.
Service participants
Node and Python worker adapters for transcription, translation, moderation, analytics, and custom AI; signaling/data clients for Go, Rust, and .NET.
Device bridges
MQTT v5, serial, BLE, and USB sidecars translate external device traffic into the same typed protocol envelope used by rooms.
Native and mobile SDKs
React Native, Flutter, Swift, Kotlin, and C++ paths share generated protocol types and signaling APIs. Media maturity varies by runtime.
Robot participants
ROS2 bridges, teleoperation controls, fleet rooms, telemetry, command topics, services, and WHIP/SFU camera integration for guided deployments.
Add media workflows without redesigning transport.
The SFU remains the transport plane. Selected streams branch into explicit processing only when a product experience requests a scene or workflow.
Demand-driven scene runtime
Room-scoped scene request, update, and release messages start pluggable workers on demand and tear them down when no longer needed.
Live and batch composition
Grid, active-speaker, PiP, and caller-defined layouts run through the Rust/GStreamer compositor path, including live layout updates and composited producer output.
Live compositor path
Bounded live composition supports mediasoup and the integrated engine with persistent pipelines, no-restart track changes, scene updates, and output published back into the room. Batch composition remains first-party-only.
Current boundaries
RTMP, SRT, HLS, and LL-HLS control-plane and composited-feed APIs have shipped, but stock end-to-end consumer validation is still open. Broader codecs, GPU backends, declarative scenes, per-viewer fan-out, SIP, and durable semantic renders remain roadmap work.
Operate real-time as product infrastructure.
Identity, observability, policy, support context, and usage data remain part of the platform rather than becoming downstream operational debt.
Identity and admission
JWT, JWKS, OIDC, OAuth introspection, RBAC mapping, waiting-room tickets, moderator approval, abuse limits, and targeted token revocation.
Diagnostics and support
Peer state, ICE path, topology, quality trends, redacted support bundles, consented connectivity checks, local bundle evaluation, and repeatable RTC test matrices.
Telemetry and usage
OpenTelemetry, Prometheus, tenant-labeled usage records, a reference rollup consumer, and a region-aware illustrative metering dashboard. Durable billing stays customer-owned.
Admin and global monitoring
Room and participant controls, worker visibility, regional capacity, process pressure, live relationships, SFU diagnostics, and TURN totals.