TomatoRTC | Real-Time Participant Fabric

One operating model for the whole live product.

TomatoRTC treats people, agents, robots, devices, applications, and services as first-class participants. A shared model for identity, permissions, media, state, commands, telemetry, and diagnostics replaces the parallel systems that otherwise accumulate around every new real-time workflow.

Humans

Operators, experts, customers, clinicians, support teams, and field workers in live product workflows.

AI agents

Assistants, transcription, translation, moderation, vision, and automation workers that join as room participants.

Robots

ROS2 robots, teleoperation sessions, fleet rooms, robot state, commands, services, and camera feeds.

Devices

MQTT, serial, BLE, USB, industrial gateways, sensors, embedded systems, and field hardware.

Services

Recording, analytics, workflow engines, backend automation, and custom product logic.

Browser SDK Server SDK Node workers Python workers React Next.js Vue Svelte Nuxt Solid React Native path Flutter path Swift/Kotlin/C++ paths ROS2 bridge MQTT bridge Serial/BLE/USB bridges WebTransport SSE+POST Connect-RPC SFU TURN WHIP/WHEP AI workers Voice AI path Observability Admin Usage telemetry

Differentiate the product without multiplying the stack.

Meetings are one room pattern. TomatoRTC supports products where people, agents, robots, devices, and services must share presence, media, state, commands, and telemetry as one coherent live system.

A

Differentiate beyond generic calling

The same room model that supports live video can coordinate operators, AI workers, robots, connected equipment, and backend services as product-native participants.

B

Consolidate the live stack

Identity, permissions, presence, state, commands, routing, telemetry, recovery, and diagnostics remain in one system instead of becoming long-term integration debt.

C

Control deployment and economics

Cloud, edge, private-network, regulated, and managed operating models preserve choices around latency, privacy, infrastructure ownership, and unit economics.

One product layer for the hard real-time primitives.

TomatoRTC brings rooms, identity, permissions, media, data, telemetry, diagnostics, deployment control, and multi-runtime SDKs into one governed platform instead of a portfolio of loosely connected services.

Rooms & lifecycle

Join, leave, reconnect, resume, recover media, detect stale participants, and keep live sessions understandable.

Identity & permissions

Tenant tokens, participant tokens, roles, duplicate participant protection, and room-scoped authority.

Signaling & connectivity

WebSocket, WebTransport, SSE+POST, Connect-RPC, long-polling, MQTT bridges, and serial, BLE, and USB sidecars.

Presence & awareness

Roster state, shared signals, late-joiner snapshots, device status, robot state, and UI coordination.

Voice, video & screen media

Browser publishing, screen share, file media sources, SFU paths, quality stats, and production RTC support.

Data channels & files

Typed payloads, structured messages, chunked file transfer, compression, and participant workflows.

Diagnostics & support bundles

Call quality, topology, timeline, peer state, browser capabilities, redaction, and exportable support context.

Telemetry & metering

Tenant-labeled events plus a reference consumer and region-aware dashboard for evaluating rollups. Pricing, invoicing, and durable billing remain customer-owned.

SFU, TURN & WHIP/WHEP

Media routing, relay connectivity, HTTP ingest and playback paths, and a swappable media-engine boundary.

Multi-language SDK core

Browser, Node, Python, Go, Rust, .NET, framework adapters, generated protocol contracts, and native SDK paths.

Extend the product without rebuilding the real-time core.

Once every actor shares the same room model, new AI, robotics, device, media, and service workflows become extensions of a stable platform rather than new integration programs.

AI as a first-class participant Foundation

Bots, transcription, translation, moderation, vision, and custom AI workflows join rooms with identity, presence, permissions, data, and telemetry.

Voice AI foundations Foundation

Voice presence, VAD, speech events, AI worker hooks, and local TTS provide reusable pieces. They are not a server-side conversational agent runtime, which remains roadmap work.

Robot participants Demo-ready

ROS2 robots join TomatoRTC rooms with telemetry, commands, services, state, and camera video in the same participant fabric.

Browser teleoperation Demo-ready

Keyboard and gamepad controls map to robot command topics with deadman behavior, rate limits, and operator HUDs.

Fleet rooms Demo-ready

Multiple robots appear in one room with targeted commands, shared state, and fleet dashboards.

ROS2 bridge Demo-ready

Native rclpy bridge for on-robot deployments plus a rosbridge sidecar path for edge and cloud deployments.

WHIP robot camera path Demo-ready

HTTP session lifecycle, SDP negotiation, trickle ICE, TURN Link headers, and producer/consumer wiring support guided camera integrations. Production-scale hardening remains in progress.

Local device bridges Production path

Serial, BLE, USB, and MQTT bridge sidecars turn physical devices into room participants through the same ProtocolEnvelope stream.

Industrial gateway pattern Production path

Run a thin sidecar on a gateway host and connect local devices to the fabric without embedding WebRTC in firmware.

First-party TURN Production path

Self-hostable relay infrastructure with tenant-aware metering and operational visibility.

Swappable SFU engine Production path

Use the production mediasoup path today while the first-party media-plane hardens behind the same control-plane boundary.

First-party media plane Demo-ready

Native UDP, ICE-lite, DTLS, SRTP, RTP/RTCP routing, pacing, replay protection, and diagnostics under first-party control; evaluate against target browsers and networks.

Typed data channels Production path

Structured real-time payloads, file transfer, compression, and participant-to-participant data workflows.

Shared signals & awareness Production path

Ephemeral signals and participant state let apps coordinate cursors, device status, robot state, UI state, and workflow events.

Service participants Production path

Node, Python, Go, and .NET provide production signaling/data paths for workers and backend participants; Rust and native media maturity remain runtime-specific.

Browser AI perception Foundation

Optional client-side AI plugins for local perception, VAD, speech recognition, MediaPipe, and browser-native AI APIs.

Recording workers Foundation

Mesh and SFU ingest adapters, controller/worker paths, and a synthetic artifact demo exist; complete live-room orchestration and durable storage do not yet.

Programmable Media Graph Foundation

Bounded frame-bus, scene-runtime, and composition slices support product-controlled workflows. Live composition supports mediasoup and the integrated engine; batch remains first-party-only.

TomatoRTC Media DSP Engine Roadmap

Optional signal processing and readiness intelligence for audio, video, screen-share, recordings, and AI capture.

Global operations dashboard Demo-ready

Observe signaling, rooms, clients, workers, regional capacity, SFU/TURN pressure, relationships, and deployment health; history is in-memory by default.

Support and test tooling Production path

Export redacted support bundles, run consented connectivity checks, inspect bundles locally, and exercise repeatable browser, topology, and codec matrices.

Licensing & policy Production path

Signed local license validation, role and capacity enforcement, redacted operator status, renewal controls, and explicitly entitled air-gapped operation.

C2PA / provenance Foundation

Content provenance signing tools for future trusted media and artifact workflows.

Digital twin / IoT rooms Foundation

MQTT and device bridge patterns let sensors, dashboards, operators, and automation share one live state model.

Where the platform creates leverage.

The strongest entry points are workflows where coordination quality, response time, differentiated automation, and operational visibility directly affect customer or business outcomes.

AI-assisted live operations

Operators and AI workers share the same room for transcription, moderation, translation, summarization, and workflow automation.

Voice AI room path

Voice presence, transcription, TTS, and conversational-agent architecture for products where AI needs to listen, speak, and react in real time.

Robot teleoperation

Browser operators control ROS2 robots while receiving video, telemetry, robot state, and service responses.

Industrial device rooms

Gateways, sensors, operators, dashboards, and backend services share state and commands in real time.

Expert support and field service

Support teams, customers, devices, diagnostics, and AI assistants join one live troubleshooting session.

Healthcare and regulated collaboration

Self-hosted rooms with controlled media, signaling, identity, diagnostics, and deployment policy.

Digital twins and simulation

Simulated and physical systems publish live state into rooms where humans and agents can observe and command.

AI workers and recording pipelines

Backend participants transcribe, summarize, moderate, translate, record, analyze, or enrich sessions.

One control plane. Fewer integration boundaries.

Product surfaces connect to one room model while TomatoRTC handles participant lifecycle, identity, permissions, media, data, bridges, telemetry, diagnostics, and operational control beneath it.

Applications

Meeting appsTeleoperation consolesFleet dashboardsAI workspacesIndustrial control roomsExpert workflows

Participant fabric

RoomsIdentityPermissionsPresenceAwarenessCommandsEventsTelemetryDiagnostics

Media & data plane

WebRTCData channelsSFUTURNWHIP/WHEPRecordingPMGE2EE controls

Bridge & SDK layer

BrowserNodePythonGoRustSwiftKotlinC++.NETROS2MQTTSerialBLEUSB

Operations

AdminMonitoringUsage telemetrySupport bundlesDeployment policyRelease checks

Simple to adopt. Inspectable in production.

A typed client keeps integration direct. When networks or devices change, TomatoRTC exposes lifecycle, selected path, recovery, and redacted support context instead of collapsing operational risk into a generic error.

browser-room.tsTypeScript

import { createRtcClient } from "@rtc-sdk/client-browser";

const client = createRtcClient({
  signalingUrl,
  authToken,
  displayName: "Field operator"
});

const room = await client.joinRoom("inspection-07");
await room.publishCamera();
await room.publishMicrophone();

// Room events, quality, topology, and diagnostics
// remain available to the application and its operators.

Operational timelinemedia healthy

10:18:31joinedidentity accepted · room state replayed

10:18:44offlinesignaling connection interrupted

10:18:45resumingcursor replay · media repair requested

10:18:46relaydirect ICE path failed · TURN selected

10:18:47restoredmedia confirmed healthy

10:18:49readyredacted support bundle available

Representative sequence using shipped reconnect, media-recovery, diagnostics, ICE-path, and support-bundle surfaces.

Developer overview Read the evidence matrix

Operational control is part of the product value.

TomatoRTC includes the controls required when real-time rooms become product infrastructure: tenant authority, identity, observability, deployment ownership, retention choices, and service enrichment.

Tenant-aware identity

JWT, JWKS, OIDC, OAuth introspection, tenant-token exchange, participant tokens, role permissions, and duplicate participant protection keep rooms scoped to the right authority.

Admin and operations surface

Admin APIs, room short links, global monitoring, health checks, support bundles, OpenTelemetry exporters, and usage trails give operators visibility without reverse-engineering session state.

Deployment ownership

Run the participant fabric in a customer cloud, at the edge, on private networks, or through a managed path, with explicit TURN, SFU, ports, firewall, health, and cost-planning guidance.

Persistence and storage adapters

Chat, room, registry, recording, and support workflows are designed around pluggable storage and customer-owned retention policies rather than a single hosted data model.

Service enrichment hooks

GeoIP enrichment, link previews, content provenance, face verification, effects, and browser AI perception can be added as product features without changing the room model.

Customer-ready packaging

Framework adapters, widgets, token endpoint patterns, SDK bundles, runbooks, and support-bundle workflows make it easier to move from demo to customer integration.

Efficient media and AI on one operational fabric.

Quality, latency, routing, and AI behavior remain part of the room model, giving product and operations teams a shared system for performance decisions instead of disconnected call and automation pipelines.

Route media, do not remix by default

Browsers handle capture, preprocessing, encoding, simulcast, and SVC. The SFU routes RTP and selects layers, avoiding the cost and latency of an always-on decode-mix-encode MCU.

Latency-aware packet path

The first-party media path includes packet pacing, audio-priority forwarding, retransmission cache, layer selection, and routing diagnostics behind the same SFU control boundary.

Quality signals operators can act on

QoE scoring separates network, device CPU, relay, codec, and routing issues, then exposes that context through diagnostics, support bundles, monitoring, and usage telemetry.

AI is native to the room

AI workers join as participants with identity, presence, permissions, data outputs, room context, telemetry, and lifecycle, instead of living as disconnected webhooks beside the session.

Model-provider freedom

Use local models, private inference, managed AI APIs, or custom STT, LLM, TTS, moderation, and vision providers through pluggable adapters. Hyperscalers and foundation models are options, not dependencies.

Privacy routing by design

Local providers can keep sensitive room context inside customer infrastructure, while external providers run behind product-defined redaction, policy, and routing controls.

Add media workflows without committing the entire product to an MCU.

TomatoRTC adds a programmable media-workflow foundation to the room and SFU core. Selected streams can branch into voice AI, recording, moderation, broadcast, replay, and composition without forcing every participant through an always-on decode-mix-encode path.

Live room People, agents, robots, devices

Media, state, commands, transcripts, quality signals, and room context.

Foundation Programmable Media Graph

Explicit workflows branch from the SFU only when a product experience requires processing.

Product outputs Scenes, agents, archives, streams

Composited scenes and broadcast control APIs today; stock end-to-end broadcast validation, broader voice workflows, and semantic replay remain expansion work.

SFU stays the transport plane Workflows are opt-in Processing is policy-gated Outputs stay tied to room context

Compose only when needed

Request a scene for a webinar stage, operator view, vertical replay, broadcast output, or compliance capture without making composition the default path.

Keep transport efficient

The SFU routes live media. PMG branches selected streams into workflow nodes for specific jobs instead of turning the room into a decode-mix-encode pipeline.

Turn media into AI workflows

Connect speech events, STT, LLM calls, TTS, interruption handling, transcript state, moderation, and room audio outputs through one explicit media workflow.

Recording with context

Capture tracks, participant state, timestamps, quality signals, and scene metadata so recorded artifacts can reflect the room workflow, not just a flat media file.

Moderate, redact, and route

Add policy-gated processing for redaction, content checks, provenance, AI review, and workflow-specific decisions before media or metadata leaves the controlled environment.

Broadcast and replay

RTMP, SRT, HLS, and LL-HLS control-plane and composited-feed APIs are shipped as a bounded path. Stock consumer-receives-output validation, SIP, and semantic replay remain future work.

Media intelligence designed to follow the room.

The proposed TomatoRTC Media DSP Engine is an optional signal-processing layer for live media: audio, video, screen-share, image, and network quality signals that become room-aware diagnostics, readiness scores, recording reports, and AI-capture context. The product story is media intelligence that follows the participant fabric from capture through routing, workflows, and artifacts.

Roadmap Media signals Capture quality at the source

Analyze audio, video, screen, and network conditions close to where media is produced, routed, or recorded.

DSP runtime Local analysis before cloud AI

Use Rust/WASM, AudioWorklet, browser APIs, native workers, and server pipelines for explainable media events.

Room intelligence Make media state actionable

Feed room UI, PMG workflows, recordings, support bundles, voice agents, and post-call reports with quality context.

Optional by default Audio, video, screen, network Local DSP before cloud AI Room-aware media context

Signal intelligence, not just stats

Turn raw media conditions into explainable events, diagnostics, scores, and recommendations for participants, operators, agents, and recordings.

Rust/WASM DSP core

The proposed engine uses shared DSP primitives across browser, server, recorder, native, and embedded runtimes, with local analysis before cloud AI.

Video and screen readiness

Extend the same model to brightness, blur, frozen frames, crop safety, screen readability, slide contrast, and visual capture quality.

Reports for recordings and AI

Use media readiness signals to improve transcription quality, voice-agent capture, recording diagnostics, post-call reports, and support bundles.

A product boundary leadership and engineering can evaluate.

Public readiness labels separate production-ready paths from demo-ready surfaces, foundations, and roadmap work so technical diligence, rollout scope, and commercial expectations can begin from the same facts.

Broad platform, explicit limits. Every production rollout still requires target-browser, network, identity, and capacity validation.

Production-ready

Browser SDK, WebSocket/WSS signaling, room lifecycle, mesh media
Production mediasoup SFU path and first-party TURN/STUN relay
Chat, typed signals, awareness, data channels, and file transfer
JWT/JWKS/OIDC auth, admin APIs, telemetry, diagnostics, support bundles
Shipped web framework adapters and MQTT, serial, BLE, USB sidecars
Node, Go, C#, and Python signaling/data SDK paths

Demo-ready

First-party SFU/media plane under controlled hardening
Admission gates and distributed revocation workflows
Admin and global monitoring user interfaces
ROS2 bridge, browser teleoperation, fleet rooms, and guided WHIP/WHEP workflows

Foundation

Programmable media graph, scene runtime, and composition workers
Recording controller, worker, muxing, and SFU ingest building blocks
Shipped shared-object and media-surface packages with server/PMG integration gaps
E2EE manager, browser AI perception, licensing policy, C2PA tools
Native and Rust/C++ SDK surfaces with runtime-specific media limits

Roadmap

Complete server-side conversational voice-agent runtime
Broad production recording and durable artifact operations
Durable multi-instance registry, pub/sub, and automated capacity routing
Stock end-to-end broadcast deployment and broader programmable-media portability
Media DSP engine and expanded hardware/device validation matrices

Evaluate TomatoRTC against a real product workflow.

Share the business outcome, participants, control boundaries, media paths, deployment constraints, and operating expectations. TomatoRTC can then be evaluated against a concrete architecture and product boundary.

Talk to us hello@tomatortc.com

Frequently asked questions

Straight answers for evaluation, procurement, and technical planning conversations.

Is TomatoRTC just a WebRTC SDK?

No. WebRTC is one transport and media layer. TomatoRTC is a real-time participant fabric: rooms, identity, permissions, presence, state, media, data channels, telemetry, diagnostics, service participants, and bridges for agents, robots, and devices.

What is a real-time participant fabric?

It is a common runtime where different real-time actors — people, AI agents, robots, devices, browsers, native apps, and backend services — join the same room model and communicate through media, data, state, commands, and events.

Is robotics actually part of the product?

Yes. TomatoRTC includes ROS2 bridge packages, teleoperation, simulation-first examples, robot state, command topics, and guided WHIP/SFU camera paths. Robotics is demo-ready and still needs target-hardware and field validation.

Is TomatoRTC self-hosted or managed?

It is designed to run on customer-controlled infrastructure, with a managed path available where appropriate. The core value proposition is ownership, visibility, deployment control, and the ability to extend the participant fabric as the product evolves.

What operational features are included?

TomatoRTC includes tenant-aware auth, participant tokens, admin APIs, room short links, health checks, global monitoring, OpenTelemetry exporters, usage telemetry, diagnostics, and support bundles. Storage, persistence, retention, and deployment policy are designed to fit customer infrastructure instead of forcing one hosted model.

What participants can join?

Humans, browsers, native apps, AI agents, backend workers, recording workers, robots, MQTT devices, local serial/BLE/USB devices, and services can participate through the shared room and protocol model.

Is AI a first-class part of the platform?

Yes. AI workers and agents are modeled as participants, not side-channel webhooks. They can join rooms with identity, presence, permissions, data-channel outputs, telemetry, and workflow state. Browser AI perception is optional and isolated from the core browser SDK.

Does TomatoRTC depend on one AI provider?

No. TomatoRTC uses a pluggable AI adapter model. Teams can bring local models, private inference, managed AI APIs, or custom STT, LLM, TTS, moderation, and vision providers. Hyperscalers and foundation models can be used, but they are not the architectural dependency.

What is the status of voice AI?

Voice-related building blocks include presence surfaces, browser VAD and speech events, AI worker hooks, and local TTS. A server-side conversational voice agent is proposed roadmap work; no runtime exists yet.

What is the TomatoRTC Media DSP Engine?

The TomatoRTC Media DSP Engine is a planned optional signal-processing layer for media diagnostics, explainable quality events, visualization, and readiness scoring. It is product direction, not a currently available runtime.

How does TomatoRTC approach performance?

The performance model keeps browsers responsible for capture, preprocessing, encoding, simulcast, and SVC while the SFU routes RTP and selects layers. The first-party media path adds low-latency pacing, audio-priority forwarding, QoE cause classification, diagnostics, support bundles, monitoring, and usage telemetry.

What connectivity paths does TomatoRTC support?

WebSocket/WSS is the primary production signaling path. Additional WebTransport, SSE+POST, Connect-RPC, and long-poll implementations exist at different maturity levels. Device paths use MQTT, serial, BLE, and USB sidecars; media uses browser WebRTC, SFU/TURN, and guided WHIP/WHEP paths.

Does TomatoRTC include an MCU?

TomatoRTC does not use a traditional always-on decode-mix-encode MCU. Composition and bounded broadcast control already converge on the Programmable Media Graph foundation; stock production egress, broader recording, moderation, and voice workflows expand from the same demand-driven model.

Which SDKs are ready?

The browser and web framework SDKs are the primary product paths today, with Node, Go, C#, Python, and Rust service/client paths for workers and backend participants. Native Swift, Kotlin, C++, React Native, and Flutter paths are available selectively, and media support varies by platform.

What is available today?

The production-ready paths include the browser SDK, WSS signaling, rooms, mesh media, mediasoup SFU, TURN, presence, chat, data channels, file transfer, tenant auth, diagnostics, telemetry, and support bundles. Robotics and the first-party SFU are demo-ready; the status page names every current limit.

Does TomatoRTC replace my SFU?

It includes a production mediasoup-backed SFU path today and a first-party media-plane path behind a swappable engine boundary. Teams can start on the stable production path and evaluate the first-party engine where deeper control is useful.

Does TomatoRTC support WHIP/WHEP?

TomatoRTC includes demo-ready WHIP/WHEP HTTP gateways with SDP negotiation, trickle ICE, TURN Link headers, and producer/consumer wiring. Use guided evaluation for ingest and playback requirements while production-scale hardening continues.

How do we get access?

Email hello@tomatortc.com and we will follow up with technical details and access.

Real-time rooms for every participant in your product.