WebRTC

Plain text can only get you so far I guess.

Sometimes, audio and video (and fancy data channels) do really make a difference. I talk about some of my experiences here.

Basics

• WebRTC, Google's intro page to the tech.
• WebRTC on Wikipedia, with a nice little overview and of course the potential to make you spend as much time as you'd care to following links.

Standards

• WebRTC on W3, interesting in large part because it describes the browser API, which ends up shaping native libraries as well.
• RFC3550 - RTP: A Transport Protocol for Real-Time Applications.
• RFC3551 - RTP Profile for Audio and Video Conferences with Minimal Control.
• RFC3711 - The Secure Real-time Transport Protocol (SRTP).
• RFC4566 - SDP: Session Description Protocol.
• RFC5245 - Interactive Connectivity Establishment (ICE).
• RFC5389 - Session Traversal Utilities for NAT (STUN).
• RFC5888 - The Session Description Protocol (SDP) Grouping Framework.
• RFC7065 - Traversal Using Relays around NAT (TURN) Uniform Resource Identifiers.

Implementations

• Google webrtc library, the most common and popular one that I'm aware of.
• WinRTC, basically Google's webrtc library but with patches so it can be used with UWP rather than in just classic Win32 applications.
• MixedReality-WebRTC, no longer under development, also a version of Google's webrtc library with some UWP support, but also Unity and C++, with some tweaks to make it run well on HoloLens devices.
• Media Graph Libraries often come into play to get audio/video in and out of webrtc.

Codecs

• Open Media Acceleration or OMX, a set of standards / pluggable libraries that allow audio/video codec acceleration.
• OpenMAX AL for Android includes a bunch of notes on the Android NDK components.
• OpenMAX DL. Implementation of OpenMAX DL spec from ARM. This is used to support WebAudio for Chromium on Android.
• Doc on webrtc video codecs including scalable design.
• webrtc/modules/video_coding/codecs in sources, where the built-in codecs come together.

Signaling

How do you get two people that want to use webrtc to find each other and start an exchange?

• WebSockets
• SignalR, including a native client.
• Azure SignalR is a hosted version of a SignalR service.
• Microsoft Orleans, a highly scalable framework based on the actor model. Even without serialization/persistence, a great way to have locatable, programmable nuggets of state.
• Janus WebRTC Server, a lovely C-based that implements a webrtc server with a whole bunch of plugins to enable meetings, conferences, recording, streaming, and more - with securable and programmable configuration and administration APIs.

Home