If you've been following along, we've walked through how to read audio files, inspect basic formats, play samples, resample them, compress them with opus, decompress them, and save them for later. That's quite a bit!
Today we're going to be dipping our toes into realtime audio communication, which makes everything even more interesting!
Actually making a connection between two computers over the internet can be surprisingly hard.
We have to deal with discovery, routing, traversing firewalls, authentication, and more. This is quite a bit more than I want to talk about today, so we're going to take a shortcut.
We can set up a simple, basic/insecure connection between devices via a redis instance, like we did in the past.
If you don't want to deal with finding a hosted server, you can set up your own. Note that Redis is not officially supported on Windows, but it's simple to get one going if you have WSL2 (Windows Subsystem for Linux) which comes in handy for other development tasks.
Instructions on how to install WSL are here.
Instructions on how to install redis are here.
If you install via apt-get, the default port is 6379 and it will listen on all local interfaces. To adjust the configuration, run sudo vi /etc/redis/redis.conf The bind 127.0.0.1 -::1 line (with plenty of comments preceding it) makes it such that you can only access that locally. Let's uncomment that and set a password (it won't be encrypted, so use this for this quick test and then turn things off or harden this further by enabling encryption and setting up firewall rules for your environment). To the set password we will use for connecting, uncomment and adjust the requirepass line in the configuration. Then save your configuration and start with sudo service redis-server start (or restart if it was already running).
You can verify you password by running redis-cli and sending a ping command. If the reply is (error) NOAUTH Authentication required., you can then run AUTH youpassword, and try ping again to get a PONG reply. Finally, exit will stop the cli.
Next, allow incoming connections through the WSL firewall - this isn't blocked by default, so you might not need to do anything here. Check if it's enabled with sudo ufw status. If enabled, allow incoming connections on the redis post, sudo ufw allow 6379, and reload the rules with sudo ufw reload.
Next, allow incoming connections through the Windows firewall. Open Windows Defener Firewall wit Advanced Security, and create a new inbound rule: type port, protocol TCP, local port 6379, action allow, and profile private (or public depending on where you'll connect from). Name the rule something like Redis Inbound and click Finish.
Finally, get the IP address of WSL by running ip addr show - you'll see the loopback at 127.0.0.1 and then something like eth0 with a diffrent address.
Like last time, sources will be available at the opusfun repo and we'll be discussing the major sections of code.
We're going to have a single program that can act as an audio sender or receiver.
The sender works by using a single thread that connects to redis and reads from the microphone, resamples, encodes, and broadcasts the data. After some time, it simply exits.
The receiver is a bit more complicated. Where we want to end up is having it work by creating a thread that connects to redis and reads from it, decodes, and places the decoded samples in a jitter buffer. A second thread will periodically read from this buffer and play back samples. After some time, the main thread will ask both receiver threads to stop.
For today's post, however, we'll simply write everything out to disk to verify that we're getting data.
The sender executes RunSender and exits. It sets up a few objects (audio client, resampler, encoder, connection), runs in a loop for a while, and then cleans up.
The data format we use in our messages consists of SenderPacketHeader that has basic configuration information including the size of opus-encoded data, followed by the opus-encoded data itself.
This should all be enough to reconstruct the signal on the receiving end. TODO: point to interesting code
Receiving data is a pretty interesting thing in and of itself, so we're going to leave this for a future post. Before we get to it, we'll do some amount of housecleaning.
Once you have redis, you can build the project with build.ps1 like we've done in the past, and then launch two programs: the sender and the receiver.
To set up the redis connectivity, we'll set up two environment variables, REDIS_HOST and REDIS_PWD. From a cmd prompt, you would use set REDIS_HOST=ip.address.whatever and from powershell, $env:REDIS_HOST = "ip.address.whatever"
Try running them in different order and check out the resulting file sizes - if the sender starts first, the receiver will miss some data, because the server isn't storing it, only broadcasting it. Once it's broadcasting, however, you should get a steady stream.
Happy audio sharing!
Tags: audio