Advanced Configuration

Ready to dive in with some more advanced features? Here are some quick start tutorials. Heads-up: assistance with advanced configurations is not covered by regular maintenance support and would constitute professional services if you require it. Remember you can always experiment, then reset the engine config and it will go back to the way it was when you started up Streamwell.

Performance Tuning

Larger-scale deployments will require some performance tuning on the server itself once viewership or stream counts reach into higher numbers.

These optimizations are all to do with OvenMediaEngine, the streaming engine that powers Streamwell. They are borrowed with gratitude from these OME guides (Troubleshooting and Performance Tuning) which any sysadmins should dive into deeper to realize the best performance for your server.

Choosing the Right Host OS

Streamwell works great on any modern Mac or Windows computer running Docker. However, there are network performance limitations in these operating systems (especially when factoring in the emulation layer they run for Docker) that limit them to serving a certain amount of concurrent viewers - in our testing we haven't met a modern machine that couldn't easily handle 25 viewers at 1080p / 4Mbps. But for anything more than that (25 simultaneous viewers or 100Mbps throughput) we recommend using a native Linux-based host OS instead of macOS or Windows. Read on for more tuning and testing advice.

Adjusting Thread Counts Across Cores

The 'StreamWorker' process is one of the hardest-working components of OME. In your engine configuration (under Administration -> Server -> Engine Control), increase the various "worker" thread counts, especially StreamWorker count to match the number of CPU cores you have available:

<StreamWorkerCount>8</StreamWorkerCount>

If you are streaming primarily to HLS viewers, you should also set the HLS Worker Counts appropriately.

It is assumed that the CPU cores provided are powerful enough to sustain single-threaded operations like decoding. Mobile CPUs, efficiency cores or virtualized processors should only be used if you are certain they are not choking at the per-thread level.

A great way to check this is to run the top command on your host, get the PID of "OvenMediaEngine" then run top -H -p 12345 where 12345 is the PID. It will show you the individual threads running within the application. If any of these are hitting 100% that means the individual threads are overloaded and your CPU cores are not fast enough for the task at hand.

Tuning the Linux Kernel

By default, the Linux Kernel cannot handle high-throughput. There are some system-level configuration files you can edit with more optimized values:

/etc/sysctl.conf

fs.file-max = 100000
net.core.somaxconn = 65535
net.ipv4.tcp_max_tw_buckets = 1440000
net.ipv4.ip_local_port_range = 1024 65000
net.ipv4.tcp_fin_timeout = 15
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_max_syn_backlog = 3240000
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.core.rmem_default = 16777216
net.core.wmem_default = 16777216
net.core.optmem_max = 40960
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
net.core.netdev_max_backlog = 50000
net.ipv4.tcp_max_syn_backlog = 30000
net.ipv4.tcp_max_tw_buckets = 2000000
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_fin_timeout = 10
net.ipv4.tcp_slow_start_after_idle = 0
​
/etc/security/limits.conf

* soft nofile 1048576
* hard nofile 1048576

Performance Testing

Here is a great tool for testing multiple connections to the server: http://files.streamwell.net/tester.zip

To run the test:

  • Install or confirm you are running GO on your test client: https://go.dev

  • You will need to disable SignedPolicy in the Engine config temporarily for the test to run. Under Administration -> Server -> Engine Control -> Configure Engine, comment out or cut the <SignedPolicy> section of the config.

  • Choose an open channel and copy the stream key only (minus the signature, everything including "?=" forward)

  • Start your test stream and run OvenRTCTester like so (use wss: if you are connecting to a hostname with HTTPS enabled):

$ go run OvenRtcTester.go -url "ws(s)://<SERVER-IP-OR-HOSTNAME>/live/streamKey" -n "25"

  • Adjust the -n value up bit by bit to add more simulated viewers as your tests complete successfully. Please note you will need to conduct these tests on a fast machine with fast networking as it is playing the role of potentially hundreds of viewers. A good way to conduct this test is to spin up a VPS on a reliable service like DigitalOcean and run the test from there. This ensures that your local network / computer is not acting as a bottleneck.

Use of Live Thumbnails

One of the coolest features in Streamwell is the 'birds-eye-view' on the home screen, however the CPU load to generate these live thumbnails can be considerable and this feature may not perform so well on resource-constrained or very high-throughput servers. Each incoming stream will use up to 1 full CPU thread to decode and 1 full CPU thread to encode thumbnails if this is enabled.

To disable live thumbnails, you just need to comment out the indicated section in the engine configuration. There is a toggle to stop displaying them in the web client, under Administration -> Customization -> Preferences.

For streams above 1440p / H264 and 1080p / H265 you almost certainly will want to disable this feature.

SRT Encryption

SRT passphrase encryption is available, along with 50+ other native SRT socket options.

To enable SRT encryption, simply uncomment this section in the engine configuration and change the passphrase to something you are familiar with:

<!-- <Options>
<Option>
<Key>SRTO_PASSPHRASE</Key>
<Value>thisismypassphrase</Value>
</Option>
</Options> -->

N.B. SRT Passphrases must be between 10-80 characters in length or the Engine may not start.

You can also add additional SRT socket options in the same manner as shown above. Only the SRTO_PASSPHRASE option has been tested with Streamwell but you can really get into fine-tuning if you are brave!

Different encoders have different ways of setting the SRT encryption. As an example using our old friend OBS, you can enter the passphrase as shown (the stream key itself stays the same):

Proxy Streaming / ABR

To enable Proxy Streaming, first check the engine configuration and uncomment the sections related to ABR:

<!-- <Rendition>
<Name>Proxy</Name>
<Video>proxy_video</Video>
<Audio>opus_audio</Audio>
</Rendition> -->
 <!-- <Video><Name>proxy_video</Name>
 <Codec>h264</Codec>
<Bitrate>500000</Bitrate>
<Profile>baseline</Profile>
<Preset>slow</Preset>
<Width>640</Width>
</Video> -->

For a quick-start, simply uncomment these and commit the new configuration, then restart the engine. To use your new Proxy Streaming config in the web client, enable Proxy Streaming under Administration -> Server -> Features.

You can edit the encoding settings and add additional <Rendition> entries, each linked to a <Video> component by name. Just remember that for each entry in the <Rendition> list your server will need to transcode another video stream for each channel that is live. This can quickly consume CPU resources at an exponential level, so please be aware of this as you enable transcoding functionality.

See more on this via the OvenMediaEngine documentation.

Timezone Configuration

By default, the server uses an internal clock set to UTC ('Coordinated Universal Time'). You can change this value with an environment variable ("TIME_ZONE") when you run the Streamwell Docker container

Besides the default value of 'UTC', the following values are supported:

America:

  • America/Argentina/Buenos_Aires

  • America/Chicago

  • America/Los_Angeles

  • America/New_York

  • America/Toronto

  • America/Vancouver

Africa:

  • Africa/Abidjan

  • Africa/Accra

  • Africa/Algiers

  • Africa/Cairo

  • Africa/Casablanca

  • Africa/Johannesburg

  • Africa/Lagos

Asia:

  • Asia/Dubai

  • Asia/Kolkata

  • Asia/Shanghai

  • Asia/Tokyo

Europe:

  • Europe/Amsterdam

  • Europe/London

  • Europe/Moscow

  • Europe/Paris

  • Europe/Rome

Pacific:

  • Pacific/Auckland

  • Pacific/Honolulu

  • Pacific/Sydney

Firewall Config

Windows and macOS offer built-in firewalls in their system settings, where you can allow connections to the Docker application which should allow Streamwell to function if you have these firewalls enabled.

On Ubuntu and many flavours of Linux, the 'ufw' firewall is built-in. Here is an example of configuring ufw for Streamwell (including SSH access on port 22 - if you enable ufw without this you may lock yourself out of your server!):

ufw allow 22,80,443,1935,1936,3333,3334,8000,8001,10010/tcp
ufw allow 9999,10010/udp
ufw enable -y

Once your system is in production, you should disable / block any unused ports as a best security practice.

Enabling Legacy HLS

Certain external players and devices (like ROKU) require a legacy version of the HLS standard when playing the stream back on those devices. A bit of manual configuration is required to enable this:

1) Under Administration -> Server -> Engine Control -> Configure Engine, there are two sections marked <!-- Legacy HLS --> with <HLS></HLS> configurations that are commented out. Uncomment these, commit the configuration, then restart the engine.

2) Toggle the legacy HLS links in the interface with cheat codes legacyHlsLinksEnabled or legacyHlsLinksDisabled (enter them under Administration -> Customization -> Set Custom Title).

3) A new 'Legacy HLS Link' will appear in the stream info panel on each channel. Ta-da!

Note: HLS streams need to fill their playlist before they are available to stream. For legacy HLS this takes up to 1 minute. You can change the segment duration and count in the engine configuration but it is not recommended to lower it beneath a duration of 3 and a count of 5.

PreviousOvenMediaEngineNextTerms and Conditions

Last updated