Guide to Reducing Latency (Delay)

Introduction:

Stream latency, the delay between the source video and the video playout at the stream destination, is the sum of several delays: time spent processing the video stream on the encoder side, and the latency and quality of the connecting network(s).

The time delay over the network is dependent on the network capacity and distance between nodes but is generally not something of which one has a lot of control. Long distance transmissions can be improved by utilizing the Streambox Cloud Services.

Below are several settings that can be changed to reduce the time spent processing the video. Most of these settings are on the Encoder, but the last pair of settings are on the Decoder/Media Player end of the stream.

Reducing these settings too greatly can affect quality of the video or reliability of the stream, so the user should find the best compromise between quality/reliability and latency.

Encoder Network Settings:

Network Type

Wired networks are superior in latency and reliability to wireless networks. Wireless networks often must re-send data to battle interference and crosstalk, and while they generally do this very well, a real-time application such as streaming can expose its flaws. The re-sending of data increases latency.

VBR Buffer

Measured in seconds and set from 0.0 to 8 seconds, VBR Buffer is a measure of the Encoder buffer size. If the value is set too low, it can result in poor motion processing; higher-motion videos require higher VBR buffers. Typically, values in the 0.0 to 0.5 range are adequate for HD video, and 0.2-0.7 for UHD/4K video.

The more powerful the machine doing the encoding, in terms of CPU and RAM available, the lower one can set the VBR buffer. Adjust higher or lower to find a value that offers the best balance given the motion-quality and latency desired.

FEC, Forward Error Correction

FEC is used in place of or in addition to LDMP as a correction method. It adds additional packets for redundancy, which results in slightly increase latency. FEC sends redundant data packets which can be converted by the decoder/player into replacements for packets that failed to arrive. The addition of redundant packets means a reduction in the rate of stream packets.

Each FEC option is designated as Parity or R-S (Reed-Solomon). Parity mode is less efficient and is more of a legacy mode. R-S is more efficient and preferred. Each entry contains a percentage amount, and that is the percentage of the target bitrate which is taken away from the video/audio stream and dedicated to redundant packets. R-S entries also contain a pair of numbers such as 16/2 or 32/5 which represent the frequency at which redundant packets are generated. So, 16/2 might be read to mean “For every 16 packets of video/audio, 2 redundant packets are added. The FEC percentage associated with 16/2 is 11%, which refers to the fact that there are 2 redundant packets out of a total (16+2) of 18, and 2/18 is 11%.

If LDMP is enabled and functioning, you can disable FEC in most cases.

Encoder Audio/Video Config Settings:

Audio Codec

By using uncompressed audio (PCM) instead of compressed (AAC, etc.), a small amount of latency can be eliminated. PCM audio requires greater bandwidth than compressed audio, though. Where AAC at full bitrate is 256 kilobits per second, per channel of audio, PCM is up to 3 times that, or 768 kbps per channel of audio.

The amount of bitrate required by the audio is included in the Max bitrate setting on the Network page, so larger audio bitrates mean a reduction in video bitrate. Be sure to take that into consideration when switching. Transmitting fewer channels of audio is also an option.

Encoder Transport Mode:

LDMP vs UDP

LDMP is our higher-reliability transport option which requires the receiving unit (Cloud server, media player, or decoder) reply to the Encoder with both latency measuring data and to acknowledge receipt of all video stream data. Data that was lost in transmission can be re-sent in due time to be part of the stream. It can be used in place of FEC for error correction, but it does have a built-in delay, as described below. (It uses the UDP network protocol, but sends UDP data bi-directionally.) LDMP settings related to latency are detailed below.

UDP mode is our legacy mode in which the Encoder sends data out without any way of knowing if the data got through, so it is best used on highly reliable networks. However, error correction (for not-so-reliable networks) can be added through Forward Error Correction (see above). Since UDP mode has no two-way communication, it is a lower-latency transport mode.

Encoder UDP Transport Settings:

UDP Mode has only one setting, a choice between Single, Broadcast, and Multicast Modes.

Select Single mode. Broadcast and Multicast are legacy modes that don’t really apply to modern streaming and have been superseded by use of the Streambox Cloud and LDMP, respectively.


Encoder LDMP Transport Settings:

ACK_TO/ACK Timeout/Acknowledgment timeout

Ack Timeout is the time (in milliseconds) the Encoder waits for replies from the Decoder. If the Encoder doesn't receive a reply that acknowledges receipt of a data packet, it re-sends the packet, and these re-sent packets may arrive late, resulting in packet loss on screen.

Measure the ping time (Round-Trip Time, RTT) from Encoder to Decoder, set the ACK Timeout to be about 1.3 times the average measured Ping/RTT, with a minimum value of 25 over the internet, 10 over local network/fiber. For example, if the RTT averages 50 ms then you set the ACK_TO to 65.

RTT Multiplier

This sets the time the Encoder will wait for replies regarding the re-sent packets as described above.

Set this to 2, or if the ping value measured above varies widely, 3 or 4. The new time that the Encoder will wait, for the re-sent packets, is the RTT Multiplier times the Ack Timeout, above.

For example, an Ack Timeout of 75 and a Multiplier set to 2 means that the first, initial packets sent to the Decoder must be acknowledged within 75ms, and if that does not happen, the Encoder re-sends and will wait 75ms x 2 = 150ms for the acknowledgment from the Decoder.

Pre-Buffer (also called Jitter2 Percentage)

If you have a wired network, this should be set to 10 (or as low as 2), but some buffer is required. For wireless networks, it may be necessary to set higher values, depending on the reliability of the network for streaming. A value of 10 means 10% of the Target Bitrate will be the size of the Decoder’s Jitter2 Buffer. The typical range is 2 to 25(%). In the case of very poor network efficiency (like over a poor cellular connection), values may have to be increased to 100, 150, even up to 400, but these values will greatly increase latency.

Jitter (Decoder) Buffers:

As packets arrive at the decoder, they may arrive with some variation in the timing of their arrival compared to the sending, due to network fluctuations, different paths taken through the Internet, and so on. This variability, known as jitter, requires compensation in order to have a smooth video playout. The Jitter buffers are Decoder buffers in which the video packets are temporarily stored to remove any variance in the timing of their arrival and placed in correct order if they arrived out of order due to a severe variation induced by the network or multiple networks in use. Those are the respective jobs of the Jitter1 and Jitter2 buffers.

The Encoder’s LDMP settings include a value to send a command to the Decoder to set the size of the Jitter2 buffer to a percentage of the bitrate and use a combination of other LDMP settings to calculate, then command, the size of the Jitter1 buffer. The size of the buffers can also be set manually on the Decoder as well. Where the Encoder Jitter and the Decoder Jitter values differ, the higher of the two values for each buffer takes precedence.

Decoder Network Settings

Jitter 1 and Jitter 2

Jitter 1 and 2 are the names of two buffers built in, and they can be manually set (number of packets). Defaults are low, in the 5-25 range. When the Encoder has LDMP enabled, the Encoder will suggest buffer sizes to the Decoder, which compares those suggested values to the values stored in Jitter1/Jitter2 and takes the higher value. So, unless these values are set in the hundreds or thousands, Jitter1 and Jitter2 are likely not being used by anything. In most cases, you can set Jitter1 and Jitter2 to 5 and 10, respectively.

Because the Encoder Pre-Buffer settings are a percentage of target bitrate, and the Decoder Jitter settings use the number of packets as the setting, it’s difficult to compare the two settings directly. The Decoder’s Info tab will display the current Jitter settings, in packets, during an active stream, having converted from the percentage automatically.

Adding it up: Lowest Latency Settings

The lowest latency one can achieve is ultimately determined by the speed and quality of the network. Below are some settings which will achieve the lowest possible latency. These settings essentially offer the lowest amount of error correction as well, and so rely on a high quality network transmission with no or almost no packets being lost in the sending.


Lowest Latency (UDP)


Using UDP will produce the absolute lowest latency. Target Bitrate doesn’t matter as long as both sender and receiver can handle the bandwidth requirement.

Encoder Settings:

  • VBR Buffer: 0.0 (for HD/2K DCI) or 0.2 (UHD/4K DCI)

  • Transport Mode: UDP (Single)

  • Forward Error Correction: Off

  • Audio Codec: PCM

Decoder settings:

  • Jitter 1, Jitter 2 both set to 5


Lowest Latency (LDMP)


Encoder Settings:

  • VBR Buffer: 0.0 (for HD/2K DCI) or 0.2 (UHD/4K DCI)

  • Transport Mode: LDMP

    • ACK Timeout: Must be set based on measure network latency, as described above, with minimum values starting at 10.

    • RTT Multiplier: 0

    • Pre-Buffer (aka Jitter2 Percentage): 0

  • Forward Error Correction: Off

  • Audio Codec: PCM

Decoder settings:

  • Jitter 1, Jitter 2 both set to 0 (the LDMP will override these settings and set higher values)