Adopting AV1 for Real-Time Communication (RTC) at Scale

Meta’s adoption of AV1 for real-time communication has been a multi-year effort focused on codec efficiency, device compatibility, latency, power consumption, and resilience to poor networks. AV1 can reduce bitrate by at least 20% versus H.264/AVC while preserving quality, especially below 100 kbps and for screen content. However, making it practical for mobile video calls required low-complexity encoding, careful decoder selection, and systems designed to avoid freezes caused by bitrate spikes and packet loss.

Why Meta Adopted AV1 for RTC

• AV1 delivers comparable visual quality at substantially lower bitrates than H.264/AVC.
• Offline testing showed at least a 20% bitrate reduction on low-end and mid-range devices under Meta’s product settings.
• Lower bitrate is especially valuable in real-time calls, where network rates may fluctuate between 10 and 400 kbps.
• At a 100 kbps limit, AV1 video remained much clearer than H.264/AVC, which appeared blurry.
• AV1 is also well suited to screen sharing:
  • Palette mode efficiently represents frames with a limited number of colors.
  • Intra-block copy predicts repeated patterns within the same frame.
  • Both tools improve the readability of text and other computer-generated content.

Challenges of Real-Time Video

• RTC requires end-to-end latency below roughly 300 milliseconds; techniques common in video-on-demand, such as multi-pass encoding and extensive buffering, can introduce unacceptable delay.
• Sudden bitrate increases can cause freezes.
• Network bandwidth changes may require adjustments to resolution or frame rate.
• Resolution changes generally require a key frame, producing a temporary bitrate spike.
• Packet loss can trigger retransmissions or additional key frames, also increasing the risk of freezes.
• Mobile devices must encode and decode video simultaneously, making power efficiency essential.

Encoder Selection and Complexity

• AV1’s advanced coding tools improve compression but can significantly increase encoding complexity.
• An open-source AV1 encoder consumed 14% more power than H.264/AVC on a Pixel 8 during testing.
• AV1 also used more memory, contributing to application crash regressions.
• Meta therefore adopted an internal low-complexity AV1 encoder with power consumption comparable to H.264 baseline.
• The encoder supports multiple presets:
  • Higher-complexity presets prioritize quality.
  • An ultra-low-complexity preset offers complexity comparable to H.264/AVC.
• Meta selects the encoder preset according to device capabilities, allowing AV1 deployment beyond high-end phones.

Decoder Selection

• Although decoding is generally less demanding than encoding, AV1 decoding was still challenging for low-end mobile devices.
• Initial tests found real-time decoding failures, video freezes, and audio/video synchronization problems on some devices.
• Meta evaluated multiple open-source decoders and selected dav1d based on its power efficiency and reliability.

Meta’s experience shows that AV1 adoption in RTC requires more than simply enabling a new codec. A practical deployment depends on device-specific complexity controls, efficient decoding, strict latency management, and mechanisms that prevent bandwidth changes or packet loss from interrupting calls.