Content Delivery Network

3 posts

cloudflare4 min readCurated summary

Dogfooding at scale: migrating cdnjs to Cloudflare’s Developer Platform

cdnjs now runs entirely on Cloudflare’s Developer Platform after a migration intended to improve maintainability rather than performance. Despite the rise of bundlers and modern JavaScript tooling, cdnjs still serves about 9 billion requests per day because it is free, familiar, immutable, auditable, and widely used by both developers and AI coding assistants. The migration replaces a fragmented GCP, GitHub, VM, and Cloudflare setup with a unified architecture built around Workers, R2, Workflows, Queues, D1, KV, Cache, and Containers. ## cdnjs’s Scale and Continued Relevance - cdnjs serves roughly: - 108,000 requests per second - 9 billion requests per day - Traffic across more than 330 Cloudflare data centers - A 98.6% cache-hit rate - It is used by approximately 12% of websites and holds a 48.3% share of the JavaScript CDN market. - Its simple `<script>`-tag model remains popular because: - URLs and versions are consistent and immutable. - Libraries are available without accounts, API keys, or rate limits. - Files include Subresource Integrity hashes. - The project is open source and community-driven. - AI assistants frequently generate cdnjs URLs because they appear throughout years of tutorials, documentation, GitHub repositories, and Stack Overflow answers. ## Why the Existing Architecture Became a Problem - Cloudflare moved cdnjs file serving to Workers and KV in 2020, improving resilience and enabling pre-compressed Brotli and gzip assets. - The publishing pipeline remained on GCP because Cloudflare previously lacked suitable tools for: - Fetching large package archives - Running CPU-intensive processing - Coordinating multi-step jobs over hours - The old pipeline combined GCP Functions, Google Cloud Storage, Pub/Sub, a git-sync VM, GitHub, Workers KV, and a bare-metal origin. - New features and bug fixes required coordinating deployments across multiple platforms, while observability required manually stitching together unrelated logs. ## Problems with the Legacy Pipeline - **No shared tracing** - Package updates could pass through several systems without a common correlation ID. - Partial failures could leave KV updated while GitHub remained stale, with no alert indicating the divergence. - **Split-brain storage** - File content existed both in Workers KV and a GitHub repository. - Neither system was cleanly authoritative, making reconciliation difficult. - **Storage-driven orchestration** - GCP Cloud Functions triggered one another through object-created events. - Storage effectively acted as a message queue without dead-letter handling, backlog visibility, or reliable replay. - **Operational fragmentation** - npm polling required 26 separately deployed Cloud Functions, one for each alphabetic shard. - Health monitoring required checking all 26 deployments and their logs. - **An oversized GitHub repository** - The repository exceeded 1.1 TB of packed storage. - GitHub could no longer generate archive downloads reliably. - Cloning and forking became impractical. - A 274-entry `.gitignore` accumulated to exclude releases the pipeline could not reject properly. - **Security overhead** - Cloud Functions, a VM, container images, storage buckets, and service-account credentials all required patching, auditing, and protection. - Retiring these components reduced the attack surface and eliminated recently exposed vulnerabilities. ## The New Cloudflare-Based Architecture - The rebuilt system uses Cloudflare’s Developer Platform end to end. - **R2** becomes the single source of truth for file content. - It can store large assets that previously did not fit comfortably in KV, including source maps, large bundles, and font packages. - Its S3-compatible API makes the catalog accessible to external tools and mirrors. - The broader platform combines: - Workers for request handling - Workflows for orchestration - Queues for reliable asynchronous processing - R2 for durable object storage - D1, KV, Workers Cache, and Containers for supporting services - Centralizing the pipeline should make processing state observable, reduce deployment complexity, and eliminate inconsistencies between edge storage and the GitHub repository. ## Practical Conclusion The cdnjs migration demonstrates that a globally critical, high-volume open-source service can evolve from a collection of legacy systems into a unified serverless platform. Its continued value comes not only from speed, but from being free, predictable, immutable, and easy for both humans and automated tools to consume.

Read original(opens in new tab)
netflix3 min readCurated summary

Smarter Live Streaming at Scale: Rolling Out VBR for All Netflix Live Events

Netflix switched all Live events from Constant Bitrate (CBR) to capped Variable Bitrate (VBR), using AWS Elemental MediaLive’s QVBR setting. VBR allocates bits according to scene complexity, reducing delivery costs and improving playback quality, but its unpredictable spikes and dips invalidate traditional capacity-planning assumptions. Netflix addressed this by reserving delivery capacity according to each stream’s nominal bitrate rather than its current traffic level. ## Why Netflix Moved Live Streaming from CBR to VBR - CBR delivers streams near a fixed target, making server capacity and traffic patterns easy to predict. - However, CBR wastes bits on simple scenes and may provide insufficient bits for complex action. - VBR targets consistent visual quality instead: - Simple scenes use substantially fewer bits. - Complex scenes receive higher bitrate to prevent artifacts. - Netflix’s tests found: - Approximately 15% fewer bytes transferred on average. - Around 10% less traffic during the peak minute. - About 5% fewer rebuffers per hour. - Lower average traffic improves Open Connect scalability and can reduce startup delays and playback interruptions. ## Why VBR Creates Stability Risks - VBR bitrate can remain well below its nominal target during simple scenes, sometimes using only 2 Mbps for a 5 Mbps stream. - Delivery systems may interpret these low-traffic periods as spare server capacity and route additional sessions to the server. - When complex content appears—such as fights, confetti, rapid camera movement, or detailed crowds—bitrate can quickly rise to 6–8 Mbps or more. - If too many sessions were admitted during the low-bitrate period, aggregate traffic can exceed link or NIC capacity, causing: - Higher latency - Packet loss - Playback stalls - Quality downshifts ## Making Capacity Planning Aware of VBR - Netflix changed traffic-steering decisions so they no longer rely solely on current throughput. - Each stream reserves capacity based on its nominal bitrate, even when its current bitrate is much lower. - This treats every stream as capable of quickly returning to its expected capacity level. - The approach prevents servers from being overfilled during low-complexity scenes and keeps delivery behavior consistent across CBR and VBR. ## Matching VBR Bitrates to CBR Quality - Identical nominal bitrates do not produce identical behavior: - CBR remains clustered around its target with frequent small variations. - VBR spends far less on simple scenes and increases bitrate only when complexity demands it. - Netflix therefore needed to revisit which nominal VBR bitrates correspond to the quality previously delivered by CBR, rather than assuming the same configured bitrate would provide equivalent results. Netflix’s rollout shows that VBR is more than an encoder setting: it requires coordinated changes to bitrate ladders, capacity reservations, and traffic steering. With those safeguards, VBR can deliver comparable or better quality while using significantly less network capacity.

Read original(opens in new tab)
netflixOriginal article

Netflix Live Origin. Xiaomei Liu, Joseph Lynch, Chris Newton | by Netflix Technology Blog | Dec, 2025 | Netflix TechBlog (opens in new tab)

The Netflix Live Origin is a specialized, multi-tenant microservice designed to bridge the gap between cloud-based live streaming pipelines and the Open Connect content delivery network. By operating as an intelligent broker, it manages content selection across redundant regional pipelines to ensure that only valid, high-quality segments are distributed to client devices. This architecture allows Netflix to achieve high resilience and stream integrity through server-side failover and deterministic segment selection. ### Multi-Pipeline and Multi-Region Awareness * The origin server mitigates common live streaming defects, such as missing segments, timing discontinuities, and short segments containing missing video or audio samples. * It leverages independent, redundant streaming pipelines across different AWS regions to ensure high availability; if one pipeline fails or produces a defective segment, the origin selects a valid candidate from an alternate path. * Implementation of epoch locking at the cloud encoder level allows the origin to interchangeably select segments from various pipelines. * The system uses lightweight media inspection at the packager level to generate metadata, which the origin then uses to perform deterministic candidate selection. ### Stream Distribution and Protocol Integration * The service operates on AWS EC2 instances and utilizes standard HTTP protocol features for communication. * Upstream packagers use HTTP PUT requests to push segments into storage at specific URLs, while the downstream Open Connect network retrieves them via GET requests. * The architecture is optimized for a manifest design that uses segment templates and constant segment durations, which reduces the need for frequent manifest refreshes. ### Open Connect Streaming Optimization * While Netflix’s Open Connect Appliances (OCAs) were originally optimized for VOD, the Live Origin extends nginx proxy-caching functionality to meet live-specific requirements. * OCAs are provided with Live Event Configuration data, including Availability Start Times and initial segment numbers, to determine the legitimate range of segments for an event. * This predictive modeling allows the CDN to reject requests for objects outside the valid range immediately, reducing unnecessary traffic and load on the origin. By decoupling the live streaming pipeline from the distribution network through this specialized origin layer, Netflix can maintain a high level of fault tolerance and stream stability. This approach minimizes client-side complexity by handling failovers and segment selection on the server side, ensuring a seamless experience for viewers of live events.