Nginx

5 posts

cloudflare3 min readCurated summary

Secure all your internal vibe-coded applications — in one click

AI-driven development makes it easy for employees to deploy applications, but also increases the risk of unintentionally exposing company data. Cloudflare’s new Access integration for Workers makes applications private by default at the Worker or account level, regardless of how they are reached. It also exposes authenticated user identity directly in Worker code and supports private-by-default internal deployment platforms. ## Worker-Level Access Protection - Access authentication is enforced before requests reach application code. - Protection applies across custom domains, routes, `workers.dev` subdomains, and preview URLs. - Policies can cover: - Preview deployments only - Every hostname associated with a Worker - Attaching policies to the Worker eliminates the need to update Access settings whenever a new domain is added. - Existing identity providers, email addresses, domains, groups, and service tokens can control access. ## Account-Wide Private Defaults - An account-level policy automatically protects all current and future Workers. - Organizations can protect preview traffic, production traffic, or both. - Public Workers can explicitly bypass the account-wide policy. - For individual applications, Worker policies provide targeted protection. - When multiple policies apply, precedence is: - Hostname policies - Worker policies - Account policies ## Accessing User Identity in Worker Code - Authenticated requests expose identity through `ctx.access`. - `ctx.access.getIdentity()` returns information such as: - Email address - Name - Groups - Developers no longer need to parse, validate, and extract claims from Access JWTs manually. - Applications can use this identity for personalization, authorization, and per-user logging. - Code should handle requests without Access metadata, for example by returning a `403` response. ## Local Development and Testing - `wrangler dev` can simulate authenticated users locally. - An `access.dev` block in `wrangler.jsonc` defines a test audience and identity: ```json { "access": { "dev": { "aud": "my-app", "identity": { "email": "admin@company.com" } } } } ``` - Developers can change the configured email to test different user experiences without repeatedly deploying and authenticating through Access. ## Private Internal Deployment Platforms - Workers for Platforms can host many applications inside a namespace. - Traffic is routed through a shared dispatch Worker. - Protecting the dispatch Worker with Access makes every application deployed through it private by default. - Cloudflare provides an open-source example of an internal drag-and-drop deployment platform using this model. ## Infrastructure Behind the Feature - The capability relies on FL2, Cloudflare’s Rust-based modular proxy. - Workers routing had to be separated from execution so Cloudflare could determine the destination Worker before applying Access. - This routing change would have been more difficult in the older NGINX- and Lua-based FL1 architecture. Cloudflare’s approach shifts application security from an optional developer-configured step to an organizational default. Teams deploying internal or experimental Workers should use account-level or dispatch-level Access policies, while using Worker-level policies and local identity simulation for application-specific control and testing.

Read original(opens in new tab)
cloudflare3 min readCurated summary

Launching Cloudflare’s Gen 13 servers- trading cache for cores for 2x edge compute performance

Cloudflare’s Gen 13 servers use AMD EPYC 5th Gen Turin processors to provide up to twice as many cores as Gen 12. However, Turin’s much smaller per-core cache caused the legacy FL1 request-handling layer to suffer severe latency increases, despite higher throughput. Cloudflare found that tuning alone could not fully solve the problem, reinforcing the need for FL2, a Rust-based rewrite designed to scale with cores rather than depend heavily on cache. ## Turin’s Core-Heavy Architecture - Gen 13 Turin processors offer: - Up to 192 cores and 384 SMT threads, compared with Gen 12’s 96 cores. - Improved instructions per cycle through the Zen 5 architecture. - Up to 32% lower power consumption per core. - DDR5-6400 support for greater memory bandwidth. - The tradeoff is substantially less cache: - Gen 12 Genoa-X provides 12 MB of L3 cache per core through 3D V-Cache. - The 192-core Turin 9965 provides only 2 MB per core. - This architecture favors aggregate throughput but challenges workloads dependent on cache locality. ## FL1’s Cache and Latency Problems - FL1, based on NGINX and LuaJIT, was optimized for Gen 12’s large cache. - AMD uProf measurements showed: - Dramatically higher L3 cache miss rates on Turin. - More requests requiring slow DRAM access. - Increasing latency as CPU utilization and cache contention rose. - An L3 hit takes roughly 50 CPU cycles, while a DRAM fetch can take more than 350 cycles. - As a result, Gen 13’s additional cores delivered throughput gains but introduced unacceptable latency penalties. ## Throughput Gains at an Unacceptable Cost - With FL1, Gen 13 produced: - 10% more throughput on the 128-core Turin 9755. - 31% more on the 160-core Turin 9845. - 62% more on the 192-core Turin 9965. - The Turin 9965 offered the strongest total-cost-of-ownership benefits. - However, latency increased by more than 50% at high CPU utilization, which would negatively affect customer experience and violate performance requirements. ## Hardware and Resource Tuning - Cloudflare tested several mitigations with AMD: - Hardware prefetcher and Data Fabric Probe Filter adjustments produced only marginal improvements. - Adding FL1 workers increased throughput but took resources away from other services. - CPU pinning and isolation provided limited benefits. - AMD’s Platform Quality of Service (PQOS) was used to control cache and memory-bandwidth sharing across Turin’s Core Complex Dies. ## Cache Isolation with PQOS - Reserving part of a single CCD’s cache for FL1 produced less than 5% additional throughput. - Configurations assigning FL1 50–75% of each CCD’s cache also delivered less than 5% improvement and caused minor degradation elsewhere. - A socket-level approach was more successful: - Six of twelve CCDs, aligned with a NUMA domain, were dedicated to FL1. - This provided more than 15% incremental throughput while keeping latency acceptable. - These results showed that workload placement and cache locality could help, but they were not a complete substitute for software designed around Turin’s cache profile. Cloudflare’s broader solution was FL2, a Rust-based rewrite of its core request-handling layer. By reducing dependence on large per-core caches, FL2 enabled Gen 13’s higher core count to translate into scalable edge-compute performance without the latency penalties seen with FL1.

Read original(opens in new tab)
netflixOriginal article

Netflix Live Origin. Xiaomei Liu, Joseph Lynch, Chris Newton | by Netflix Technology Blog | Dec, 2025 | Netflix TechBlog (opens in new tab)

The Netflix Live Origin is a specialized, multi-tenant microservice designed to bridge the gap between cloud-based live streaming pipelines and the Open Connect content delivery network. By operating as an intelligent broker, it manages content selection across redundant regional pipelines to ensure that only valid, high-quality segments are distributed to client devices. This architecture allows Netflix to achieve high resilience and stream integrity through server-side failover and deterministic segment selection. ### Multi-Pipeline and Multi-Region Awareness * The origin server mitigates common live streaming defects, such as missing segments, timing discontinuities, and short segments containing missing video or audio samples. * It leverages independent, redundant streaming pipelines across different AWS regions to ensure high availability; if one pipeline fails or produces a defective segment, the origin selects a valid candidate from an alternate path. * Implementation of epoch locking at the cloud encoder level allows the origin to interchangeably select segments from various pipelines. * The system uses lightweight media inspection at the packager level to generate metadata, which the origin then uses to perform deterministic candidate selection. ### Stream Distribution and Protocol Integration * The service operates on AWS EC2 instances and utilizes standard HTTP protocol features for communication. * Upstream packagers use HTTP PUT requests to push segments into storage at specific URLs, while the downstream Open Connect network retrieves them via GET requests. * The architecture is optimized for a manifest design that uses segment templates and constant segment durations, which reduces the need for frequent manifest refreshes. ### Open Connect Streaming Optimization * While Netflix’s Open Connect Appliances (OCAs) were originally optimized for VOD, the Live Origin extends nginx proxy-caching functionality to meet live-specific requirements. * OCAs are provided with Live Event Configuration data, including Availability Start Times and initial segment numbers, to determine the legitimate range of segments for an event. * This predictive modeling allows the CDN to reject requests for objects outside the valid range immediately, reducing unnecessary traffic and load on the origin. By decoupling the live streaming pipeline from the distribution network through this specialized origin layer, Netflix can maintain a high level of fault tolerance and stream stability. This approach minimizes client-side complexity by handling failovers and segment selection on the server side, ensuring a seamless experience for viewers of live events.

lineOriginal article

Flexible Multi-site Architecture Designed with N (opens in new tab)

LINE NEXT optimized its web server infrastructure by transitioning from fragmented, manual Nginx setups to a centralized native Nginx multi-site architecture. By integrating global configurations and automating the deployment pipeline with Ansible, the team successfully reduced service launch lead times by over 80% while regaining the ability to use advanced features like GeoIP and real client IP tracking. This evolution ensures that the infrastructure can scale to support over 100 subdomains across diverse global services with high reliability and minimal manual overhead. ## Evolution of Nginx Infrastructure * **PMC-based Structure**: The initial phase relied on a Project Management Console using `rsync` via SSH; this created security risks and led to fragmented, siloed configurations that were difficult to maintain. * **Ingress Nginx Structure**: To improve speed, the team moved to Kubernetes-based Ingress using Helm charts, which automated domain and certificate settings but limited the use of native Nginx modules and complicated the retrieval of real client IP addresses. * **Native Nginx Multi-site Structure**: The current hybrid approach utilizes native Nginx managed by Ansible, combining the speed of configuration-driven setups with the flexibility to use advanced modules like GeoIP and Loki for log collection. ## Configuration Integration and Multi-site Management * **Master Configuration Extraction**: Common directives such as `timeouts`, `keep-alive` settings, and `log formats` were extracted into a master Nginx configuration file to eliminate redundancy across services. * **Hierarchical Directory Structure**: Inspired by Apache, the team adopted a `sites-available` structure where individual `server` blocks for different services (alpha, beta, production) are managed in separate files. * **Operational Efficiency**: This integrated structure allows a single Nginx instance to serve multiple sites simultaneously, significantly reducing the time required to add and deploy new service domains. ## Automated Deployment with Ansible * **Standardized Workflow**: The team replaced manual processes with Ansible playbooks that handle everything from cloning the latest configuration from Git to extracting environment-specific files. * **Safety and Validation**: The automated pipeline includes mandatory Nginx syntax verification (`nginx -t`) and process status checks to ensure stability before a deployment is finalized. * **Rolling Deployments**: To minimize service impact, updates are pushed sequentially across servers; the process automatically halts if an error is detected at any stage of the rollout. To effectively manage a rapidly expanding portfolio of global services, infrastructure teams should move toward a "configuration-as-code" model that separates common master settings from service-specific logic. Leveraging automation tools like Ansible alongside a native Nginx multi-site structure provides the necessary balance between rapid deployment and the granular control required for complex logging and security requirements.

discord2 min readCurated summary

Discord Update: December 19, 2024 Changelog

Discord’s December 19, 2024 changelog recaps the year’s most useful performance and reliability improvements through a festive poem. The updates focused on making mobile apps more stable and responsive, speeding up servers, GIFs, chat, and APIs, and improving image quality and storage usage. Discord concludes by encouraging users to join voice, play games, or Go Live, while promoting Nitro gifting. ## Mobile Stability and Performance - iOS crash fixes reduced the overall crash rate by **84%**. - Android’s chat renderer reduced slow frames by up to **60%** and lowered chat-list memory usage by about **12%**. - The Android Expression Picker—covering emojis, GIFs, and stickers—saw up to **50% fewer dropped frames** and approximately **7.5% lower memory usage**. - Foldable-device support and multitasking were significantly improved. - Mobile server lists were virtualized so only visible servers remain loaded, improving scrolling performance. - Server switching on Android and iOS became more than **30% faster**. - Mobile GIF Picker loading times improved by up to **80%**. ## Backend and Media Improvements - Infrastructure changes, including moving to Google Cloud C3 instances and removing an Nginx layer, reduced p90+ API latency by roughly **25%**. - Discord’s media proxy now preserves ICC color-profile data when resizing images into WebP, improving color accuracy across platforms. - iOS data-storage changes reduced disk usage substantially; the most affected 1% of users went from approximately **5.7 GB to 1.7 GB**, with some users recovering up to **4 GB**. ## Seasonal Extras - The recap is presented as a “Fixmas” poem summarizing fixes from Discord’s 2024 Patch Notes. - Users who gift Nitro during the promotion also receive an Avatar Decoration. Together, these changes show Discord’s emphasis on incremental improvements that make everyday communication faster, more reliable, and less demanding on mobile devices.

Read original(opens in new tab)