Dogfooding at scale: migrating cdnjs to Cloudflare’s Developer Platform (opens in new tab)
cdnjs now runs entirely on Cloudflare’s Developer Platform after a migration intended to improve maintainability rather than performance. Despite the rise of bundlers and modern JavaScript tooling, cdnjs still serves about 9 billion requests per day because it is free, familiar, immutable, auditable, and widely used by both developers and AI coding assistants. The migration replaces a fragmented GCP, GitHub, VM, and Cloudflare setup with a unified architecture built around Workers, R2, Workflows, Queues, D1, KV, Cache, and Containers. ## cdnjs’s Scale and Continued Relevance - cdnjs serves roughly: - 108,000 requests per second - 9 billion requests per day - Traffic across more than 330 Cloudflare data centers - A 98.6% cache-hit rate - It is used by approximately 12% of websites and holds a 48.3% share of the JavaScript CDN market. - Its simple `<script>`-tag model remains popular because: - URLs and versions are consistent and immutable. - Libraries are available without accounts, API keys, or rate limits. - Files include Subresource Integrity hashes. - The project is open source and community-driven. - AI assistants frequently generate cdnjs URLs because they appear throughout years of tutorials, documentation, GitHub repositories, and Stack Overflow answers. ## Why the Existing Architecture Became a Problem - Cloudflare moved cdnjs file serving to Workers and KV in 2020, improving resilience and enabling pre-compressed Brotli and gzip assets. - The publishing pipeline remained on GCP because Cloudflare previously lacked suitable tools for: - Fetching large package archives - Running CPU-intensive processing - Coordinating multi-step jobs over hours - The old pipeline combined GCP Functions, Google Cloud Storage, Pub/Sub, a git-sync VM, GitHub, Workers KV, and a bare-metal origin. - New features and bug fixes required coordinating deployments across multiple platforms, while observability required manually stitching together unrelated logs. ## Problems with the Legacy Pipeline - **No shared tracing** - Package updates could pass through several systems without a common correlation ID. - Partial failures could leave KV updated while GitHub remained stale, with no alert indicating the divergence. - **Split-brain storage** - File content existed both in Workers KV and a GitHub repository. - Neither system was cleanly authoritative, making reconciliation difficult. - **Storage-driven orchestration** - GCP Cloud Functions triggered one another through object-created events. - Storage effectively acted as a message queue without dead-letter handling, backlog visibility, or reliable replay. - **Operational fragmentation** - npm polling required 26 separately deployed Cloud Functions, one for each alphabetic shard. - Health monitoring required checking all 26 deployments and their logs. - **An oversized GitHub repository** - The repository exceeded 1.1 TB of packed storage. - GitHub could no longer generate archive downloads reliably. - Cloning and forking became impractical. - A 274-entry `.gitignore` accumulated to exclude releases the pipeline could not reject properly. - **Security overhead** - Cloud Functions, a VM, container images, storage buckets, and service-account credentials all required patching, auditing, and protection. - Retiring these components reduced the attack surface and eliminated recently exposed vulnerabilities. ## The New Cloudflare-Based Architecture - The rebuilt system uses Cloudflare’s Developer Platform end to end. - **R2** becomes the single source of truth for file content. - It can store large assets that previously did not fit comfortably in KV, including source maps, large bundles, and font packages. - Its S3-compatible API makes the catalog accessible to external tools and mirrors. - The broader platform combines: - Workers for request handling - Workflows for orchestration - Queues for reliable asynchronous processing - R2 for durable object storage - D1, KV, Workers Cache, and Containers for supporting services - Centralizing the pipeline should make processing state observable, reduce deployment complexity, and eliminate inconsistencies between edge storage and the GitHub repository. ## Practical Conclusion The cdnjs migration demonstrates that a globally critical, high-volume open-source service can evolve from a collection of legacy systems into a unified serverless platform. Its continued value comes not only from speed, but from being free, predictable, immutable, and easy for both humans and automated tools to consume.