Service Mesh | Techlist.io

lineNov 21, 2025Original article

Connecting thousands of LY Corporation services (opens in new tab)

LY Corporation developed a centralized control plane using Central Dogma to manage service-to-service communication across its vast, heterogeneous infrastructure of physical machines, virtual machines, and Kubernetes clusters. By adopting the industry-standard xDS protocol, the new system resolves the interoperability and scaling limitations of their legacy platform while providing a robust GitOps-based workflow. This architecture enables the company to connect thousands of services with high reliability and sophisticated traffic control capabilities. ## Limitations of the Legacy System The previous control plane environment faced several architectural bottlenecks that hindered developer productivity and system flexibility: * **Tight Coupling:** The system was heavily dependent on a specific internal project management tool (PMC), making it difficult to support modern containerized environments like Kubernetes. * **Proprietary Schemas:** Communication relied on custom message schemas, which created interoperability issues between different clients and versions. * **Lack of Dynamic Registration:** The legacy setup could not handle dynamic endpoint registration effectively, functioning more as a static registry than a functional service mesh control plane. * **Limited Traffic Control:** It lacked the ability to perform complex routing tasks, such as canary releases or advanced client-side load balancing, across diverse infrastructures. ## Central Dogma as a Control Plane To solve these issues, the team leveraged Central Dogma, a Git-based repository service for textual configuration, to act as the foundation for a new control plane: * **xDS Protocol Integration:** The new control plane implements the industry-standard xDS protocol, ensuring seamless compatibility with Envoy and other modern data plane proxies. * **GitOps Workflow:** By utilizing Central Dogma’s mirroring features, developers can manage service configurations and traffic policies safely through Pull Requests in external Git repositories. * **High Reliability:** The system inherits Central Dogma’s native strengths, including multi-datacenter replication, high availability, and a robust authorization system. * **Schema Evolution:** The control plane automatically transforms legacy metadata into standard xDS resources, allowing for a smooth transition from old infrastructure to the new service mesh. ## Dynamic Service Discovery and Registration The architecture provides automated ways to manage service endpoints across different environments: * **Kubernetes Endpoint Plugin:** A dedicated plugin watches for changes in Kubernetes services and automatically updates the xDS resource tree in Central Dogma. * **Automated API Registration:** The system provides gRPC and HTTP APIs (e.g., `RegisterLocalityLbEndpoint`) that allow services to register themselves dynamically during the startup process. * **Advanced Traffic Features:** The new control plane supports sophisticated features like zone-aware routing, circuit breakers, automatic retries, and "slow start" mechanisms for new endpoints. ## Evolution Toward Sidecar-less Service Mesh A major focus of the project is improving the developer experience by reducing the operational overhead of the data plane: * **Sidecar-less Options:** The team is working toward providing service mesh benefits without requiring a sidecar proxy for every pod, which reduces resource consumption and simplifies debugging. * **Unified Control:** Central Dogma acts as a single source of truth for both proxy-based and proxyless service mesh configurations, ensuring consistent policy enforcement across the entire organization. For organizations managing large-scale, heterogeneous infrastructure, transitioning to an xDS-compliant control plane backed by a reliable Git-based configuration store is highly recommended. This approach balances the need for high-speed dynamic updates with the safety and auditability of GitOps, ultimately allowing for a more scalable and developer-friendly service mesh.

Service Mesh Kubernetes Gitops Envoy+4

airbnbSep 15, 20254 min readCurated summary

Viaduct, Five Years On: Modernizing the Data-Oriented Service Mesh

Viaduct, Airbnb’s data-oriented service mesh, has evolved substantially over five years while retaining its core model: a central schema, hosted business logic, and re-entrant composition through GraphQL. Its usage has grown eightfold, supporting more than 130 teams and over 1.5 million lines of production code, without increasing operational overhead. Viaduct Modern now aims to simplify its developer API and establish stronger architectural boundaries, alongside the project’s release as open source. ## Adoption and Evolution - Viaduct traffic has increased by a factor of eight since 2020. - More than 130 teams now host code in Viaduct, supported by hundreds of weekly active developers. - The hosted codebase has grown to over 1.5 million lines, with roughly the same amount of test code. - Operational overhead has remained constant, incident-minutes have been cut in half, and costs have grown linearly with QPS. - Viaduct is now available as open-source software. ## Core Principles That Remain - **Central schema:** Viaduct provides one integrated schema connecting domains across Airbnb. - More than 75% of requests are internal. - The schema is developed by many teams but exposed as a connected graph. - **Hosted business logic:** Teams run business logic directly in Viaduct rather than maintaining separate microservices. - This reduces operational overhead and can allow standalone services to be retired. - Viaduct provides a serverless environment so developers can focus on application logic. - **Re-entrancy:** Hosted logic composes with other hosted logic through GraphQL fragments and queries. - This supports modularity. - It helps avoid the tightly coupled structure and maintenance problems associated with traditional monoliths. ## Problems with the Earlier Design - Viaduct’s APIs evolved reactively in response to individual use cases. - Multiple mechanisms emerged for accomplishing similar tasks, creating confusion for developers. - Some capabilities were well supported while others were not. - The framework’s layers had loose, inconsistent interfaces. - The boundary between Viaduct and hosted application code was weak. - These issues made framework improvements increasingly risky because changes could disrupt existing users. ## Simplifying the Tenant API - Viaduct Modern overhauls the developer-facing API and execution engine. - The new Tenant API reduces the implementation choices to two mechanisms: - **Node resolvers** - **Field resolvers** - The choice is determined by the schema rather than by ad hoc behavioral distinctions. - Resolver APIs have been unified wherever possible. - The goal is a smaller, more consistent surface that preserves successful ideas from the old API while removing unnecessary alternatives. ## Tenant Modules and Re-Entrant Composition - Viaduct uses modules and re-entrancy to provide boundaries similar to service definitions and RPC APIs in microservice architectures. - A tenant module combines: - Schema owned by a team - The code implementing that schema - Modules can create rich connections in the shared graph, but direct code dependencies between teams are discouraged. - Instead, teams declare their data requirements through GraphQL fragments and queries. ### Example: Extending the `User` Type - A Core User team owns the base `User` type and resolves fields such as `firstName` and `lastName`. - A Messaging team can extend `User` with a `displayName` field. - Its resolver declares that it needs `firstName` and `lastName`. - Messaging does not depend directly on Core User’s implementation or need to know where those fields originate. - This declarative model lets teams collaborate through the schema while preserving ownership and modularity. ## Framework Modularity - Viaduct Modern also restructures the framework itself. - The system consists of: - The GraphQL execution engine - The Tenant API - Hosted application code - Historically, the interfaces between these layers were weak, making performance and reliability improvements difficult to introduce safely. - The redesign focuses on stronger abstraction boundaries so the framework can evolve independently of application code. Viaduct’s modernization is intended to preserve its centralized, data-oriented model while making development simpler and framework evolution safer. The open-source release provides an opportunity for other organizations to evaluate or adopt this approach to schema-driven, modular service composition.

Read original

Service Mesh Distributed Systems Serverless Microservices+3

figmaAug 7, 20243 min readCurated summary

How We Migrated onto K8s in Less Than 12 months | Figma Blog

Figma migrated most of its core services from AWS ECS to Kubernetes in under 12 months because ECS was increasingly limiting its platform ambitions. Kubernetes offered better support for stateful workloads, Helm-based software, autoscaling, service networking, and the broader CNCF ecosystem. The migration was considered worthwhile because Figma had relatively few core services and had already containerized its workloads, making the transition more manageable. ## Figma’s Existing Compute Platform - By early 2023, Figma was already running all services in containers on Amazon ECS. - ECS had enabled rapid adoption of containerized workloads, but Figma’s growing infrastructure team began evaluating a more capable long-term platform. - Figma is not organized around thousands of microservices: - A small set of powerful core services provides modularization and traffic isolation. - New product capabilities are usually added to existing services rather than creating new ones. - This limited service count made a Kubernetes migration more practical. ## Limitations of ECS - ECS lacked Kubernetes primitives needed for complex workloads. - Running `etcd` on ECS required fragile custom startup code to manage cluster membership because ECS does not provide StatefulSets or persistent pod identity. - Kubernetes StatefulSets provide stable identities and stateful networking for systems such as `etcd`. - ECS did not natively support deploying groups of services packaged as Helm charts. - Open-source tools such as Temporal would require manual conversion into Terraform configurations. - This increased installation and maintenance effort. - ECS also made routine infrastructure operations more cumbersome. - For example, safely removing a malfunctioning EC2 instance was difficult. - EKS can cordon a node and move its pods elsewhere while respecting graceful shutdown behavior. ## Access to the CNCF Ecosystem - Kubernetes would give Figma access to a larger ecosystem of open-source cloud-native tools. - Autoscaling was a major motivation: - Figma was provisioning services for peak demand, wasting resources during lower-traffic periods. - Kubernetes tooling such as KEDA supports scaling based on CPU, SQS queue length, and custom Datadog metrics. - Figma expected to adopt a service mesh eventually. - Existing AWS load balancer routing created operational drawbacks: - Network Load Balancers could take several minutes to register or remove targets. - This slowed emergency deployments and increased incident remediation time. - Envoy offered more customization than AWS load balancers, including custom filters for shedding load during incidents. - Figma had already deployed standalone Envoy machines for a major service and saw Kubernetes ecosystems such as Istio as a path toward fleet-wide service-mesh adoption. Figma’s experience suggests that Kubernetes was justified not simply as a replacement for ECS, but as a foundation for more capable operations and broader platform tooling. Organizations considering a similar move should first assess their workload complexity, existing container maturity, and whether Kubernetes capabilities will materially reduce infrastructure work.

Read original

Service Mesh Kubernetes Docker & Containers Terraform+4

airbnbNov 10, 20203 min readCurated summary

Taming Service-Oriented Architecture Using A Data-Oriented Service Mesh

Airbnb’s Viaduct rethinks the service mesh as a data-oriented layer rather than a network for routing procedural service calls. Built on GraphQL, it presents a unified data graph that hides microservice dependencies from consumers and improves modularity in large SOAs. The central schema can also coordinate service APIs, database models, and serverless data transformations, making system-wide changes more agile. ## The Problem with Large SOAs - Modern organizations may operate thousands of microservices connected through highly tangled dependency graphs. - These graphs resemble “spaghetti code” at the service level: - Changes become difficult to plan. - Teams must coordinate across many service boundaries. - Consumers often depend directly on multiple underlying services. - Airbnb argues that microservice architectures need stronger organizing principles and technical mechanisms for enforcing modularity. ## From Procedure-Oriented to Data-Oriented Design - Traditional procedural design groups procedures into modules with public APIs and hidden implementation details. - Data-oriented design instead organizes software around encapsulated data objects and the methods that operate on them. - Microservices have largely returned SOA to a procedural model: - Each service exposes collections of remote procedural endpoints. - Consumers must know which services provide the data they need. - Viaduct applies data-oriented principles to the service mesh itself. ## Viaduct’s GraphQL Data Mesh - Viaduct defines the mesh through a GraphQL schema containing: - Types and interfaces representing managed data. - Queries and subscriptions for reading data. - Mutations for updating data. - The schema forms a single graph spanning data owned by many microservices. - A consumer can navigate related data through one query, such as: - `productById { manufacturer }` - `productById { reviews }` - `productById { reviews { author } }` - Viaduct determines which services provide each requested field. - This hides service dependencies from consumers and prevents every client from building its own cross-service orchestration logic. ## The Central Schema - Unlike distributed GraphQL approaches that split schemas across modules or federated services, Viaduct treats the schema as one central artifact. - Airbnb uses schema-management primitives to let multiple teams collaborate while preserving a unified model. - Portions of the central schema can define individual microservice APIs. - Airbnb ultimately aims to use the same schema to define database structures. - This could improve “data agility”: - Database changes would no longer need manual translation through several API layers. - A single schema update could propagate changes from storage through services to clients. - Cross-team coordination and delivery times could be reduced. ## Serverless Derived Fields - Many SOAs contain stateless services that transform backend data for particular clients or presentation layers. - Viaduct supports derived fields computed by serverless cloud functions. - These functions operate on the graph without needing direct knowledge of the underlying microservices. - Moving transformation logic into stateless containers can: - Reduce the number of services. - Lower operational overhead. - Keep the core service graph simpler. ## Implementation and Operational Features - Viaduct is built on `graphql-java`. - It supports fine-grained field selection through GraphQL selection sets. - It uses data-loading techniques and an intra-request cache. - Reliability features include short-circuiting and soft dependencies. - Field-level observability shows which services consume particular data. - Its GraphQL interface enables use of established open-source tooling and interactive development tools. Viaduct’s practical recommendation is to place a unified data schema at the center of the architecture, allowing the mesh—not individual consumers—to manage service composition. This can make large SOAs more modular, easier to evolve, and better suited to serverless execution.

Read original

Service Mesh Microservices Serverless GraphQL+3