Flexible Multi-site Architecture Designed (opens in new tab)

LINE NEXT optimized its web server infrastructure by transitioning from fragmented, manual Nginx setups to a centralized native Nginx multi-site architecture. By integrating global configurations and automating the deployment pipeline with Ansible, the team successfully reduced service launch lead times by over 80% while regaining the ability to use advanced features like GeoIP and real client IP tracking. This evolution ensures that the infrastructure can scale to support over 100 subdomains across diverse global services with high reliability and minimal manual overhead.

Evolution of Nginx Infrastructure

• PMC-based Structure: The initial phase relied on a Project Management Console using rsync via SSH; this created security risks and led to fragmented, siloed configurations that were difficult to maintain.
• Ingress Nginx Structure: To improve speed, the team moved to Kubernetes-based Ingress using Helm charts, which automated domain and certificate settings but limited the use of native Nginx modules and complicated the retrieval of real client IP addresses.
• Native Nginx Multi-site Structure: The current hybrid approach utilizes native Nginx managed by Ansible, combining the speed of configuration-driven setups with the flexibility to use advanced modules like GeoIP and Loki for log collection.

Configuration Integration and Multi-site Management

• Master Configuration Extraction: Common directives such as timeouts, keep-alive settings, and log formats were extracted into a master Nginx configuration file to eliminate redundancy across services.
• Hierarchical Directory Structure: Inspired by Apache, the team adopted a sites-available structure where individual server blocks for different services (alpha, beta, production) are managed in separate files.
• Operational Efficiency: This integrated structure allows a single Nginx instance to serve multiple sites simultaneously, significantly reducing the time required to add and deploy new service domains.

Automated Deployment with Ansible

• Standardized Workflow: The team replaced manual processes with Ansible playbooks that handle everything from cloning the latest configuration from Git to extracting environment-specific files.
• Safety and Validation: The automated pipeline includes mandatory Nginx syntax verification (nginx -t) and process status checks to ensure stability before a deployment is finalized.
• Rolling Deployments: To minimize service impact, updates are pushed sequentially across servers; the process automatically halts if an error is detected at any stage of the rollout.

To effectively manage a rapidly expanding portfolio of global services, infrastructure teams should move toward a "configuration-as-code" model that separates common master settings from service-specific logic. Leveraging automation tools like Ansible alongside a native Nginx multi-site structure provides the necessary balance between rapid deployment and the granular control required for complex logging and security requirements.