Automating RDS Postgres to Aurora Postgres Migration

Netflix standardized on Amazon Aurora PostgreSQL after finding that PostgreSQL already supported most relational workloads and that Aurora offered stronger scalability, availability, elasticity, and ecosystem alignment. To migrate nearly 400 RDS PostgreSQL clusters efficiently, Netflix built a self-service workflow that automates replication, traffic quiescence, validation, and cutover while minimizing downtime and eliminating data loss. The Aurora read-replica method is preferred over snapshot migration because it keeps the target nearly synchronized while production continues running.

Why Netflix Chose Aurora PostgreSQL

• PostgreSQL already supported the majority of Netflix’s relational workloads.
• Internal evaluations found Aurora PostgreSQL could support more than 95% of workloads running on other relational database systems.
• PostgreSQL benefits from:
  • A broad open-source ecosystem
  • Strong community adoption
  • Compatibility with modern data platforms
• Aurora’s distributed, cloud-native architecture provides:
  • Better scalability and elasticity
  • High availability
  • Support for globally distributed applications
• The migration effort began with RDS PostgreSQL and is intended to expand to other relational systems.

Database Migration Requires More Than Data Copying

A safe migration must move both data and database functionality while preserving correctness, availability, and performance.

• Data replication: Copy existing data and continuously apply source changes to the destination.
• Quiescence: Stop writes to the source so the destination can catch up completely.
• Validation: Confirm that source and destination data are synchronized.
• Cutover: Redirect applications to the new Aurora database as the system of record.

Operational and Technical Challenges

• Manually migrating almost 400 PostgreSQL clusters would be slow, error-prone, and operationally expensive.
• Coordinating downtime across dependent services is difficult.
• Netflix therefore created a self-service workflow that handles orchestration, safety checks, and correctness guarantees automatically.
• The system must guarantee:
  • Zero data loss
  • Extremely short downtime, especially for critical services
  • No performance degradation during or after migration
  • Migration of related resources such as parameter groups, read replicas, and replication slots
• Application teams control database clients, so the platform cannot depend on them manually pausing writes.
• The migration system must provide control-plane mechanisms to halt traffic safely during validation and cutover.
• The workflow must operate without obtaining RDS credentials from users, since databases may be tightly secured and the migration platform may lack direct database access.
• Because non-experts operate the process, the experience must be self-guided and require minimal user effort.

Snapshot-Based Migration

The snapshot approach is straightforward but requires stopping writes before migration.

• Halt write traffic to the RDS PostgreSQL source.
• Create a manual snapshot.
• Convert the snapshot into an Aurora-compatible format.
• Create an Aurora PostgreSQL cluster from the converted snapshot.
• Validate the new cluster.
• Redirect applications to the Aurora endpoint.

This method is simple but can involve a longer interruption because the target is not continuously updated while the snapshot is created and converted.

Aurora Read-Replica Migration

The read-replica approach reduces downtime by continuously replicating the RDS database into Aurora.

• Create an Aurora PostgreSQL read replica from the RDS source.
• Stream changes asynchronously from RDS to Aurora while applications continue using the source.
• Provision and validate Aurora configuration, connectivity, and performance in advance.
• When replication lag is sufficiently low, briefly pause writes.
• Allow the replica to catch up fully.
• Promote it to a standalone Aurora PostgreSQL cluster.
• Redirect application traffic to the Aurora endpoint.

This approach keeps the destination nearly synchronized before cutover, making it substantially less disruptive than snapshot-based migration.

Netflix’s automation focuses on making the read-replica migration process safe, repeatable, and self-service, with the platform handling replication, traffic control, validation, and cutover rather than relying on manual application-team coordination.