Database Migration from GCP into AWS Using AWS Database Migration Service
Explore how we overcame multi-cloud management challenges by designing and implementing a seamless database migration from GCP to AWS.
Our Customer
Greenfence is a blockchain platform eco-system developed for the consumer goods industry to enable cost-efficient, transparent and trustworthy commercial relationships at scale. Greenfence empowers stakeholders along the end-to-end value chain to create individualized and secured blockchain networks that can connect, collaborate and transact on a private or public basis.
The Obstacles They Faced
Greenfence encountered challenges in managing its multi-cloud environment due to the use of different providers. More precisely, Greenfence’s application layer was hosted on AWS as a containerized application running on an ECS cluster, while their database layer resided in Google Cloud (GCP) using MySQL as the database engine. The variety of cloud providers presented Greenfence with several obstacles:
- The complexity of application management across cloud platforms.
- The security concerns related to the public-facing data endpoints.
- High total costs of the multi-cloud IT infrastructure.
How We Helped
We analyzed the client’s existing infrastructure, identified pain points, and devised a migration strategy from GCP (Google Cloud Platform) to AWS using AWS Database Migration Service (DMS) to streamline their database layer.
Greenfence’s primary challenge revolved around migrating the database layer from GCP to AWS while ensuring minimal downtime. The migration needed to maintain data integrity, synchronize data in near real-time, preserve the platform’s performance and functionality and avoid disruption to the live web platform. The secondary technical challenge was dealing with the limitations of AWS DMS service when migrating the AUTO_INCREMENT attribute.
Key challenges Greenfence faced included:
- Data synchronization
Managing data consistency and replication between the AWS and GCP environments was a significant complication. The lack of a unified database environment made it difficult to ensure real-time data availability and synchronization across the multi-cloud setup. - Network connectivity and security
Connecting the application layer in AWS to the DB layer in GCP requires establishing secure and reliable network connectivity between the two clouds. Configuring and managing virtual private networks (VPNs), firewalls, and security policies across different cloud providers can be complex, requiring careful planning and coordination. - Performance optimization
Greenfence’s teams needed to optimize the performance of their database layer to meet the growing demands of their application. However, the distributed nature of the database across different cloud providers made the implementation of efficient performance-tuning strategies demanding. - Cost management
Maintaining a multi-cloud setup can lead to increased operational costs due to the need for separate management and infrastructure resources. Greenfence sought a solution to optimize costs while ensuring the required functionality and scalability of its platform.
As we at Romexsoft settled on addressing the obstacles Greenfence faced and simultaneously meeting the no-downtime migration requirement, we implemented the following solution:
- Database migration
We recommended using AWS Database Migration Service (DMS) to perform a homogeneous migration from the existing MySQL database in GCP to Amazon RDS for MySQL in AWS. This approach allowed for a smooth transition of the database layer while minimizing downtime. - Target database engine
Considering the compatibility and familiarity with the existing MySQL engine, we used Amazon RDS for MySQL as the target database engine. This choice ensured a seamless migration process and minimized any potential compatibility issues. - Manual configuration
In order to overcome the limitations of DMS in migrating the AUTO_INCREMENT attribute, we assisted Greenfence in manually configuring the AUTO_INCREMENT attribute on the relevant columns in the target Amazon RDS for MySQL instance. This step was crucial in maintaining the continuity of auto-incremented primary keys. - Configuration for ongoing replication
We configured AWS DMS for ongoing replication to obtain near real-time data synchronization between the source GCP MySQL database and the target Amazon RDS for MySQL in AWS. The achieved seamless continuous replication minimized data latency and enabled the migration process without disrupting the live web platform. - Increasing high availability
In the target database in AWS RDS, we enabled data replication across different availability zones (multi A-Z). - Enhancing security
We implemented data encryption at rest in the AWS RDS database and configured network topology with private subnets and corresponding security groups to provide secured private access to the DB only from the entire VPC where the application cluster is located.