5 Tips to Migrate from MongoDB to Amazon DocumentDB

Organizations primary focus during database migrations is successful movement of production to new databases however, this should not consume all resources in the organization. Data migration is the most crucial step during the migration project and the end objective is to get a perfect replica of the production database onto a new system so that maintenance window can be reduced to bare minimum and minimal interruption to end user services.

In today’s topic we will learn about data migration from MongoDB to Amazon DocumentDB database, important key factors to consider during the data migration and actual data migration.

Key Areas for Data Migration: MongoDB to Amazon DocumentDB

Compatibility

The very first step here is to understand the application using the MongoDB Atlas database currently compatible with the new database Amazon DocumentDB? Will application after porting work As-Is or would need some changes? Migration of databases might involve changes into the hosting application. It is important to understand what changes are required and resources needed to fulfill this requirement. Amazon DocumentDB having compatibility with (MongoDB Atlas 3.6 and 4.0 tools / drivers.) Responses are emulated by Amazon DocumentDB expected by clients from the MongoDB Atlas server using Apache 2.0 open source MongoDB Atlas 3.6 and 4.0 APIs.

These APIs key characteristics support a fault-tolerant, highly scalable, distributed, self-healing system for critical workloads. Amazon DocumentDB has a compatibility tool in-built to examine MongoDB log files or source code from MongoDB Atlas application to understand if there are any queries of users that are not supported by Amazon DocumentDB. The tool output is a report on unsupported operators and filenames to investigate further.

Migrate from MongoDB to Amazon DocumentDB

Sizing

What should be the ideal sizing of the Amazon DocumentDB cluster? Sizing requires extensive testing as there is no perfect way to size the cluster. It all depends on if given enough data as well as learning from past such exercise of sizing is available. To do near to perfect sizing it is crucial to determine some key factors as under:

Existing database structure – MongoDB Atlas server mention (CPU, memory, IO and networking) and resources utilization (CPU, memory, IO (avg. peak read/write), transmit / receive bandwidth) which could be a good starting point.

Data details – number of documents, document size number averaged out, number of indexes, size of index, query per second or per minute (insert/update/delete) actions

Application service level agreements (SLAs) – query response time and application critical data change requirements

Amazon DocumentDB sizing calculator is available to support this.

Testing

Application performance during peak, capability to handle load and speed. Once compatibility and sizing review is done, testing by performing one-off data migration. There are several testing methodologies that can be used – automated vs manual. The entire testing methodology is broken down into three areas to focus on majorly:

Correctness – Application behaviour on a new database platform. Application need to pass all correctness tests
Performance – Application KPIs and metrics are based on CRUD operations / per second which is a good criterion to measure application performance on a new database. If an application is underperforming as per laid down KPIs and metrics it is time to relook at cluster sizing, fix it and perform testing again.
Load – If Amazon DocumentDB passes the performance test then we need to know how it is going to perform in worst case peak load situations. In vertical scaling Amazon DocumentDB can scale when the primary node size is increased. In horizontal scaling 15 read-replicas can be added to the primary AWS region. Five additional regional read-only replicas and 16 instances are supported by Amazon DocumentDB clusters.

Actual Data Migration

What disruption do we expect during the migration? Can migration be undertaken with zero downtime or impact ? Do the migration activities fit in an acceptable maintenance window? All migrations are supported using AWS database migration service (DMS). It is any easy to use tool with two major components which aid in migration:

DMS full load – Using this option once can migrate one or more MongoDB Atlas databases or specific databases to the DMS console. Full load option synchronizes simultaneously multiple collectors and parallelly load of a single collection using segmenting functionality in DMS.

DMS Change Data Capture (CDC) – Once initial point in time synchronization is accomplished. DMS supports keeping Amazon DocumentDB in sync with MongoDB Atlas deployment. On MongoDB Atlas 3.x CDC uses replica set Oplog. MongoDB Atlas 4.x uses a change stream for mirroring all changes to Amazon DocumentDB.

From DMS console the complete migration process is monitored.