Start of Main Content

Migrating to new technology is a reality that all companies face from time to time. While migrations may appear daunting, when they’re well thought out and effectively executed, performing migrations at the right time and for the right reasons can be a competitive advantage. Whether modernizing your infrastructure, adjusting to new regulatory standards, or embracing emerging technologies, migrating to new systems is vital to keep your organization competitive and innovative.

With the increasing costs of Modern Data Stack solutions, particularly those offering software as a service (SaaS), the rapid evolution of tools, and changing data best practices, organizations are grappling with when and how to migrate their data platforms.

Data migrations are unique in many ways since organizations must consider specific challenges such as data validation, data availability during migration, and the change management needed to roll out new organizational workflows, among others. Due to these challenges, migrations can be long and expensive initiatives, often requiring assistance from external resources with experience in the new technologies to help ensure the existing team remains focused on their roadmap. Moreover, migrations sometimes lack a clear ROI, since quantifying and communicating the value proposition may take a long time, and different stakeholders will value different things.

Because planning and execution are so important to data migrations, we’ve created this blog series to highlight our approach to performing migrations. Our overarching framework is based on a tried-and-true product development cycle: Plan — Build — Ship. We’ll highlight the pillars of this cycle as it relates to migrations, and in future posts, we’ll go into greater detail on each one.


The first phase of data migration is planning. A well-thought-out plan should be able to address questions and cater to different types of audiences in the organization such as:

  • Why are we performing this data migration?
  • What is the scope of this migration?
  • What are the migration’s constraints and requirements?
  • Who is involved in this migration, and how?
  • What does success look like when we’ve finished the migration?
  • What are the risks involved and how are we managing them?

Much of the value in answering these questions is in the process. Understanding why you’re undertaking this project, going through the process of assembling the right team, ensuring the right workflows are in place, and getting stakeholders involved from the beginning are core elements to a successful migration. Ultimately, these and other considerations should be documented, socialized, and aligned on before continuing. The output of this phase will be, minimally, a single document that doubles as both (1) a roadmap with answers to the above, and (2) a tool that you can use to project manage the migration. Keeping these artifacts tightly coupled ensures fewer duplicative efforts and a single source of truth for the migration.


  • Strike a balance between planning too much and planning too little. All plans change, so it’s important to identify which elements your team should have a strong perspective on, and which can be iterated on as you go. Your plan should have just enough flexibility to adjust throughout the migration.
  • Prioritize well. Successful migrations establish momentum but can’t build it if you start in the wrong place. Collaborate with your stakeholders and show them how you’ve decided on which workloads to migrate in what order, and why. Incorporate their feedback in your prioritization and be transparent so relevant parties understand your process.
  • Thoroughly document data dependencies. In a self-service environment, it can be difficult to identify how data is being used. Without complete visibility into data consumer behavior, there are major risks of operational interruption. No migration is worth fracturing trust between your data team and its key stakeholders.


After planning is complete and the roadmap has been socialized and aligned upon, ideally by a steering committee, you’re ready to enter the next phase of the migration — Build. This is the longest phase in the process and it's where the magic happens. The first step in the Build phase is setting up a “landing zone” to determine your data's destination. Then you move the code and data, followed by thorough testing and validation, a critical piece involving numerous feedback loops across your teams. After passing QA, you deploy to a staging environment before proceeding to the Ship phase where your migration gets fully completed.


  • Your teams must align on a successful “QA pass” before migrating workloads. This is especially critical when business logic needs to be refactored. In “lift and shifts” this is less of an issue as you have a source to compare against. For “refactors” it becomes extremely hard to pin down exactly what "done" looks like.
  • Validation often follows a Pareto-like distribution — the final 5% takes as much effort as the original 95%. The business value of the remaining 5% will vary based on your use cases.
  • Be cautious of “moving the goalposts.” Otherwise, you’ll be stuck in endless validation cycles like with refactors.
  • Invest in developer tools and templating up front. Without standardization migrated workloads will look disconnected. Tools and templates can ensure that no matter how your team scales the work delivered is consistent, repeatable, and easier to maintain.

Want help planning your data migration?

We can share details more about our Plan—Build—Ship approach to data migrations and use it to ensure a successful migration.


After the Build phase is completed, you’re onto the final stage — Ship. Shipping changes with extra TLC is important due to the potential impact of data migration work. As your teams look to promote the migrated system to production and begin deprecating old workloads, they should do so gradually.

While the benefits of a technology migration are often clear to data teams, your consumers still rely on the old system and may not realize the benefits of the migration until long after it’s completed. With this imbalance in mind, it’s integral to provide minimal disruption to existing operations and maintain hard-won trust between data and the rest of the organization. It’s also important to maintain existing operations while the migration is underway.


  • Change management is among the hardest elements of a successful migration. During the Build phase, your teams should consider how to onboard the rest of the organization onto the new platform. This includes recruiting evangelists early, showcasing demos, and bringing folks along the journey, so that when the time comes to deprecate old workloads, most folks are already aligned and ready to work in the new platform.
  • Allow time for upskilling your customers/data consumers on how to interact with the new platform.
  • Things will change. Platforms are rarely backward compatible. Maintain a data changelog and over-communicate any expected drift in metrics or data assets that result from the migration.

The Importance of Artifacts and Documentation

Lastly, from beginning to end, it’s critical to create usable artifacts and documentation at each stage of the migration process. Documentation enables the migration team to scale beyond itself, by socializing their work and helping teams onboard themselves. Artifacts, such as training videos, best practices, a data changelog, and validation reports, demonstrate that care was taken during the migration to mitigate operational disruptions. This promotes trust and visibility into the project, which is critical when the change management happens in the back half of the migration process.

As with every project, there are nuances and best practices to each phase which we’re excited to share with you as this series goes on. Stay tuned for more details on how to succeed through each phase of a data migration. Set yourself and your team up for success the next time you upgrade from a legacy system, refactor your data models, or consolidate tooling.

Want help planning your data migration? Contact us. Our data experts can share details more about our Plan—Build—Ship approach to data migrations and use it to ensure a successful migration for your organization.

  • Data Strategy and Governance
  • Data Platform Migration
  • Data Governance

Take advantage of our expertise on your next project