Businesses can no longer count on using the same technologies for decades. Hardware and software become obsolete sooner than ever before. So data migration is an unavoidable challenge each company faces once in a while.
Transferring data from one computer environment to another is a time-consuming, multi-step process involving such activities as planning, data profiling, and testing, to name a few. You can read more about it in our previous article Data Migration: Process, Types, and Golden Rules to Follow. Here, we’ll focus on tools that can save you the lion’s share of tedious tasks — namely, key types of data migration software, selection criteria, and some popular options available in the market.
Types of data migration tools
Data migration projects range from a run-of-the-mill database upgrade to a global shift of the enterprise system to the cloud. Consequently, automation solutions differ as well. There are three major types of data migration software to choose from.
Three types of data migration tools.
Self-scripted tools
Use cases: small projects, specific source and target locations not supported by other solutions
Automation scripts can be written by data engineers or ETL developers in charge of your migration project. This makes sense when you move a relatively small amount of data and deal with simple requirements. Besides, self-scripted tools become a necessity if a source or target location is not supported by any other software.
Among cons of the do-it-yourself approach is the need for coding skills, extra time your engineers have to spend on scripting, and scalability issues.
On-premise tools
Use cases: data migration within an enterprise network, on-premise mergers and acquisitions
On-premises software is meant for migrations within an enterprise network or when mergers and acquisitions take place and moving to the cloud is not on the agenda. Such tools are a natural choice if compliance or security requirements restrict companies from using cloud-based technologies.
The bright side of the on-premise tools is the ability of the company’s IT department to take full control over every single data element and migration step. On the dark side, your engineers must keep everything up and running on their own. Besides, this type of software has limited scalability compared to cloud solutions.
Cloud-based tools
Use cases: moving data from on-premises to cloud or between cloud environments
These tools automate data migration to the cloud or from one cloud location to another. They are usually very flexible and handle different types of data. Besides, cloud-based solutions easily scale up and down depending on your current needs. They typically offer a pay-as-you-go pricing model that wards off wasting money on unused resources.
The greatest concern about cloud solutions is whether it can guarantee protection of data in the move.
Now, that you’ve got a general picture, it’s time to explore key selection criteria in more detail.
How to choose the right data migration solution
When choosing migration software, there many aspects from your project budget to the quality of customer support offered by the provider. Here’s the list of the most essential things to be considered.
Functionality
Fully-fledged data migration software covers the following parts of the migration project:
planning including job scheduling and workflow orchestration,
pre-migration data profiling,
migration execution — or Extract, Transform, Load (ETL) processes,
testing, and
post-migration audit (validation).
Phases of the data migration process.
If a tool lacks functionality, you have to complete it with another solution or task your data engineers with creating scripts to fill the gaps.
Data sources and destinations
Self-scripted migration can deal with any data location and type. However, it will take tons of engineering time to write automation scripts if you are moving files in various formats from multiple sources to different destinations.
Ready-to-use tools, both on-premises and cloud-based, cover a wide range of systems and formats. Nevertheless, you should carefully check whether your files to be migrated and the environments are on the list of supported options.
Performance and scalability
Performance is a top priority for large projects with tons of data to be moved within a reasonable timeframe. As for scalability, that’s where cloud solutions show their best side. They scale up and down according to dynamic needs of data migration. On-premise software, on the other hand, is restricted by the hardware on which it runs.
Ease of use
You can’t count on a user-friendly interface when dealing with free options. But commercial tools should be easy to use. Though not a key factor, intuitive design saves you a lot of time and can signal that the vendor is well-aware of the migration process. Of course, a good look doesn’t necessarily mean that you will face no issues while moving the data. And that’s where customer support comes into play.
Customer support
When selecting a commercial tool, make sure that in case of any issues you’ll get timely help from the provider. Most often, vendors provide support:
by email,
via a forum, or
via a live chat.
With e-mail or a forum, you ask a question and just wait for someone to answer it which can take hours or even days. Live chats are better options to get a quick response. But this is to be checked beforehand. You can read reviews by other customers to understand if the quality and speed of support meet your requirements.
Security
Opt for vendors who clearly state the level of protection they provide for your data in the move. This is especially relevant if you migrate sensitive information that is subject to compliance requirements.
Pricing
The cost of data migration tools vary from zero to thousands of dollars. The final price will heavily depend on the quantity of data migrated, project duration, types of source and target locations, level of support, and other factors.
Free solutions are typically developed by large corporations as a supplement for their main product. They are of a high quality but with limited functionality intended for their core system.
Open-source solutions are either free or significantly cheaper than commercial alternatives. But you get what you pay for. Such as, they can be poorly maintained and updated, have limited functionality, and require coding skills to work with them. Some open-source products are really good — however, you’ll get access to only a few features while the rest is fee-based. Among most popular open-source tools used for migrations are Apache NiFi, CloverDX, and Pentaho Data Integration.
Commercial products offer a wide range of features along with user-friendly interface, training, and technical support, which is most suitable for organizations lacking expertise in data migration. They also tend to cover numerous data sources and targets.
Best data migration software overview
It’s worth noting that tools used to run migrations also cover other types of data transition, such as
data replication — or continuous duplication or making copies of data to store the content in different locations;
data integration — or gathering data from different sources to provide users with a single view of information.
Similar to migration, data integration involves Extract, Transform, and Load procedures. So, products supporting data transfer are often called integration or ETL tools.
With this in mind, let’s examine popular end-to-end, system-agnostic solutions — in light of the mentioned-above selection criteria.
Data migration software comparison.
Astera Centerprise: focus on migrating complex and legacy data
Astera Centerprise is an on-premise data integration solution running on the Windows platform. It is mostly used by medium and large enterprises to migrate complex datasets and modernize legacy systems. Among key clients are Novartis (number 4 in Fortune Magazine’s Most Admired Pharmaceutical companies), Bank of America, and Wells Fargo Bank, to name a few.
Functionality. The software covers the entire migration process, from profiling the source data to validating the content loaded to the target location. It provides a variety of tools to schedule, orchestrate, preview, and monitor the migration flow. Particular attention is paid to the mapping of complex hierarchical data.
Data sources, and destinations. Astera offers pre-built connectors with a wide range of sources and destinations — for example, Oracle, SQL Server, MySQL, PostgreSQL, SAP HANA, Salesforce and more. It features over 200 built-in tools to enable processing any type of data, including complex formats like XML, JSON, EDI, COBOL as well as with unstructured files (texts or PDFs).
Astera Centerprise use cases, components, and supported data sources/destinations.
Centerprise is great for database migrations and processing complex datasets. However, when it comes to metadata, the solution leaves room for improvement as it requires some manual post-processing. Besides that, many clients wish Astera had more pre-built connections with popular cloud storage services and apps.
Performance and scalability. The parallel-processing ETL engine ensures high performance even when you move large datasets. To boost the speed of migration, there is a Job Optimizer feature that allows you to modify the flow at run-time (say, to remove unnecessary operations) and thus reduce execution time. As with any other on-premise software, Astera Centerprise scalability depends on your hardware.
Ease of use. The drag-and-drop interface enables no-coding data migrations, so the process can be handled by data analysts or even business users with no technical background. Simple to set up, configure, and get started with, Centerprise is often referred to as “the most intuitive data migration tool.”
Security. The special Security Manager feature enables you to assign user roles and create different access levels to specific resources and operations.
Customer support. Astera is known for superior support services provided via email or conference calls.
Pricing. The product is rather pricey for small companies, but you can try it no matter your business size — just download a free trial version from the provider’s website.
Talend: a fast shift from anywhere to AWS and other cloud locations
Talend flagship product — Talend Data Fabric — combines data migration, integration, and governance functions in a single cloud-based platform. Along with commercial products, the vendor offers a suite of open source tools named Open Studio. They can be downloaded for Windows or Mac OS.
Functionality. Free solutions are enough to prepare data for migration, orchestrate tasks, and run a simple ETL process. Commercial products cover the entire migration cycle and boast a large number of features including automated quality checks as data moves between locations and visual mapping for complex data structures.
Data sources and destinations. As an AWS Migration Competency Partner, Talend guarantees smooth data transportation from anywhere to Amazon cloud facilities. But it also deals with numerous other on-premise and cloud sources and destinations.
Its free suite supports Oracle, Teradata, Microsoft SQL Server, Salesforce, AWS, Microsoft Azure, Google Cloud platform, and more. Commercial products offer a far longer list of available connectors. It works particularly well with databases and CRM software.
Performance and scalability. Talend is praised for its ability to transfer millions of records at a time without any issues. Even its free ETL tool shows the impressive speed of migration and scalability.
Ease of use. The software features an intuitive drag-and-drop interface. However, to perform complex migrations you will need tech specialists with expertise in Java.
Security. The company has a dedicated team of security experts to ensure data protection at each level, from preventing physical damage of data centers to cloud workload monitoring. Talend products are compliant with main industry standards including HIPAA, GDPR, and CSA STAR (the Cloud Security Alliance Security Trust Assurance and Risk program).
Customer support. Clients can get free support from the Talend Community and find the documentation they need in the Talend Help Center. Professional tech support is fee-based, and the response time depends on the support level (Silver/Gold, Platinum, or Mission Critical) you have purchased.
Pricing. Talend has an open-source version with 900+ free components and connectors that you can download from the vendor’s website. Commercial products come with a 14-day free trial and the price depends on the package you choose and the number of users. Its data integration suite costs $1,170 per user/monthly or $12,000 per user/annually.
Informatica PowerCenter: enterprise-grade migration from multiple sources
A leader in the data integration space for nearly three decades, Informatica PowerCenter helps over 5000 enterprises worldwide manage their data assets in the cloud and on-premises. It is widely used for large data migration projects, especially for moving historical data from legacy systems into new applications. Informatica partners with AWS, Google Cloud, Microsoft Azure, Salesforce, and many other platforms.
Functionality. PowerCenter provides end-to-end functionality for data migration projects. It offers parallel management and monitoring mechanisms, load balancing, repeatable audit and validation of data, and other enhancements.
But if you see room for improvement, your data engineers can create rule-based workflows to optimize the overall process.
Data sources and destinations. The platform easily connects with all leading database management systems (DBMSs), cloud services, application systems, etc. It creates advanced data transformations for XML, JSON, unstructured documents, industry formats (like HL7 or EDI), legacy formats (COBOL). IoT machine data, and more.
The platform is unrivaled for migrating data from different sources But if you deal with a single data source, you can opt for one of the numerous cheaper options.
Performance and scalability. Due to the parallel ETL processing, the solution demonstrates great performance even when dealing with large datasets. It’s scalable enough to cover the needs of enterprise-grade projects.
Ease of use. Though Informatica provides a graphical interface to create data migrations, simplicity is not the strong point of the solution. PowerCenter requires more learning than many other ETL tools. According to product reviews, set up and integration with existing services can be a hassle, while training may take a few weeks.
Security. The vendor provides documentation to manage security within the Informatica domain. The measures encompass authentication of users and services, protection of data transmitted and stored, the security of connections for external components, access control to data and services. You may also employ an additional tool to ensure compliance with regulations like GDPR, CCPA, and CCAR.
Customer support. Customer support is provided via email, phone, or Informatica network. The regional support team is available during local business hours.
Pricing. The high pricing on connectors and overall usage makes this solution unsuitable for mid-size and small businesses. To make sure that the cost is worth it, enterprises can take advantage of a 30-day free trial.
StarfishETL: safe switching between CRMs and ERPs
Founded in 2009 by CRM experts, StarfishETL is built to move core business data between widely-used CRMs and ERPs, both on-premises and in the cloud.
Functionality. The system simplifies the most popular migrations — based on customer feedback. It performs commonly requested mappings, ensures data quality, and allows you to schedule and control workflows. But you can go beyond pre-built features and run more complex migration projects using self-scripted tools. StarfishETL supports Python, C#, JavaScript, and several other scripting languages to expand your possibilities.
Data sources and destinations. The solution focuses on CRM systems ensuring quick move from one to another. However, it also offers pre-built connectors with popular databases (MongoDB, SQLite, PostgreSQL, MySQL, Oracle, Amazon Redshift, and others) and supports Excel and XML files.
Performance and scalability. The platform runs on AWS and takes advantage of Amazon Elastic Cloud capabilities that make the migration tool suitable for companies of different sizes. It is powerful enough to quickly move information from legacy CRM and ERP systems to the cloud for midsize businesses. Complaints arise when dealing with huge amounts of data. Also, many clients wish the platform could run more than one scheduled job at a time.
Ease of use. The tool aims at computer-savvy users who have scripting experience and a basic understanding of data types, databases, and web services. For those without а technical background, it won’t be quite simple to make the most of the system and its capabilities.
Security. To ensure safety, the system applies strict data encryption creating two-fold security protocols between their internal servers and AWS. StarfishETL also protects information from system outages by backups.
Customer support. You can find technical documentation, ask questions to the community, or leave a ticket describing your problem on the Support Portal page. Another option is to submit a consultation request to the dedicated support team. The response time depends on the severity of your problem and the level of your subscription. It ranges from one business hour to one week.
Pricing. The fee depends on the number of connectors and transactions and starts from $495 per migration from a single source to a single target. Some additional options are really expensive for small businesses.
There is still a lot of work for data engineers
Every data migration project is unique, and its success relies heavily on the right software. So, don’t make a hasty decision — take your time and do deep research, weighing up pros and cons. It’s strongly recommended using a free trial to understand if the solution fits your case.
But even with the best software in place, you will still need data engineers to define a migration strategy, run the migration process, validate results, and fix any issues. A tool is as good as the hands using it — this statement is fair enough for migration of any complexity.
