Data Migration in the

Enterprise World
Dissecting the What, the Why, and the How
 Summary
Data migration enables organizations to expand their data store and management capabilities, and make full use of
data to drive business decisions. It is a widely used process of moving data, and almost every organization undertakes
one at some point.

This eBook will give you insights into the data migration process. It will further discuss why this process is essential for
organizations, the challenges they might encounter during the process, and finally list down the techniques and tools that
can simplify and streamline data migration.

Data Migration in the Enterprise World: Dissecting the What, the Why, and the How
Table of Contents
THE WHAT ....................................................................................................................... 04
What Is Data Migration? 05
Types of Data Migration 06

THE WHY ......................................................................................................................... 09

Situations That Prompt Data Migration 10

Challenges in Data Migration 10

Factors that Impact the Success of a Data Migration Process 11

THE HOW ......................................................................................................................... 13

Data Migration Techniques 14

The 7 R’s of Data Migration 15

Data Migration Tools 15

Finding the Right Migration Software: Features to Consider 16

Astera Centerprise: Making the Data Migration Process Painless 17

Planning to Migrate? Steps to A Successful Enterprise Data Migration 17

CONCLUSION ................................................................................................................. 23
 The What
Know the Basics

Data Migration in the Enterprise World: Dissecting the What, the Why, and the How | 04
What Is Data Migration?
According to the International Data Corporation (IDC), data
migration represents 60% of any large enterprise IT project.
But what is it?
Simply put, it refers to the data transfer between different types of file formats, databases, and storage systems.
However, ‘transfer’ is not the only aspect of data migration. The success rate of any data migration project is directly
dependent on the diversity, volume, and quality of data being transferred. If the data is diverse, the migration process
includes mappings and transformations between source and target systems. Above all, data quality must be assessed
before migration to ensure a successful implementation.

Data Migration in the Enterprise World: Dissecting the What, the Why, and the How | 05
Types of Data Migration
Data migration can be broadly categorized into four types:

Database Migration
This type of migration involves moving data between two
database engines.
However, when the original data changes, it simultaneously affects the data language or protocol. The change is reflected
in the application layer as well. Database migration modifies data without modifying the schema.

The circumstances that demand this type of migration include:

• When the database software requires an update
• To migrate a database to the cloud
• In case the organization needs to change database vendors

It requires careful planning and testing as there are several small tasks involved in the process, such as determining the storage
capacity of the target database, testing applications, and ensuring data confidentiality.

Application Migration

This type of migration occurs when an organization switches

from one platform or vendor application to another.

Data Migration in the Enterprise World: Dissecting the What, the Why, and the How | 06
Each application has a unique data model. Moreover, applications are not portable. Therefore, the operating systems,
virtual machine configurations, and management tools of each application can be different in the development and
deployment environment.

These factors can contribute to the complexity of the migration process. Introducing middleware products in the
process simplifies it; as they help bridge the technology gap.

Storage Migration
The process deals with moving data from one storage
system to another, such as a hard disk or the cloud.
The need for storage migration arises due to technology upgrades. It is easy to implement data management features
like data validation, cloning, reducing corrupt or old information, etc., during this process. In addition, storage migration
enables cost-effective scaling and faster performance.

Data Migration in the Enterprise World: Dissecting the What, the Why, and the How | 07
Cloud Migration
Moving to the cloud ensures scalability, requires fewer storage resources, and is cost-effective. This is why cloud
migration has become one of the latest trends in the data management industry.

In cloud migration, an organization’s complete or partial

information assets, applications, or services are deployed
to the cloud.
The cloud’s firewall protects the migrated data.

Data Migration in the Enterprise World: Dissecting the What, the Why, and the How | 08
 The Why
Reasons for Migrating Data

Data Migration in the Enterprise World: Dissecting the What, the Why, and the How | 09
Situations That Prompt Data Migration
The most common reason for migration is the need
to move data to a new system to scale up and accommodate
volume growth.
However, other circumstances may also prompt organizations to opt for a data migration project. The reasons include:

• To replace legacy systems that are unable to keep up with the evolving performance requirements of the organization

• To reduce storage footprint by migrating to a system that consumes less power and space. Consequently, cutting down
the operational costs.

• To stay competitive by adopting state-of-the-art technologies, paving the path for data migration

• To prepare a disaster recovery plan; as data migration can help create a secure

• To transfer data to the cloud, eliminating the cost of on-premise IT infrastructure

Challenges in Data Migration

Migrating enterprise data from one system to another is always challenging, especially when moving millions of
records across systems.

To help simplify this process, we’ve identified the most common data migration challenges and ways to avoid them:

1. The Complexity of Source Data

One of the main challenges in data migration is the complexity of source data. Often, data integration has to be performed
on the source data to work through any intricacies. For example, when migrating from a legacy data storage system, data
needs to be transformed to ensure compatibility with the target system.

2. Loss of Data or Corrupt Data

Migrating data from one system to another might result in data loss or corruption. This issue can be tackled by ensuring that
the number of source records matches the number of migrated records. Various tools are available in the market that can
help authenticate the migrated data.

3. Need for In-Depth Testing and Validation

For complex data migrations that involve thousands of records, the cost of failure is simply too high. Therefore, the need to
perform in-depth data validation and testing cannot be ignored. Keep in mind any events that could affect data quality. Also,
test a large chunk of data (10 – 20%) to assure quality.

Data Migration in the Enterprise World: Dissecting the What, the Why, and the How | 10
Factors that Impact the Success of a
Data Migration Process
A fully functional data migration process must include the following considerations:

Is Your Migration Project Getting the Attention It Needs?

Selecting a new enterprise-wide system is a strategic business undertaking that deals with new technologies and
professionals. Migration is usually a small part of a much bigger project. An average business typically focuses on system
configuration and other technicalities rather than ensuring the data that will populate the new system is fit for purpose.

Holistically speaking, out of the plethora of tasks, data migration falls low on the list of priorities to the management that
considers it to be a simple task of transferring data, and does not find the high costs and administrative burden justifiable.
Thus, data migration, the resources it requires as well as the difficulty of the task is often underestimated and put on the
backburner until it’s too late. Make sure that’s not the case.

Thoroughly Understand the Design Requirements

Firm grasp of the design requirements is critical, namely migration priorities and schedules, replication and backup settings,
and capacity planning. A simple miscalculation in this area can have lasting repercussions that may impact the cost. This is
also the stage where the IT department decides which migration strategy is suitable for the project – Trickle or Big Bang.
Selecting the right one is highly subjective to the firm’s existing data and future requirements. Let’s have a look.

Big Bang – This type of migration initiates and completes full transfer within a limited time frame. Expect some downtime as
the data is moved and processed, however the project execution is rather quick.

Trickle – This migration strategy conducts the project in multiple phases. The target and source systems run simultaneously,
which keeps the migration running in real-time. This type of migration is more complex and time-consuming as compared to
Big Bang, however it takes lesser downtime and eliminates operational efficiencies.

Budget for the Field Expert

Quite a few tech firms prefer a hands-on approach, and migration budgets simply don’t allow for an expert’s input.
Nevertheless, unless the firm happens to have an in-house migration specialist, they will need to spend money to
save resources money by hiring an expert.

Collaborate with the End Users

The data migration process should be considered a business project that involves end-users, rather than a technical set of
steps. Your staff or customers will have a stake in the migration and understandable anxiety over its success.

Data Migration in the Enterprise World: Dissecting the What, the Why, and the How | 11
Involve them: Depending on the specific data rules you plan to implement, consider which data should receive priority to
migrate first. In addition, try to understand and implement what the end-users are hoping from the migration: Better perfor-
mance? Analytics? A simpler way to issue requests?

Following this approach will result in a unified and comprehensive approach towards migration that takes into account the
concern of the stake holders, and saves considerable time and cost in the long run.

Migration Isn’t Done in One Go

It’s quite common for a data migration project to have multiple phases. This is a known engineering practice which breaks
down the entire process of migration into manageable chunks instead of attempting to resolve migration, risk errors, and
redundancies in one go. In some migration cases, the first attempt fails and the process has to be restarted. With multiple
phases, it performs easier to perform integrity checks at the end of each interval as the migration progresses.

When compared to data migration, storage migration is far simpler as you don’t need to update old storage and map it to the
new system. However, migrating data between vastly different storage systems is the real challenge. Using software that have
a code-free environment with built-in data quality features, advanced transformations, and support for a wide variety of data
formats simplifies storage migrations.

Backup Source Data

With massive amounts of data and restricted storage space, some data can’t be backed up. Should the worst happen, losing
invaluable data during migration becomes inevitable. therefore, you must be well-prepared with backups to restore data and
try again. Since most databases usually have terabytes upon terabytes of data, it is understandable to run out of storage
space. In such a case, arrange for more backup space without hesitating. Afterall, it’s hard to put a price on the data you’ve
spent years accumulating.

Migration Doesn’t Make Old Systems Useless

It’s quite likely that the old and new enterprise systems might work in collision. Taking the old system offline immediately can
prove to be counterproductive as the new rollout may experience some hiccups until it’s completely operational for the end user.

Plan for the Future

Once all the data has migrated successfully, attempt to test the migration again using a mirror of the production environment.
Once it checks out, the next step is to go live and carry out final tests. When the environment is operating smoothly, it’s safe to
shut off the old systems.

In addition, it’s better to take initiatives to simplify the post-migration process. Rather than spending costly resources for
updating source data before migration, establish analytics and governance controls in this new environment. Continuously
monitoring migrated data to look for unusual access patterns, orphaned work sets, and security can also help you in the long
run. This will ensure that effective utilization of enterprise data.

Data Migration in the Enterprise World: Dissecting the What, the Why, and the How | 12
 The How
Perform Successful Data Migrations Easily

Data Migration in the Enterprise World: Dissecting the What, the Why, and the How | 13
Data Migration Techniques
Several factors determine the right migration technique for an organization, such as the available resources, data volume,
data sensitivity, and business requirements. The method that offers a blend of reliability, migration efficiency, and
minimum impact on users and business processes is the one that can best serve the organization’s needs.

TDWI conducted a survey to identify the technologies

organizations use to migrate or consolidate databases. ETL
emerged as the preferred technology with 41% respondents
voting in its favor. Other answers included: manual coding
(27%), database replication (11%), and Enterprise Application
Integration (EAI) (3.5%).

ETL Emerges as the Preferred Choice for Data Migration

Data Migration in the Enterprise World: Dissecting the What, the Why, and the How | 14
Extract, Load, Transform (ETL)
ETL tools can handle the complex requirements of the data migration process. These include processing big data sets,
in-depth data profiling, and integration between multiple platforms. Some ETL tools even provide the capability of
automating the standard ETL tasks, such as retrieving data from operational systems, transforming it into a unified
format, and loading it into a destination database.

The 7 R’s of Data Migration

When planning a data migration strategy, it’s imperative to keep in mind these seven critical characteristics:

• Robust: Efficiently manage all phases of data migration ranging from data extraction and transformation to
authentication and loading into the destination system, even when data volume is high
• Resilient: Easy to modify and handle problems in the operating environment

• Rapid: Efficient process execution, leveraging the present source or destination facilities to support quick processing

• Reporting: Progress pointers during the migration process should be available

• Reconciliation: Reconcile the completed process

• Recoverable: The data should be retrievable from the point of failure when needed
• Reusable: Should have reusable process modules, including transformation utilities, error control, and data cleaning
procedures that can be used in other tasks without impacting the already migrated data

Data Migration Tools

Data migration tools are broadly divided into three types:

On-Premise Open-source Cloud-based

Installed on-premise to enable Community-developed tools Available on the cloud, these
data transfer within the available at a low price or free tools move data from different
organization of charge systems (including applica-tions,
cloud and on-premise data
stores, services) to the cloud

Data Migration in the Enterprise World: Dissecting the What, the Why, and the How | 15
Finding the Right Migration
Software: Features to
Consider
The key to selecting the right data migration tool is research.
Browsing through review websites like Capterra, G2 Crowd, etc. will
give you a fair idea of what options are available in the industry.
The next step is to know about the basic features that will help you perform successful enterprise data migration. Here are the
important features that you should consider when selecting a data migration tool:

Easy Data Mapping

To carry out a successful data migration process, it’s essential to correctly map data from source to destination. To enable business
personnel and data professionals use these processes easily, it’s best to go for a tool that offers a code-free, drag-and-drop,
graphical user interface, and preferably auto-mapping.

Advanced Data Integration and Transformation Capabilities

Executing a successful migration project involves extracting data from the desired source, identifying quality issues and errors
through profiling, and transforming it to follow the destination schema. A data management solution that offers connectivity to a
wide range of heterogeneous sources, ETL, data validation, and transform features restructures data for targeted delivery and
ensures that the enterprises can execute complex migration projects and get maximum value from the migrated data.

Enhanced Connectivity
A data migration tool should seamlessly connect with disparate source and destination data structures, such as XML, JSON, EDI, etc.
If your business requires connectivity to business applications (Marketo, Salesforce, etc.) or databases (IBM DB2, PostgreSQL), look
for a tool that offers pre-built connectors for these applications.

Automated Data Migration

Features like workflow orchestration, dynamic layouts, and job scheduling eliminate manual work, streamline data processes, and
enable faster data migration. Especially if your migration project involves large data sets, automating the process offers additional
benefits, such as reduced implementation time, lower project costs, and improved ROI.

Data Migration in the Enterprise World: Dissecting the What, the Why, and the How | 16
Astera Centerprise:
Making the Data Migration
Process Painless
Astera Centerprise is an all-inclusive data integration solution
that enables businesses to accomplish faster data migrations in
an easy-to-use, code-free environment.
With the capability to process large data volumes and complex hierarchal structures like EDI, XML, JSON, and more, Astera
Centerprise has successfully carried out complex data migration projects for several Fortune 1000 companies.

Also, the software offers a combination of automation, built-in data quality, profiling, and cleansing features,
advanced transformation capabilities, and extensibility to add custom functions.

Planning to Migrate?
Steps to A Successful
Enterprise Data
Migration
Data migration is more than just moving data for one
repository to another. Executing a successful transfer
includes the following steps:

Data Migration in the Enterprise World: Dissecting the What, the Why, and the How | 17
1. Design a Strategy

Selecting a strategy depends on the need for data migration. Is

it because the organization is consolidating systems following
an acquisition or a merger or is it because of data overload?
Solidifying the business objectives will lay the groundwork for the data migration plan.
Next, the organization needs to decide whether to go for data migration services or enterprise-grade ETL tools
designed to facilitate data migration. There are several service providers that will identify the source and destination
systems, generate workflows, and migrate the content. Alternatively, purchasing a code-free data migration solution
gives you the liberty to carry out the transfer at your convenience, speeds up the project, and reduces risks.
Simultaneously, it can lower the overall project cost.

By strategizing beforehand, the organization can inform business and IT users regarding the goals of the migration
project and ensure enterprise-wide successful adoption and implementation.

2. Assess and Analyze

The next step is to analyze enterprise data. Answering the following questions can help with this assessment:

• How much of the enterprise data requires migration?

• Is the data structured or unstructured?
• Is data redundancy going to be an issue?
• How much of the data is ROT (redundant, obsolete, or trivial)?
• Is the information old or recent?

Comb through all the workflows and shared drives that store data to figure out its use throughout the organization.

3. Collect and Cleanse Data

This step involves removing ROT data, compressing the

remaining content, and converting enterprise-wide data
into a single format.
For example, in the figure below, data profiling transformation is applied on the order data of a company to examine
the available information and remove duplicate/redundant records. The figure following that shows the details of the
data set.

Data Migration in the Enterprise World: Dissecting the What, the Why, and the How | 18
 Data Profiling

Data Profiling Result

Manual execution of these tasks can be time and resource intensive for large datasets. Therefore, finding a solution
that automates these tasks can make the migration process efficient and error-free.

Data Migration in the Enterprise World: Dissecting the What, the Why, and the How | 19
4. Sort Data
Once you have profiled the data into a high-quality and usable
form, the next phase is to categorize it according to the
migration requirements.
The categorization can be done on the basis of product type, ID, or any other criteria. It makes easier to route data
into the right buckets.

Continuing the above-mentioned example, figure. 4 shows the profiled data sorted with respect to the Product ID.

Applying Sort Transformation

Data Migration in the Enterprise World: Dissecting the What, the Why, and the How | 20
5. Validate Data

This step involves reviewing the execution process. Assess the

data rules and check if they are working the way they are
supposed to, and map out any exceptions in your dataflow.
Even if the process is automated, it’s best to keep a check on it to ensure that you don’t encounter any unexpected
challenges when implementing the process.

After sorting the data, we will apply the data quality rules to determine the errors based on defined rules. In this scenario,
data quality rules are used to identify and eliminate the records in which the field ‘Quantity’ has a zero value.

Applying Data Quality Rules

Data Migration in the Enterprise World: Dissecting the What, the Why, and the How | 21
6. Migrate

This is the final step in which everything falls into place. The steps
mentioned above will generate well-organized and clean datasets;
all that is left is migrating them from one system to another.
Once the data is migrated to the targeted destination, which can be a database or a data repository, such as a data mart or a
data warehouse. The employees can then access it to fuel data analytics, streamline workflows, enhance data security, and
more.

In the example below, the data is migrated to SQL server after being profiled, sorted, and checked for errors.

Data Migrated to the Destination Database

Data Migration in the Enterprise World: Dissecting the What, the Why, and the How | 22
Conclusion
Migrations that are not completed on time can prove to be costly, both in terms of revenue and reputation. To expedite data
migration projects and minimize the risk of failure, your business needs an enterprise-grade tool that can automate the
repetitive tasks associated with data migration while ensuring data quality and minimum downtime.

Astera Centerprise is an end-to-end data integration solution that is designed to cater to the complex data needs of
businesses. When migrating data between two different versions of a database engine or performing a heterogeneous
migration, the software can help you with its extensive range of features.

Interested in learning more? Watch this quick demo, download a Astera Centerprise trial, or contact us to discuss your
data migration use-case.

Data Migration in the Enterprise World: Dissecting the What, the Why, and the How | 23