ECS ™ Procedure Generator

Solution for Validating your engagement

Centera to ECS Migration - ECSSync

Topic
ECS PS Procedures
Selections
ECS Professional Services Procedures: Centera to ECS Migration - ECSSync

Generated: December 10, 2019 4:39 PM GMT

REPORT PROBLEMS

If you find any errors in this procedure or have comments regarding this application, send email to
[email protected]

Copyright © 2019 Dell Inc. or its subsidiaries. All Rights Reserved.

THIS DOCUMENT IS RESTRICTED FOR THE INTERNAL USE OF EMC CORPORATION (“EMC”)
ONLY. THIS DOCUMENT AND THE INSTRUCTIONS CONTAINED HEREIN ARE FOR USE BY
APPROVED EMC SERVICE PERSONNEL ONLY. ANY USE OF THIS DOCUMENT AND/OR ITS
INSTRUCTIONS BY ANYONE OTHER THAN AUTHORIZED EMC SERVICE PERSONNEL
(“UNAUTHORIZED USER”) IS NOT AUTHORIZED, LICENSED OR PERMITTED BY EMC.
UNAUTHORIZED USER ASSUMES SOLE AND COMPLETE RESPONSIBILITY AND RISK FOR ANY
AND ALL LOSSES OR DAMAGES ARISING OUT OF OR IN CONNECTION WITH ANY SUCH
UNAUTHORIZED USE.

Dell, EMC, Dell EMC and other trademarks are trademarks of Dell Inc. or its subsidiaries. Other
trademarks may be the property of their respective owners.

Publication Date: December, 2019

EMC Confidential Information- for internal use only version: 2.0.4.0

Page 1 of 26
 Contents
Preliminary Activity Tasks .......................................................................................................3
Read, understand, and perform these tasks.................................................................................................3

How to Perform a Centera to ECS Migration using ECSSync 3.x ..........................................4

Overview:.................................................................................................................................................4
Reference sites:.......................................................................................................................................5
Design:.....................................................................................................................................................6
Migration speed. ......................................................................................................................................6
Deploy and Configure the OVA: ..............................................................................................................6
Centera Preparation ................................................................................................................................7
ECS Preparation....................................................................................................................................10
Generate a list using JCAS....................................................................................................................17
Start the Migration .................................................................................................................................21
Working with MariaDB ...........................................................................................................................23
Troubleshooting .....................................................................................................................................25

EMC Confidential Information- for internal use only version: 2.0.4.0

Page 2 of 26
Preliminary Activity Tasks
This section may contain tasks that you must complete before performing this procedure.

Read, understand, and perform these tasks

1. [ ] Table 1 lists tasks, cautions, warnings, notes, and/or knowledgebase (KB) solutions that you
need to be aware of before performing this activity. Read, understand, and when necessary perform
any tasks contained in this table and any tasks contained in any associated knowledgebase solution.

Table 1 List of cautions, warnings, notes, and/or KB solutions related to this activity

2. [ ] This is a link to the top trending service topics. These topics may or not be related to this activity.
This is merely a proactive attempt to make you aware of any KB articles that may be associated with
this product.
ECS Top Service Topics

EMC Confidential Information- for internal use only version: 2.0.4.0

Page 3 of 26
How to Perform a Centera to ECS Migration using ECSSync 3.x
This procedure provides the steps to migrate from Centera to ECS/CAS using ECSSync
Prerequisites:
 Minimum software versions:
 Sync 3.1
 ECS 3.0
 Centera 4.2
 Application SDK 3.1.544
 Required ports: 3218,22 and 25
 Delivery folks had to have gone through the ECS migration workshop and shadowed up to 2 sync
migrations

Note: Sync does not support migrating advanced retention data. Native must be used.

Note: CUA is not supported on ECS. Send an email to [email protected] if you require assistance
on the replacement options. Sync does have a CUA plugin to migrate onto another gateway or NAS.

Note: Refer to https://inside.emc.com/docs/DOC-197329 this includes the pros and cons between using
Native and Sync

Overview:
ECSSync is an external tool developed by the Object EMC Development team. The tool comes pre-
configured as an OVA to be easily deployed into a customer’s ESX environment. The tool supports
Centera and Atmos/CAS to ECS CAS, S3 to S3, Atmos Rest to ECS and nas to nas. Unless it’s a dark
site this service can be performed remotely.
ECSSYNC flow:
 Object is read from Clip List
 Object is transferred
 Object is verified
 Database is updated as an error or verified
 Next object is processed
 ECS Sync is multi-threaded to ensure parallelism and performance
 ECS Geo replication will maintain VDC synchronization
 ECSSync OVA
 OVA bundle includes.
 CentOS7
 16B RAM
 200 GB disk (only 80GB is used up front)
 Single NIC at 1GB or 10GB

EMC Confidential Information- for internal use only version: 2.0.4.0

Page 4 of 26
  Quad core
 MariaDB
 SDK binaries
 Sync scripts
 Xml templates
 Java

Note: Reach out to [email protected] for assistance on a manual install If the customer cannot
support the OVA

 ECSSync supports the following functionality:

 Migration Pause, Resume, Stop and Status
 Thread control
 Clip list
 Pool level migration
 MD5 checksum Validation
 Quite, verbose, silent and debug logging
 Retry of a failed clip
 Database to log all migration transactions
 Xdoctor for email home
 Runs as a service
 Log files are rolled over and compressed
 Multiple jobs per sync server
 If the sync stops it will resume where it left off as long as the DB table it was populating is still
present.

Reference sites:
Following sites will assist with gaining information or assistance
 PS assistance: [email protected]
 ECS CSE: [email protected]
 ECS Community [email protected]
 ISV Validation: http://vr.solarch.lab.emc.com
 Centera Community tools https://community.emc.com/docs/DOC-2393
 PS Services breakdown: https://inside.emc.com/docs/DOC-197329
 ECS Hardware: https://support.emc.com/products/36913_ECS-Appliance-
Hardware/Documentation
 OVA download location: https://github.com/EMCECS/ecs-sync/releases


EMC Confidential Information- for internal use only version: 2.0.4.0

Page 5 of 26
Design:
Below is an example of how to design a migration to gain optimal speed.
 Multiple sync servers can be deployed; rule of thumb is 1 sync server for every 2 Centera access
nodes.
 Best practice is try not go over 4 servers per site with each server running no more than 2 jobs
 To speed up the migration ask the customer for more access nodes if they have less than 4, this is
non-intrusive and only requires an Ethernet cable to be run.
 Always obtain the health reports prior to the migration to build out the migration configuration guide
 Design example: 4 applications, Centera A and B and VDC C and D, 8 access nodes
 Request 4 sync servers for each site. Prod site has sync 1-4 and dr has sync 5-8
 Obtain a clip list from the 4 applications
 Run 2 jobs on sync 1, 2. 5 and 6 for app 1 and 2 ,one job is app 1 and the other job is app 2
on each server, use the other 4 servers for app 3 and 4
 If the customer wants to do 1 app at a time, split the list by 8,run 1 job per server

Migration speed.
 For objects less than 250KB expect 1.5 Million clips a day per job per server to get migrated and
validated
 For objects over 250KB Expect 2TB a day per job per server
 These are conservative numbers, best be low then high, speeds up to 8TB and 3M objects a day per
server has been observed

Note: To perform the migration remotely establish an ESRS connection to either the Centera or ECS and
ssh over to the sync server or open a webex session

Deploy and Configure the OVA:

Note: The OVA deployment is typically performed by the customer

3. [ ] Upload the OVA to a datastore

4. [ ] From a vsphere client connected to either an ESX host or VCenterselect select File > Deploy
OVF Template
5. [ ] Power on the VM
6. [ ] Open a Virtual Machine Console to the VM
7. [ ] Login is ecssync:ecs-sync-ova
8. [ ] Change the FQDN hostname to ecs_sync1, for the 2nd server ecs_syn 2 for an example Enter:
vi /etc/hostname
9. [ ] Add ecs_sync1 to /etc/hosts, Enter:
vi /etc/hosts,
127.0.0.1 ecs_syncX localhost localhost.localdomain
 Where ‘X’ is the number of the ecs_sync server you are working on.
10. [ ] Configure the network, Enter:
cd /etc/sysconfig/network-scripts/

EMC Confidential Information- for internal use only version: 2.0.4.0

Page 6 of 26
 mv ifcfg-ensxx ifcfg-ensxx.org
vi ifcfg-ensxx enter the following parameters
DEVICE="ensxx"
ONBOOT=yes
BOOTPROTO=static
IPADDR=
NETMASK=
GATEWAY=
DNS1=
 Reboot
 Verify the hostname is correct
 Bring up the interface Enter:
ifup /etc/sysconfig/network-scripts/ifcfg-ensxx
 Ping the gateway to verify network connectivity

Centera Preparation
For every pool you will need to update each profile to create a merged pea file for the source and target.
11. [ ] Using Centera viewer login into the Centera’s as admin:centera, if the admin password does not
work ask the customer to log you in
12. [ ] Validate if advanced retention is enabled, if it is sync cannot be used, you must use native. The
health report will have this information as well. Enter show features

13. [ ] Enter show pool list.

Note: Cluster, AuditArchive ,SystemArchive and Console are system pools, they do not get migrated.

14. [ ] For each pool enter show pool detail <pool> to get the profile mapping.

EMC Confidential Information- for internal use only version: 2.0.4.0

Page 7 of 26
15. [ ] Enter update profile <profile name>. Take all of the defaults and generate a pea file

16. [ ] Repeat the steps above on the replica, to get the ip of the replica enter
show cluster replication
1. [ ] Once you have the 2 pea files, open both pea files with wordpad,
2. [ ] On the DR pea file copy what is highlighted below

EMC Confidential Information- for internal use only version: 2.0.4.0

Page 8 of 26
 3. [ ] On the source pea file paste in the highlighted area from the DR pea file. The secret should be
the same with different cluster id’s, save the file as the final pea file.

1. [ ] Test the pea file using JCenteraVerify

EMC Confidential Information- for internal use only version: 2.0.4.0

Page 9 of 26
 2. [ ] Login into the sync server, ecssync:ecs-sync-ova

3. [ ] Create a directory in /home/ecssync called pea, Copy or paste the pea file to that directory.
Repeat for all other pea files.
4. [ ] Verify if CE+ is enabled Enter
show security all
5. [ ] Verify if retention classes are configured Enter
show retention all
Retention class name Period
-------------------------------------------------
dx01 5 years
-------------------------------------------------
Note: If there are retention classes, you have to configure the same class name on the ECS
namespace

ECS Preparation
1. [ ] Login into ECS using a browser: Default is root:ChangeMe
2. [ ] To get the ECS sn# go into cli as admin:ChangeMe enter:
sudo xdoctor –topology

3. [ ] Create the VDC’s, replication group and storage pool if it hasn’t been done so already. Refer to
the ecs install procedure for guidance

Note: All configuration changes made on ECS will automatically get pushed to the other VDC(s)

4. [ ] Ensure D@RE (server side encryption) is licensed, this is a free license which has to be enabled
on the namespace, go to settings > Licensing, if the license is not visible it hasn’t been installed DO
NOT proceed if this license is not enabled. Reach out to the account team to inform them to obtain a
D@RE license for both VDC’s

Note: You cannot enable D@re after a namespace has been created

5. [ ] Create a namespace, customer should apply the naming convention, Keep it lowercase with only
an underscore with no spaces

Note: If the cluster is at CE+ create a namespace with compliance enabled, for GE and basis clusters
leave compliance disabled on the namespace.

EMC Confidential Information- for internal use only version: 2.0.4.0

Page 10 of 26
 Note: Once the namespace is created you cannot enable compliance or D@RE, make sure you have
that correct

 Enable Access during outage and server-side Encryption

 If retention classes are configured on Centera you have to configure the classes when you
create the namespace. The names have to match if not the app will not be able to delete its
object. The value does not have to match. Once you save the namespace you cannot delete
the retention policy but you can go back in and add another class


EMC Confidential Information- for internal use only version: 2.0.4.0

Page 11 of 26
 6. [ ] Save the namespace
7. [ ] Ask the customer if they require a management user for each application or just one to avoid
having to use root
8. [ ] Go to users, create a Management user for the application or for a universal user.

9. [ ] Logout and log back in as that new management user. Typically the customer will give EMC it’s
own management user.
10. [ ] Create an Object user which is equivalent to a profile on Centera for every application to be
migrated,
 The name does not have to match the Centera profile name, refer to the customer for the
naming convention. You must use lowercase
 Select the Namespace for that user
 Select save

11. [ ] Create a bucket, which is equivalent to a pool on Centera

 Select your namespace from the dropdown
 Select the replication group from the drop down
 Set the object user for that bucket
 Bucket tagging is for S3 only
 Enable CAS and server side encryption

Note: When you first create a bucket metadata search is grayed out, per design, when the bucket is
saved it will automatically get enabled. This was designed to prevent it from staying disabled which will
disable query. After you save it go back into the bucket, you will see metadata search is enabled.

 Enable access during outage which will allow failover to VDC2

 Bucket retention: The application will always set the retention. The bucket retention has to be
equal to or greater than the retention on the application.

EMC Confidential Information- for internal use only version: 2.0.4.0

Page 12 of 26
 1. [ ] Enter the bucket again, from the drop down select edit ACL (access control list). Verify the object
user is in full control. Root is not required for ACL.

2. [ ] Go back to users and edit the object user

EMC Confidential Information- for internal use only version: 2.0.4.0

Page 13 of 26
  Generate the pea file by selecting generate > set password, this will create a tab for generate
pea file, generate the file
 Copy the pea file to wordpad, you do not need the pea file from the other VDC at 3.x, the id’s
are now the same.
 Set the bucket that was created for your user

3. [ ] Login to the Centera to test the ecs pea file, login is dev:we!dev!code?
4. [ ] From an access node enter the following to get to JCAS
export LD_LIBRARY_PATH=/home/filepool/bin:.; /usr/java/`cd /usr/java;ls |grep
j`/bin/java -jar /home/filepool/bin/JCASScript.jar

5. [ ] At the casscript prompt enter po <ecs ip>?pea file

6. [ ] If the pea file authenticates you will see the connection

7. [ ] Enter c for capability, if read and write are disabled, the user or bucket was not configured
correctly. You should not be using root as the bucket owner

EMC Confidential Information- for internal use only version: 2.0.4.0

Page 14 of 26
 8. [ ] Test a write and read, for CAS once you write a file to a bucket you can never delete the bucket
even if you delete the file, a reflection is always left behind

Eee

9. [ ] Copy the pea file to the sync servers, at this point there should be 2 pea files named for example
evault_Centera.pea and evault_ecs.pea

ECSSYNC Preparation
MariaDB is a database used to log all clips migrated, validated and errored. These tables will assist with
troubleshooting and reporting

EMC Confidential Information- for internal use only version: 2.0.4.0

Page 15 of 26
Note: Create a table for every clip list or application your migrating

Note: Never drop a table after a migration, the table is required for tracking purposes

1. [ ] Vi a file called the create_script.sql, copy in the below lines,at the top line enter the name of the
table to be created and save.
CREATE TABLE IF NOT EXISTS evault (
source_id VARCHAR(54) PRIMARY KEY NOT NULL,
target_id VARCHAR(54),
is_directory INT NOT NULL,
size BIGINT,
mtime DATETIME,
status VARCHAR(32) NOT NULL,
transfer_start DATETIME null,
transfer_complete DATETIME null,
verify_start DATETIME null,
verify_complete DATETIME null,
retry_count INT,
error_message VARCHAR(2048),
is_source_deleted INT NULL,
INDEX status_idx (status)
);
2. [ ] To create the table enter:
mysql ecs_sync -p < create_table.sql
at the prompt enter ecs-sync-db
3. [ ] Login into mysql enter
mysql ecs_sync –p
4. [ ] Verify the table was created enter
show tables;
5. [ ] To view the table enter
describe <name of the table>;
+--------------+---------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+--------------+---------------+------+-----+---------+-------+
| source_id | varchar(54) | NO | PRI | NULL | |
| target_id | varchar(54) | YES | | NULL | |
| started_at | timestamp | YES | | NULL | |
| completed_at | timestamp | YES | | NULL | |
| verified_at | timestamp | YES | | NULL | |
| status | varchar(32) | NO | | NULL | |
| message | varchar(2048) | YES | | NULL | |

Note: Be sure that every application’s clip list has its own table

Prepare the clip list

Clip list notes:
 A cliplist should always be provided by the application
 If the application is DiskXtender or Sourceone you will need to follow the procedure located in
the PS section of the Centera Solve procedure generator to generate the cliplist
 Most lists are too large to open in vi so we have scripts listed in this section to clean the lists.
 If the customer cannot provide the clip list follow the steps in this section to query the pool
6. [ ] VI a file called clean.pl, copy in the following:

EMC Confidential Information- for internal use only version: 2.0.4.0

Page 16 of 26
 #!/usr/bin/perl

while(<>){
/([A-Z0-9]{13}e[A-Z0-9]{13,39})/g;
print "$1\n";
}
7. [ ] Make the script executable Enter:
chmod +x clean.pl
8. [ ] Clean.pl will remove everything that is not a clip except blank lines, Enter
./clean.pl clip list > cliplist.clean
1. [ ] To remove blank lines enter
grep ‘e’ cliplist.clean > cliplist.cleaner

2. [ ] Verify if you have any hidden windows characters which are special
characters at the end of the clips such as ^M Enter:
cat –v cliplist.cleaner | more
E7DO27H1TD1T1eF583RBTK8U44DG41A60F2BTJ0MBP96C339KVSPJ^M
1. [ ] If there are hidden characters enter.
dos2unix cliplist.cleaner cliplist.final

Note: If dos2unix is not on the server enter yum install dos2unix

2. [ ] The final list should have 1 clip per line

3. [ ] If you have for an example 4 vm’s do a wc-l on the list and split the
list by 4 and copy the lists to the other vm’s.
Split –lines 1000000 dx01.clips dx01, this will generate dx01aa and
up.

Note: There is no limit to the number of clips in a list

Generate a list using JCAS

If the customer cannot generate a list for you’re going to have to query the pool using JCAS

Note: If there’s multiple apps in the default ask [email protected] for assistance to breakdown the
apps in the pool

1. [ ] Login to the centera backend

2. [ ] Got to a mnt/?/service partition on an access node that has the most space, Enter:

EMC Confidential Information- for internal use only version: 2.0.4.0

Page 17 of 26
 fpshell -t access –c ‘df –lh’ | more

3. [ ] CD to the /mnt/?/service directory

4. [ ] Enter screen to run query in the background
5. [ ] Launch JCAS Enter:
Export LD_LIBRARY_PATH=/home/filepool/bin:.; /usr/java/`cd /usr/java;ls |grep
j`/bin/java -jar /home/filepool/bin/JCASScript.jar

1. [ ] At the casscript prompt Enter

po localhost?<name of the pea file for the pool you will be querying>
1. [ ] To dump the pool Enter:
qtf <name of the list>

Note: The query can process up to 15M clips a hour, the list comes in a
ready state so no need to clean it

1. [ ] If you have to perform a delta after the first pass enter JCAS, for an
example the customer gave you the list on sept 2, set the lower boundary
to the day before and do another qtf
CASScript>qslb
usage: querySetLowerBound "yyyy.mm.dd hh:mm:ss"
qtf 2nddelta
1. [ ] Use cldiff to get the delta from the original list against the delta.
Both lists will need to sorted, once sorted enter ./cldiff delta orginal,
this will generate a source_only and target_only , rename source_only that
will be your new delta list

X-doctor setup
2. [ ] To configure xdoctor Enter:
This should be done on the ecssync server
sudo su

xdoctor -c

xDoctor Configuration Menu

--------------------------
(1) Overview
(2) Reporting
(3) Scheduling
(4) Archiving
(5) Repository
(6) Auto Update

(9) Migration Info

(0) Exit

Please make a choice: 2

xDoctor Notification

EMC Confidential Information- for internal use only version: 2.0.4.0

Page 18 of 26
 --------------------
(1) Overview
(2) SMTP Settings
(3) Reporting Settings

(0) Main menu

Please make a choice: 2 à put in a proper SMTP

Dedicated SMTP Server for the System [Server_name or IP_address:port]
[mailhub.lss.emc.com:25]:
Enable TLS? [No]:

[SMTP Settings]
SMTP Server = mailhub.lss.emc.com:25
TLS = False
TLS Username =
TLS Password =

xDoctor Notification
--------------------
(1) Overview
(2) SMTP Settings
(3) Reporting Settings

(0) Main menu

Please make a choice: 3

Enable xDoctor Reporting? [Yes]:

Notify on [REPORT, CRITICAL, FIXED, ERROR, WARNING, STATUS, VERBOSE] [REPORT]:

Go back to main page 0 and Select the migration info 9 and put in the details
important part is the Target product S/N but all details should be filled…

Migration Info
----------------
Customer Name [DevSetup4]:
Source Product Name [Centera]:
Source Product Version [4.3]:
Source SerialNumber [APM00000000426]:
Target Product Name [ECS]:
Target Product Version [3.0]:
Target SerialNumber [ECS00000000001]:

1. [ ] To validate if the reports are reaching the xdoctor portal, go to http://xdoctor.isus.emc.com/. If

reports are not reaching the xdoctor portal, check your SMTP setting to make sure that they are
correct.

EMC Confidential Information- for internal use only version: 2.0.4.0

Page 19 of 26
 1. [ ] Select ECS, browse to Other > double click you customer > Double click the system > select
migration from the left most option

Edit the cas_migration.xml

2. [ ] Copy the cas_migration.xml from /home/ecssync/ecs-sync-3.1/sample to /home/ecssync/ecs-
sync-3.1. Copy the file to the name of the application your migrating
Edit the cas_migration.xml. Highlighted in yellow are the sections that need to modified
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!--
This is a sample configuration to migrate a CAS application that has a clip ID
list.
It uses 30 threads, verifies data using MD5 checksums and tracks status of all
objects in a database table.
-->
<syncConfig xmlns="http://www.emc.com/ecs/sync/model"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.emc.com/ecs/sync/model model.xsd">
<options>
<threadCount>30</threadCount>
<verify>true</verify>
<!-- optionally provide a clip list (one clip ID per line) -->
<sourceListFile>/home/ecssync/lists/evault.clips</sourceListFile>

<!-- uncomment the following if you are *not* submitting to the ecs-sync
service running on the OVA -->
<!--

EMC Confidential Information- for internal use only version: 2.0.4.0

Page 20 of 26
 <dbConnectString>jdbc:mysql://localhost:3306/ecs_sync?user=ecssync&amp;password=ecs
-sync-db&amp;autoReconnect=true&amp;characterEncoding=UTF-8</dbConnectString>
-->
<dbTable>evault</dbTable>
<!-- comment above and uncomment below if you are *not* connecting to a
mysql DB -->
<!--
<dbFile>cas_migration.db</dbFile>
-->
<casConfig>
<!-- modify hosts and pea file or optionally use user/secret params ala
CAS SDK spec -->

<connectionString>hpp://10.246.22.131?/home/ecssync/pea/evault_Centera.pea</connect
ionString>
</casConfig>
</source>
<target>
<casConfig>
<!-- modify hosts and pea file or optionally use user/secret params ala
CAS SDK spec -->

<connectionString>hpp://10.241.172.43?/home/ecssync/pea/evault.pea</connectionStrin
g>
</casConfig>
</target>
</syncConfig>

1. [ ] Enter xmllint cas_migration.xml to verify if there were any syntax errors. If it comes back
showing you the content of the entire xml it’s good if not if will tell you what lines have a syntax error

Start the Migration

2. [ ] Start the Migration. This runs as a service, running it in the background is not required Enter:
ecs-sync-ctl --submit cas_migration_lab.xml

01-29 13:20:03INFO [main] EcsSyncCtl:206 - Command: Submit file

cas_migration_lab.xml
01-29 13:20:03DEBUG [main] EcsSyncCtl:416 - HTTP Response 201:Created
Submitted Job 2

Note: If your migrating another application on the same server you should have a separate xml file and
table, repeat the command above to kick off another job

1. [ ] To monitor the job enter: ecs-sync-ctl –status <job #>

EMC Confidential Information- for internal use only version: 2.0.4.0

Page 21 of 26
 2. [ ] If the job instantly says complete, you have an issue, look at the ecssync.log in /var/log/ecs-
sync/ecs-sync.log for the reason it failed
 Some common problems are:
 ecs-sync service is not running enter: sudo service ecs-sync –status
 Source or target authentication failed, check the pea files
 Xml can’t find the path to the pea file or cliplist
 Mariadb is not running enter: service mariadb status
 Syntax error in the xml file
 LD libraries are not exported, enter:
export LD_LIBRARY_PATH=/usr/local/Centera_SDK/lib/64, then a set | grep LD

3. [ ] The /var/log/ecs_sync/ecssync/ecssyn.log will only log errors. To monitor the migration, use the
DB or the ctl status
4. [ ] To pause and resume the migration Enter:
ecs-sync-ctl –list-jobs to get the job number running
ecs-sync-ctl --pause <job id>
ecs-sync-ctl --resume <job id>
1. [ ] To change the thread count you do not have to edit the xml file, stop or pause the migration
Enter:
ecs-sync-ctl --threads <thread-count> --set-threads <job-id>
1. [ ] By default the sync server reserves 200GB and only show 80GB, if you’re
doing a large migration, keep an eye on the capacity by entering df-lh, if
more capacity is required enter: (this is instant no service restart is
required)
sudo xfs_growfs /dev/mapper/centos-root

EMC Confidential Information- for internal use only version: 2.0.4.0

Page 22 of 26
Working with MariaDB
Once the migration is in progress you can run sql commands to get a status
2. [ ] To get into the MariaDB Enter
mysql ecs_sync -p password is ecs-sync-db)

3. [ ] SQL commands

### View the table is correct

describe <table name>;

+-------------------+---------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------------+---------------+------+-----+---------+-------+
| source_id | varchar(54) | NO | PRI | NULL | |
| target_id | varchar(54) | YES | | NULL | |
| is_directory | int(11) | NO | | NULL | |
| size | bigint(20) | YES | | NULL | |
| mtime | datetime | YES | | NULL | |
| status | varchar(32) | NO | | NULL | |
| transfer_start | datetime | YES | | NULL | |
| transfer_complete | datetime | YES | | NULL | |
| verify_start | datetime | YES | | NULL | |
| verify_complete | datetime | YES | | NULL | |
| retry_count | int(11) | YES | | NULL | |
| error_message | varchar(2048) | YES | | NULL | |
+-------------------+---------------+------+-----+---------+-------+

### Show all of the tables configured

show tables;

+--------------------+
| Tables_in_ecs_sync |
+--------------------+
| EDI |
| EDI_error |
| <table> |
| objects |
+--------------------+

### Get the status of the migration

select status,count(*) from <table> group by status;

+-------------+----------+
| status | count(*) |
+-------------+----------+
| Error | 1977 |
| In Transfer | 29 |
| Retry Queue | 51 |
| Verified | 218955 |
+-------------+----------+

### Get the total amount of capacity migrated

select sum(size) from <table> where status in ('Transferred','Verified');

+-------------+
| sum(size) |
+-------------+

EMC Confidential Information- for internal use only version: 2.0.4.0

Page 23 of 26
 | 17836530274 |
+-------------+

### Get the SDK error

select LEFT(error_message,80), count(*) from <table> where status='Error' group by
LEFT(error_message,80);

| [java.lang.RuntimeException: CAS Error -10021/FP_CLIP_NOT_FOUND_ERR: Clip CDF is

| 86 |

### DUMP A specific field

select source_id from <table> where status='Error' into outfile 'sync1_error_1124';

Note: The outfile goes into /var/lib/mysql/ecs_sync

### DUMP specific errors

select source_id from <table> where status = 'Error' and error_message like
'%10021%' into outfile 'sync3_10021.clips';

###list the clips that errored

select source_id from <table> where status = 'Error' and error_message like
'%10005%' limit 10;

+-------------------------------------------------------+
| source_id |
+-------------------------------------------------------+
| 3O1C8Q2N73CJHe5HLSRPJSSIH91G415OO1FOOR0G54O34IHC2RMOV |
| 3O1ESBPOP7TMBeC3U5LA4LPROOOG410T45FL2G0406DH3HVQV1P4J |
| 3O1HKJDBC8V7Ge2MKCQ8E8I4OKPG415NF4674K0S7QQO4QLBL6C0K |
| 3O1M23TCTVCN6e3O3F348F2UHIMG414DC6EM8T03FE8JHGBN2JGIE |
| 3O1MVRPUU02CDe5OE4CN8DMUCMEG412JHJMFI20V53N478OVSPOG2 |
| 3O1N5I2KJTB0PeBHPB48B13AM9VG415LJS7DJU0KDQCAGDSG165C1 |
| 3O1OF4TVCH8MSeAO5PGKUELFIFMG410RT2B7V70J3A0NTCDS5V4LS |
| 3O1R908MQHBBIeEVHF5AC18FSCLG4137DGVHIA01BIAR957G3EIRK |
| 3O1STDDM67ES0e9GMOV7G1JAEJ9G410U812GGB0MB19D7CCHUKU4J |
| 3O1VL6VE9O1DGeE4NI99DHLBH27G4150MCLFM60BC42QFDS420RN2 |
+-------------------------------------------------------+
10 rows in set (0.00 sec)

### Get the entire trace of an error

select error_message from <table> where status='Error' limit 2;

### Get the count for the clips that failed

select count(*) from <table> where status = 'Retry Queue';

select source_id from <table> where status = 'Retry Queue';

### Get the time the migration started

select min(transfer_start), max(verify_complete) from <table>;

+---------------------+----------------------+

EMC Confidential Information- for internal use only version: 2.0.4.0

Page 24 of 26
 | min(transfer_start) | max(verify_complete) |
+---------------------+----------------------+
| 2017-01-29 10:48:09 | 2017-01-29 14:40:12 |

Troubleshooting
This section will describe the troubleshooting steps
One of most common errors when doing a migration from Centera is 10021, this
is typically clips that were deleted on Centera before it was migrated, 10014
network issue and 10005 store error.
4. [ ] After the migration dump the DB of all errors, take that list and try
migrating it again. After the 2nd pass take all of the errors that failed
again and point the tool to the DR Centera. After this take what still
failed to migrate into an errors.final list.
5. [ ] Go to the JCAS directory and enter screen
6. [ ] Launch JCAS and open a pool to the pool you were migrating from
7. [ ] Enter qsd to query for all deletes
CASScript>qsd
Query Expression Properties:
Start Time: UNBOUNDED
End Time : UNBOUNDED
Type : DELETED

1. [ ] Start the query Enter

qtf dx01.deletes,
1. [ ] When the query completes, sort both the query and the errors.final list
sort dx01.deletes > dx01.deletes.sort
sort errors.final > errors.sort
1. [ ] Copy the cldiff tool into the JCAS directory, enter
2. [ ] Next we will diff the 2 lists using cldiff. Cldiff will produce 2 files called
source_only and target_only
In this example we compared the Centera deletes against the migration errors,
472 matched which means the errored clips are not on Centera which means the
application deleted the clip during the migration, these can be ignored,
target only has 6 which is the errored list which means these 6 clips have
other issues.
./cldiff dx01.sort errors.final.sort
Writing clips only in dx01.sort to ./source_only
Writing clips only in errors.final.sort to ./target_only
100.0% ETC:0.00m 482 clips in 0s ( 0 cps).
Done.
Clips that exist on both clusters (472):
462 match

EMC Confidential Information- for internal use only version: 2.0.4.0

Page 25 of 26
 Clip differences (10):
0 source only
10 target only
3. [ ] With the 6 clips you can open a pool to the Centera and try reading the
BLOB, if it generated a file it’s readable, if not it’s corrupted, at this
point open a SR

1. [ ] If every clip is accountable for you can line up a window to cutover the applications which is an
outage where you give them the ECS pea files and ip’s and they have to stop the services on the app
and point the the app to ECS.
2. [ ] Have the customer test recalls if all looks good submit a final report for that application.

EMC Confidential Information- for internal use only version: 2.0.4.0

Page 26 of 26