SOFTWARETESTINGSPACE

S O FT W A R E T ES T I NG S P A CE O NE O F T H E MO S T P O P U LA R S O FT W A R E T ES T I NG B LO G S W I T H Q U A LI T Y
A R T I CLES O N T ES T A U T O MA T I O N A ND P ER FO R MA NCE T ES T I NG U S I NG T O O LS , A U T O MA T ED T ES T I NG , T ES T
S T R A T EG I ES , NEW S U P DA T ES A ND G ENER A L S ELF H ELP G U I DE

Home

Videos

SeleniumTestAutomationTraining

manualsoftwaretesting

softwaretestingjobs

JMeterPerformanceTestingTraining

softwaretestingqa

softwaretestingtool

agilesoftwaretesting
softwaretestingtutorial

testingdata
FRIDAY , MARCH 19, 2010

T O T AL P AG EVI EW S

Howtododatabasemigrationtestingeffectivelyandquickly?

803529
S EAR CH T H I S B LO G

Search

G O O G LE+ FO LLO W ER S

InderPSingh
Addtocircles

My earlier post, How to do real database testing (10 tips to perform serious database
tests)?, turned out to be quite popular. However, that post did not discuss the topic of
databasemigrationtesting.Hence,thispost.
Databasemigrationtestingisneededwhenyoumovedatafromtheolddatabase(s)toa
newdatabase.Theolddatabaseiscalledthelegacydatabaseorthesourcedatabaseand
the new database is called the target database or the destination database. Database
migrationmaybedonemanuallybutitismorecommontouseanautomatedETL(Extract
TransformLoad)processtomovethedata.Inadditiontomappingtheolddatastructure
tothenewone,theETLtoolmayincorporatecertainbusinessrulestoincreasethequality
ofdatamovedtothetargetdatabase.

1,062havemeincircles

Now,thequestionarisesregardingthescopeofyourdatabasemigrationtesting.Hereare
thethingsthatyoumaywanttotest.
1.Allthelive(notexpired)entitiese.g.customerrecords,orderrecordsareloadedintothe
target database. Each entity should be loaded just once i.e. there should not be a
duplicationofentities.
2. Every attribute (present in the source database) of every entity (present in the source

FO LLO W ER S

Jointhissite
withGoogleFriendConnect

Members(465) More

database)isloadedintothetargetdatabase.
3. All data related to a particular entity is loaded in each relevant table in the target
database.
4.EachrequiredbusinessruleisimplementedcorrectlyintheETLtool.
5.Thedatamigrationprocessperformsreasonablyfastandwithoutanymajorbottleneck.
Next,letusseethechallengesthatyoumayfaceindatabasemigrationtesting.
1.Thedatainthesourcedatabase(s)changesduringthetest.
2.Somesourcedataiscorrupt.
3.Themappingsbetweenthetables/fieldsofthesourcedatabases(s)andtargetdatabase
arechangedbythedatabasedevelopment/migrationteam.

ABOU T ME

4.Apartofthedataisrejectedbythetargetdatabase.
5.Duetotheslowdatabasemigrationprocessorthelargesizeofthesourcedata,ittakesa
longtimeforthedatatobemigrated.
Thetestapproachfordatabasemigrationtestingconsistsofthefollowingactivities:
I N D E R P SI N G H

I.Designthevalidationtests
Inordertotestdatabasemigration,youneedtouseSQLqueries(createdeitherbyhandor
usingatoole.g.aquerycreator).Youneedtocreatethevalidationqueriestorunagainst
boththesourceaswellasthetargetdatabases.Yourvalidationqueriesshouldcoverthe
scopedefinedbyyou.Itiscommontoarrangethevalidationqueriesinahierarchye.g.you
want to test if all the Orders records have migrated before you test for all OrderDetails
records. Put logging statements within your queries for the purpose of effective analysis
andbugreportinglater.
II.Setupthetestenvironment
The test environment should contain a copy of the source database, the ETL tool (if
applicable) and a clean copy of the target database. You should isolate the test
environmentsothatitdoesnotchangeexternally.
III.Runyourvalidationtests
Depending on your test design, you need not wait for the database migration process to
finishbeforeyoustartyourtests.

Follow

1k

Enjoyfreetestingvideosonour
YouTubechannel
Iamanexperiencedprofessional
withcoreexpertiseinproviding
powerfulandeconomical
softwaretestautomationand
manualtesting/QAservices.If
yourequirethesetestingservices,
pleasecontactme.
Theopinionsexpressedinmy
videosandblogsaremypersonal
opinions.

V I EW MY CO MP LET E P R O FI LE

IV.Reportthebugs
Youshouldreportthefollowingdataforeachfailedtest:
a.Nameoftheentitythatfailedthetest
b.Numberofrowsorcolumnsthatfailedthetest

SubscribetoSoftwareTesting
SpacebyEmail

c.Ifapplicable,thedatabaseerrordetails(errornumberanderrordescription)
d.Validationquery
d.Useraccountunderwhichyourunyourvalidationtest
e.Dateandtimethetestwasrun
Keepthetipsbelowinmindtorefineyourtestapproach:
1.Youshouldtakeabackupofthecurrentcopiesofthesourceandtargetdatabases.This
would help you in case you need to restart your test. This would also help you in
reproducinganybugs.
2.Ifsomesourcedataiscorrupt(e.g.unreadableorincomplete),youshouldfindoutifthe
ETLtooltakesanyactiononsuchdata.Ifso,yourvalidationtestsshouldconfirmthese
actions.TheETLtoolshouldnotsimplyacceptthecorruptdataassuch.
3. If the mappings between the tables/ fields of the source and target databases are
changedfrequently,youshouldfirsttestthestablemappings.
4. In order to find out the point of failure quickly, you should create modular validation
tests. If your tests are modular, it may be possible for you to execute some of your tests
beforethedatamigrationprocessfinishes.Runningsometestswhilethedatamigration
processisstillrunningwouldsaveyoutime.

CAT EG O R I ES

Software
Testing
(161) software testing
tools
(48)
Functional
Testing (43) software
testing jobs (41) Web
Testing (39) Freshers/
Beginners/ Getting Started
(36)TestingMethodologies
(29) manual software
testing (28) software testing
tutorial (23) Test Management
(20) load testing (14) Open
SourceTools(13)QuizTime (13)
performance
testing
(13)
Database Testing (12) Estimation
(10) Security Testing (9) Test Data
(6)Training(5)agilesoftwaretesting(5)

5. If the database migration process is manual, you have to run your validation queries
externally.However,iftheprocessusesanETLtool,youhavethechoicetointegrateyour
validationquerieswithintheETLtool.

BLOG ARCHIVE

2015(2)
Ihopethatyouarecomfortablewiththeconceptofdatabasemigrationtesting.(whether
dataismigratedbetweenbinaryfilesandanRDBMSorbetweenRDBMSs(Oracle,SQL

2014(15)

Server, Informix or Sybase)). According to you, what is the main problem faced while

2013(35)

testingdatabasemigration?Whatisagoodwaytohandlethisproblem?

2012(4)

P O S T ED B Y I NDER P S I NG H A T 7: 52 P M

2011(35)
2010(66)
December2010(2)
November2010(1)

19 comments

October2010(4)
September2010(1)
August2010(3)

Add a comment

June2010(5)
May2010(13)
April2010(13)
March2010(16)

Top comments

Saba Naaz 1 year ago - Shared publicly

very nice blog
+1
2

1 Reply

Inder P Singh 1 year ago

Thank you for your comment.

Wiliams Smith 2 weeks ago - Shared publicly

SoftwareTestAutomation
Estimation(10+factors
w...
RolesandResponsibilities
ofaSoftwareTestLead...
Howtoimproveyourskills
asasoftwaretester?
Performancetestingusing
HPLoadRunner
software
FreeeBookonSoftware
Testing

Howtofindmoreorbetter

Thaks
for posting this useful information,
bugs(12tipstoexplod...
It is very useful to me and who are searching for the <a href= http://hyderabadsys.com/etl-testing-online-training/
online training.</a>
AutomatedTesting:Howto
1 Reply

Quazi Ahsan 3 weeks ago - Shared publicly

Clear written and helpll link
1 Reply

Aditya Bhuyan 10 months ago - Shared publicly

writeautomatabletest
c...
MicrosoftMSDNsoftware
testingforums
Howtododatabase
migrationtesting
effectivelya...
Whatisthebestplaceto
storetestdataforyour...

I was hitting my head from wall to wall for a data migration test validation. It had givenWhatknowledgedoyou
many clear ideas.
needtohaveinorderto
1 Reply
do...

Prakhar Jain 7 months ago - Shared publicly

Nice and valuable information
1 Reply

Katy Perez 7 months ago - Shared publicly

Thank you, this helps me a lot!!!
1 Reply

MultilingualTesting:How
totestmultilingualapp...
TestingReports:Howto
testreportswitha
checkl...
Whyyourbugsarerejected
(20+reasonswhy
develo...
SoftwareTestEstimation:
Howtoestimatetesting
...
Howtodorealdatabase

Balakrishnan Sadhasivam 8 months ago - Shared publicly

testing(10tipsto

What type of Queries used? with Example

+1
2

1 Reply

perfor...
February2010(6)
January2010(2)

Suraj Khalate 1 year ago - Shared publicly

very good

2009(4)

1 Reply

Inder P Singh 1 year ago

Thank you, Suraj.

Mohammed Alam 1 year ago - Shared publicly

This post is very helpful for me. Thanks a lot.
+1
2

1 Reply

3 years ago

Thanks Inder,, one more question : What categories does Data Migration and Data Conversion testing comes and why?

Inder P Singh 3 years ago

These should fall under the Database testing categories. Additionally, depending on how its performed, these ma
Automated or Manual, Functional or Non-functional etc. categories of testing. Thank you for your question.

3 years ago

can you please explain the batch processing and how to test it. Thanks.

Inder P Singh 3 years ago

There may be batches of jobs/ statements within the ETL process. Batches may even be nested within other bat
thing to test within a batch is its transactional execution. That is, the batch must succeed or fail as a unit. Thank
question.

3 years ago

Great outline of Database Migration Testing. Good job Inder.

Sreenivasu 4 years ago

Thanks inder .. this article is really helpful to understand DMT.

Vipul Saxena 4 years ago

This article is very useful having major key points for database testing. Thanks INDER

KoolDude 4 years ago

This is nice Article. I have a specic question regarding DB migration. Let us say If source DB contains 10 tables, wh
Hi,
to migrate the data to a single table in destination (in our case, source is oracle, destination is sybase).
migration testing // All data related to a particular entity is loaded in each relevant table in the target database// it is no

test all the records manually. Do we have any specic tool to compare the source and destination data in this case? Ac
long back with DBSOLO, but this tool can only compare the data If source and Destination DB schema's are equivalent a
of tables should be same

NewerPost

Home

OlderPost

Subscribeto:PostComments(Atom)

CO P Y RIGH T20 0 9 20 1 5 SO F TW A RETESTIN GSP A CEA L L RIGH TSRESERV ED .