Beruflich Dokumente
Kultur Dokumente
PostsRSS
CommentsRSS
Edit
Searchhere...
KnowInformatica
DataWareHousingETLProjectLifeCycle
PostedbySureshat1:08AM Labels:DataWarehouse,ETL,LifeCycle
Me
ThefollowingarethedifferentphasesinvolvedinaETLprojectdevelopmentlifecycle.
S UR ES H
1)RequirementGathering
VI EW M Y C OM PL ETE
PROF I L E
a)BusinessRequirementCollection(BRD)
b)SystemRequirementCollection(SRD)
2)DesignPhase
a)HighLevelDesignDocument(HRD)
b)LowlevelDesignDocument(LLD)
3)DevelopmentPhase
a)MappingDesign
b)CodeReview
Labels
c)PeerReview
4)Testing
DataWarehouse(3)
a)UnitTesting
b)SystemIntegrationTesting.
ETL(1)
c)USerAcceptanceTesting(UAT)
5)PreProduction
6)Production(GoLive)
RequirementGathering:
FactTable(1)
LifeCycle(1)
BusinessRequirementCollection(BRD):
SQL(2)
>ThebusinessrequirementgatheringstartbybusinessAnalyst,onsitetechnicalleadandclient
businessusers.
>Inthisphase,aBusinessAnalystpreparesBusinessRequirementDocument(BRD)(or)Business
RequirementSpecifications(BRS)
>BRcollectiontakesplaceatclientlocation.
>Theo/pfromBRAnalysisare
BlogArchive
>BRSBusinessRequirementSpecifications
>SRSSystemRequirementSpecifications
SystemRequirementCollection(SRD):
Seniortechnicalpeople(or)ETLarchitectwillpreparetheSRSwhichcontainss/wandh/wdetails
2011(6)
March(3)
TheSRSwillincludes
a)O/Stobeused(windowsorunix)
b)RDBMSrequiredtobuilddatabase(oracle,Teradataetc)
c)ETLtoolsrequired(Informatica,Datastage)
February(3)
DataWareHousingETLProjectLife
d)OLAPtoolsrequired(Cognos,BO)
TheSRSisalsocalledasTechnicalRequirementSpecifications(TRS)
DesignPhase:
>Theo/pfromdesignandplanningphaseis
a)HLD(HighLevelDesign)Document
PoweredbyBlogger.
FactTable
DataWarehouseDefinition
SearchThisBlog
b)LLD(LowLevelDesign)Document
Search
HLD(HighLevelDesign)Document:
AnETLArchitectandDWHArchitectparticipateindesigningasolutiontobuildaDWH.
AnHLDdocumentispreparedbasedonBusinessRequirement.
LLD(LowLevelDesign)Document:
BasedonHLD,aseniorETLdeveloperprepareLowLevelDesignDocument
TheLLDcontainsmoretechnicaldetailsofanETLSystem.
AnLLDcontainsdataflowdiagram(DFD),detailsofsourceandtargetsofeachmapping.
Resources
AnLLDalsocontainsinformationaboutfullandincrementalload.
AfterLLDthenDevelopmentPhasewillstart
1Keydata
DevelopmentPhase:
Mapping:
>BasedanLLD,theETLteamwillcreatemapping(ETLCode)
>Afterdesigningthemappings,thecode(Mappings)willbereviewedbydevelopers.
Followers
CodeReview:
>CodeReviewwillbedonebydeveloper.
Jointhissite
>Incodereview,thedeveloperwillreviewthecodeandthelogicbutnotthedata.
withGoogleFriendConnect
>Thefollowingactivitiestakesplaceincodereview
Members(2)
>Youhavetocheckthenamingstandardsoftransformation,mappingsofdataetc.
>Sourceandtargetmapping(Placedthecorrectlogicornotinmapping)
PeerReview:
>Thecodewillreviewedbyyourteammember(thirdpartydeveloper)
Testing:
Alreadyamember?Signin
Thefollowingvarioustypestestingcarriedoutintestingenvironment.
1)UnitTesting
2)DevelopmentIntegrationTesting
3)SystemIntegrationTesting
AddSubscribtion
4)UserAcceptanceTesting
UnitTesting:
>AunittestfortheDWHisawhiteBoxtesting,ItshouldchecktheETLprocedureandMappings.
>ThefollowingarethetestcasescanbeexecutedbyanETLdeveloper.
1)Verifydataloss
2)No.ofrecordsinthesourceandtarget
3)Dataload/Insert
4)Dataload/Update
5)Incrementalload
6)Dataaccuracy
7)verifyNamingstandards.
8)VerifycolumnMapping
>TheUnitTestwillbecarriedbyETLdeveloperindevelopmentphase.
>ETLdeveloperhastodothedatavalidationsalsointhisphase.
DevelopmentIntegrationTesting
>Runallthemappingsinthesequenceorder.
>FirstRunthesourcetostagemappings.
>Thenrunthemappingsrelatedtodimensionsandfacts.
SystemIntegrationTesting:
>Afterdevelopmentphase,wehavetomoveourcodetoQAenvironment.
>Inthisenvironment,wearegivingreadonlypermissiontotestingpeople.
>Theywilltestalltheworkflows.
>Andtheywilltestourcodeaccordingtotheirstandards.
UserAcceptanceTesting(UAT):
>Thistestiscarriedoutinthepresenceofclientsidetechnicaluserstoverifythedata
migrationfromsourcetodestination.
ProductionEnvironment:
>MigratethecodeintotheGoLiveenvironmentfromtestenvironment(QAEnvironment).
Thanks:shivakrishnas
2 0C OMME NTS :
Anonymoussaid...
thanksforsuchausefulpost,reallyappreciateit.
JUNE 14 , 2 011 AT 9 : 57 AM
Anonymoussaid...
GoodpostonETLlifecycle.
Mahesh
AUG US T 10, 2 011 AT 10: 4 0 PM
Sandyasaid...
Posts
Comments
Hi,Thislifecycleisreallyveryclearandnice
thankyouforthis.
AUG US T 17, 2 011 AT 9 : 52 PM
Sureshsaid...
ThanksforyourcommentsSandya...
O C TO B E R 14 , 2 011 AT 5: 16 PM
Anonymoussaid...
thanxforpostingithelpsalottome..........
thislifecycleissameforabinitioalso?
plzgivereplaytomemyemailidisrcreddy15.e@gmail.com.......
FE B RUARY 2 2 , 2 012 AT 1: 18 AM
Alekhyasaid...
ThanksforthePosting.....Ithelpsmelot...
Canupleasesaywatarethecomplexissuesusuallyfacewhilemapping....
Canupleasesharetheinformationtomygmailaccountalekhya.rudra@gmail.com
MARC H 4 , 2 012 AT 7: 07 PM
cmtopinkasaid...
What'sthedifferencebetweenUnittestingandDataValidation?
MAY 1, 2 012 AT 12 : 12 PM
Anonymoussaid...
HiAll,
Iamnewininformatica,canyoupleasesharewithmethesampledocumentslikeSRS,FS,HLD,LLD,etcon
mymailidisgajraj.meena@gmail.com
Thanksinadvance.
Gajraj
D E C E MB E R 3 1, 2 013 AT 11: 17 PM
Nitinsaid...
thankYouSoMuch
thispostisreallyveryuseful...
JANUARY 2 1, 2 014 AT 4 : 02 AM
NaveenKumarsaid...
ThankYouSuresh
FE B RUARY 5, 2 014 AT 9 : 04 AM
sruthisaid...
Reallyagreatwork,ithelpedmealot
FE B RUARY 9 , 2 014 AT 11: 2 1 PM
jakegeorgesaid...
ETLTestingOnlineTraining,ONLINETRAININGITSUPPORTCORPORATETRAINING
http://www.21cssindia.com/courses/etltestingonlinetraining249.htmlThe21stCenturySoftwareSolutions
ofIndiaoffersoneoftheLargestconglomerationsofSoftwareTraining,ITSupport,CorporateTraining
instituteinIndia+919000444287+917386622889Visakhapatnam,HyderabadETLTestingOnline
Training,ETLTestingTraining,ETLTesting,ETLTestingOnlineTraining|ETLTestingTraining|ETLTesting|
"Coursesat21stCenturySoftwareSolutions
TalendOnlineTrainingHyperionOnlineTrainingIBMUnicaOnlineTrainingSiteminderOnlineTraining
SharePointOnlineTrainingInformaticaOnlineTrainingSalesForceOnlineTrainingManymore|Call
Us+917386622889+919000444287contact@21cssindia.com
Visit:http://www.21cssindia.com/courses.html"
S E PTE MB E R 3 0, 2 014 AT 9 : 2 4 PM
DanielMasonsaid...
ItwasreallyanicearticleandIwasreallyimpressedbyreadingthisarticleWearealsogivingallsoftware
CourseOnlineTraining.TheDataWarehousingCoursesOnlineTrainingisoneoftheleadingOnlineTraining
instituteintheworld.
D E C E MB E R 2 6 , 2 014 AT 11: 18 AM
jhansiranisaid...
thankuSuresh,..reallyveryusefull..givemeurmailidplz
APRI L 2 , 2 015 AT 10: 10 PM
venkatesankalyanasundaramsaid...
ThankusomuchMrSureshforthewonderfularticle.
canusendthesampledocumentforBRD,SRD,HLD,LLDforreference(ssaivenkat89@gmail.com)
APRI L 6 , 2 015 AT 5: 08 AM
Bapisaid...
ThankyouSuresh,foralltheseValuableinfo.
Canyouplzzprovidemetherelevantmatrialsanddocumentstomyfollowingemailid:
bapisahoo194@gmail.com
JULY 6 , 2 015 AT 12 : 2 1 AM
CatMindssaid...
Thisinformationwhichyouprovidedisverymuchusefulforus.Itwasveryinterestingandusefulforqaonline
training.WealsoprovidingETLTestingOnlineTraininginstituteinUSA.
AUG US T 10, 2 015 AT 11: 54 AM
rajeshkannayadhavansaid...
I'mhappytojoiningthisblog,Itisaveryniceexperienceforme...Thankyouforallyourpostings.
TestingTrainingwithLiveProject
AUG US T 2 8, 2 015 AT 9 : 4 5 PM
Anonymoussaid...
HiSuresh,AsamnewfortheINFORMATICA,itwillbeveryhelpfulforme,thankyouforthisinformative
blog.AndplscanyousharethesampledocumentsofBRD,SRD,etc...wouldbesothankfullforthat.thankyou..
O C TO B E R 1, 2 015 AT 2 : 11 AM
Unknownsaid...
hiplsprovidedocumentsforhldandlld
ramesh.lokonda@gmail.com
D E C E MB E R 2 6 , 2 015 AT 6 : 4 7 PM
Enteryourcomment...
Commentas:
Publish
GoogleAccount
Preview
Home
Subscribeto:PostComments(Atom)
KnowInformatica
BloggerthemebyBloggerThemesDesignbyFreeThemeLayouts
OlderPost