8/1/2015

Phase7:TestHardwareMultipleReleaseProject

MENU

PhoneDirectory

StateAgencies

HOME
COTSPROJECTS
CUSTOMPROJECTS
HARDWARE/NETWORKPROJECTS

Entersearchterm

Hardware/NetworkProjects
RolesandResponsibilities
GlossaryandAcroynms
Templates
AlternateMethodologies
FAQS

SDLCPhasesHardwareMultiple
1Initiation
2ConceptDev
3Planning
4Requirements
5Design
6Development
7Test
8Implementation
9Ops&Maint
10Disposition
JumptoHardwareSingle

RelatedLinks
PDFVersion
TestAnalysisApproval
TestProblemReportTemplate
ITSecurityPolicyandStandards

Phase7:TestHardwareMultipleRelease
http://doit.maryland.gov/SDLC/Hardware/Pages/Phase07Multiple.aspx

1/13

8/1/2015

Phase7:TestHardwareMultipleReleaseProject

Project
TheTestPhasefocusesonanempiricalinvestigationinwhichtheresultsdescribethequalityofthe
system:testingcannotconfirmasystemfunctionsproperlyunderallconditionsbutcanestablish
thatitfailsunderspecificconditions.Theearlieradefectisfoundinthedevelopmentprocessthe
lessexpensivethefix.Testingearlyinthesystemlifecyclereducesriskssuchasscheduledelays
orcostoverrunsduetoincompleteorunacceptablecomponents.IntheTestPhase,testingofthe
systemprovesthatthesystemmeetsallrequirements,includingthoseforperformanceand
security.Theindepthsecuritytestingofthisphaseidentifiesanypartsofthesystemthatwillnot
satisfyaccreditationcriteria.Finally,acceptancetestingconfirmsthedevelopedsystemsatisfiesthe
enduserswhoidentifiedthebusinessneedandtherequirements.Multiplereleaseprojectsrequire
multipleiterationsoftheTestPhaseoneforeachrelease.

Contents
1.0:Objectives/Goals
2.0:DeliverablesandApprovals
3.0:Roles
4.0:TasksandActivities
5.0:Conclusions

1.0Objectives/Goals
Objectives
SuccessfulcompletionoftheTestPhaseshouldcomprise:
Proofthroughsystem,security,anduseracceptanceteststhatthesystemmeetsall
requirements,functionsaccordingtodesignparameters,andsatisfiesallbusiness,technical,
andmanagementstakeholders
AssurancethatthesystemfunctionsasdescribedintheOperationsManual
Migrationofdatafromthelegacysystem(ifapplicable)
ApprovaltoprogresstotheImplementationPhase

Goals
ThepurposeoftheTestPhaseistoguaranteethatsystemsuccessfullybuiltandtestedinthe
DevelopmentPhasemeetallrequirementsanddesignparameters.Afterbeingtestedand
accepted,thesystemmovestotheImplementationPhase.
BACKTOTOP

2.0DeliverablesandApprovals
SDLCdeliverableshelpStateagenciessuccessfullyplan,execute,andcontrolITprojectsby
providingaframeworktoensurethatallaspectsoftheprojectareproperlyandconsistentlydefined,
planned,andcommunicated.TheSDLCtemplatesprovideaclearstructureofrequiredcontent
alongwithboilerplatelanguageagenciesmayutilizeandcustomize.Stateagenciesmayuse
formatsotherthanthetemplates,aslongasthedeliverablesincludeallrequiredcontent.
ThedevelopmentanddistributionofSDLCdeliverables:

http://doit.maryland.gov/SDLC/Hardware/Pages/Phase07Multiple.aspx

2/13

8/1/2015

Phase7:TestHardwareMultipleReleaseProject

EnsurecommonunderstandingamongDevelopmentTeammembersandstakeholders,
Serveasareminderofspecifiedplansasprojectsbecomeincreasinglycomplex,
ProvideagencyseniormanagementandotherStateofficialsinsightintoprojectrisksand
ongoingperformance,
Encouragetheexecutionofrepeatableandconsistentprocesses,
FacilitatetheimplementationofprojectmanagementandagencyITbestpractices,and
Resultinacomprehensiverecordofprojectperformanceusefulformanypurposes(e.g.staff
knowledgetransfer,budgetaryandotherassessmentactivities,lessonslearned).
Duringthedevelopmentofdocumentation,theDevelopmentTeamshould:
Writecomprehensive,easytounderstanddocumentswithnoredundantinformation.
Developanorganizeddocumentrepositoryforcriticalprojectinformation,soDevelopment
Teammemberscaneasilyaccess,store,andreferenceprojectdocumentsandother
deliverablesfromalllifecyclephases.
Implementroutinedeliverablereviewstocorrectinaccuracy,incompleteness,and
ambiguities.
Recognizethatsampletemplatesfordeliverablesareavailableagenciesmightaccept
deliverablesindifferentformatsaslongasallrequiredinformationispresent.Thecontentof
thesedeliverablesmightexpandorshrinkdependingonthesize,scope,andcomplexityof
theproject.
Recycleorreferenceinformationfromearlierdocumentswherepossibleandbeneficial.
Thefollowingisalistingofdeliverablesrequiredofallprojectsforthisphaseofwork.

Deliverable
TestAnalysisApprovalDeterminationsummarizesthesystemsperceivedreadinessandis
attachedtotheTestAnalysisReportasafinalresultofthetestreviews.
Goals
Documenttheperceivedproductionreadinessofthesystem
ServeasaninputtotheprojectReadinessDocumentdescribedbelow

DevelopedBy
DevelopmentTeam
ApprovedBy
ProjectSponsor
AgencyCIO

http://doit.maryland.gov/SDLC/Hardware/Pages/Phase07Multiple.aspx

3/13

8/1/2015

Phase7:TestHardwareMultipleReleaseProject

Deliverable
TestProblemReportsdocumentproblemsencounteredduringtestingarealsoattachedto
theTestAnalysisReport.
Goals
Documentdetailedresultsoftesting
DevelopedBy
DevelopmentTeam
ApprovedBy
ProjectSponsor
AgencyCIO
Deliverable
InformationTechnologySystemsCertification&Accreditationincludescompletionofa
SecurityRiskAssessment,SensitiveSystemSecurityPlan,SecurityOperatingProcedures,
SecurityTestandEvaluation,andCertificationStatements.
Goals
Assesstechnicalandnontechnicalsafeguardstodeterminetheextenttowhichthesystem
meetssecurityrequirements
ObtainformaldeclarationbyaDesignatedApprovalAuthority(DAA)thataninformation
systemisapprovedtooperateinaparticularsecuritymodeusingaprescribedsetof
safeguardsatanacceptablelevelofrisk
DevelopedBy
DevelopmentTeam
ApprovedBy
ProjectSponsor
AgencyCIO
Deliverable
DefectLogtracksandsummarizesinatabularformatdefectsorbugsfoundduringtesting.
Defectsmaybedocumentedviamultiplecommerciallyavailablebugtrackingtoolsormanuallyin
aspreadsheet.

http://doit.maryland.gov/SDLC/Hardware/Pages/Phase07Multiple.aspx

4/13

8/1/2015

Phase7:TestHardwareMultipleReleaseProject

Goals
Allowteammemberstotrackreportedbugs,ordefects
Clearlycommunicatesummaryofdefectsfound
Recordfactsregardingknownbugs,suchastimesreported,individualswhoreported
them,defectstatuses,andteammembersresponsibleforaddressingthebugs
DevelopedBy
DevelopmentTeam
ApprovedBy
N/ATheDefectLogdoesnotrequireapproval.
Deliverable
ReadinessDocumentconsolidatessummaryinformationregardingthecurrentstatusofthe
systemandtheprojectandprovidesdecisionmakerswiththeinformationnecessarytomakea
GoNoGodecision.Itshouldincludeachecklistforallworkproducts,UserAcceptanceTest
results,otherqualitycontrolcheckssuchapeerreview,andresultsofthesystemwalkthroughs.
Goals
ProvideinformationnecessarytomaketheGoNoGodecision
Consolidatestatusinformationregardingtheeffectivecompletionoftheprojectand
achievementofprojectobjectivesandSDLCrequirements
Affirmachievementofalldeliverableacceptancecriteria
DevelopedBy
DevelopmentTeam
ApprovedBy
AgencyCIO

AlldeliverablesotherthanthoseidentifiedasUpdatesshouldbedevelopedinthisphase.
DeliverablesidentifiedasUpdatesshouldberevisitedandenhancedasnecessaryasprescribedin
thisphase.
Deliverablesproducedduringthisphasemustbereviewedindetailandshouldfollowtheapproval
pathasdefinedintheabovetable(foreachiteration).Asignaturepageorsectionshould
accompanyeachdeliverablerequiringapproval.DoITwillperiodicallyrequestcopiesofthese
documentsaspartofitsoversightresponsibilities..
BACKTOTOP

3.0Roles
http://doit.maryland.gov/SDLC/Hardware/Pages/Phase07Multiple.aspx

5/13

8/1/2015

Phase7:TestHardwareMultipleReleaseProject

Thefollowingpersonnelparticipateintheworkactivitiesduringthisphase:
ProjectSponsor
ExecutiveSponsor
AgencyCIO
ProjectManager
DevelopmentTeam
ProjectStakeholders

RACIKey
ResponsibleDescribesrolethatexecutestheactivitiestoachievethetask.
AccountableDescribesrolesthatownthequalityofthedeliverableandsignoffonworkthat
Responsibleprovides.
ConsultedDescribesrolesthatprovidesubjectmatterexpertise.
InformedDescribesrolesthatreceiveinformationaboutthetask.

TheRolesandResponsibilitiespagehasdetaileddescriptionsoftheserolesandtheirassociated
responsibilities.
BACKTOTOP

4.0TasksandActivities

http://doit.maryland.gov/SDLC/Hardware/Pages/Phase07Multiple.aspx

6/13

8/1/2015

Phase7:TestHardwareMultipleReleaseProject

http://doit.maryland.gov/SDLC/Hardware/Pages/Phase07Multiple.aspx

7/13

8/1/2015

Phase7:TestHardwareMultipleReleaseProject

4.1ReviewPhasePrerequisites.
TheProjectManagerensuresthefollowingprerequisitesforthisphasehavebeencompleted:
ThePMPiscurrent,andthescheduleshowingthetargetterminationdateforthesystemis
current.Allhardwareandnetworkcomponentshavebeenconstructedandtested.
Unitandintegrationtestplansandresultarefinal.
TheConversionPlanformigratingdatacompletelyandaccuratelyfromthelegacysystemto
thenewsystemiscomplete(ifapplicable).
DuringtheTestPhase,theDevelopmentTeamfrequentlymaydiscoverproblemswithinterfaces
andfunctionsthatrequirerepair.
TheProjectManagershouldconfirmandreviewanytestingtoolsanddefecttrackingmechanisms
andthechangemanagementtoolusedinthesystemdevelopment.
http://doit.maryland.gov/SDLC/Hardware/Pages/Phase07Multiple.aspx

8/13

8/1/2015

Phase7:TestHardwareMultipleReleaseProject

4.2MonitorProjectPerformance.
TheProjectManagermonitorsprojectperformancebygatheringstatusinformationabout:
Allchangestobaselinedata
Changemanagementinformation
Activityprogresswithstatusdetails
Listofcompleteandincompletedeliverables
Activitiesinitiatedandfinished
Testingperformedandtestresults
Estimatedtimetocompletion
Resourceutilizationdata
Changestoprojectscope
TheProjectManageralsoorganizesandoverseessystematicqualitymanagementreviewsof
projectworkasapartofmonitoringtheprojectperformance.
Tomeasureprojecteffortatallphasesofthelifecycle,theProjectManagerestablishestimelines
andmetricsforsuccessateachphaseofworkwhenplanningprojecttasks.
ThePMBOK,fourthedition,providesadditionaldetailsoncontrollingprojectworkinsections4.4
and4.5andonprojectscopecontrolinsection5.5.

4.3UpdatePMPandCommunicationManagementPlan.
TheProjectManagerupdatesthePMProutinely(atleastquarterly)toensurethePMPreflects
projectperformanceaccurately.Reviewprojectperformancecontrolsandrisksfordeviationsfrom
thebaseline.
InformationdistributionisoneofthemostimportantresponsibilitiesoftheProjectManager.The
ProjectManagerreviewsandupdatestheCommunicationManagementPlanatleastquarterlyto
documentpotentialstakeholderchanges.TheProjectManagerredistributestheupdatedPMPand
riskmanagementinformationaccordingtotherevisedCommunicationManagementPlan.PMBOK,
fourthedition,section10containsadditionaldetailsonprojectcommunicationsandinformation
distribution.

4.4PerformRiskManagementActivities.
TheProjectManagerconductsriskassessmentsduringtheTestPhasetheseactivitiesinclude:
Identificationdeterminationofinitialandemergingrisksthatmightaffecttheprojectaswell
aseachriskcharacteristic
RiskAnalysisconductingquantitativeand/orqualitativeanalysisofeachidentifiedrisk.
Usually,qualitativeriskmanagementtechniquesaremostapplicableforStateprojects.These
riskanalysismethods,aswellastheconditionsunderwhicheachmethodmightbeused,are
describedindetailinsection11ofPMBOK.
ResponsePlanningplanningofmethodsfordevelopingmitigation,transfer,oravoidance
strategiestoreducerisk
MonitoringandControltrackingrisks,monitoringresidualrisk,identifyingnewrisks,
executingresponseplans,andevaluatingriskmanagementeffectiveness
Theseactivitiesoccurthroughouttheprojectdurationtotrackandmitigateanyneworupdated
http://doit.maryland.gov/SDLC/Hardware/Pages/Phase07Multiple.aspx

9/13

8/1/2015

Phase7:TestHardwareMultipleReleaseProject

projectrisks.PMBOK,fourtheditionhasdetailsforriskmanagementactivitiesinsection11,
particularlyinsections11.2through11.6.

4.5MigrateLegacyData.
Ifdatamigrationisinscope,theProjectManagerreviewsandexecutestheConversionPlanto
migratelegacydatatothenewsystem.TheDevelopmentTeammayrepeatdatamigrationfor
eachiterationassociatedwithareleasetoproduction.Themigrationinvolvesenteringdataintothe
newsystemandverifyingthatthemigrateddataiscorrect.Havingcorrectdatainthenewsystem
isessentialtoitsfunctioningasintended.Adatamigrationplanisimportantfortransformation,
migration,andmodernizationprojectsentirelynewsystemswilllikelyhavenodatatoconvertand
havenoConversionPlan.
Afewpointsaboutdataconversion:
Informationcanbediscardedeasily,butaddinginformationrequireseffort.
Computerscanonlyaddinformationaccordingtorulespeoplecanaddinformationinany
manner.
Convertingdatatoanewformatwithadditionalfeaturesdoesnotcreatethenewinformation
butaddsspaceforit.Apersonmustenterthenewinformation.
Somedataconversioncanoccurdirectlyotherdataconversion,howeverrequiresseveral
timeconsumingsteps.
Dataconversionmayresultinthelossofinformationifthetargetformathasfewerfeatures
thanthesourceformat.
Onemethodofverifyingdataintegrityisparalleloperationsduringwhichtheoldsystemruns
simultaneouslywiththenewsystem.Theoutputfromeachsystemiscomparedifalliscorrect,the
newsystemiscertified.Ifthenewsystemfailsinanyway,continueoperationsontheoldsystem
untilallproblemsareresolved.

4.6ConductSystemTesting.
TheDevelopmentTeamconductsthesystemtestsaccordingtothetestplansanddocumentsall
resultsontheTestAnalysisReport,TestProblemReports,andTestAnalysisApproval
Determination.Systemtestingisconductedonacomplete,integratedsystemtodetermine
compliancewithallrequirements.Systemtestingincludesteststoensurethatthedeveloped
systemmeetsallrequirements.TheDevelopmentTeamwillrepeatsystemtestingforeach
iterationassociatedwithareleasetoproduction.
ReturnanyfailedcomponentstotheDevelopmentTeamforcorrectionmovethepassing
componentsontosecuritytesting.
Testingmaybeoneoftwoapproaches:
Static,ormanualtestingconductingreviews,walkthroughs,andinspection
Dynamic,orautomatedtestingexecutingasetoftestscripts
Testingmayfolloweitherablackboxtestingorawhiteboxtestingmethodology.
Blackboxtestingapproachesthesystemwithnoknowledgeoftheinternalcomponents,
structure,orfunctions.Blackboxtestingprovidesanunbiasedopinionaboutthesystembut
hasthedisadvantageofbeingblindtointerconnectionsandtherestofthesystem.
http://doit.maryland.gov/SDLC/Hardware/Pages/Phase07Multiple.aspx

10/13

8/1/2015

Phase7:TestHardwareMultipleReleaseProject

Whiteboxtestingallowsatestertohaveknowledgeoftheinfrastructuretobetested,
frequentlyincludingnetworkdiagrams,domainnames,andIPaddressinginformation.
Combiningwhiteboxtestingwithblackboxtestingallowsevaluationofthecompletenessof
thetestsuiteandinfrequentlytestedpartsofthesystemandensurescriticalfunctionshave
beentested.
Regressiontestingfocusesonrevealingerrorsinfunctionsthatdidworkcorrectlybutstopped
workingduetomodifications.Regressiontestingtypicallyinvolvesrepeatingentiretestscriptsto
ensureallfunctionalityoperatescorrectlyafteraunitorcomponenthasbeenmodified.
TheDevelopmentTeamshouldpreparetoperformnonfunctionaltestssuchasload,usability,and
securitytesting.Duringloadtesting,performancetestsstressthesystemandindicateifthesystem
canhandlelargequantitiesofdataorendusers.TheDevelopmentTeamcanautomatetestingto
expeditetheprocessandensureconsistency.
Regardlessofthetestingmethodology,theDevelopmentTeamupdatestheRTMtoreflectalltest
resultsandensuretraceabilitybacktotheoriginalrequirements.Whenalltestingisfinished,an
auditofthetestingshouldshowtestresultsforeveryelementofthesystemandtraceabilitytoits
correspondingrequirement.

4.7ConductSecurityTesting.
TheDevelopmentTeamexecutesthesecurity/penetrationtestsanddocumentsalltestresults.
Returnanyfailedcomponentstothedevelopersforcorrectionmovethepassingcomponentson
toacceptancetestingafterallcomponentshavepassedintegrationandsecuritytesting.The
DevelopmentTeamwillrepeatsecuritytestingforeachiterationassociatedwithareleaseto
production.
Testsecuritycontrolspriortothesystemdeploymenttouncoveralldesignandimplementation
flawsthatmightviolatetheDoITssecuritypolicy.Securitytestinginvolvesnumerousmethods,such
asanalyzingsystemdesigndocumentation,inspectingtestdocumentation,andindependently
executingfunctionalandpenetrationtesting.
StatepolicyforITsystemsrequiresthatallExecutiveBranchagenciescertifyandaccreditIT
systemsandsitesundertheirownershipandcontrol.TheDevelopmentTeamshouldreviewthe
DoITInformationTechnologySecurityCertificationandAccreditationGuidelinesandtheprojects
SSCDforanyactionsnecessarytoenablethesystemtobecomecertifiedandaccreditedpriorto
implementation.ThesedocumentsareavailableattheDoITStateInformationTechnologySecurity
PolicyandStandardswebpage.

4.8ConductAcceptanceTesting.
TheDevelopmentTeamstartsanddocumentstheacceptancetesting.Confirmthatthedeveloped
systemmeetsallrequirementsasstatedintheSRD.Review,rework,andretestanyfailed
components.Whenallcomponentspassacceptancetesting,thesystemisreadyfor
implementation.TheDevelopmentTeamwillrepeatacceptancetestingforeachiteration
associatedwithareleasetoproduction.

4.9UpdateSystemDocumentation.
DuringtheTestPhase,problemswithinterfacesandfunctionalityarefrequentlydiscoveredand
requirefixes.TheProjectManagerensuresthatthedocumentationreflectsanychangesfromall
http://doit.maryland.gov/SDLC/Hardware/Pages/Phase07Multiple.aspx

11/13

8/1/2015

Phase7:TestHardwareMultipleReleaseProject

previousphasesaswellaschangesthatoccurredduringthisPhase.Thisdocumentationincludes
theConversionPlan,theOperationsorSystemsAdministrationManual,theMaintenanceManual,
andtheTrainingPlan.TheProjectManagercoordinatestheseupdates.

4.10ReviewImplementationProcedures.
Bearinginmindanymodificationsthatresultfromtesting,theProjectManagerandDevelopment
Teamreviewimplementationprocedures,includinganynecessaryresourcesandinformation,for
deployingthesysteminitstargetenvironment.

4.11MakeSystemGoNoGoDecision.
TheAgencyCIOandProjectSponsordecidewhethertoperformadditionaltestingortoproceedto
thenextphase,Implementation.Theyreviewthesystemrequirements,theuseracceptance
criteria,andtheuseracceptancetestresultsandconsultwithothersabouttheconditionofthe
projectandthestateofcompletenessofthesystem.Usingtherequirementsandacceptance
criteriaasaquantitativebase,theyconsiderotherqualitativefactorsthroughacollaborative
discussiontoarriveattheGoNoGodecision.Thiscriticalprojectdecisioncanmovethesystem
toaproductionstate.

4.12PerformPhaseClosureActivities.
TheProjectManagerandtheDevelopmentTeamprepareandpresentaprojectstatusreviewfor
theAgencyCIO,ProjectSponsor,ExecutiveSponsor,andotherprojectstakeholdersafter
completingallTestPhasetasks.Thisreviewaddresses:
StatusofTestPhaseactivities
Planningstatusforallsubsequentlifecyclephases,withsignificantdetailaboutthenext
phase
Statusofresourceavailability
ProjectscopecontrolasdescribedinthePSS/(SBDandanyrequiredadjustmentstothe
scope
Changestotheprojectscheduleandestimatedcompletiondate
"GoNoGo"decisionmadetoproceedtonextphase,basedonTestPhaseinformation
VerificationthatallchangesareconductedinaccordancewiththeapprovedChange
ManagementPlan
TheProjectManagercomparesactualprojectperformancetothePMPandtheprojectedcostof
theprojecttodetermineanyvariancesfromthecostbaselineduringthephaseendreview.The
ProjectManageralsoperformsacomprehensiveriskassessmentoftheprojecttoupdatetheRisk
Registerbeforebeginningthenextphase,Implementation.
TheProjectManagermustobtaindeliverableapprovalsignaturesbeforeproceedingtothe
ImplementationPhase.
Updatetheprojectdocumentationrepositoryuponcompletionofthephaseclosureactivities..
BACKTOTOP

5.0Conclusions
AttheendoftheTestPhase,theDevelopmentTeamhascompletedaworking,fullytested
informationsystemthatmeetsallbusinessandtechnicalrequirements.TheapprovaloftheTest
http://doit.maryland.gov/SDLC/Hardware/Pages/Phase07Multiple.aspx

12/13

8/1/2015

Phase7:TestHardwareMultipleReleaseProject

Phasedeliverables,thecompletionoftheTestPhaseprojectstatusreview,andtheapprovalto
proceedtothenextphase,signifytheendoftheTestPhase.

ContactUs
Privacy
Accessibility
45CalvertStreet,Annapolis,MD21401
300301WestPrestonStreet,BaltimoreMD21201
TollFree:877.634.6361

http://doit.maryland.gov/SDLC/Hardware/Pages/Phase07Multiple.aspx

13/13