Sie sind auf Seite 1von 55

Upgrade Case Study: Database Replay, Snapshot Standby and Plan Baselines

Upgrade Case Study: Database Replay, Snapshot Standby and Plan Baselines Arup Arup Nanda Nanda

ArupArup NandaNanda

About Me

OracleOracle DBADBA forfor 1616 yearsyears andand countingcounting SpeakSpeak atat conferences,conferences, writewrite articles,articles, 44 booksbooks BroughtBrought upup thethe GlobalGlobal DatabaseDatabase GroupGroup atat StarwoodStarwood Hotels,Hotels, inin WhiteWhite Plains,Plains, NYNY

Starwood Hotels, Hotels, in in White White Plains, Plains, NY NY 22 Database Database 11g 11g
Starwood Hotels, Hotels, in in White White Plains, Plains, NY NY 22 Database Database 11g 11g
22
22

DatabaseDatabase 11g11g UpgradeUpgrade

What You will Learn

AA rehashrehash ofof ourour 11g11g UpgradeUpgrade ExperienceExperience WhatWhat challengeschallenges laylay duringduring ourour upgradeupgrade WhatWhat toolstools areare availableavailable withwith OracleOracle HowHow wewe usedused thesethese toolstools toto meetmeet thesethese challengeschallenges

The information is for educational purpose only; not professional advise or consultation. Starwood and the speaker make no warranty about the accuracy of the content and assume no responsibility for the consequence of the actions shown in the slides.

responsibility for the consequence of th e actions shown in the slides. Database Database 11g 11g

DatabaseDatabase 11g11g UpgradeUpgrade

33

Must Read

MetaLinkMetaLink NoteNote 429825.1429825.1 showsshows thethe stepssteps forfor aa manualmanual upgradeupgrade MetaLinkMetaLink NoteNote 601807.1601807.1 UpgradeUpgrade CompanionCompanion forfor 11gR1:11gR1: aa oneone stopstop paperpaper forfor upgrade.upgrade. NoteNote 837570.1837570.1 forfor 11gR211gR2 AA veryvery importantimportant stepstep [[NeverNever skipskip it]it] checkcheck forfor danglingdangling dictionarydictionary objectsobjects MetaLinkMetaLink 579523.1579523.1

dictionary objects objects – – MetaLink MetaLink 579523.1 579523.1 Database Database 11g 11g Upgrade Upgrade 4

DatabaseDatabase 11g11g UpgradeUpgrade

44

Database Details

AA lotlot ofof applications;applications; notnot justjust oneone AA lotlot ofof businessbusiness processes;processes; notnot justjust aa fewfew VeryVery criticalcritical businessbusiness functionalitiesfunctionalities AA highhigh $$ amountamount attributedattributed toto downtimedowntime oror slownessslowness (which(which alsoalso translatestranslates toto downtimedowntime sincesince thethe appsapps timetime out)out) VersionVersion 10.2.0.410.2.0.4 waswas prepre--upgradeupgrade

Version Version 10.2.0.4 10.2.0.4 was was pre pre - - upgrade upgrade Database Database 11g 11g

DatabaseDatabase 11g11g UpgradeUpgrade

55

Pre/Post-Upgrade

PrePre--UpgradeUpgrade

PostPost--UpgradeUpgrade

10.2.0.410.2.0.4

11.1.0.711.1.0.7

NoNo FlashFlash RecoveryRecovery AreaArea

FlashFlash RecoveryRecovery AreaArea

FlashbackFlashback notnot EnabledEnabled

FlashbackFlashback EnabledEnabled

NonNon--OMFOMF

OracleOracle ManagedManaged FilesFiles

OlderOlder hardwarehardware

NewerNewer hardwarehardware

NoNo partitioningpartitioning NoNo CompressionCompression SomeSome parametersparameters LinuxLinux RHASRHAS 44
NoNo partitioningpartitioning
NoNo CompressionCompression
SomeSome parametersparameters
LinuxLinux RHASRHAS 44

PartitioningPartitioning

CompressionCompression

ChangedChanged paramsparams

LinuxLinux RHASRHAS 55

DatabaseDatabase 11g11g UpgradeUpgrade

66

The $zillion Question

IfIf itit ainaintt brokenbroken dondontt fixfix itit isis generallygenerally thethe mantramantra MustMust HaveHave AnswersAnswers WhatWhat willwill happenhappen willwill thethe databasedatabase atat leastleast performperform asas muchmuch asas rightright now,now, oror itit mightmight bebe worse?worse? HowHow dodo wewe know?know? HowHow certaincertain areare we?we?

do do we we know? know? How How certain certain are are we? we? Database Database

DatabaseDatabase 11g11g UpgradeUpgrade

77

Why Worse?

OptimizerOptimizer PlansPlans couldcould bebe changechange forfor betterbetter (or,(or, worseworse)) performanceperformance relatedrelated FunctionalityFunctionality maymay havehave changed,changed, producingproducing unexpectedunexpected resultsresults NewNew bugsbugs maymay bebe encounteredencountered forfor whichwhich therethere willwill bebe nono patches,patches, atat leastleast notnot immediatelyimmediately SomeSome newnew functionalityfunctionality maymay requirerequire furtherfurther attentionattention

may may require require further further attention attention Database Database 11g 11g Upgrade Upgrade 8 8

DatabaseDatabase 11g11g UpgradeUpgrade

88

The Usual Plan

CreateCreate newnew environmentenvironment (pre(pre--prod)prod) andand runrun thethe productionproduction--likelike eventsevents therethere andand examineexamine thethe performanceperformance TheThe keykey isis itit isis productionproduction--likelike;; notnot actualactual eventsevents thatthat occurredoccurred inin production.production. UsuallyUsually synthetic,synthetic, concoctedconcocted

production. Usually Usually synthetic, synthetic, concocted concocted Database Database 11g 11g Upgrade Upgrade 9 9

DatabaseDatabase 11g11g UpgradeUpgrade

99

So, what’s Problem?

SyntheticSynthetic transactionstransactions areare notnot faithfulfaithful reproductionsreproductions ofof thethe eventsevents thatthat actuallyactually happenedhappened TheyThey areare mechanizedmechanized andand repeatable,repeatable, butbut dodo notnot capturecapture productionproduction dynamicsdynamics ConcoctedConcocted onesones dodo notnot taketake intointo accountaccount uniqueunique datadata values.values. Example:Example: namename searchessearches areare moremore onon JohnsonJohnsoninin NewNew YorkYork whilewhile inin LosLos Angeles,Angeles, ititss LeeLee

Los Los Angeles, Angeles, it it ’ ’ s s “ “ Lee Lee ” ”

DatabaseDatabase 11g11g UpgradeUpgrade

1010

The Ugly Truth

WillWill thethe databasedatabase workwork thethe samesame wayway (if(if notnot better)better) afterafter thethe upgrade?upgrade? SyntheticSynthetic transactionstransactions willwill notnot givegive youyou thethe answeranswer ToTo getget that,that, youyou mustmust askask thethe usersusers toto redoredo thethe activitiesactivities exactlyexactly howhow theythey diddid inin realreal productionproduction InIn thethe samesame order,order, usingusing thethe samesame breaksbreaks inin between!between!

using using the the same same breaks breaks in in between! between! Database Database 11g 11g

DatabaseDatabase 11g11g UpgradeUpgrade

1111

Challenges

BuildingBuilding aa testtest systemsystem quickly,quickly, easily,easily, accurately,accurately, repeatedlyrepeatedly DryDry runsruns ofof UpgradesUpgrades EnsuringEnsuring performanceperformance RepeatingRepeating thethe activitiesactivities ofof thethe productionproduction

accuratelyaccurately withoutwithout impactingimpacting productionproduction

without without impacting impacting production production Impact Impact of of new new parameters parameters

ImpactImpact ofof newnew parametersparameters

DatabaseDatabase 11g11g UpgradeUpgrade

1212

Additional Challenges

YouYou wantwant toto changechange somethingsomething elseelse duringduring thethe upgradeupgrade (since(since youyou havehave anan outage)outage) ConvertConvert toto RACRAC StorageStorage toto ASMASM ChangeChange bufferbuffer poolspools ChangeChange somesome parameter,parameter, suchsuch asas cursor_sharingcursor_sharing TakeTake advantageadvantage ofof newnew features,features, e.g.e.g. LOBsLOBs toto SecurefilesSecurefiles

features, e.g. e.g. LOBs LOBs to to Securefiles Securefiles Database Database 11g 11g Upgrade Upgrade 13

DatabaseDatabase 11g11g UpgradeUpgrade

1313

Tools at your Disposal

DatabaseDatabase ReplayReplay SQLSQL PerformancePerformance AnalyzerAnalyzer SQLSQL TuningTuning AdvisorAdvisor SQLSQL PlanPlan ManagementManagement EasierEasier StandbyStandby BuildingBuilding SnapshotSnapshot StandbyStandby SwitchingSwitching betweenbetween PhysicalPhysical andand LogicalLogical

Switching between between Physical Physical and and Logical Logical Database Database 11g 11g Upgrade Upgrade 14

DatabaseDatabase 11g11g UpgradeUpgrade

1414

Concocting Prod Work

WorkloadWorkload generatorgenerator toolstools suchsuch asas LoadLoad RunnerRunner cancan simulatesimulate useruser actions,actions, CaptureCapture clickstreamclickstream onon aa webpagewebpage DatabankDatabank parametersparameters toto simulatesimulate loadload CoverageCoverage forfor importantimportant workflowsworkflows onlyonly UpgradeUpgrade involvesinvolves onlyonly oneone changedchanged partpart ApplicationApplication -->> AppApp ServerServer -->> DatabaseDatabase So,So, therethere isis nono needneed toto testtest thethe entireentire stackstack CostCost ofof QAQA isis notnot insignificantinsignificant AvailabilityAvailability ofof QAQA isis notnot automaticautomatic

Availability of of QA QA is is not not automatic automatic Database Database 11g 11g Upgrade

DatabaseDatabase 11g11g UpgradeUpgrade

1515

Parts of a System

App Server

Database

Parts of a System App Server Database Applications These are the only parts that are changing
Parts of a System App Server Database Applications These are the only parts that are changing

Applications

Parts of a System App Server Database Applications These are the only parts that are changing
These are the only parts that are changing
These are the
only parts that
are changing

DatabaseDatabase 11g11g UpgradeUpgrade

O/S

Storage

1616

Workload Generation Tools

App Server

Database

They may work here; but in that case, it becomes a pure SQL “runner”
They may work
here; but in that
case, it becomes a
pure SQL “runner”
here; but in that case, it becomes a pure SQL “runner” O/S Applications Synthetic Workload Generation

O/S

but in that case, it becomes a pure SQL “runner” O/S Applications Synthetic Workload Generation Tools

Applications

Synthetic Workload Generation Tools Usually Work Here
Synthetic
Workload
Generation Tools
Usually Work Here

Storage

What are the SQLs? When to run? How to sequence?

DatabaseDatabase 11g11g UpgradeUpgrade

1717

Questions for “SQL Running”

WhatWhat SQLsSQLs werewere executedexecuted HowHow oftenoften waswas eacheach oneone executedexecuted DeterminesDetermines parsing,parsing, bufferbuffer cachecache hits,hits, etc.etc. InIn whatwhat orderorder werewere theythey executedexecuted DeterminesDetermines bufferbuffer hitshits HowHow muchmuch waswas thethe timetime betweenbetween themthem DeterminesDetermines bufferbuffer hits,hits, parsingparsing WhatWhat optimizeroptimizer environmentenvironment waswas inin effecteffect SomeoneSomeone setssets DBFMBRCDBFMBRC beforebefore runningrunning anan SQLSQL andand thenthen resetsresets itit toto defaultdefault AreAre sequencesequence numbersnumbers guaranteed?guaranteed?

DatabaseDatabase 11g11g UpgradeUpgrade

Are Are sequence sequence numbers numbers guaranteed? guaranteed? Database Database 11g 11g Upgrade Upgrade 18 18

1818

Capturing the Work

SQLSQL TraceTrace CapturesCaptures thethe SQLSQL statements,statements, inin order,order, withwith plansplans 1004610046 TraceTrace CapturesCaptures thethe SQLSQL statementsstatements withwith timestamptimestamp BindBind variablesvariables 1005310053 TraceTrace CapturesCaptures thethe optimizeroptimizer environmentenvironment But,But, howhow willwill youyou putput thethe informationinformation fromfrom allall thisthis togethertogether toto produceproduce somethingsomething thatthat is:is:

ExecutableExecutable RepeatableRepeatable

DatabaseDatabase 11g11g UpgradeUpgrade

1919

Database Replay

ThisThis isis wherewhere DatabaseDatabase ReplayReplay reallyreally shinesshines ItIt capturescaptures thethe actualactual transactionstransactions fromfrom thethe productionproduction system,system, inin thethe samesame order,order, withwith thethe samesame breaksbreaks inin betweenbetween ItItss asas ifif thethe usersusers areare redoingredoing thethe samesame activitiesactivities inin frontfront ofof thethe testtest systemsystem EvenEven sequencesequence numbersnumbers areare fetchedfetched thethe samesame wayway theythey occurredoccurred inin productionproduction NoNo primaryprimary keykey violationviolation

production No No primary primary key key violation violation Database Database 11g 11g Upgrade Upgrade 20

DatabaseDatabase 11g11g UpgradeUpgrade

2020

Workload Capture

TheThe packagepackage dbms_workload_capturedbms_workload_capture capturescaptures workloadworkload fromfrom currentcurrent productionproduction TheThe packagepackage existsexists inin 11g,11g, soso whatwhat aboutabout 10g?10g? InIn 10.2.0.410.2.0.4 itit existsexists ForFor earlierearlier versions,versions, aa patchpatch needsneeds toto bebe appliedapplied ReferRefer toto MetaLinkMetaLink NoteNote 560977.1560977.1 forfor detailsdetails TheThe easiesteasiest isis toto useuse EnterpriseEnterprise ManagerManager GridGrid ControlControl GridGrid ControlControl 10.2.0.510.2.0.5 hashas thethe toolkittoolkit

Control Control 10.2.0.5 10.2.0.5 has has the the toolkit toolkit Database Database 11g 11g Upgrade Upgrade

DatabaseDatabase 11g11g UpgradeUpgrade

2121

Steps

CaptureCapture WorkloadWorkload ItIt producesproduces aa setset ofof filesfiles withwith extensionextension *.*.recrec MoveMove themthem toto thethe 11g11g systemsystem UseUse ReplayReplay featurefeature inin commandcommand lineline oror EMEM toto replayreplay thethe activitiesactivities BothBoth thesethese activitiesactivities taketake AWRAWR snapshotssnapshots beforebefore andand afterafter events.events. UseUse AWRAWR CompareCompare PeriodPeriod ReportReport toto comparecompare thethe performance.performance. CheckCheck sessionsession S311835S311835 forfor DatabaseDatabase ReplayReplay DemoDemo

S311835 S311835 for for Database Database Replay Replay Demo Demo Database Database 11g 11g Upgrade Upgrade

DatabaseDatabase 11g11g UpgradeUpgrade

2222

Capture from 10g

CreateCreate aa directorydirectory toto holdhold thethe recrec filesfiles

createcreate directorydirectory RATRAT asas /oracle/rat/oracle/rat

AddAdd aa FilterFilter

BEGINBEGIN

dbms_workload_capture.add_filterdbms_workload_capture.add_filter((

fnamefname

fattributefattribute =>=> 'USER','USER',

=>=> ''abcd_filterabcd_filter',',

fvaluefvalue

=>=> 'ABCD');'ABCD');

END;END; AllowsAllows youyou toto capturecapture onlyonly tthosehose forfor thethe useruser calledcalled ABCD.ABCD.

DatabaseDatabase 11g11g UpgradeUpgrade

2323

StartStart thethe CaptureCapture ProcessProcess

BEGINBEGIN DBMS_WORKLOAD_CAPTURE.START_CAPTUREDBMS_WORKLOAD_CAPTURE.START_CAPTURE ((

 

namename

=>=> capture1',capture1', =>=> 'RAT','RAT', =>=> 3600,3600,

 

dirdir

durationduration

default_actiondefault_action

=>=> 'EXCLUDE','EXCLUDE', =>=> TRUE);TRUE);

auto_unrestrictauto_unrestrict

   

END;END;

 

ItIt willwill generategenerate aa lotlot ofof filesfiles inin thethe formatformat wcrwcr_* *.recrec /oracle/rat/oracle/rat directory.directory.

inin thethe

 
 

DatabaseDatabase 11g11g UpgradeUpgrade

2424

GetGet thethe capturecapture IDID

selectselect IDID fromfrom dba_workload_capturesdba_workload_captures wherewhere statusstatus == COMPLETEDCOMPLETED

ExportExport thethe AWRAWR

beginbegin dbms_workload_capture.export_awrdbms_workload_capture.export_awr ((capture_idcapture_id =>=> <<captureidcaptureid>);>);

end;end;

//

AWRAWR willwill alsoalso bebe exportedexported asas aa dumpfiledumpfile inin thethe /oracle/rat/oracle/rat directory.directory. CopyCopy allall thethe filesfiles inin thatthat directorydirectory toto thethe targettarget systemsystem

DatabaseDatabase 11g11g UpgradeUpgrade

2525

Replay Steps

Replay Steps 1. 1. Create Create directory directory on on the the target target 2. 2.
Replay Steps 1. 1. Create Create directory directory on on the the target target 2. 2.

1.1. CreateCreate directorydirectory onon thethe targettarget 2.2. PrePre--processprocess thethe capturedcaptured workloadworkload 3.3. ReplayReplay thethe workloadworkload 4.4. FromFrom thethe commandcommand lineline $$ wrcwrc system/managersystem/manager

line line $ $ wrc wrc system/manager system/manager replaydir replaydir =/u01/oracle/rat =/u01/oracle/rat

replaydirreplaydir=/u01/oracle/rat=/u01/oracle/rat

DatabaseDatabase 11g11g UpgradeUpgrade

2626

During Replay

Gives you an idea about how much is left DatabaseDatabase 11g11g UpgradeUpgrade 2727
Gives you an idea
about how much is
left
DatabaseDatabase 11g11g UpgradeUpgrade
2727

Get the Reports

DatabaseDatabase 11g11g UpgradeUpgrade 2828
DatabaseDatabase 11g11g UpgradeUpgrade
2828
This “compare” report, aka “Diff- diff Report” is the most important. It shows the system
This “compare”
report, aka “Diff-
diff Report” is the
most important. It
shows the
system stats on
the target and the
source when the
same activities
were occurred
there.
important. It shows the system stats on the target and the source when the same activities

SQL Performance Analyzer

SomeSome SQLsSQLs showedshowed regression,regression, i.e.i.e. theythey underperformedunderperformed comparedcompared toto 10g10g YouYou needneed toto knowknow whywhy optimizeroptimizer environment,environment, bindbind variables,variables, etc?etc? SPASPA allowsallows youyou toto runrun capturedcaptured SQLsSQLs inin differingdiffering environmentsenvironments InIn thethe samesame databasedatabase butbut

In In the the same same database database but but Different Different optimizer optimizer parameters

DifferentDifferent optimizeroptimizer parametersparameters DifferentDifferent waysways ofof collectingcollecting stats,stats, WithWith pendingpending statsstats inin 11g,11g, cancan validatevalidate onon PRODPROD duringduring maintenancemaintenance windows/nonwindows/non--peakpeak DifferentDifferent indexes,indexes, oror MVsMVs

DatabaseDatabase 11g11g UpgradeUpgrade

2929

Source of SQLs

SharedShared PoolPool CapturedCaptured fromfrom ProductionProduction duringduring aa workloadworkload StoredStored inin aa SQLSQL TuningTuning SetSet (STS)(STS) ContinuousContinuous CaptureCapture functionalityfunctionality toto capturecapture allall SQLsSQLs

Source
Source

STS

Export

And

Import

DatabaseDatabase 11g11g UpgradeUpgrade

STS Target
STS
Target
Replay 3030
Replay
3030

Capture from 10g

TheThe followingfollowing capturescaptures thethe SQLSQL StatementsStatements intointo aa SQLSQL TuningTuning SetSet (STS)(STS) inin 10g.10g. BEGINBEGIN dbms_sqltune.capture_cursor_cache_sqlsetdbms_sqltune.capture_cursor_cache_sqlset(( sqlset_namesqlset_name =>'10GSTS',=>'10GSTS',

time_limittime_limit

=>=> '3600','3600',

repeat_intervalrepeat_interval=>'300',=>'300',

repeat_interval =>'300', =>'300', sqlset_owner sqlset_owner =>'SYS');

sqlset_ownersqlset_owner =>'SYS');=>'SYS');

END;END; ThisThis incrementallyincrementally capturescaptures thethe SQLSQL statementsstatements everyevery 55 minsmins forfor 1010 hours.hours. YouYou cancan exportexport thisthis STSSTS andand importimport intointo 11g.11g.

DatabaseDatabase 11g11g UpgradeUpgrade

3131

SPA Tasks

SPA Tasks Create Create an an SPA SPA Task Task on on the the STS STS
SPA Tasks Create Create an an SPA SPA Task Task on on the the STS STS

CreateCreate anan SPASPA TaskTask onon thethe STSSTS importedimported ReplayReplay withwith OptimizerOptimizer == 10.2.0.410.2.0.4 ReplayReplay withwith OptimizerOptimizer == 11.1.0.711.1.0.7 CompareCompare andand makemake adjustmentsadjustments RepeatRepeat 22 throughthrough 44 asas neededneeded

Repeat Repeat 2 2 through through 4 4 as as needed needed Database Database 11g 11g

DatabaseDatabase 11g11g UpgradeUpgrade

3232

SPA Optimizer Change

CreateCreate anan SPASPA TaskTask onon thethe STSSTS importedimported

an an SPA SPA Task Task on on the the STS STS imported imported Database Database
an an SPA SPA Task Task on on the the STS STS imported imported Database Database

DatabaseDatabase 11g11g UpgradeUpgrade

3333

Compare

Elapsed time significantly reduced DatabaseDatabase 11g11g UpgradeUpgrade
Elapsed time
significantly
reduced
DatabaseDatabase 11g11g UpgradeUpgrade
Majority of SQLs didn’t see their plan changed!
Majority of
SQLs didn’t
see their plan
changed!

3434

Compare …

Shows the SQL_IDs, we can find from v$sql
Shows the SQL_IDs,
we can find from
v$sql
Plan changed for this SQL, Using SQL_ID, check from v$sql DatabaseDatabase 11g11g UpgradeUpgrade 3535
Plan changed for this SQL, Using
SQL_ID, check from v$sql
DatabaseDatabase 11g11g UpgradeUpgrade
3535
You can call upon SQL Tuning Advisor to suggest possible tuning options on this SQL
You can call upon SQL Tuning
Advisor to suggest possible
tuning options on this SQL

Clicking on the SQL_ID you can see the various stats on the SQL

SeeSee thethe SPASPA DemoDemo atat SessionSession S311324S311324 The report continues with the plans before and
SeeSee thethe SPASPA DemoDemo atat SessionSession
S311324S311324
The report continues with the plans
before and after the upgrade, so you
can compare them
DatabaseDatabase 11g11g UpgradeUpgrade
3636

SQL Plan Management

WhatWhat happenshappens whenwhen thethe planplan isis actuallyactually worse?worse? PerhapsPerhaps thethe planplan isis betterbetter whenwhen aa differentdifferent optimizeroptimizer environmentenvironment parameterparameter isis used?used? InIn thatthat case,case, wewe usedused SQLSQL PlanPlan ManagementManagement toto letlet thethe optimizeroptimizer pickpick thethe rightright planplan fromfrom thethe poolpool ofof plansplans

right plan plan from from the the pool pool of of plans plans Database Database 11g

DatabaseDatabase 11g11g UpgradeUpgrade

3737

SPM

AnalogousAnalogous toto StoredStored OutlinesOutlines ButBut unlikeunlike outlines,outlines, baselines:baselines:

CalculateCalculate thethe planplan anyway;anyway; butbut dondontt useuse it.it. TheThe DBADBA mustmust checkcheck andand markmark aa planplan goodgood byby acceptingacceptingitit aa processprocess calledcalled evolvingevolving HaveHave multiplemultiple plansplans inin thethe baselinebaseline andand choosechoose thethe bestbest SoSo itit isis thethe bestbest ofof bothboth worldsworlds CheckCheck sessionsession S309133S309133 forfor SPMSPM DemoDemo

Check session session S309133 S309133 for for SPM SPM Demo Demo Database Database 11g 11g Upgrade

DatabaseDatabase 11g11g UpgradeUpgrade

3838

Strategy with SPM

IfIf aa planplan isis fixedfixed,, thatthat isis used,used, regardlessregardless ofof thethe presencepresence ofof otherother plansplans CaptureCapture allall thethe plansplans fromfrom 10g10g toto anan SQLSQL TuningTuning SetSet LoadLoad themthem toto 11g11g afterafter upgradeupgrade MarkMark allall ofof themthem asas fixedfixed So,So, thethe plansplans willwill bebe thethe samesame asas 10g10g TurnTurn onon capturecapture baselines;baselines; thethe newnew plansplans willwill bebe storedstored inin thethe baselinesbaselines EvolveEvolve themthem toto seesee ifif anyany planplan isis betterbetter

them to to see see if if any any plan plan is is better better Database

DatabaseDatabase 11g11g UpgradeUpgrade

3939

Test System Creation

10g 10g Data Guard Original New System System DatabaseDatabase 11g11g UpgradeUpgrade 4040
10g
10g
Data Guard
Original
New
System
System
DatabaseDatabase 11g11g UpgradeUpgrade
4040

Test System Creation

10g
10g
11g
11g
X Data Guard removed Original System
X
Data Guard
removed
Original
System

upgraded

New

System

DatabaseDatabase 11g11g UpgradeUpgrade

1. Start the DB Workload Capture Process

2. Simultaneously break Data Guard

3. Convert the Standby to snapshot standby

4. Upgrade the standby to

11g

4141

Converting 10gR2 Standby to RW

PrimaryPrimary

StandbyStandby

1.1. alteralter databasedatabase recoverecoverr managedmanaged standbystandby databasedatabase cancel;cancel;

2.2. createcreate restorerestore pointpoint goldgold guaranteeguarantee flashbackflashback database;database;

1.1. alteralter systemsystem archivelogarchivelog current;current; 2.2. alteralter systemsystem log_archive_dest_slog_archive_dest_s tate_2tate_2 == defer;defer;

log_archive_dest_s tate_2 tate_2 = = defer; defer; 4. 4. 1 . 1 . alter alter database

4.4.

1.1. alteralter databasedatabase activateactivate standbystandby database;database;

2.2. shutdown/startupshutdown/startup mountmount

3.3. alteralter databasedatabase setset stastandbyndby databasedatabase toto maximizemaximize performance;performance;

alteralter systemsystem log_archive_dest_state_2log_archive_dest_state_2 == defer;defer;

DatabaseDatabase 11g11g UpgradeUpgrade

4242

5.5. alteralter databasedatabase open;open;

Test System Creation

Enable Database for Flashback

10g

11g
11g

Original

System

New

System

DatabaseDatabase 11g11g UpgradeUpgrade

4343

Test System Creation

Take a Restore Point 10g 11g Replay Capture Workload Workload Original New System System
Take a
Restore
Point
10g
11g
Replay
Capture
Workload
Workload
Original
New
System
System

DatabaseDatabase 11g11g UpgradeUpgrade

4444

Test System Creation

Flashback the database to the Restore Point

Creation Flashback the database to the Restore Point 10g 11g Replay Workload Original System New System
10g
10g
11g Replay Workload
11g
Replay
Workload
database to the Restore Point 10g 11g Replay Workload Original System New System Database Database 11g
Original System
Original
System

New

System

DatabaseDatabase 11g11g UpgradeUpgrade

The workload has been captured only once; and replayed several times.

Repeat this as often as needed

4545

Actual Upgrade

1. 10g 10g Standby

2. Stop Data Guard

3. Upgrade the standby to 11g

4. This becomes the new production

5. The old prod is still available as of that point in time

10g
10g
11g
11g
X Data Guard removed Original System
X
Data Guard
removed
Original
System

upgraded

New

System

DatabaseDatabase 11g11g UpgradeUpgrade

4646

Post Upgrade Tweaking

11g
11g

11g

11g Physical Standby, Converted from the old system
11g Physical Standby, Converted from the old system

Physical Standby, Converted from the old system

Standby

Physical Standby, Converted from the old system Standby Data Guard New System Database Database 11g 11g
Physical Standby, Converted from the old system Standby Data Guard New System Database Database 11g 11g

Data Guard

New

System

DatabaseDatabase 11g11g UpgradeUpgrade

4747

Post Upgrade Tweaking

1. What should the value of cursor_space_for_time should be?

2. What will be the effect of the I/O constraining Resource Manager?

3. What will be effect of the Patch Update?

11g
11g

11g

11g Convert to Snapshot Standby
11g Convert to Snapshot Standby

Convert to Snapshot Standby

Standby

Update? 11g 11g Convert to Snapshot Standby Standby Data Guard stopped New Prod Database Database 11g
Update? 11g 11g Convert to Snapshot Standby Standby Data Guard stopped New Prod Database Database 11g

Data Guard stopped

New

Prod

DatabaseDatabase 11g11g UpgradeUpgrade

4848

Post Upgrade Tweaking

1. Take a Restore Point on the standby

Make changes on the standby

2.

11g 11g Capture Replay Workload Workload
11g
11g
Capture
Replay
Workload
Workload

3.

4.

Standby

Capture workload from production

Replay against the standby

workload from production Replay against the standby New 5. Flashback the standby to the Restore Point

New

5. Flashback the standby to the Restore Point

6. Repeat steps 2-5

Prod

DatabaseDatabase 11g11g UpgradeUpgrade

4949

Convert to Active Data Guard

11g
11g

11g

Convert to Active Data Guard 11g 11g Data Guard New Standby Prod Database Database 11g 11g
Convert to Active Data Guard 11g 11g Data Guard New Standby Prod Database Database 11g 11g

Data Guard

New

Standby

Prod

DatabaseDatabase 11g11g UpgradeUpgrade

1. Convert the standby back to normal from snapshot

2. Stop Managed Recovery Process

3. Open the standby in Read Only mode

4. Restart the MRP

5. Pure Read Only queries can be directed at the Standby

5050

Maintaining 2 Versions

1. 10g 10g Standby

2. Break Data Guard

3. Upgrade the standby to 11g

4. This becomes the pre- production

5. Set up Golden Gate replication (or Streams) to apply SQLs to the 11g DB from 10g

(or Streams) to apply SQLs to the 11g DB from 10g 10g 11g Golden Gate Replication

10g

11g
11g
(or Streams) to apply SQLs to the 11g DB from 10g 10g 11g Golden Gate Replication

Golden

Gate

Replication

Original

System

New

System

DatabaseDatabase 11g11g UpgradeUpgrade

5151

Cutover

1. Stop the apply

2. Redirect clients to the new DB

3. Reverse the replication direction.

to the new DB 3. Reverse the replication direction. 10g 11g Golden Gate Replication Original System

10g

11g
11g
the new DB 3. Reverse the replication direction. 10g 11g Golden Gate Replication Original System New

Golden

Gate

Replication

Original

System

New

System

DatabaseDatabase 11g11g UpgradeUpgrade

5252

Tools Used

DatabaseDatabase ReplayReplay SQLSQL PerformancePerformance AnalyzerAnalyzer SQLSQL TuningTuning AdvisorAdvisor SnapshotSnapshot StandbyStandby ActiveActive DataData GuardGuard GoldenGolden GateGate

Active Active Data Data Guard Guard Golden Golden Gate Gate Database Database 11g 11g Upgrade Upgrade

DatabaseDatabase 11g11g UpgradeUpgrade

5353

Conclusion

UpgradeUpgrade isis justjust goinggoing toto happen,happen, youyou cancantt preventprevent itit ThisThis isis thethe bestbest youyou cancan dodo toto mitigatemitigate thethe risks,risks, byby replayingreplaying thethe activitiesactivities asas faithfullyfaithfully asas youyou cancan OracleOracless RealReal ApplicationApplication TestingTesting SuiteSuite allowsallows youyou exactlyexactly thatthat faithfullyfaithfully replayingreplaying thethe activitiesactivities UsingUsing StandbyStandby databasedatabase youyou cancan minimizeminimize thethe riskrisk ofof failurefailure duringduring upgrade.upgrade. SnapshotSnapshot StandbyStandby allowsallows youyou toto tweaktweak thethe parametersparameters andand setssets thethe stagestage forfor futurefuture upgradesupgrades

sets the the stage stage for for future future upgrades upgrades Database Database 11g 11g Upgrade

DatabaseDatabase 11g11g UpgradeUpgrade

5454

DatabaseDatabase 11g11g UpgradeUpgrade 5555
DatabaseDatabase 11g11g UpgradeUpgrade
5555