Beruflich Dokumente
Kultur Dokumente
https://karlarao.wordpress.com/2010/01/31/workload-characterization-...
REPLY
1 de 15
27/09/2015 11:54
https://karlarao.wordpress.com/2010/01/31/workload-characterization-...
(h p://lh3.ggpht.com/_F2x5WXOJ6Q8/S2hR6V8NjCI/AAAAAAAAAo0/YM_c7VhFKiI
/dba_hist3.png)
Now time for some action! :)
As you can see from above.. there is a highlighted portion (in green). By running the script, we have
identied that there was a sudden spike on the following:
DB Time (7th column).. which means there is some signicant user level activity going on!
IOPs read (16th column) and IO read MB/s (19th column).. which means that the stresser is
possibly more on SELECT operations
Oracle and OS CPU% utilization (last 6 columns) which means the spike on activity is
a ributed by Oracle and not other operations on the OS, also you could see that it is not a ributed by
an RMAN activity (because its zero!)
Plus the Average Active Sessions (AAS) (11th column) and the OS Load (13th column)
2 de 15
27/09/2015 11:54
https://karlarao.wordpress.com/2010/01/31/workload-characterization-...
(h p://lh4.ggpht.com/_F2x5WXOJ6Q8/S2WdQ-v_hxI/AAAAAAAAAnQ/DwfXAlbyWPg/ksar1.png)
Also youll see the Load Average, which is the same from the output of the AWR script above.
(h p://lh6.ggpht.com/_F2x5WXOJ6Q8/S2WdQ-oJbBI/AAAAAAAAAnU/9oqBrfLGNfc
/loadaverage1.png)
Now, lets check out the IO devices the database is using ASM with 3 ASM disks, I would get the
major and minor numbers and correlate these with the SAR data
3 de 15
27/09/2015 11:54
1
2
3
4
5
6
7
8
9
10
https://karlarao.wordpress.com/2010/01/31/workload-characterization-...
GROUP_NUMBER NAME
HEADER_STATU MOUNT_S
------------ -------------------- -----------1 DATA_1_0000
UNKNOWN
OPENED
1 DATA_1_0001
UNKNOWN
OPENED
1 DATA_1_0002
UNKNOWN
OPENED
[oracle@dbrocaix01
brw-rw---- 1 root
brw-rw---- 1 root
brw-rw---- 1 root
STATE
TOTAL_MB
FREE_MB LA
------- -------- ---------- ----NORMAL
3067
0
NORMAL
3067
0
NORMAL
3067
0
osm]$ ls -l /dev/sd[c,d,f]
disk 8, 32 Jan 17 2010 /dev/sdc
disk 8, 48 Jan 17 2010 /dev/sdd
disk 8, 80 Jan 17 2010 /dev/sdf
You can see from the images of the block devices below that they are mainly on Read Activity.. which
validates the output of IOPs read (16th column) and IO read MB/s (19th column) of the AWR
script cool!
(h p://lh6.ggpht.com
/_F2x5WXOJ6Q8
/S2WdRDdc4gI
/AAAAAAAAAnY
/N2z2ZTOSeko/dev8-32.png)
(h p://lh5.ggpht.com
/_F2x5WXOJ6Q8
/S2WdRJb_7UI
/AAAAAAAAAnc
/wVH42EOo4-8/dev8-48.png)
(h p://lh6.ggpht.com
/_F2x5WXOJ6Q8
/S2WdaPSE5RI
/AAAAAAAAAng
/gsQIGeSqZY8/dev8-80.png)
Another thing When you get to run the script on your environment also try to generate AWR
report on particular SNAP_IDs youll notice that the values you got from the query will be the
same on the AWR report
Update (20100201):
Id like you to see the output of the Enterprise Manager Performance Page on the time when the
sudden workload spike happened.
(h p://lh4.ggpht.com
/_F2x5WXOJ6Q8/S2a45qQX4aI
/AAAAAAAAAoA
/qPg_TrYxxlA
/PerformancePage1.png)
(h p://lh6.ggpht.com
/_F2x5WXOJ6Q8/S2a45hDdpJI
/AAAAAAAAAoE
/zUzo6tLbOE8
/PerformancePage2.png)
(h p://lh4.ggpht.com
/_F2x5WXOJ6Q8
/S2a45gvDB7I
/AAAAAAAAAoI
/LOv7qMAaOUo
/TopActivity.png)
The rst and second images above came from the root page, youll see the following similarities on
the AWR script output:
Load Average, which is on the range of 3.2 to 4.4
Average Active Sessions (AAS), which is on the range of 2.2 to 3.5
Disk IO read MB/s see that the graph is reaching the 80000 Physical Reads KB/s.. check the
4 de 15
27/09/2015 11:54
https://karlarao.wordpress.com/2010/01/31/workload-characterization-...
SNAP_ID 338 (Time 6:50) from the AWR script output youll see that it generated 73.372 MB/s!
Also, see the output of the ADDM run below which further validates the output of the script:
FINDING 2: 18% impact (236 seconds)
----------------------------------The throughput of the I/O subsystem was significantly lower than expected.
RECOMMENDATION 1: Host Configuration, 18% benefit (236 seconds)
ACTION: Consider increasing the throughput of the I/O subsystem.
Oracle's recommended solution is to stripe all data file using the
SAME methodology. You might also need to increase the number of disks
for better performance. Alternatively, consider using Oracle's
Automatic Storage Management solution.
RATIONALE: During the analysis period, the average data files' I/O
throughput was 74 M per second for reads and 102 K per second for
writes. The average response time for single block reads was 13
milliseconds.
SYMPTOMS THAT LED TO THE FINDING:
SYMPTOM: Wait class "User I/O" was consuming significant database time.
(37% impact [488 seconds])
The third image is the Top Activity section you can see that most of the activity are SELECTs and
the database is waiting on User and System I/O (see the Top SQLs and Top Sessions)
BTW, this script could also be used on a RAC environment you have to run it on each node. Ive made it
pull the DBID and instance_number automatically..
Update (20100202):
On Windows environment for the DBA_HIST_OSSTAT view it does not contain the LOAD and
IOWAIT_TIME stat_name so I removed these columns from the query. See the sample output below:
(h p://lh4.ggpht.com/_F2x5WXOJ6Q8/S2hR6URKt1I/AAAAAAAAAow/HgTEtkuD4sU
/dba_hist_windows.png)
Thats all for this post! Now youll know be er about the workload of your environment :)
5 de 15
27/09/2015 11:54
https://karlarao.wordpress.com/2010/01/31/workload-characterization-...
17 comments
6 de 15
7 de 15
https://karlarao.wordpress.com/2010/01/31/workload-characterization-...
CASE
WHEN e.stat_name = AVG_IOWAIT_TIME
THEN CASE
WHEN s.begin_interval_time = s.startup_time
THEN e.VALUE
ELSE e.VALUE Lag(e.VALUE)
OVER(PARTITION BY e.stat_name,e.dbid,e.instance_number,s.startup_time ORDER BY
e.snap_id)
END
ELSE 0
END avg_iowait_time,
patrick
karlarao says:
MARCH 1, 2010 AT 2:23 AM
Thanks for the feedback Patrick Ill look into this :)
Pingback: Blogroll Report 29/01/2009 05/02/2010 Coskans Approach to Oracle
Pingback: Seeing Exadata in action Karl Araos Blog
Pingback: Graphing the AAS with Perfsheet a la Enterprise Manager Karl Araos Blog
Pingback: Using Grid to display database CPU usage Oracle DBA A lifelong learning
experience
Arun says:
JANUARY 20, 2011 AT 8:58 PM
I am unable to download the script from the link. Can you provide me the script.
karlarao says:
JANUARY 27, 2011 AT 11:25 AM
I have already sent it on your mail :)
Kamus says:
MARCH 18, 2011 AT 10:51 AM
Hi Karl
The script mentioned in this post is no longer available, I knew you have moved all amazing
scripts into a new google docs page, but update the URL in this post would be be er.
karlarao says:
MARCH 18, 2011 AT 4:18 PM
Hi Kamus,
Yes, Ive had several emails regarding the access of the scripts. Ill probably move all of them
on my DropBox and share the entire folder.
27/09/2015 11:54
8 de 15
https://karlarao.wordpress.com/2010/01/31/workload-characterization-...
9 de 15
https://karlarao.wordpress.com/2010/01/31/workload-characterization-...
10 de 15
https://karlarao.wordpress.com/2010/01/31/workload-characterization-...
11 de 15
https://karlarao.wordpress.com/2010/01/31/workload-characterization-...
END
ELSE NULL
END physical_writes,
CASE
WHEN stat_name = redo writes THEN CASE
WHEN begin_interval_time = startup_time THEN VALUE
ELSE VALUE Lag(VALUE, 1) over(PARTITION BY stat_id, dbid, instance_number,
startup_time ORDER BY snap_id)
END
ELSE NULL
END redo_writes,
CASE
WHEN stat_name = redo size THEN CASE
WHEN begin_interval_time = startup_time THEN VALUE
ELSE VALUE Lag(VALUE, 1) over(PARTITION BY stat_id, dbid, instance_number,
startup_time ORDER BY snap_id)
END
ELSE NULL
END redo_size,
CASE
WHEN stat_name = physical read IO requests THEN CASE
WHEN begin_interval_time = startup_time THEN VALUE
ELSE VALUE Lag(VALUE, 1) over(PARTITION BY stat_id, dbid, instance_number,
startup_time ORDER BY snap_id)
END
ELSE NULL
END physical_read_io_requests,
CASE
WHEN stat_name = physical write IO requests THEN CASE
WHEN begin_interval_time = startup_time THEN VALUE
ELSE VALUE Lag(VALUE, 1) over(PARTITION BY stat_id, dbid, instance_number,
startup_time ORDER BY snap_id)
END
ELSE NULL
END physical_write_io_requests
FROM dba_hist_snapshot
natural join dba_hist_sysstat
WHERE stat_name IN ( logons current, execute count, physical reads, physical writes,
redo writes, redo size, physical read IO requests, physical write IO requests ))
GROUP BY dbid,
snap_id,
instance_number),
sub_osstat
AS (SELECT dbid,
snap_id,
instance_number,
MAX(load) load,
SUM(busy_time) busy_time,
SUM(user_time) user_time,
SUM(sys_time) sys_time,
SUM(io_wait_time) io_wait_time,
MAX(cpu) cpu,
27/09/2015 11:54
12 de 15
https://karlarao.wordpress.com/2010/01/31/workload-characterization-...
MAX(phy_mem) phy_mem,
SUM(avg_sys_time),
SUM(avg_busy_time),
SUM(avg_iowait_time),
SUM(avg_idle_time),
SUM(avg_user_time)
FROM (SELECT e.snap_id,
e.dbid,
e.instance_number,
CASE
WHEN e.stat_name = LOAD THEN e.VALUE
ELSE 0
END load,
CASE
WHEN e.stat_name = NUM_CPUS THEN e.VALUE
ELSE 0
END cpu,
CASE
WHEN e.stat_name = AVG_BUSY_TIME THEN CASE
WHEN s.begin_interval_time = s.startup_time THEN e.VALUE
ELSE e.VALUE Lag(e.VALUE) over(PARTITION BY e.stat_name, e.dbid, e.instance_number,
s.startup_time ORDER BY e.snap_id)
END
ELSE 0
END avg_busy_time,
CASE
WHEN e.stat_name = AVG_SYS_TIME THEN CASE
WHEN s.begin_interval_time = s.startup_time THEN e.VALUE
ELSE e.VALUE Lag(e.VALUE) over(PARTITION BY e.stat_name, e.dbid, e.instance_number,
s.startup_time ORDER BY e.snap_id)
END
ELSE 0
END avg_sys_time,
CASE
WHEN e.stat_name = AVG_USER_TIME THEN CASE
WHEN s.begin_interval_time = s.startup_time THEN e.VALUE
ELSE e.VALUE Lag(e.VALUE) over(PARTITION BY e.stat_name, e.dbid, e.instance_number,
s.startup_time ORDER BY e.snap_id)
END
ELSE 0
END avg_user_time,
CASE
WHEN e.stat_name = AVG_IOWAIT_TIME THEN CASE
WHEN s.begin_interval_time = s.startup_time THEN e.VALUE
ELSE e.VALUE Lag(e.VALUE) over(PARTITION BY e.stat_name, e.dbid, e.instance_number,
s.startup_time ORDER BY e.snap_id)
END
ELSE 0
END avg_iowait_time,
CASE
WHEN e.stat_name = AVG_IDLE_TIME THEN CASE
WHEN s.begin_interval_time = s.startup_time THEN e.VALUE
27/09/2015 11:54
13 de 15
https://karlarao.wordpress.com/2010/01/31/workload-characterization-...
14 de 15
https://karlarao.wordpress.com/2010/01/31/workload-characterization-...
https://karlarao.wordpress.com/2010/01/31/workload-characterization-...
Also Ive presented this topic at Hotsos and there are also presenters doing mining on AWR
and we are aware of dierent things to do it and kind of agree that it depends. So in my
case, the way I pull the data from various tables the snap_id + 1 approach is the winner.. and I
have evidence (test case) to prove it.
-Karl
Create a free website or blog at WordPress.com. | The Newsy Theme. Design by Themify.
Follow
15 de 15
27/09/2015 11:54