Monitoring Hardware, Database and R/3 Basis Chapter II R/3 Instance: logical unit which consist of R/3 Work

Processes that are administrated by a dispatcher process. It also has a set of R/3 buffers located in the computer-shared memory and accessed by the R/3 work processes. An application server is a computer with one or more R/3 instances. (P.27) Every R/3 system has only one Database Monitoring hardware, use transaction ST06 SWAP space: Should be 3 times as large as the physical main memory and at least 2GB Guideline for CPU and Paging:
Unused CPU should normally be at least 20% on average per hour. 35% is even better. Paging rate should not become too large. The paging rate is not critical if every hour 20% of the physical memory is being paged.

Hardware Bottleneck potential causes: - Non-optimal Workload Distribution (look at other computers in the system) - Individual Processes that consume too much CPU (look at ABAP Program using trace)
Use the OS Monitor ST06. If R/3 Work Processes are causing a high CPU load call the Local Work Process Overview SM50. If a database process is causing a high CPU consumption over a long period of time call the Database Monitor ST04

- Database processes causing High CPU load use ST04 and analyze SQL Statement External processes that consume too much CPU Use ST06 ( terminate or optimize use of these processes) Individual Programs that use too much Memory use ST06 and SM50 The File System Cache is too large this is an O/S file system cache located in the main memory where it competes for memory space with the R/3 System and the database. If too big, it will leave insufficient memory space for R/3 and the database thus causing high paging rate.

I/O on the hard disk The Hard Disk Monitor: located within the OS Monitor enables to monitor the I/O read/write on the hard disk, nut also, the wait time and response time on the hard disk. A heavy load on an individual disk is signified by a value greater than 50% in the column Util and it may indicate an I/O Bottleneck

Analyzing the database buffers (quality of): Physical Read Accesses: Number of read accesses to the hard disk and indicates how many blocks or pages must be loaded to satisfy the user queries being processed in an R/3 work process. Logical Read Accesses: This value is the total number of read accesses and indicated how many block or pages are read from the buffer and hard disk Buffer Quality Hit Ratio: (Logical access physical accesses) / logical accesses x 100

Expensive SQL Statements (P50 55): Thy cause a high CPU utilization percentage and a high I/O load. This can lead to an acute hardware bottleneck on the DB server and reduces performance of other programs. They Block R/3 Processes for a long time They read many data blocks into the data buffer or the database server

Expensive SQL Statements are indicated by a high number of logical read accesses (which place a high load on the CPU) and physical read accesses (which place a high load on the I/O system). I/O Bottlenecks: An individual disk has such a heavy load, a value greater than 50% is in the column Util Frequently accessed data files reside on these disks Wait situation occur when you access these files I/O bottlenecks can be resolves by improving the table distribution on the file system. In particular that the disk with a high load does not contain any more frequently accessed files that could be relocated. Exclusive Lock Waits: When a user blocks a row in a table through an SQL Statement like UPDATE or SELECT UPDATE. If another user tries to lock it he has to wait until the first one finishes.

Analyzing R/3 Buffers: If the buffers are too small the data is displaced and the DB has to be unnecessarily accessed to reload the buffer. The number of displacements is referred to as the number of SWAPS. There should be no SWAPS in the production system except for the program buffer where you can have 10000 per day., To avoid subsequent swaps ensure each buffer has sufficient memory. Invalidation: When a buffered object such as a program or table is declared invalid because it has been changed. Semaphores: Every application has resources that can only be used by only one work process at the time. If an R/3 work process wants to use these resources it sets a semaphore. If other processes also require this resource they have to wait until the process that is holding the process has completed the action. Long Database Response time (p.88): Look at expensive SQL Statements. Work Process is status complete: If you cannot restart the work process there may be a problem with the R/3 kernel or with logging on to the DB.
IF more than 20% of the work processes are in PRIV mode this indicates a memory management problem IF more than 40% of the work processes are performing a database action such as sequential read or commit this indicates a database problem If more than 20% of the work process are reading the same table at the same time there may be a problem with expensive SQL Statements.

Workload Analysis Chapter III Dispatcher Wait Time: The request spent in the dispatcher queue; indicated by Av. Wait time in the Workload Monitor User Context: Data such as Variables, internal Tables and report lists stored in the main memory of the Application server. Load Time: Time required for loading and generating ABAP programs and R/3 Screens that may not be in the buffers Database Time: Time required when data is read or changed in the DB. Defined as Av. DB request time and is measured from the moment of sending the database request to the database server and continues until the moment at which data is returned to the application server. It is measured by the application server, it includes also the time required by the network to transfer the data. P.110. Processing Time: Total Response time minus (Dispatcher wait time + Roll Time + Load Time + Enqueue Time + DB Request Time) Or Rt (Dwt + Rt + Lt + Et + DBRt). CPU Time: Measured by the O/S whenever a work process is occupying CPU. IT is not an additive component of Transaction Response Time but is consumed during Load Time, Roll Time and Processing Time. P.111 The Difference between Processing Time and CPU Time should not be greater than 100% (P. 114):
The problem may stem from a CPU Bottleneck where you may not have enough CPU Capacity available to the R/3 Work Processes, which must wait for the CPU to become available. Or, can be caused by Wait time in the R/3 work process.

Roll Wait Time: The time in witch a user context was not in a work process. Activity or Throughput: Number of transaction steps per unit of time (P.117) Load: Think of it as the time during which a task occupies the R/3 system and can be proportional to Response Time. Rule of Thumb to qualify which Response time is too large: Dispatcher Wait time > 50ms Database Time is greater than 40% of Response time Dispatcher Wait Time Database Time is > 400ms: A large DBt slows performance for all transactions Processing Time is > 2X CPUt: Maybe caused by CPU Bottleneck or communication between systems

 Avg. response time > 1s Hardware Bottleneck (P.124): If hourly average of free CPU is < 20% Paging rate per hour exceeds 20% of Physical Main Memory Look at processing time on the application server and see if the processing time is more than double the CPU time. This indicates that the work processes are waiting for CPU. Check on DB Server if there is a hardware Bottleneck Make sure the virtually allocated memory is smaller than 1.5times the physical main memory

Causes for Hardware Bottleneck (P.126) Poor load distribution Individual processes causing a high CPU load (expensive SQL Statements, Background processes or external processes) Insufficient Hardware Capacity

General Database Performance Problem: DBt much greater than 40% of Rt Wt Direct Reads > 2ms Sequential Reads > 10ms Change and Commits > 25ms

Performance Caused by R/3 Memory Configuration (P128) Increase of load time > 50ms indicates R/3 Program buffers, CUA Buffers or screen buffers are too small Increase of the avg. roll in or roll out > 20ms indicates that the R/3 extended memory or the R/3 buffer is full

Transaction Profile Which Transaction causes the greater DB load Which Transaction causes the greatest CPU Load IS there a transaction where the DB load or CPU time is greater than 60% of the total response time? Any Customer developed programs?

Performance Analysis for ABAP Programs Chapter IV Single Statistic Record: Statistical Record specific to each transaction steps in the R/3 System. It is recorded in files on the application servers and includes response tines, memory requirements, database accesses and so on. They are recorded hourly by the collector program RSCOLL00. Typical Single Statistic Record Problems: Large Database times Large Database times occurring only sporadically: usually due to tables being displaced or invalidated in the table buffer. Large CPU times: indicate complex calculation in the ABAP code or frequent accessing on the table buffer. Other Problems: such as loading processes from program buffers, R/3 lock administration (Enqueue) or RFC Calls

Direct Reads: the operation REOPEN transfers the concrete values in the WHERE clause to the database. The operation FETCH locates the database data and transfers it to the application server. The number of records transferred from the database by a FETCH is calculated as follows: Number of Records= Value of dbs/io_buf_size / bytes of one record Network Problem: Always perform a second and third trace of an expensive SQL Statement with a long runtime. If the response time is only slow at a particular point in time it may be a network problem or database access problem. The processing times of SQL Statements are measured on the application server and the runtime comprises not only the time required by the DB to furnish the data but also the time required to transfer the data between the database and application server. ABAP Trace: You do a trace when the runtime of the programs to be analyzed comprises mainly of CPU time (If programs require high CPU time). An ABAP trace measures not only the runtime of database accesses but also the time required by: Individual modularization Internal table operations Other ABAP statement

Configuring R/3 Memory Areas Chapter V All OS system combined cannot allocate more memory than the sum of Physical memory plus the available swap space Each OS cannot allocate more memory than the maximum addressable memory area (address space). This reflects the OS Architecture 32-bit or 64 bit.

Local Memory: assigned to one OS process Shared Memory: Available to multiple OS processes R/3 Buffers are in Shared Memory User Context: Data such as Variables, internal Tables and report lists stored in the main memory of the Application server. The initial part of User Context is stored in R/3 Roll Memory. R/3 Roll Memory ztta/roll_area The total size of the roll area in bytes R/3 Extended Memory

Em/initial_size_MB specifies the size of R/3 extended memory at instance start up Ztta/roll_extension specifies the maximum size of a single user context in R?3 extended memory

R/3 Heap Memory abap/heap_area_dia and abap/heap_area_nondia: Specify the maximum amount of R/3 heap memory that can be allocated by a dialog and non-dialog work process respectively abap/heap_area_total: How much R/3 heap memory can be allocated in the sum of all work processes. The maximum value is 2GB for a 32-bit kernel and should be at least greater than 600MB The Database instance requires 20% TO 30% of the total memory, which is the sum of the main memory of all servers. Available Memory: Physical Memory + Swap Space Address Space: Sum of all memory addresses that are available to a work process
Per work Process you need 7.5MB of Memory. For R/3 Extended Memory (p.207) you need 70% to 100% of the physical memory available. The largest Buffers are R/3 program buffers and are allocated per R/3 instance.

Workload Distribution Chapter VII ST03: Workload Monitor SM50: Work Process Overview Message Server: Service that enables messages to be exchanged between the various R/3 instances. There is only one per R/3 system. AN example of communication between R/3 instances managed by the Server is the distribution of update request between R/3 instances. Enqueue Service: Service that manages R/3 specific locks. It is called the Enqueue server. ATP Server: Only one for each R/3 System. Users, WPs and CPU Capacity: Number of Users / Number of Dialog WPs Number of WPs / Number of Processors

If CPUs are in full use adding more Work Processes will not improve performance but may even decrease it. R/3 Benchmark for Work Processes: Reference values for CPU needs of various R/3 Modules but: Do not reflect if there are more OLTP or OLAP. They assume you have more OLTP Does not reflect possible background load

Single Points of Failure: If The Message, Enqueue or ATP Service fails, there will be a general system failure for ht entire R/3 System. These services should be put centrally on one R/3 central instance Background Processing: Distributed amongst the application servers (Common mistake is to put it on the DB server) Update Work Process Distribution Spread it out over the application servers to avoid system failure in case the server goes down and it will be easier for load adjustments

Too many WPs on the DB server can cause CPU Bottlenecks.

Fundamentals of Table Buffering Chapter VII 3 Types Single Record Full Generic

Buffer Synchronization: Process whereby a change to an entry in a buffered table on one R/3 instance is updates in the table buffer of other R/3 instances. It depends if the ABAP statement has a WHERE clause or not. Buffer Prerequisites: So: Never buffer Transaction Data Master Data is normally not buffered Customizing Data is normally buffered Small Tales and Frequently Read Tables must not occur frequent write accesses Tables are not typically accesses using a secondary index Short term inconsistencies do not matter

Table that should not be Buffered: Have an Invalidation rate of over 1% (Total Rows affected divided By Total ABAP/IV Processors requests) A Table That Should be Buffered: IS Small and has relatively little changes (Under Column Total in ABAP/IV Processors requests.

Database Locks and Enqueue Chapter VIII Database Locks: Typically a row in a database table. TH SQL Statement COMMIT finalizes all changes than removes the corresponding database lock. Exclusive Lock Wait: When a work process tries to lock an object that already has a lock on it. Table Locks: If a work process places single row locks on more, for example, 10% of the single rows in a table, the locks are automatically replaced by a table lock. Table locks affect parallel processing in background jobs and will become serial processing. Occurs in Period Closing Programs. Avoiding Bottlenecks by Buffering Number Ranges (p.313): Buffering In Main Memory
Numbers are lost if R/3 is shut down Numbers are not in a numerical sequence

Buffering through an additional database table (NRIV_LOKAL) for each R/3 instance where name of instance is part of the primary key
When same docs are being created on several instances Numbers are not assigned in Numerical order Some numbers may not be assigned

The ATP Server: The following may reduce system performance during Availability Check: Locks: The material checked must be locked by an Enqueue. It may block other users who need to deal with the Material Read Accesses for Table RESB and VBBE. These tables are large and can cause high run time for the availability check

Optimizing SQL Statements Chapter IX Two Forms of Optimization (p.339): 1. The ABAP Code 2. The Database e.g. creating indexes Factors for evaluating SQL Statements: Number of logical read accesses for each execution (Gets/Execution) Number of logical read accesses for each transferred record

Missing Secondary Indexes: Can cause performance problems if the index belongs to table that is larger than around 1MB. Generating Table Access Statistics using DBA Planning calendar through DB13. Rules for creating Secondary indexes: 1. 2. 3. 4. 5. 6. Include only selective fields Include few fields in the index no more than 4 Position selective fields to the left of the index Exception to rule 1 & 3 Indexes should be Disjunct (largely the same fields) Create few indexes per field Also never change an index for tables larger than 10MB during business hours Check the Optimizer uses it the way you expected