Ovpi Master 51 BP v10

HP SGBU Services
Title Description File Owner

HP Software OVPI Project Deployment Best Practices
OVPI_Master_51_BP_v10.doc Michael P. Davies
Category Version Status
All 1.0 Final
HP Software Support
Authors Editor/ Primary Contributor Version Date Status
Multiple Michael Davies 1.0 10/31/2007 Final
Classification
HP SGBU Services
All 1.0 Final
HP Software Support
1 Document History
Version
0.05 0.1 0.2 1.0
Date
3/28/2007 9/28/2007 9/30/2007 10/31/2007
Revised by
Michael Davies Michael Davies Michael Davies Michael Davies P. P. P. P.
Description
Initial Draft Version Initial Version Minor Modifications to Format Final
2 Document Contributors
Contributor Clarke, Kevin Davies, Michael Davies, Michael Majumdar, Shamik Unknown HP Author Source Document OVPI_Best_Practices_v1.0_Guide.doc OVPI_PS_5.1_BP_1.pdf Base Document New Material System Performance analysis using Glance.ppt
OVPI Port Numbers.htm
HP SGBU Services
All 1.0 final
HP Software Support
Contents
1 2 3 Document History.................................................................................................................................2 Document Contributors .........................................................................................................................2 Introduction .........................................................................................................................................7 3.1 3.2 3.3 3.4 Document Overview ..................................................................................................................7 Intended Audience ....................................................................................................................7 Document Layout.......................................................................................................................7 Outline of Document Topics including Intended audience...............................................................7
OVPI Theory........................................................................................................................................9 4.1 4.2 4.3 Executive Summary....................................................................................................................9 Technical Summary of Limiting Factors ........................................................................................9 Performance and System Utilization...........................................................................................10
Pre Installation ...................................................................................................................................11 5.1 5.2 Executive Summary..................................................................................................................11 Project Scheduling...................................................................................................................11 5.2.1 5.2.2 Project Milestones and tasks .............................................................................................11 OVPI Project Schedule Case study .....................................................................................13
Installation.........................................................................................................................................17 6.1 6.2 Executive Summary..................................................................................................................17 Documentation........................................................................................................................17
Oracle Database installations ..............................................................................................................17 7.1 Dsi_Dpipe User privileges ........................................................................................................17 7.1.1 7.2 Production Dsi_Dpipe user privileges .................................................................................18
Database configurations ..........................................................................................................18
OVPI Installation ................................................................................................................................23
HP SGBU Services
All 1.0 final
HP Software Support
8.1 9
Installation Process ..................................................................................................................23
Installing OVPI on Oracle....................................................................................................................23 9.1 Installation Wizard..................................................................................................................24 RNS Installation ..........................................................................................................................30 10.1 10.2 10.3 10.4 Install RNS packages ...........................................................................................................30 Start the Package Manger GUI..............................................................................................32 Using the Package Manager GUI to install Report Packs ..........................................................33 Installation Notes.................................................................................................................38 10.4.1 Downloading the Service Pack..........................................................................................39 10.4.2 Performing Prerequisite Tasks............................................................................................39 10.4.3 Installing the Service Pack ................................................................................................41
10
11 11.1
Distributed Architecture Configurations ..........................................................................................42 Poller Configuration .............................................................................................................44 11.1.1 Areas of Configuration ....................................................................................................44 11.1.2 Collection Parameters ......................................................................................................44 11.2 Satellite Server Configuration ................................................................................................47 11.2.1 Process Serialization and load balancing...........................................................................47 11.2.2 Process Scheduling ..........................................................................................................50
12
Central Server Configurations ......................................................................................................51 12.1.1 Process Serialization........................................................................................................51 12.1.2 Process Scheduling ..........................................................................................................55
13 13.1
Standalone OVPI Server Configurations ........................................................................................56 Polling configuration ............................................................................................................56 13.1.1 Areas of Configuration ....................................................................................................56 13.1.2 Collection Parameters ......................................................................................................56 13.2 13.3 Process Serialization ............................................................................................................57 Sample Trendtimer.sched......................................................................................................60 Interface Reporting Configurations ................................................................................................62 4
14
HP SGBU Services
All 1.0 final
HP Software Support
14.1 14.2 15 15.1 15.2 16 16.1 16.2 16.3
Pre Configuration Steps. .......................................................................................................62 Configuring Directed Instance Polling.....................................................................................63 Additional Post Installation Steps...................................................................................................66 Periodically Restarting the Application Server..........................................................................66 Rebuilding Table Indexes on a Central or Standalone Server ....................................................66 Ongoing Maintenance ................................................................................................................67 Executive Summary ..............................................................................................................67 Change Controls and Documentation.....................................................................................67 SNMP Node Management ...................................................................................................67 16.3.1 SNMP Discoveries...........................................................................................................68 16.3.2 NNM Node Synchronizations ..........................................................................................68 16.3.3 Adding New Managed Nodes .........................................................................................68
16.4
Database Backup and Recovery ............................................................................................71 16.4.1 Hot Vrs Cold Database Backups .......................................................................................71 16.4.2 RMAN Oracle Database Recovery Manager ......................................................................71 16.4.3 Backup Procedure ...........................................................................................................72
16.5
System Monitoring ...............................................................................................................72 16.5.1 Maintenance Procedures ..................................................................................................72 16.5.2 Process Completion Monitoring.........................................................................................73 16.5.3 Tacking system processing times .......................................................................................73
16.6
Expanding the Environment...................................................................................................74 16.6.1 Standalone Server has reached capacity............................................................................74 16.6.2 Distributed Environment has reached its maximum capacity.................................................75
17 17.1 17.2 17.3 17.4 17.5 17.6
Appendix A: Sybase to Oracle Syntax Conversions .......................................................................76 TERMINOLOGY / EQUIVALENCES .......................................................................................76 ENVIRONMENT VARIABLES .................................................................................................78 UNIX COMMANDS.............................................................................................................78 Datatypes ...........................................................................................................................79 SQL*PLUS ..........................................................................................................................80 sqlplus commands ...............................................................................................................81 5
HP SGBU Services
All 1.0 final
HP Software Support
17.7
ADMINISTRATION ..............................................................................................................84 17.7.1 HOW TO STOP ORACLE.................................................................................................85 17.7.2 STARTUP COMMANDS ...................................................................................................85 17.7.3 DEFAULT LOGINS ...........................................................................................................87
17.8 18 19
Frequently Asked Questions ..................................................................................................87 Appendix B: OVPI Default Port Numbers .......................................................................................88 Appendix C: Advanced Baseline Procedures..................................................................................90
HP SGBU Services
All 1.0 final
HP Software Support
3 Introduction
3.1 Document Overview
The intention of this document is to provide OVPI deployment and maintenance advice that gives a clear understanding of the resources, processes, and hardware requirements needed to manage a successful OVPI project. This document is intended to be a living document and ever changing; it contains the most up-to-date advice on deploying and managing OVPI environments.
3.2 Intended Audience

This document is intended to be read by all levels of the deployment project team.
3.3 Document Layout

This document contains vital information that is separated into sections based on its intended audience. The audience of the sections is outlined as follows: Senor Management or Purchasing Officer Executive Summary; which contains information on hardware and personnel requirements. Project Manager Technical Summary; which includes an overview of the tasks, processes and personnel required to deploy and maintain OVPI during its program life cycle. OVPI Administrator or Deployment Specialist Technical detail; which includes detailed process information for installing and maintaining OVPI.
3.4 Outline of Document Topics including Intended audience
HP SGBU Services
All 1.0 final
HP Software Support
HP SGBU Services
All 1.0 final
HP Software Support
4 OVPI Theory
4.1 Executive Summary
OVPI is a complex solution. It is difficult to install and maintain, and if the system is not installed and maintained properly, it will fail to meet expectations. It is important to note that OVPI is a data mining application, and due to this, it has a large system footprint. OVPI should not be installed on a system that contains any other high system utilization software. In addition, OVPI is I/O process intensive, due to constant database accesses. It must be installed on a system with multiple disks or a SAN. First, it is best to design an architecture that processes less than the maximum advertised OVPI processing loads. The best way to think of this is: If you have a car, you would never drive it around at redline all day. You know you would destroy the engine. OVPI reacts in the same way to excess utilization. If you process too much data, the system will breakdown constantly. As a safeguard, you should deploy a system that is designed to run at 70% of capacity. For example, if you know OVPI can collect a maximum of 30,000 interfaces on a single server, you should deploy the system in such a way that it will be collecting only 70% of the maximum load or 21,000 interfaces. This will allow for additional processing head room should a failure occur and increases the projected reliability. NOTE - The bottom line with OVPI is if it is over utilized, it will not perform up to expectations. However to that note, if it is over utilized, it will not work properly. Do not attempt to stretch the capabilities outlined in this document, or your installation will not be successful.
4.2 Technical Summary of Limiting Factors

It is important for us to note at this time that there are limiting factors that must be taken into account when setting up a large OVPI installation. I have listed some of these factors here: 1) Data table size The most limiting factor for OVPI is data-table size. When configured properly, OVPI can scan and manage rate tables of about 8 to 10 million rows and summarization tables that are between 10 and 12 million. There are three factors that affect the size of the data tables: a) Data retention For a 20,000 IR element 5-minute collection, the data table size is about 11 million rows with a retention time of 1 day. If the data is kept for an additional day, the data size doubles to 22 million rows, which is far above a manageable size.
HP SGBU Services
All 1.0 final
HP Software Support
b)
Polling interval For a 40,000 IR Element 15-minute collection, the data size is about 7.6 million rows. However, if we increase this collection to 5-minute intervals, we are now collecting 3 times the amount of data, which results in a 21 million row table, which far exceeds the manageable size. Number of elements For each 1 element of IR data collected during a polling period, you get one row of data. Thus, each additional element adds additional rows in the data tables.
c)
2) Disk Configurations Because OVPI is a data mining application, it is very I/O intensive. In order to ensure optimal throughput, it is recommended that you utilize a SAN or local raid array. If possible, it is best to split the segments of the database across at least 3 physical devices by placing the transaction segment, the temp segment and the data segments on their own drive. 3) Number of CPUs OVPI has a number of different core processes, and because of this, it will utilize multiple CPUs. It is recommended that an OVPI server have a minimum of 2 CPUs. 4) Number of report packs installed - It is important to understand that there is a limited amount of processing cycles on any system and each additional report pack will take away from those processing cycles. HP support recommends that no more than five report packs are run on any given installation. 5) Type of polling used OVPI utilizes multiple polling methods to collect data. Of these, SNMP is the most optimized, allowing the customer to collect roughly 30,000 elements on a given system. However, when using other collections, such as pa_collect, the collections will be much more limited. (Please contact an OVPI solutions architect for the polling limitations of other collection methods than SNMP.)
4.3 Performance and System Utilization

When deploying OVPI, it is crucial that the project team understand that OVPI must be installed in such a way that it does not become over utilized. If an OVPI system is over utilized, it will limit the system availability. In order to understand OVPI system performance, it needs to be understood that all processing is interdependent. Data must be collected, then summarized, then deleted. Every step must be completed in its entirety before the next process can occur. Taking this into account, process scheduling is extremely important. The administrator must know exactly how long each process is taking and needs to understand that each process is taking available processing cycles from the system. If too much multi-processing is being performed, it will slow down the installation. And to that fact if too little multi-processing is being performed, the system will be underutilized and will not be returning the value expected. Finally, detailed information on process management and scheduling will be included in the installation portion of this guide.
10
HP SGBU Services
All 1.0 final
HP Software Support
5 Pre Installation
As stated in the first section of this document, OVPI is a complex solution. While there are some out-of-the-box capabilities, customers often feel that they would like to expand on them by adding additional reports and collections. To this effect, the customer needs to understand that they must have this in mind when they size and install the system. OVPI scales easily, as long as the client has the correct hardware and architecture to perform the tasks. WARNING Do not stage your deployment by investing in minimal hardware and then plan on growing it. From the start, you must have a good understanding of your final goals, or you could end up having to buy all new hardware and rebuilding your solution. Realize that if you are planning on growing from one standalone server to a distributed environment, invest in the hardware to make the standalone server into the central server when the time comes.
5.2 Project Scheduling

When looking at deploying OVPI, it should be understood that there are a series of milestones that must be outlined and met to ensure a working installation. In order to understand OVPI project management, this document will present a case study to give a solid understanding of a deployment. However, before the case is presented, we must first understand the milestones and their respective tasks. 5.2.1 Project Milestones and tasks
1) Pre-installation Planning Before starting to deploy an OVPI environment, every requirement must be laid out and planed for. The number of collections, summarizations, and reports must be accurately estimated in order to decide on the correct architecture. Planning Meetings A) The brainstorming meeting - Start by having a meeting of set length with an OVPI solutions architect, your management and your end-users, if possible. Brainstorm on the reporting requirements. Gather the information from this meeting and allow the OVPI solutions architect to match those requirements to the correct hardware and licensing structure. (Warning Do not attempt to design a solution without the assistance of an OVPI solutions Architect. While you may feel that you have plenty of technical experts on your team, they may not have the experience with OVPI to make the correct architecture decisions) B) The meeting of the decision-makers (Looking at cost vs requirements) After the solution architect has had a chance to design a solution to meet the requirements set forth in the brainstorming meeting, have a meeting to take a look at what requirements present the best ROI. You may find that some of the requirements require too much time to build or too large of an initial investment in hardware. Refine the list in this meeting and decide to attempt only to deploy on the requirements that make fiscal sense.
11
HP SGBU Services
All 1.0 final
HP Software Support
C) The Presentation (Presenting the solution to the end-users) - start by setting end-user expectations when deploying the accepted solution. After having the brainstorming and the decision-making meeting, present the final solution to everyone that was in the fist brainstorming meeting. Make sure to outline the deliverables that were decided on in the first meeting and also outline those deliverables that will not be included in the solution because they were not cost effective. (Warning make sure the end-users accept the final solution. This is your last chance to perform proper sizing. If changes are made later in the project, the system may not meet expectations.) 2) Hardware Start by ordering the hardware required to meet the final solution. Do not attempt to run the OVPI installation on hardware that does not meet the requirements set forth in the OVPI sizing sheets. (Note If you invest in the right hardware, you will be rewarded with a more stable solution) 3) Database Installations The first step in installing OVPI is installing the databases. Allow adequate time for each database installation. Employ a certified DBA to perform your installations for you. You will find that they may have valuable input into the configuration of the database installation on the systems. Also, if you are deploying on a SAN, make sure that you have made the SAN administrator aware of the fact that you are deploying a data mining application on the SAN. (Please note: Normal database installation will take about a day per OVPI installation.)
4)
OVPI Core Product Installation The core product installation should be installed by an OVPI expert. However, if installing multiple systems, you can have the OVPI expert install and document the first install and have a competent system administrator perform the remaining installations using the directions created by the OVPI expert. (Note: You should allow time in your project plan for documenting the install). Normally, a core product install will take about a half day to perform.
5) OVPI Package Installation OVPI Package Installation should be performed by a certified OVPI expert. OVPI Reporting Package installation is the most critical part of the installation process. If mistakes are made during the installation, the core product may need to be redeployed. (WARNING: Do not attempt to allow a novice to install the reporting solutions). (Depending on the amount of report packs being installed, it should take about half a day to install the package.) 6) Post Installation Configuration steps After installing the core product and the reporting solutions, extensive configurations will need to be made to ensure that the installation functions properly. These tasks include process scheduling and collections configurations. Normally, we would allot 3 man-days to each system to be spread over two weeks. The first day will include the configuration of scheduling, and the other two days will be utilized for collection configurations and tuning.
12
HP SGBU Services
All 1.0 final
HP Software Support
5.2.2
OVPI Project Schedule Case study
To start with, this case study is only meant as a starting place. It represents a simple project that does not focus on custom reporting solutions. Case Acme Telco has decided that they would like to monitor roughly 120,000 Interfaces on their customer network. They have spoken with HP and have decided that they will be happy using the OEM release of the Interface Reporting Report Pack and the Device Resources Cisco Solution. All databases will be installed on the local disks. After sizing, HP has recommended the following architecture:
Satellite Satellite1 Satellite2 Satellite3 Satellite4 Satellite5 Total Nodes
# Managed Elements 30000 30000 30000 30000 30000 150000
5.2.2.1
Architecture Decision Process
The first thing that someone may notice is the above architecture supports 150,000 elements. When the OVPI documents say that each OVPI server is capable of collecting a maximum of 30,000 elements, they are referring to this as a maximum load. You would never run a car at maximum speed, so the same care needs to be taken with OVPI. The customer has only requested a system that could handle 120,000 elements. While the system technically could run at full speed on 4 servers, it would leave little, if any, room for recovering from down databases; thus, the addition of an additional satellite server. It cannot be stressed enough that if you want a stable solution, you cannot push the capabilities of the product.
5.2.2.2
Project Schedule
The following tasks must be completed prior to starting the OVPI install: 13
HP SGBU Services
All 1.0 final
HP Software Support
1) All hardware must be installed. 2) The OS with all patches should be loaded on the hardware. 3) The hardware must be connected to the network, and all ACLs must be configured for the servers to reach the managed nodes.
5.2.2.2.1 The following is a project overview for the proposed architecture:

Day one
1) Install Database, OVPI Core product, reporting solutions on the Central server. Day Two 1) Install Database, OVPI Core product, reporting solutions on the Satellite1. Day Three 1) Install Database, OVPI Core product, reporting solutions on the Satellite 2 and 3. (Note: That by the time the 3rd install is performed, the bugs should be worked out, and the installs should go in quicker.) Day Four 1) Install Database, OVPI Core product, reporting solutions on the Satellite 2 and 3. Day Five (Assuming it is a Friday) 1) 2) 3) Configure Distributed Processing. Upload all property data to the systems. Import Node lists (Start by importing about a quarter of the nodes. The rest will be loaded over the next week). DO NOT START POLLING ON A FRIDAY!!!! (Unless you are willing to watch the system all weekend)
4)
Day Six 1) Perform an initial backup of the systems.
14
HP SGBU Services
All 1.0 final
HP Software Support
2) 3)
Start all scheduled tasks (OVPI timers should be started at this point). Start documenting the installations performed; gather information on software configurations, polling lists, property data and user information. (This information will be used if a system failure occurs and the system needs to be rebuilt.)
Day Seven 1) Perform a system audit; record the time of collections and summarization processes. At this point only a quarter of the nodes have been added, so if you are seeing processing issues, you will need to address them before moving on by performing additional system tuning. Record and address any system failures, if they exist. Review reports and make sure that the expected data is being populated. At this point you should expect to see historical daily reports and NRT reports. Forecasting and exception reporting may or may not be displaying data, depending on if exceptions were met or not. Add another quarter of the nodes.
2) 3)
4)
Day Eight (Same as day seven) 1) Perform a system audit; record the time of collections and summarization processes. At this point only half of the nodes have been added, so if you are seeing processing issues, you will need to address them before moving on by performing additional system tuning. Record and address any system failures, if they exist. Review reports and make sure that the expected data is being populated. At this point you should expect to see historical daily reports and NRT reports. Forecasting and exception reporting may or may not be displaying data, depending on if exceptions were met or not. Add another quarter of the nodes.
2) 3)
4)
Day Nine(Same as day seven and eight) 1) Perform a system audit; record the time of collections and summarization processes. At this point of the nodes have been added, so if you are seeing processing issues, you will need to address them before moving on by performing additional system tuning.
15
HP SGBU Services
All 1.0 final
HP Software Support
2) 3)
Record and address any system failures, if they exist. Review reports and make sure that the expected data is being populated. At this point you should expect to see historical daily reports and NRT reports. Forecasting and exception reporting may or may not be displaying data, depending on if exceptions were met or not. Add another quarter of the nodes.
4)
Day 10 (Final Configurations and Tuning) 1) Perform a system audit; record the time of collections and summarization processes. At this point all nodes have been loaded. You should have fairly consistent collection times. While the larger data tables will not grow to their full size for many months, the more heavily utilized tables, such as rate and hour tables, should be close to their full size. Record and address any system failures, if they exist. Review reports and make sure that the expected data is being populated. At this point you should expect to see historical daily reports and NRT reports. Forecasting and exception reporting may or may not be displaying data, depending on if exceptions were met or not. Perform final documentation and backup polling lists one last time.
2) 3)
4)
Day 11 and 12 1) Release the system to the end-users for them to check their reports. They may be more familiar with the managed nodes and their properties than the OVPI administrator is. The end-users should at this time make the OVPI admin aware of any missing nodes or improperly configured properties. Address any complaints received. Release the installation into production.
2) 3)
16
HP SGBU Services
All 1.0 final
HP Software Support
6 Installation
At this point the pre-system planning should be done and project schedule should be ready. Its now time to release the team to perform the install. The only problem is: Who is the team? The team should contain at least one OVPI certified professional. It is never recommended that a large OVPI install be performed by an OVPI novice. The lead installer should have at least attended OVPI admin training and should have experience in installing production OVPI environments. It is possible to utilize a qualified system admin for cloning tasks and as an apprentice. In addition to the OVPI expert, your team should include the following: 1) 2) A qualified system admin to configure the OS and systems. A qualified DBA, if using Oracle. HP does not provide direct Oracle support. And, therefore, it is advised that the team include a DBA. A project manager to ensure that deadlines are met.
3)
6.2 Documentation
During the installation, the lead installer should take time to document every step in the installation, so that it can be cloned by others. Each configuration step should be recorded; including directory locations, database configurations, and environment variables. The goal of the documentation should be that if a qualified system admin should pick up the guide, he should be able to follow the steps to recreate the installation, if necessary.
7 Oracle Database installations

7.1 Dsi_Dpipe User privileges
During the initial install and any additional service pack or OVPI binary upgrade, it is a requirement to have the DBA privilege for dsi_dpipe. The reason for this is that dsi_dpipe will attempt to change its own privileges, and the installation could fail.
17
HP SGBU Services
All 1.0 final
HP Software Support
7.1.1
Production Dsi_Dpipe user privileges
The following will produce dsi_dpipe privileges that will allow OVPI user to function in production without DBA privileges being given to dsi_dpipe:
GRANT "CONNECT" TO DSI_DPIPE GRANT "RESOURCE" TO DSI_DPIPE / ALTER USER DSI_DPIPE DEFAULT ROLE "CONNECT" / GRANT ALTER SESSION TO DSI_DPIPE / GRANT CREATE PROCEDURE TO DSI_DPIPE / GRANT CREATE SEQUENCE TO DSI_DPIPE / GRANT CREATE TABLE TO DSI_DPIPE / GRANT CREATE TRIGGER TO DSI_DPIPE / GRANT CREATE VIEW TO DSI_DPIPE / GRANT SELECT ANY DICTIONARY TO DSI_DPIPE / GRANT SELECT ANY TABLE TO DSI_DPIPE / GRANT UNLIMITED TABLESPACE TO DSI_DPIPE / GRANT EXECUTE ON SYS.DBMS_LOCK TO DSI_DPIPE /
7.2 Database configurations

Database configurations should be performed by a qualified Oracle DBA. In addition, the following information can be utilized to help make configuration decisions. This information is sourced from the OVPI Best Practices Document written by Kevin Clarke, HP Technical Consultant. (Some formatting has been edited, due to incompatibilities in document types and removal of duplicate data between sources.)
Source: MSB_OVPI_Best_Practices_V10_Guide.pdf
3. Oracle Configuration
3.1. Partnerships with Oracle DBAs
If your organization has an Oracle DBA group, spend time to establish a working relationship with the DBAs that make up this group. Understand the following:
18
HP SGBU Services
All 1.0 final
HP Software Support
you will likely have to convince the DBAs about specific requests. - Set clear expectations on response times to requests for help. - Understand your recovery options and scenarios. - Clearly communidate that OVPI does not like to run out of space. It is critical that the Oracle DBA group respond to space increase needs of OVPI in a timely fashion. A good relationship with your DBA team will help you be successful and help you to avoid frustration.
- What will they allow? Can you have DBA privileges? Unlimited tablespace? - What can they help with? - Communicate what OVPI needs? - Take input from them. Remember the following section is a Guideline, and
3.2. Software
Software for Oracle 9i can be downloaded using the following links: The recommended patch level is Oracle 9.2.0.7. HP-UX (64-bit) Software File Name Oracle 9i Enterprise Oracle9i-9.2.0.1.0-64.1of4.tar.gz Oracle9i-9.2.0.1.0-64.2of4.tar.gz Oracle9i-9.2.0.1.0-64.3of4.tar.gz Oracle9i-9.2.0.1.0-64.4of4.tar.gz Oracle 9.2.0.7 Patch Set p4163445_9207_HP64.zip Oracle Security Patch Set p4193295_9206_HP64.zip Windows (32-bit) Software File Name Oracle 9i Enterprise 92010NT_Disk1.zip 92010NT_Disk2.zip 92010NT_Disk3.zip Oracle 9.2.07 Patch Set p4163445_92070_WINNT.zip Oracle Security Patch Set p4269928_9206_WINNT.zip Oracle software can also be downloaded from the following locations: - ftp://dimecvsapppro01.americas.cpqcorp.net/ORACLE/Oracle9i-Server/ - http://www.oracle.com/support/premier/global-support-resolution/metalink.html
3.2.1. 32-bit vs. 64-bit Software

Be sure to use the proper software that supports the chipset architecture of the server. Do not use 64-bit software versions on 32-bit hardware platforms.
19
HP SGBU Services
All 1.0 final
HP Software Support
3.3. Install Oracle Client on non-Data Base Servers

Installing the Oracle client software on all OVPI servers provides a way to test the data base connections and also allows for the use of other Oracle software components, if needed, such as SQL*Plus.
3.4. Setup Default Environment Variables

Configure a local environment variable to provide a default connection string for Oracle logins. This is not set up by default during the Oracle installation. The variable names are different for Unix and Windows.
TWO_TASK (Unix) LOCAL (Windows)
3.5. System Global Area (SGA) Allocation

The following distribution of memory can be used as a starting point for large OVPI instances. Changes may be necessary once data is loaded and normal processing has commenced, but these are reasonable starting points. Recommended Parameter Value Comments Shared Pool 1,024 MB db_cache_size (Buffer Cache) 1,024 MB Specifies the size of the cache for standard block size buffers. large_pool_size 256 MB Specifies the size of the large pool allocation heap, which is used by Shared Server for session memory, parallel execution for message buffers, and RMAN backup and recovery for disk I/O buffers. java_pool_size 256 MB Specifies the size, in bytes, of the Java pool memory, which stores shared in-memory representations of Java methods and class definitions, as well as Java objects migrated to the Java session space at end-of-call.
20
HP SGBU Services
All 1.0 final
HP Software Support
pga_aggregate_target
512 MB
sga_max_size
4,096 MB
Specifies the target aggregate PGA memories of all server processes attached to the instance. Set this parameter to a positive value before enabling the automatic setting of working areas. This memory does not reside in SGA. The database uses this parameter as a target amount of PGA memory it uses. When setting this parameter, subtract the SGA from the total memory on the system available to the Oracle instance. The remaining memory can be assigned to the pga_aggregate_target. Specifies the maximum size of the System Global Area for the lifetime of the instance.
INIT.ORA Parameters
The following distribution of memory can be used as a starting point for large OVPI instances. Changes may be necessary once data is loaded and normal processing has commenced, but these are reasonable starting points. Parameter open_cursors Recommended Value 2,000 Comments Specifies the maximum number of open cursors (context areas) a session can have at once, and constrains the PL/SQL cursor cache size which PL/SQL uses to avoid reparsing statements re-executed by a user. Set this value high enough to prevent your applications from running out of open cursors. The maximum number of table locks obtained by all users. A DML lock is required for each table undergoing a DML operation. For example, three users modifying two tables require a value of six.
dml_locks
500
21
HP SGBU Services
All 1.0 final
HP Software Support
processes
200
Sessions
225
Specifies the maximum number of operating system user processes that can simultaneously connect to an Oracle Server. This value should allow for all background processes such as Job Queue (SNP) and parallel execution (Pnnn) processes. Specifies the total number of user and system sessions. The default number is greater than PROCESSES to allow for recursive sessions.
3.9.4. Physical Distribution for Local Disk Systems

Use SAN whenever possible. Otherwise use either six, four or two drive configurations, or worst case, a single disk. Performance will degrade as fewer disks are used to distribute the data. For example, in a case where you have two controllers, and four disks, the following strategy helps to distribute the I/O load between controllers and physical disks. It attempts (as best as possible), to keep data and indexes on separate disks and tries to force as many disks as possible to be active during summarization (two upload disk, two rate disks, and then two rate disks, two summary disks). OVPI Table Space Distribution Tablespace Name Controller Disk Initial Size (MB) DPIPE_DEFAULT_SEG 2 2 2,000 DPIPE_OVERFLOW_SEG 1 1 8,000 DPIPE_PROPERTY_IND_SEG 1 1 2,000 DPIPE_PROPERTY_SEG 2 2 2,000 DPIPE_RATE_IND_SEG 1 3 4,000 DPIPE_RATE_SEG 2 4 4,000 DPIPE_SUMMARY_IND_SEG 1 1 4,000 DPIPE_SUMMARY_SEG 2 2 4,000 DPIPE_UPLOAD_IND_SEG 1 3 2,000 DPIPE_UPLOAD_SEG 2 4 2,000
3.9.5. Physical Distribution for SAN Disk Systems

If SAN is used, a single LUN can be allocated to hold all files, and I/O distribution strategy decisions can be made by the SAN software.
22
HP SGBU Services
All 1.0 final
HP Software Support
8 OVPI Installation
Root Privileges SUDO WILL NOT WORK FOR INSTALLATIONS!!!!

The bottom line is that SUDO commands do not perform the installation tasks correctly. They cause additional delays for hunting down installation failures. It is best, if at all possible, to use a root user account to perform the installation tasks for OVPI, Service Packs and RNS installations.
8.1 Installation Process

OVPI consist of multiple components, including the database, binaries and reporting solutions. The portions must be installed in the order described below for the installation to complete without errors. All patches for the binaries and reporting solutions can be found at http://support.hp.com. Installation should be performed in the following order: 1) 2) 3) 4) 5) 6) Install and configure Oracle. Install the OVPI binaries. Install the OVPI Service Packs. Install the Reporting Solutions. Install any available RNS patches. Configure the polling and processing of the report packs.
9 Installing OVPI on Oracle

This section describes how to install HP OpenView Performance Insight (OVPI) in a standalone configuration on an Oracle database using the setup program. Configuration of the server to be a Satellite or Central server is done after the software has been installed. The installation is done through an installation wizard.
23
HP SGBU Services
All 1.0 final
HP Software Support
Your system must meet the prerequisites in Chapter 2, Preparing for Installation of the HP OpenView Performance Insight Install Guide. If more detailed information is needed, please read the HP OpenView Performance Insight Release Notes and the HP OpenView Performance Insight Install Guide.
9.1 Installation Wizard

The OVPI setup program runs the installation wizard. This section describes how to start the setup program. This involves supplying information to the installation wizard after the setup program has been initiated. Task 1: Before starting the OVPI setup wizard, ensure that the Oracle listener and database are running. Task 2: Start the setup program: Log on to the local system as root. Insert and mount the installation CD in the CD-ROM drive or extract the OVPI 5.1 tarball to the temporary directory. Locate the setup program on the CD or in the temporary directory that the tarball was extracted into, and start it by typing the following command:
./setup
The Setup window opens.
Figure 1 Setup Window
24
HP SGBU Services
All 1.0 final
HP Software Support
Select OVPI 5.1.0 from the Setup window
Task 3: Acknowledge the OVPI Welcome screen.
Figure 2 Welcome Window

Read the Welcome window. Click Next to continue the installation.
Task 4: Accept the license agreement.
25
HP SGBU Services
All 1.0 final
HP Software Support
Figure 3 License Agreement Window
To accept the license agreement and continue installing OVPI, click the I accept the terms in the license agreement option, and then click Next.
26
HP SGBU Services
All 1.0 final
HP Software Support
27
HP SGBU Services
All 1.0 final
HP Software Support
Task 5: Select the OVPI components you want to install.
Figure 4 Component Selection Window
Make sure that all components are checked; all components are required for a standalone installation and then click Next.
28
HP SGBU Services
All 1.0 final
HP Software Support
Figure 5 Installation Progress window

Do one of the following: If the OVPI installation was successful, click Done. If the OVPI installation was unsuccessful, click the Details button at the bottom of the Installation. Progress window for additional information, and then contact Support.
After the installation is complete, run the following commands as root to fix installation permission problems: chown trendadm:trendadm $DPIPE_HOME/odbc/ovpi_odbc.ini chown trendadm:trendadm $DPIPE_HOME/data/systems.xml chmod 644 /etc/trend.conf
29
HP SGBU Services
All 1.0 final
HP Software Support
Post installation Steps Verify the environment and make sure the variables are setup. For Unix run the following command: # env * This should return the correct entries for: DPIPIE_TMP DPIPE_LOG DPIPE_HOME ORACLE_SID
10 RNS Installation
Remember to install only the needed report packs, as additional packages take away from system performance and can lead to confusion on the part of the administrators when updates are being made.
10.1 Install RNS packages

Start the setup program: Log on to the local system as root (SUDO may not be used to perform this installation). Stop the trendtimer process if it is running: o /etc/init.d/ovpi_timer stop Setup your DISPLAY environment variable as some of this installation requires a GUI * #setenv DISPLAY IP:DISPLAY Insert and mount the RNS 7.0 installation CD in the CD-ROM drive Locate the setup program on the CD and start it by typing the following command:
./setup
The following message will appear:
30
HP SGBU Services
All 1.0 final
HP Software Support
WARNING: This installation may overwrite previously installed packages with the same directory/filenames. Before continuing with this installation, please be sure you have taken backups of any changes or customizations you may have made.
Do you want to continue with the installation(Y/N)?__
Type Y and hit Enter to continue.
The following message will appear:
--------------------------------------------------------| Reporting and Network Solutions 7.0 Main Menu
--------------------------------------------------------| 0. Exit |
| 1. Install OV Performance Insight Report Packs | 2. Install OV NNM Value-add Components | 3. Install OV NNM Smart Plug-ins (SPIs)
31
HP SGBU Services
All 1.0 final
HP Software Support
--------------------------------------------------------Please enter your choice:
Type 1 and hit Enter to continue This will extract the RNS packages to the default location of $DPIPE_HOME/packages The Package Manager GUI will automatically start. If you wish to continue with the installation of the Report Pack, skip to the Using the Package Manager GUI to install Report Packs in the Install Report Packs section. Or choose Exit to exit the Package Manager GUI. After you have completed using the Package Manager GUI and are returned to the menu. Exit the menu
Install Report Packs
10.2 Start the Package Manger GUI

To start Package Manager, do one of the following:
Using the Management Console Log on as trendadm. Type the following at the command line: piadmin The Management Console window opens. From the Tools menu, choose Package Manager. The Package Manager Welcome window opens.
Command Line Log on as trendadm. Type the following at the command line:
32
HP SGBU Services
All 1.0 final
HP Software Support
packagemgr The Package Manager Welcome window opens.
10.3 Using the Package Manager GUI to install Report Packs

Once you have started package manager either from the Management Console or the command line:
Figure 6 Welcome Window
Choose Next to continue
33
HP SGBU Services
All 1.0 final
HP Software Support
Figure 7 OVPI Packages Location Window
Do the following: Select Install, and then enter the directory in which you installed the package or packages. This should be the packages folder of the directory into which you installed OVPI. For example: /usr/OVPI/packages
Click Next.
34
HP SGBU Services
All 1.0 final
HP Software Support
Figure 8 OVPI Report Deployment Window
(DO NOT Select the Deploy Reports check box to deploy the reports to the Web Access Server if you are installing a satellite server) Perform the following tasks: In the OVPI Web Access Server Name box, type the name of the system (the Web Access Server system) to which you want to deploy the reports. This should be the local system name. In the OVPI Web Access Server Port Number box, type the port number of the system (the Web Access Server system) specified in the OVPI Web Access Server Name box. This will be port 80 unless it was changed during the OVPI install. Enter your Administrator user name and password, which will be the user trendadm unless you changed this during the OVPI install. Click Next.
35
HP SGBU Services
All 1.0 final
HP Software Support
NOTE: If the packages you are installing cannot be installed on the current database, a message will display, indicating that you cannot install the selected report packs on this system. For example, you will receive a message if you try to install a report pack that is only available for Sybase on a system that uses Oracle for its database.
Next perform the following tasks: Select the check box next to each package that you want to install, ensuring that a red check mark ( ) appears in the box next to each package. When you select a package that has dependencies on other packages, Package Manager automatically selects those packages for installation. A black check mark will appear in the box next to each dependent package.
36
HP SGBU Services
All 1.0 final
HP Software Support
37
HP SGBU Services
All 1.0 final
HP Software Support
Service Pack 1 Install OVPI 5.1 SP1 Installation Notes
10.4 Installation Notes

This section describes how to download the Service Pack; the installation tasks you need to perform before you install the Service Pack; and the procedure to install the Service Pack.
NOTE: This Service Pack requires OVPI version 5.1. It contains all bug fixes from version 5.1. To view the previous list of bug fixes, please see the Performance Insight Release Notes for version 5.1.
38
HP SGBU Services
All 1.0 final
HP Software Support
In addition, the Performance Insight Installation Guide and the Performance Insight Administration Guide have been updated and contain important information not included in the 5.1 (February) documentation.
See Support for information about how to obtain OVPI documentation.
10.4.1 Downloading the Service Pack To download the Service Pack:

1. Go to the following website:
http://support.openview.hp.com/cpe/ovpi/patch_ovpi.jsp
2. From your web browser, right-click on the link for your operating system. 3. From the menu, select Save Target As. The Save As dialog box opens. 4. Navigate to the location where you want to save the service pack, and click OK. The final files should be named as follows: HP-UX: OVPI1_sp1_hpux.sh Linux: OVPI1_sp1_linux.sh Solaris: OVPI1_sp1_solaris.sh Windows: OVPI1_sp1_windows.exe
10.4.2 Performing Prerequisite Tasks

Before you install this Service Pack, make sure that you perform the following prerequisite tasks: 1. Notify users that the OVPI software will be temporarily unavailable while you install the Service Pack. 2. Back up the directory into which you installed OVPI before you install the Service Pack. 3. Use binary mode whenever you FTP the Service Pack.
39
HP SGBU Services
All 1.0 final
HP Software Support
4. Install the Service Pack only once on each system. 5. Stop OVPI Timer and wait for all OVPI processes to stop. To stop OVPI Timer, do one of the following depending on your operating system:
HP-UX Type the following command:
/sbin/init.d/ovpi_timer stop
Linux Type the following command:
/etc/init.d/ovpi_timer stop
Solaris Type the following command:
/etc/init.d/ovpi_timer stop b
e. Windows From the Windows taskbar, click the Start button, point to Settings, and choose Control Panel. The Control Panel window appears. f. Double-click Administrative Tools, and then double-click Services. The Services window opens. g. h. Right-click the service (OVPI Timer). Select Stop from the shortcut menu. The Status column indicates if the selected service was successfully stopped. 6. Stop any running OVPI clients (Builder, Viewer, Management Console); the Service Pack stops Web Access Server.
40
HP SGBU Services
All 1.0 final
HP Software Support
7. Wait for all OVPI processes to stop.
NOTE: There is no utility to uninstall this Service Pack. If for some reason you need to remove the Service Pack, you will have to revert to the backup you made of the OVPI directory before you began this Service Pack installation.
10.4.3 Installing the Service Pack

Before you install the Service Pack, make sure that you complete the prerequisite tasks described in the Performing Prerequisite Tasks section. Should errors occur during the installation, see the log file ovpi_51_sp1_install.log, which is created in the default log folder. 10.4.3.1
UNIX Systems
To install the Service Pack on UNIX: 1. Log on as root. 2. If the database is installed, ensure that it is running. 3. Change directory to the location of the Service Pack file. 4. At the command line, type one of the following commands, depending on your operating system:
sh OVPI1_sp1_hpux.sh sh OVPI1_sp1_linux.sh sh OVPI1_sp1_solaris.sh
The Welcome page opens; follow instructions.
41
HP SGBU Services
All 1.0 final
HP Software Support
The Service Pack is installed.
11 Distributed Architecture Configurations
42
HP SGBU Services
All 1.0 final
HP Software Support
43
HP SGBU Services
All 1.0 final
HP Software Support
The key to creating and maintaining an optimized distributed install is understanding that every process on every OVPI installation affects the behavior of the entire environment. For example, the data transfers from Satellite B and C run in serial with Satellite B transferring data first. When Satellite B is running slow due to database issues, the copy from Satellite C starts later. If this behavior occurs during the morning summarization period, the necessary data from Satellite C may not be transferred in time to be included in the daily summarizations. It is now easy to understand how the inter-dependences of a distributed environment can cause system failure. In order to minimize the chances of a system failure, balancing and synchronizing of processes across the environment can be performed. This is performed by configuring the processing in order from the pollers to the satellites and finally the central server.
11.1 Poller Configuration

The main purpose of poller tuning is to be able to collect as many elements as possible within 70% of the polling interval period. 11.1.1 Areas of Configuration The collection process can be split into two processes, each of which requires its own configuration steps. The collection process first goes into the database selecting what is to be polled from internal views. Then it takes that list and performs the collections. After that it passes the collection lists to sub-collectors, which then perform the SNMP Gets.
11.1.2 Collection Parameters There are a couple of main areas of consideration when dealing with polling. SNMP Timeouts It is best to minimize the number of unreachable network devices that are to be polled because that as long as a sub-collection process is waiting on a response, it will not be released to perform additional collections. To adjust the timeout amount, the o option will need to be used; however, it is recommended that this timeout period NOT be increased. SNMP retries Once a SNMP request has timed out, OVPI will attempt to collect that data 5 additional times, each time waiting for the device to respond. HP recommends that you decrease the number of timeouts to lower setting.
GETBULK When ever possible, it is best to attempt to utilize the SNMP GETBLUK collection option. It is important to note that the SNMP versions must be set properly for each device in order for the GETBLUK option to function properly. The 44
HP SGBU Services
All 1.0 final
HP Software Support
command line option for SNMP GETBULK is e. In addition, this option should be invoked with the j option, which will tell mw_collect to check the database for SNMP version for each device. Raw2Delta Parameters and configurations If using multiple pollers, it is important that only one kicks off the raw2delta process to minimize the chance of table contention occurring. The x option, which informs mw_collect not to process raw2delta, should be included on all but one system from which SNMP polls occur. Number of Sub-collectors mw_collect calls a sub collector process for each managed node and raw collection combination. By increasing the number of available sub collectors, you increase the bandwidth of the collection, which decreases the total amount time it takes to collect data. The c option controls the number of sub collector processes. This number can be configured as high as 150 on a large UNIX system; however, on a windows box, it should be kept at around 50 due to OS limitations.
The following is a properly configured Trendtimer.sched configuration for 5.1 poller on an environment where it is the only polling system:
# # trendtimer.sched: # # NOTE: MAX COMMANDS that trendtimer will load to run is 100. # 24:00+1:00 - - {DPIPE_HOME}/bin/log_backup -f {TREND_LOG}/audit.log 24:00+1:00 - - {DPIPE_HOME}/bin/log_backup -f {TREND_LOG}/trend.log 24:00+1:00 - - {DPIPE_HOME}/bin/log_backup -f {TREND_LOG}/metrics.log 60 - - {DPIPE_HOME}/bin/trendexec -i 60 24:00+1:00 - - {DPIPE_HOME}/bin/trendexec -i 1440 MO+1:00 - - {DPIPE_HOME}/bin/trendexec -i 10080
45
HP SGBU Services
All 1.0 final
HP Software Support
MONTH+1:00 - - {DPIPE_HOME}/bin/trendexec -i 44640 5 - - {DPIPE_HOME}/bin/mw_collect -n -i 5 -r 2 -e j -K 1 10 - - {DPIPE_HOME}/bin/mw_collect -n -i 10 -r 2 -e -j -K 1 15 - - {DPIPE_HOME}/bin/mw_collect -n -i 15 -r 2 -e -j -K 1 20 - - {DPIPE_HOME}/bin/mw_collect -n -i 20 -r 2 -e -j -K 1 60 - - {DPIPE_HOME}/bin/mw_collect -n -i 60 -r 2 -e -j -K 1 24:00+1:00 - - {DPIPE_HOME}/bin/mw_collect -n -i 1440 -r 2 -e -j -K 1
# 5 - - {DPIPE_HOME}/bin/ee_collect -i 5 # 10 - - {DPIPE_HOME}/bin/ee_collect -i 10 # 15 - - {DPIPE_HOME}/bin/ee_collect -i 15 # 20 - - {DPIPE_HOME}/bin/ee_collect -i 20 # 60 - - {DPIPE_HOME}/bin/ee_collect -i 60
60 - - {DPIPE_HOME}/bin/disk_space # 24:00+23:00 - - {DPIPE_HOME}/bin/indexmaint -K # 24:00+1:00 - - {DPIPE_HOME}/bin/ee_collect -i 1440 # 24:00+2:00 - - {DPIPE_HOME}/bin/trend_discover -t # 24:00+1:00 - - {DPIPE_HOME}/bin/transform_maint -remove # 24:00+24:00 - - {DPIPE_HOME}/bin/db_delete_data
46
HP SGBU Services
All 1.0 final
HP Software Support
# 24:00+1:00 - - {DPIPE_HOME}/bin/tpmaint
11.2 Satellite Server Configuration
11.2.1 Process Serialization and load balancing Due to the inter dependencies between satellite servers, it is recommended that each satellite server in a distributed environment be standardize across the hardware utilized, the processing schedules and the polling loads. By standardizing the configurations, it will be easier to plan for server capacity and replacement. In addition, this type of server configuration will allow for easier expansion and system recoveries.
11.2.1.1
Process Serialization
In order to minimize the amount of process contention, it is important to serialize satellite server processing. This can be done by separating the processing into three sections: Hourly Scripts Combining all the hourly processes into one proc file will minimize the amount of process contention. We do this by creating a script called OVPI_HOURLY_SCRIPT.pro and place it in the $DPIPE_HOME/scripts directory. This file will contain all calls for hourly summarizations.
Please note that the following script includes information from multiple report packs and will need to be modified to include scripts from your environment. # Michael Davies # 08/04/2005 # Performs Hourly procs # OVPI_HOURLY_SCRIPT.pro
47
HP SGBU Services
All 1.0 final
HP Software Support
begin: Hourly wait
{DPIPE_HOME}/bin/trend_proc -f {DPIPE_HOME}/scripts/COBRA/rate_IR_cobra.pro {DPIPE_HOME}/bin/trend_proc -f {DPIPE_HOME}/scripts/IR_DevPort_Hourly_Process.pro {DPIPE_HOME}/bin/trend_proc -f {DPIPE_HOME}/scripts/populate_index_by_ifindex.pro {DPIPE_HOME}/bin/trend_proc -f {DPIPE_HOME}/scripts/DeviceResourceReporting_Hourly.pro {DPIPE_HOME}/bin/trend_proc -f {DPIPE_HOME}/scripts/DeviceResourceBackplaneReporting_Hourly.pro {DPIPE_HOME}/bin/trend_proc -f {DPIPE_HOME}/scripts/Netscreen_and_Nokia_Hourly.pro end: Hourly
Maintenance Script Combining all the maintenance procedures will decrease the amount of table contention. All maintenance procedures should be combined into a script named OVPI_MAINT_SCRIPT.pro and placed in $DPIEP_HOME/scripts. This script should be run once nightly.
Please note that indexmaint has been modified in order to rebuild all indexes daily. This is required because of the shortened retention times on the satellite servers. # Michael Davies # 08/04/2005 # Performs system Maintenance # OVPI_MAINT_SCRIPT.pro
48
HP SGBU Services
All 1.0 final
HP Software Support
begin: maint wait
{DPIPE_HOME}/bin/indexmaint r -f {DPIPE_HOME}/bin/db_delete_data {DPIPE_HOME}/bin/transform_maint -remove {DPIPE_HOME}/bin/tpmaint end: maint
Daily Processing Data should only be summarized to the hourly level on the satellite server; however, there are some daily processes that must be run for different datapipes to collect. Create the proc file $DPIPE_HOME/scripts/OVPI_DAILY_SCRIPT.pro and place all daily processes in the file. Please note that the following script includes information from multiple reports and will need to be modified to include scripts from your environment. # Michael Davies # 08/04/2005 # Performs daily procs # OVPI_DAILY_SCRIPT.pro
begin: DAILY wait {DPIPE_HOME}/bin/trend_proc -f {DPIPE_HOME}/scripts/CommonPropertyTables_importdata.pro
49
HP SGBU Services
All 1.0 final
HP Software Support
{DPIPE_HOME}/bin/trend_proc -f {DPIPE_HOME}/scripts/CommonPropertyTables_delete_nodes.pro
{DPIPE_HOME}/bin/trend_proc -f {DPIPE_HOME}/scripts/CiscoVLANDailyReporting.pro {DPIPE_HOME}/bin/trend_proc -f {DPIPE_HOME}/scripts/DevResCiscoSwitchDailyReporting.pro {DPIPE_HOME}/bin/trend_proc -f {DPIPE_HOME}/scripts/Netscreen_and_Nokia_Daily.pro end: DAILY
11.2.2 Process Scheduling After completing the serialization files comment out of $DPIPE_HOME/lib/trendtimer.sched all entries for the processes included in the serialization scripts. Next add the calls for the scripts to trendtimer.sched using the following entries:
24:00+17:00 - - {DPIPE_HOME}/bin/trend_proc -f {DPIPE_HOME}/scripts/OVPI_MAINT_SCRIPT.pro 24:00+2:00 - - {DPIPE_HOME}/bin/trend_proc -f {DPIPE_HOME}/scripts/OVPI_DAILY_SCRIPT.pro 1:00+20 - - {DPIPE_HOME}/bin/trend_proc -f {DPIPE_HOME}/scripts/OVPI_HOURLY_SCRIPT.pro
11.2.2.1
Load Balancing
Baselining System Performance Baselining the OVPI installation is an important first step to understanding the capacity of each server. Each OVPI environment will behave differently, depending on the number of reporting solutions installed, as well as the number of collected elements. Due to this fluctuation, it is important to baseline the satellite and poller combination by slowly adding elements and monitoring system behavior. Items that will need to be baselined
50
HP SGBU Services
All 1.0 final
HP Software Support
1)
2)
The maximum number of elements that can be polled within the targeted collection interval. The recommended procedure is to start with a limited amount of nodes then gradually increase the amount every two days until the collection takes a maximum of 70% of the interval. For a 15 minute interval, this means that the collection must finish within 10 minutes. The amount of time it takes to complete hourly summarizations should be recorded. It is important to verify that these summarizations complete on fully populated tables within an hour.
12 Central Server Configurations

12.1.1 Process Serialization The Primary thing to remember when dealing with process serialization and scheduling on the central server is that all data is being fed by the satellites servers. Therefore, the primary consideration is to ensure that all data is copied form the satellites prior to the commencement of summarizations. Due to this data dependency, copies should be included in daily processing scripts.
12.1.1.1
TrendCopies
During the TRENDcopy process two sets of sub -transfers occur, one for adding additional property entries and one for data transfers. I t is possible to separate these sub-processes, which will result in a more optimized processing schedule than utilizing the two functions together during the day. 12.1.1.2
Process Serialization
In order to minimize the amount of process contention, it is important to serialize the central server processing. This can be done by separating the processing into three sections: 1) Hourly Scripts Combining all the hourly processes into one proc file will minimize the amount of process contention. We do this by creating a script call OVPI_HOURLY_SCRIPT.pro and place it in the $DPIPE_HOME/scripts directory. This file will contain all calls for dataonly trend copies. TRENDcopy is called with the D optiuon specifying that only reporting data will be copied and the I 0 option, which informs trendcopy to not perform indexmaint after the completion of the copy process.
Please note that the following script includes information from the Interface Reporting report pack and will need to be modified to include scripts from your environment. # Michael Davies # 08/04/2005
51
HP SGBU Services
All 1.0 final
HP Software Support
# performs Hourly procs # OVPI_HOURLY_SCRIPT.pro
begin: Hourly wait
{DPIPE_HOME}/bin/perl {DPIPE_HOME}/scripts/IR_Performance.pl -p "Satellite_1_Copy" -t hour -v start {DPIPE_HOME}/bin/trendcopy -t SHIRDevPorts -s SATELLITE_SERVER_1_DATABASE -S THIS_MACHINE_DATABASE D I 0 {DPIPE_HOME}/bin/trendcopy -t RIRRouterSystem -s SATELLITE_SERVER_1_DATABASE -S THIS_MACHINE_DATABASED I 0 {DPIPE_HOME}/bin/perl {DPIPE_HOME}/scripts/IR_Performance.pl -p "Satellite_1_Copy" -t hour -v stop
end: Hourly
2) Maintenance Script Combining all the maintenance procedures will decrease the amount of table contention. All maintenance procedures should be combined into a script named OVPI_MAINT_SCRIPT.pro and placed in $DPIPE_HOME/scripts. This script should be run once nightly.
Please note that the following script includes information from multiple reports and will need to be modified to include scripts from your environment. # Michael Davies # 08/04/2005 # Performs system Maintenance
52
HP SGBU Services
All 1.0 final
HP Software Support
# OVPI_MAINT_SCRIPT.pro
begin: maint wait
{DPIPE_HOME}/bin/indexmaint {DPIPE_HOME}/bin/db_delete_data {DPIPE_HOME}/bin/transform_maint -remove {DPIPE_HOME}/bin/tpmaint end: maint
3) Daily Processing The Daily processing script should include calls to all daily processing summarizations, as well as a full set trendcopies. The trendcopies should be called prior to the summarizations for their corresponding report packs. Please note that the following script includes information from Interface Reporting Only and will need to be modified to include scripts from your environment. # Michael Davies # 08/04/2005 # Performs daily procs # OVPI_DAILY_SCRIPT.pro
begin: IR_DMF wait
53
HP SGBU Services
All 1.0 final
HP Software Support
# Include a full set of TRENDcopies prior to performing the daily {DPIPE_HOME}/bin/trendcopy -t SHIRDevPorts -s SATELLITE_SERVER_1_DATABASE -S THIS_MACHINE_DATABASE {DPIPE_HOME}/bin/trendcopy -t RIRRouterSystem -s SATELLITE_SERVER_1_DATABASE -S THIS_MACHINE_DATABASE end: DAILY_COPIES {DPIPE_HOME}/bin/perl {DPIPE_HOME}/scripts/IR_Performance.pl -p "Property_Import" -t day -v start {DPIPE_HOME}/bin/trend_proc -f {DPIPE_HOME}/scripts/IR_importdata.pro {DPIPE_HOME}/bin/perl {DPIPE_HOME}/scripts/IR_Performance.pl -p "Property_Import" -t day -v stop
# Start all processes for Report Pack Summarizations {DPIPE_HOME}/bin/perl {DPIPE_HOME}/scripts/IR_Performance.pl -p "DevPort_DMF_Summary" -t day -v start {DPIPE_HOME}/bin/trend_sum -t SHIRDevPorts {DPIPE_HOME}/scripts/SDIRDevPorts.sum {DPIPE_HOME}/bin/trend_sum -t SDIRDevPorts {DPIPE_HOME}/scripts/SMIRDevPorts.sum {DPIPE_HOME}/bin/trend_sum -t SDIRDevPorts {DPIPE_HOME}/scripts/SDIRDevPorts_forecast.sum -e SDIRDevPorts -f
-e SMIRDevPorts -f
-e fore -y 42 -f
{DPIPE_HOME}/bin/trend_sum -t SDIRDevPorts -e dow {DPIPE_HOME}/scripts/SDIRDevPorts_forecast_dow.sum
-y 42 -f
54
HP SGBU Services
All 1.0 final
HP Software Support
{DPIPE_HOME}/bin/trend_sum -t SDIRDevPorts {DPIPE_HOME}/scripts/SDIRCust_Cust.sum {DPIPE_HOME}/bin/trend_sum -t SDIRDevPorts {DPIPE_HOME}/scripts/SDIRCust_All.sum
-e SDIRCust -f
-e SDIRCust -f
{DPIPE_HOME}/bin/perl {DPIPE_HOME}/scripts/IR_Performance.pl -p "DevPort_DMF_Summary" -t day -v stop
end: IR_DMF
12.1.2 Process Scheduling After completing the serialization files comment out of $DPIPE_HOME/lib/trendtimer.sched, all entries for the processes included in the serialization scripts. Hourly processes that include data copies should be performed every 4 to 6 hours, except during times that daily summarizations are occurring. Add the calls for the scripts to trendtimer.sched using the following entries:
24:00+17:00 - - {DPIPE_HOME}/bin/trend_proc -f {DPIPE_HOME}/scripts/OVPI_MAINT_SCRIPT.pro 24:00+2:00 - - {DPIPE_HOME}/bin/trend_proc -f {DPIPE_HOME}/scripts/OVPI_DAILY_SCRIPT.pro 24:00+6:00+40 - - {DPIPE_HOME}/bin/trend_proc -f {DPIPE_HOME}/scripts/OVPI_HOURLY_SCRIPT.pro 24:00+6:00+40 - - {DPIPE_HOME}/bin/trend_proc -f {DPIPE_HOME}/scripts/OVPI_HOURLY_SCRIPT.pro 24:00+10:00+40 - - {DPIPE_HOME}/bin/trend_proc -f {DPIPE_HOME}/scripts/OVPI_HOURLY_SCRIPT.pro 24:00+14:00+40 - - {DPIPE_HOME}/bin/trend_proc -f {DPIPE_HOME}/scripts/OVPI_HOURLY_SCRIPT.pro 24:00+20:00+40 - - {DPIPE_HOME}/bin/trend_proc -f {DPIPE_HOME}/scripts/OVPI_HOURLY_SCRIPT.pro
55
HP SGBU Services
All 1.0 final
HP Software Support
13 Standalone OVPI Server Configurations

For a standalone installation all processing will be performed on the same physical system. Standalone installations are far simpler to configure then a distributed environment due to the fact that process dependencies do not exist across numerous systems. However, there are some draw backs, a standalone system is not capable of performing the same level of collections as a distributed environment because there are not as many processing cycles available as there are in t he distributed environment. Due to the finite amount of processing cycles available the same processes of poll timing and process serialization will need to be followed as for a distributed environment.
13.1 Polling configuration

Polling calls in $DPIPE_HOME/lib/trendtimer.sched will need to be modified to allow polling optimization. 13.1.1 Areas of Configuration The collection process can be split into two processes, each of which requires its own configuration steps. The collection process first goes into the database selecting what is to be polled from internal views. Then it takes that list and performs the collections. After that it passes the collection lists to sub-collectors, which then perform the SNMP Gets.
13.1.2 Collection Parameters There are a couple of main areas of consideration when dealing with polling. SNMP Timeouts It is best to minimize the number of unreachable network devices that are to be polled because that as long as a sub-collection process is waiting on a response, it will not be released to perform additional collections. To adjust the timeout amount, the o option will need to be used; however, it is recommended that this timeout period NOT be increased. SNMP retries Once a SNMP request has timed out, OVPI will attempt to collect that data 5 additional times, each time waiting for the device to respond. HP recommends that you decrease the number of timeouts to lower setting. GETBULK When ever possible, it is best to attempt to utilize the SNMP GETBLUK collection option. It is important to note that the SNMP versions must be set properly for each device in order for the GETBLUK option to function properly. The command line option for SNMP GETBULK is e. In addition, this option should be invoked with the j option, which will tell mw_collect to check the database for SNMP version for each device.
56
HP SGBU Services
All 1.0 final
HP Software Support
Raw2Delta Parameters and configurations If using multiple pollers, it is important that only one kicks off the raw2delta process to minimize the chance of table contention occurring. The x option, which informs mw_collect not to process raw2delta, should be included on all but one system from which SNMP polls occur. Number of Sub-collectors mw_collect calls a sub collector process for each managed node and raw collection combination. By increasing the number of available sub collectors, you increase the bandwidth of the collection, which decreases the total amount time it takes to collect data. The c option controls the number of sub collector processes. This number can be configured as high as 150 on a large UNIX system; however, on a windows box, it should be kept at around 60 due to OS limitations.
13.2 Process Serialization

Just like with a distributed environment it is important to minimize the amount of process contention by serializing processing. This can be done by separating the processing into three sections: Hourly Scripts Combining all the hourly processes into one proc file will minimize the amount of process contention. We do this by creating a script called OVPI_HOURLY_SCRIPT.pro and place it in the $DPIPE_HOME/scripts directory. This file will contain all calls for hourly summarizations.
Please note that the following script includes information from multiple report packs and will need to be modified to include scripts from your environment. # Michael Davies # 08/04/2005 # Performs Hourly procs # OVPI_HOURLY_SCRIPT.pro
begin: Hourly wait
{DPIPE_HOME}/bin/trend_proc -f {DPIPE_HOME}/scripts/COBRA/rate_IR_cobra.pro {DPIPE_HOME}/bin/trend_proc -f {DPIPE_HOME}/scripts/IR_DevPort_Hourly_Process.pro

57
HP SGBU Services
All 1.0 final
HP Software Support
{DPIPE_HOME}/bin/trend_proc -f {DPIPE_HOME}/scripts/populate_index_by_ifindex.pro {DPIPE_HOME}/bin/trend_proc -f {DPIPE_HOME}/scripts/DeviceResourceReporting_Hourly.pro {DPIPE_HOME}/bin/trend_proc -f {DPIPE_HOME}/scripts/DeviceResourceBackplaneReporting_Hourly.pro {DPIPE_HOME}/bin/trend_proc -f {DPIPE_HOME}/scripts/Netscreen_and_Nokia_Hourly.pro end: Hourly
Maintenance Script Combining all the maintenance procedures will decrease the amount of table contention. All maintenance procedures should be combined into a script named OVPI_MAINT_SCRIPT.pro and placed in $DPIEP_HOME/scripts. This script should be run once nightly.
# Michael Davies # 08/04/2005 # Performs system Maintenance # OVPI_MAINT_SCRIPT.pro
begin: maint wait
{DPIPE_HOME}/bin/indexmaint {DPIPE_HOME}/bin/db_delete_data
58
HP SGBU Services
All 1.0 final
HP Software Support
{DPIPE_HOME}/bin/transform_maint -remove {DPIPE_HOME}/bin/tpmaint end: maint
Daily Processing Create the proc file $DPIPE_HOME/scripts/OVPI_DAILY_SCRIPT.pro and place all daily report pack processes in the file. Please note that the following script includes information from multiple reports and will need to be modified to include scripts from your environment. # Michael Davies # 08/04/2005 # Performs daily procs # OVPI_DAILY_SCRIPT.pro
begin: DAILY wait {DPIPE_HOME}/bin/perl {DPIPE_HOME}/scripts/IR_Performance.pl -p "DevPort_DMF_Summary" -t day -v start {DPIPE_HOME}/bin/trend_sum -t SHIRDevPorts {DPIPE_HOME}/scripts/SDIRDevPorts.sum {DPIPE_HOME}/bin/trend_sum -t SDIRDevPorts {DPIPE_HOME}/scripts/SMIRDevPorts.sum {DPIPE_HOME}/bin/trend_sum -t SDIRDevPorts {DPIPE_HOME}/scripts/SDIRDevPorts_forecast.sum -e SDIRDevPorts -f
-e SMIRDevPorts -f
-e fore -y 42 -f
59
HP SGBU Services
All 1.0 final
HP Software Support
{DPIPE_HOME}/bin/trend_sum -t SDIRDevPorts -e dow {DPIPE_HOME}/scripts/SDIRDevPorts_forecast_dow.sum {DPIPE_HOME}/bin/trend_sum -t SDIRDevPorts {DPIPE_HOME}/scripts/SDIRCust_Cust.sum {DPIPE_HOME}/bin/trend_sum -t SDIRDevPorts {DPIPE_HOME}/scripts/SDIRCust_All.sum
-y 42 -f
-e SDIRCust -f
-e SDIRCust -f
{DPIPE_HOME}/bin/perl {DPIPE_HOME}/scripts/IR_Performance.pl -p "DevPort_DMF_Summary" -t day -v stop end: DAILY
13.3 Sample Trendtimer.sched

The following is a sample $DPIPE_HOME/lib/trendtimer.sched for a standalone server.
# # trendtimer.sched: # # NOTE: MAX COMMANDS that trendtimer will load to run is 100. # 24:00+1:00 - - {DPIPE_HOME}/bin/log_backup -f {TREND_LOG}/audit.log 24:00+1:00 - - {DPIPE_HOME}/bin/log_backup -f {TREND_LOG}/trend.log 24:00+1:00 - - {DPIPE_HOME}/bin/log_backup -f {TREND_LOG}/metrics.log 60 - - {DPIPE_HOME}/bin/trendexec -i 60 24:00+1:00 - - {DPIPE_HOME}/bin/trendexec -i 1440 MO+1:00 - - {DPIPE_HOME}/bin/trendexec -i 10080
60
HP SGBU Services
All 1.0 final
HP Software Support
MONTH+1:00 - - {DPIPE_HOME}/bin/trendexec -i 44640 5 - - {DPIPE_HOME}/bin/mw_collect -n -i 5 -K 1 -r 2 -e j c 140 10 - - {DPIPE_HOME}/bin/mw_collect -n -i 10 -r 2 -e -j c 140 -K 1 15 - - {DPIPE_HOME}/bin/mw_collect -n -i 15 -r 2 -e j c 140 -K 1 20 - - {DPIPE_HOME}/bin/mw_collect -n -i 20 -r 2 -e j c 140 -K 1 60 - - {DPIPE_HOME}/bin/mw_collect -n -i 60 -r 2 -e j c 140 -K 1 24:00+1:00 - - {DPIPE_HOME}/bin/mw_collect -n -i 1440 -r 2 -e -j -K 1
# 5 - - {DPIPE_HOME}/bin/ee_collect -i 5 # 10 - - {DPIPE_HOME}/bin/ee_collect -i 10 # 15 - - {DPIPE_HOME}/bin/ee_collect -i 15 # 20 - - {DPIPE_HOME}/bin/ee_collect -i 20 # 60 - - {DPIPE_HOME}/bin/ee_collect -i 60
60 - - {DPIPE_HOME}/bin/disk_space # 24:00+23:00 - - {DPIPE_HOME}/bin/indexmaint -K # 24:00+1:00 - - {DPIPE_HOME}/bin/ee_collect -i 1440 # 24:00+2:00 - - {DPIPE_HOME}/bin/trend_discover -t # 24:00+1:00 - - {DPIPE_HOME}/bin/transform_maint -remove # 24:00+24:00 - - {DPIPE_HOME}/bin/db_delete_data
61
HP SGBU Services
All 1.0 final
HP Software Support
# 24:00+1:00 - - {DPIPE_HOME}/bin/tpmaint SU+1:00 - - {DPIPE_HOME}/bin/indexmaint -f 24:00+17:00 - - {DPIPE_HOME}/bin/trend_proc -f {DPIPE_HOME}/scripts/OVPI_MAINT_SCRIPT.pro 24:00+2:00 - - {DPIPE_HOME}/bin/trend_proc -f {DPIPE_HOME}/scripts/OVPI_DAILY_SCRIPT.pro 1:00+20 - - {DPIPE_HOME}/bin/trend_proc -f {DPIPE_HOME}/scripts/OVPI_HOURLY_SCRIPT.pro
14 Interface Reporting Configurations

If the total number of IR elements deployed in the environment exceeds 40,000, it is best to utilize directed instance polling. Directed instance polling will allow the administrator to poll only desired data from the managed nodes, thus decreasing the total size of the collections.
14.1 Pre Configuration Steps.

In order to configure Directed Instance polling, an export of IR property data will need to be performed. The export file sizes are limited to 20,000 element by default; however, this can be modified to produce a larger file. The maximum number of lines that can be exported is 50,000 using the automated processes. To modify the export size, the following changes will need to be made: Modify the $DPIPE_HOME/scripts/IR_exportdata.pro file updating the p option to the desired size. In this case the size is 20,000 rows.
62
HP SGBU Services
All 1.0 final
HP Software Support
14.2 Configuring Directed Instance Polling

(For exact directions please refer to the IR_ifentryDP_userGuide.pdf )
The following genral steps will need to be followed in order to configure directed instance polling: 1) from the $DPIPE_HOME/scripts directory run the IRConfig.pl script using the following command ../bin/perl ./IRConfig.pl Select option , 1 configure polling. Select Poll Flagged Interfaces Only. Exit the configuration screen Rerun the Config.pl script
2) 3) 4) 5)
63
HP SGBU Services
All 1.0 final
HP Software Support
6) 7) 8) 9)
Select import/export property data. Export the property data. Modify the exported data file under $DPIPE_HOME/data/PropertyData deleting any unwanted interfaces. Open Return to the command line and once again run the IRConfig.pl script from the $DPIPE_HOME directory
10) Select Import/Export Property Data then select import property data. 11) Wait one hour then run the following command
64
HP SGBU Services
All 1.0 final
HP Software Support
12) the consul and run the Administrate Interface Reporting Form setting the data filtering to true
13) At this point the system will be configured to collect only flagged interfaces. To verify that the correct number of interfaces are included, you can run the following command an hour after the configuration:
65
HP SGBU Services
All 1.0 final
HP Software Support
The counts for POLL_FLAG_IFENTRY relate to the elements that will be polled.
15 Additional Post Installation Steps

The installation program for OVPI does not perform all necessary configurations that will ensure proper running behavior. In order to produce an optimized system, the following post installation steps are recommended:
15.1 Periodically Restarting the Application Server

Due to internal JAVA memory management issues, it is necessary to restart the OVPI Application server on a regular basis. If the application server remains up for extended periods of time, it will be slow to respond to incoming requests and may fail to respond all together. HP recommends that the OVPI Application server process be restarted on a daily basis. In order to do so, a cron process should be scheduled to restart the OVPI Application Server using the following call /etc/rc3.d/**piweb restart.
15.2 Rebuilding Table Indexes on a Central or Standalone Server

OVPI is continually adding and removing data to table structures causing the indexes to become stale. In order to resolve this issue, it is recommended that on a standalone or central server indexmaint be run once a week with the f options, which informs it to forcefully rebuild the indexes on all tables.
66
HP SGBU Services
All 1.0 final
HP Software Support
16 Ongoing Maintenance
After installing OVPI continual system maintenance must be performed. An OVPI environment is in a state of constant fluctuation, which requires that an administrator have full control of the architecture at all times. Logs should be reviewed daily for errors. If this is too difficult of a task, PVLmon can be used to e-mail a report of errors to the administrator. In addition, the administrator must have a solid understanding of the OVPI environment and what changes in polling loads may do. HP recommends that any customer considere hiring a full-time OVPI administrator for larger installs. This administrator should have experience in enterprise OVPI deployments and have attended the OVPI training courses offered by HP.
16.2 Change Controls and Documentation

It is imperative that the OVPI team keep close control over their environment. Making minor changes in configurations or polling loads can lead to dire consequences. In order to avoid calamity, change control mechanisms must be put in place to allow for control of processing loads. A short List of Change Controls (NOTE: This is a partial list, and additional change controls should be added in the customer environment) 1) End-users should not be allowed to make modifications to polling configurations. Managed node additions should be requested by end-users and implemented by the OVPI administrator. OVPI administrators should have a logged account of performance metrics for the OVPI installation. This log should be used in deciding if additional polling can be performed on a given system. Administrative tasks, such as reboots, should be arranged with the OVPI admin ahead of time. All new reporting solutions should be tested in a development environment before being rolled into production. No additions of nodes or reporting solutions should be made on a Friday, unless the OVPI administrator will be monitoring the solutions over the weekend.
2)
3) 4) 5)
16.3 SNMP Node Management

Due to the fact that each additional managed node on OVPI will reduce the amount of available system processing cycles, it is imperative that the importation of nodes be closely monitored. The addition of mass numbers of managed nodes through manual or automated processes could result in system overload and eventually lead to an unrecoverable system failure. 67
HP SGBU Services
All 1.0 final
HP Software Support
16.3.1 SNMP Discoveries

SNMP Network Discoveries - In an enterprise environment, the types of networks that are usually monitored contain a great deal of interfaces that may not need to be reported on. Due to the existence of large numbers of undesired elements, it is best to avoid using the SNMP Network Discoveries; the additional collections that stem for this type of discovery will cause system slowing. SNMP Type Discoveries SNMP Type discoveries can be utilized after adding managed nodes via the GUI or node manager.
16.3.2 NNM Node Synchronizations

It is best to avoid using NNM node synchronizations in cases where an unknown number of interfaces or managed nodes may be added to a functional OVPI system. This is recommended, due to the possibility of mass numbers of new managed nodes being adding, which could overload an OVPI installation.
16.3.3 Adding New Managed Nodes

Nodes should be added in small numbers not exceeding 1000 managed elements for SNMP or 10 OVPA collections per addition, taking time to monitor the system after the additional elements have been added. It is recommended that the following procedures be performed for adding nodes to a new installation: 1) Get an accurate count of the number of OVPI elements contained on each node which will need to be utilized in the creation of Node Manager Import file. Create a group of node import files that contain 1000 managed OVPI elements each. For example, if each of the network routers contain 10 interfaces, all of which are Ethernet only, you would create a list of 100 routers to be added at a time. Follow the instructions in the OVPI reference guide under node manager to create according to the required format. Import the first file using the following command: node_manager import file <import_file> Verify the node count has increased by the expected amount of nodes using the following sql command.
2)
3)
4)
68
HP SGBU Services
All 1.0 final
HP Software Support
select count(1) from ksi_managed_node;
5) Common Properties Import The Import Utility The import utility performs three tasks: Reads property data files containing records for the elements to be updated Stores the data in temporary tables in the database Uses imported data to update Common Property Tables (K_Node, K_Customer, and K_Location) If the data in the property data file is for an element (node, customer, or location) that already exists in the property table, the data in the property table is overwritten by the imported data. If an element does not exist, a new row is inserted in the appropriate property table. The default directory for property import and export files is: {DPIPE_HOME}/OVPI/data/PropertyData. You never need to have all four property files in this directory. This directory can be empty, or it can contain just one or two files. When you want to update just one property table, edit just one file and store just one file. 69
HP SGBU Services
All 1.0 final
HP Software Support
To run the import utility manually, before the utility is scheduled to run, do the following: 1 Open a command prompt at the system level. 2 Navigate to the {DPIPE_HOME}/scripts directory. 3 Type the following command: trend_proc -f CommonPropertyTables_importdata.pro
5) Next run SNMP type discovery using the PIADMIN consul. (Do not run this typing via the command line as it takes longer than through the GUI.)
Verify if the correct amount of nodes were typed by running the following SQL commands Isql select count(1) from dl_type;
70
HP SGBU Services
All 1.0 final
HP Software Support
Alternate query select count(1), from dl_type group by type_;
7)
If interface reporting is present, configure the Interface Reporting datapipe to utilize Directed Instance polling, as it will avoid the collection of a DHCP or unwanted elements. (For Interface Reporting configuration recommendations, please utilize Section 12 of this document) Monitor the satellite server, verifying that collections are occurring via table manager. Also, after 2 hours compare the completion times of multiple collections in $DPIPE_HOME/logs/trend.log looking for consistent running times. The polls need to complete within 70% of the allotted interval. If the collection exceeds this amount, nodes will need to be removed. If the system is completing within the recommended time frame, additional nodes can be added following steps 2 through 7. In addition, do not add additional nodes if the total number of collected elements will result in a fully populated rate table above 8 million rows. This can be calculated by utilizing the following formula rate_table_size_row = ((number of elements)) *(((Number_Of_Collection_Per_Hour)*(24))*((Number_of_days_of_retnetion)+1)
8)
9)
16.4 Database Backup and Recovery

The OVPI database should be backed up on a regular basis. The following is a brief overview of the Oracle backup process and the usages of RMAN, the oracle Recovery Management tool.
16.4.1 Hot vs. Cold Database Backups

OVPI is constantly inserting new data in the database, which makes it difficult to backup the database using a cold technique in which the Oracle database is taken offline. Instead, it is recommended that the backup be performed using a hot backup technique in which the datafiles and the archived redo logs can be dumped to disk.
16.4.2 RMAN Oracle Database Recovery Manager

RMAN is a command line utility provided with the Oracle database installation. This utility provides an interface from which to initiate database backups. Due to the fact that RMAN is a native Oracle program, it contains all the necessary processes to backup a database, even while the database is being written to.
71
HP SGBU Services
All 1.0 final
HP Software Support
16.4.3 Backup Procedure

16.4.3.1
RMAN Configurations
In order to backup the Oracle database, RMAN will need to be configured prior to calling the backup. Information regarding destination files and directories, as well as datafile retention times, will need to be set. To get a full list of the configurable parameters of RMAN, please reference the Oracle online documentation
16.4.3.1.1
Backup Procedure
The following procedure can be run to provide a full backup of the database. No additional database integrity checks are required, as RMAN will perform these checks using the following calls. In addition, you should never attempt to back up the live redo log groups. #contents of backup.rcv. "#" denotes a comment line and will be ignored by RMAN. # The following performs the backup of the database and the archived log backup database plus archivelog; # The following verifies the integrity of the data that was backed up and verifies that the files can be #restored. restore database validate; # The following informs RMAN not to prompt the user for deleting old files. delete noprompt obsolete; # The following section copies the required Oracle files host 'cp /oracle/ora92/database/pwd(database_name).ora /backup'; host 'cp /oracle/ora92/network/admin/tnsnames.ora /backup'; host 'cp /oracle/ora92/network/admin/listener.ora /backup'; exit;
16.5 System Monitoring
When properly configured, OVPI can run without error for an extended period of time; however, it is possible that either by a hardware fault or an improper configuration change that the system could fail. It is, therefore, important to continue to monitor OVPI after its deployment.
16.5.1
Maintenance Procedures
It is recommended that daily monitoring procedures be generated, and that a dedicated resource be assigned that will be responsible for reporting and responding to system failures. The following procedures should be generated:
72
HP SGBU Services
All 1.0 final
HP Software Support
16.5.1.1
Log Scan
A procedure should be generated for scanning the OVPI and Oracle logs for errors. HP recommends that PLMON is configured to scan the OVPI logs for errors and send out a report via e-mail to a primary and secondary OVPI administrator. 16.5.1.2
PVLMON Configuration
PVLmon should be configured to monitor the following: 1) 2) 3) 4) Report any errors that occur. Provide information on the age of last polled data. Provide information on the size of database tables and segments. Provide disk usage information.
16.5.2 Process Completion Monitoring

To ensure that all processing occurs on the system in the allotted time frames, it is important to periodically baseline the amount of time it takes for processes to complete. Then compare the current processing times to the baseline. If the times exceed the baseline times, this could mean that the system is becoming overloaded and should be closely monitored for errors.
16.5.3 Tacking system processing times

In order to track processing times, you will need to scan the $DPIPE_HOME/logs/audit.log start and stop times of collection, roll-up and deletion processes. You should use auditreport.pl, a freely distributed audit log parser to obtain the start and end times for the following processes. trendsum mw_collect db_delete_data trendcopy indexmaint ee_collect Record you entries in a spreadsheet to compare over time. Any changes in the times should be noted, and the explanation for any increases or decreases should be added. For example, mw_collect takes 15 seconds longer to complete because the number of interfaces polled has increased.
73
HP SGBU Services
All 1.0 final
HP Software Support
16.6 Expanding the Environment

At some point, if you continue to grow your infrastructure, your OVPI environment will reach maximum capacity. At that time it will be important to understand that capacity cannot be increased by simply adding additional space in the database. You will need to add additional systems. The question then becomes how best to add additional systems to the mix to maximize your ROI. HP has provided the following examples; however, your installation may differ. If you have any questions on how to best deploy your environment, please contact a local HP solutions architect. They will be able to provide you with a solid expansion plan.
16.6.1
Standalone Server has reached capacity
In this case, a standalone server has reached maximum capacity at 30,000 elements every 15 minutes. You would like to add an additional 20,000 elements. Architecture actions In this case, it would be best to re-task the original standalone server as the central and add two additional satellites. By taking this action, you will not need to remove any property information from the existing server. In addition, this will allow the administrator to easily rebalance the loads between the new systems.
74
HP SGBU Services
All 1.0 final
HP Software Support
16.6.2 Distributed Environment has reached its maximum capacity In this case, a distributed OVPI environment has reached its capacity at about SNMP 150,000 elements. The customer has requested that they would like to add 300 OVPA agent collections and start reporting on system performance. Architecture actions In this instance, the best architecture decision would be to add a single standalone server to report and collect on the OVPA data. The original OVPI central server will be utilized as an application server.
75
HP SGBU Services
All 1.0 final
HP Software Support
17 Appendix A: Sybase to Oracle Syntax Conversions

This information has been lifted from EdsOracleCribSheet and provides useful information on the equivalencies of Sybase and Oracle. Mike Davies has made modifications to the original information to allow for better flow to this OVPI document.
17.1 TERMINOLOGY / EQUIVALENCES
SYBASE TERM server server
ORACLE TERM database instance
Notes (files / redo logs, control files ) instance is associated with a database via mounting
76
HP SGBU Services
All 1.0 final
HP Software Support
database
table space
schema
(object owner?)
collection of objects readable by the user
transaction log
redo log
oracle has a job (lgwr) that periodically writes to rollback disk segments Archiver writes rollback segments totape/backup Write modified data in memory to disk asynchronously. manages database buffer cache Basic memory containing data from disk dirty and clean pages. Both use LRU queue strategy
System Global Area: sh memory pool, db buffer cache, and redo log buffer Program Global Area: non shared memory area allocated to each user connection Daemons to cleanup / reclaim Recovery Process Lock Process System Table
dump tran checkpoint
arch (archiver) dbwr
default data cache
database buffer cache (DBC)

SGA
PGA
PMON,SMON RECO LCKN Sequence $SYBASE $ORACLE_HOME
$DSQUERY
$ORACLE_SID
77
HP SGBU Services
All 1.0 final
HP Software Support
17.2 ENVIRONMENT VARIABLES

# your DISPLAY is set to your pc for the java stuff in unix to work export DISPLAY=x.x.x.x:0.0 # oracle product directory (equivalent of $SYBASE) export ORACLE_HOME=/optapp1/oracle/product/8.1.7 # equivalent of sybases $DSQUERY export ORACLE_SID=WPGDWH export ORACLE_BASE=/optapp1/oracle/product export ORACLE_TERM=vt100 LD_LIBRARY_PATH=/usr/openwin/lib:$ORACLE_HOME/lib PATH=:$ORACLE_HOME/bin
17.3 UNIX COMMANDS

ORACLE CMD sqlplus SYBASE CMD isql DESCRIPTION sqlplus system/manager@wpgdwh
srvmgr
netasst
dbassist lsnrctl status
create name server WPGNS.wpginvest.com
create server db
78
HP SGBU Services
All 1.0 final
HP Software Support
lsnrctl start lsnrctl stop namesctl status sqlplus sql*loader bcp data loading tool, oracle sql*loader is good at getting data in but not so good at getting it out copy in binary/native mode
export/import
bcp n
17.4 Datatypes
Oracle
Sybase
Notes
Char(x)
char(x)
oracle max 255
79
HP SGBU Services
All 1.0 final
HP Software Support
Varchar2(x) Number(s,p)
varchar(x) numeric(s,p ) datetime
max 2000: much larger than Sybase's max of 255
Date
sysdate = getdate() to get current date/time. oracle default dateformat dd-mon-yy (use to_date())
Variable length data upto 2gb in size; only one per table. Can't be put in RI constraints or indexes. Kinda like a Sybase text field, but less functional (Sybase has no text size limit and can have more than one per table)
Long
Raw
binary or image store's a Row's address. No sybase system equivalent < 11.9.2 (however its not static; its internal only)
ROWNUM
column names are typically case insensitive. System data is mostly uppercase. Sequence: a system table that stores a single unique number across a database; like having a master identity value for a whole Server, not just one particular table.
17.5 SQL*PLUS
sql*plus is oracles version of isql. It is far better than isql, but nowhere near as nice as more robust editors like sqsh. Unlike Sybase the commands terminate in ( ; ) instead of ( go) to run a command input from the editor you may also type / If you edit something, it will cut off your last char unless you put in a line at the end consisting of "/" by itself. Sql*plus defaults to autocommit off (where as Sybase defaults on; once you hit "go" the statement is committed to the database). There is an option though to have autocommit on. on some sysetms sqlplus is named sqlplusw
80
HP SGBU Services
All 1.0 final
HP Software Support
syntax sqlplus login/password@server_instance
17.6 sqlplus commands

COMMAND edit edit filename save filename get filename DESCRIPTION calls up a notepad-ish editor to work in opens named file saves sql to your file (eg. save testquery.sql). Only 1 sql command per file. retrieves query from named file, which must contain only 1 sql statement (use start if >1 statement in file) retrieve and run command show errors on previously run command history in sqsh; can recall previous commands by line number piping output through more in sqsh to have null values appear as NULL instead of blanks re-executes the last known command (like hitting go again in sqsh) Stores query results in an operating system file, or sends the file to a printer. * show var show all rerun last command (use numbers otherwise) show settings for variable show list of current settings
start filename show errors l[ist] set pause on/off set null "NULL" / SPO[OL] [file_name[.ext] | OFF | OUT]
81
HP SGBU Services
All 1.0 final
HP Software Support
@filename.ext
runs stuff just as if you did isql I filename.ext
You can have defined Synonyms (Aliases) in Oracle
Metadata: you can store comments on a table and a column right in the database. VERY nice. Imports to the comment field in Erwin too.
Working w/ dates in oracle is a bit clunky; Oracle stores the date and time together like sybase but defaults to only show date. To get the time, you must do something like to_char('date',mm-dd-yy hh:mm:ss) to get the time out. Plus, when inserting any date value not in the exact form 01-jan-99 you'll have to use a to_date reverse engineered function b/c Oracle (unlike sybase) cannot interpret and convert dates on the fly. [ Also, times are only stored to the second -- no miliseconds. Not a big deal for many apps, but scientific applications frequently care. :-) ]
create (or replace) [object] feature: nice.
- create table statement has many different options compared to Sybase. You can cap the size of a table w/ Maxextents option. "create table as select.." == select into table creation. [ Try not to think of the various SIZE and MAX parameters as options, even though they are. If you don't define values, then Oracle creates objects with the defaults set up for the tablespace, which can cause performance problems or waste space.]
82
HP SGBU Services
All 1.0 final
HP Software Support
- You will need to use the Oracle copy command to dynamically copy a table, where in Sybase you used select into. - alter table modify column feature (though you can't change datatype or shrink the size unless its empty). You also cannot drop the column this way in 8.0 - decode: a simulated if/then/else. Has for/loop and while/loop constructs. When/then construct for errorhandling. - dynamic sql: exists! very straightforward. - grant, drop table, truncate, insert, update, delete all work the same more or less; different options of course. - drop table cascade constraints: very nice way to not worry about the foreign key constraints on a table when dropping it. - Create table; you specify space allocation at create table time. As you extend, Oracle will allocate more space, but this leads to extreme fragmentation.
SYBASE COMMAND sp_help [tablename] select user_name() sp_password sp_rename sp_who sp_helpcode [view_name] print "text" convert(varchar,num)
ORACLE COMMAND desc[ribe] show user password (pre Oracle 8.0; alter user) alter table a rename to b OR rename a to b select * from v$session select * from user_view prompt "text" to_char(num)
83
HP SGBU Services
All 1.0 final
HP Software Support
select a+b (string concatonation) convert(datetime,num)
select a||b (string concatonation) to_date(string,format) eg. to_date(10/14/87,mm/dd/yy)
17.7 ADMINISTRATION
Oracle typically stores in all data and log devices in flat files Oracle startup files : init${ORACLE_SID}.ora and config${ORACLE_SID}.ora THE INITWPGDWH.ORA FILE (FOR SID WPGDWH) IS IN /PRODUCT/8.1.7/DBS THIS IDENTIFIES CONTROL FILE AS WELL AS SPECIFIC ITEMS (MEMORY, NUMBER FILES ETC NECESSARY FOR RUNNING) Generally, my impression is that NOBODY uses direct oracle commands to administer their server the syntax is error prone and the number of options is large. Tools like dbartisan are required for oracle administration (actually, the other side of this is that oracle comes with a large body of administration tools.) dbastudio - not bad good schema generator,
SYSTEM TABLES V$LOG V$LOGFILE V$DATAFILE
84
HP SGBU Services
All 1.0 final
HP Software Support
alter database rename file xxx to yyy
# change any file except control file
alter database add login group filename size 24M; alter database open;
17.7.1 HOW TO STOP ORACLE

# the following script moves to the next log file prior to shutting down $ORACLE_HOME/bin/svrmgrl << EOF connect internal alter system switch logfile; shutdown immediate; exit EOF
17.7.2 STARTUP COMMANDS
# oracle can startup in 2 modes restricted (singluser/dbo) or normal
# start restricted svrmgrl << EOF connect internal startup restrict
85
HP SGBU Services
All 1.0 final
HP Software Support
exit EOF
# start normal svrmgrl << EOF connect internal startup exit EOF
# STARTUP LISTENER lsnrctl start
# STARTUP NAMES namesctl start
# TEST YOUR STARTUP sqlplus
system/manager@WPGDBH
select * from dba_tablespaces;
86
HP SGBU Services
All 1.0 final
HP Software Support
17.7.3 DEFAULT LOGINS

SYS SYSDBA this sa login default password is manager
17.7.3.1
ROLES IN ORACLE
DBA SYSTEM can assign roles, add/remove users, and modify disks etc subrole of dba
RESOURCE allow users to create their own sets of tables and relate them to a schema. The user who creates a table is the owner of the table CONNECT allow user to connect to database (cant create tables though)
17.8 Frequently Asked Questions
How do you get data OUT of Oracle? Can Sql*Loader do it? You can spool file output in Sql*plus... but [ Nope. Spooling the output from Sql*Plus is it. SQL*Loader is just what its name implies -- a loading utility. :-( ]
What is the Oracle directory structure; where do important files reside? where does the server start from? [Start looking at $ORACLE_HOME and $ORACLE_BASE. Then check out the $HOME directory for the "oracle" user -- it is frequently different than $ORACLE_HOME and $ORACLE_BASE. ]
87
HP SGBU Services
All 1.0 final
HP Software Support
18 Appendix B: OVPI Default Port Numbers

Source: (Unknown HP Author) OVPI Port Numbers.htm
OVPI uses the following default port numbers for its components: OVPI database The OVPI Database stores and processes raw and summarized data and it uses following port number for communication. Sybase uses port 5000 (Windows) or port 2052 (UNIX). Oracle uses port 1521 (Windows and UNIX). [These port numbers can be any port number higher than 1023] Web Access Server The Web Access Server provides configuration information to Report Viewer and Report Builder, report viewing, scheduling, and administrative functions through a Web browser. The Web Access Server uses port 80 when SSL is not enabled or port 443 when SSL is enabled. This port is also used for remote OVPI administration through the Management Console. If you are using the Web Access Server, you will also need to use the Management Server, which also uses port 80 (the default). The Management Server is used by remote OVPI processes to gather information about the current system in order to configure it. It is primarily used by the Management Console, but other OVPI applications use it also. If port 80 is not available, then you can specify any other free port for Web Access Server. OVPI SOAP interface also uses the Web Access Server port. Port numbers used for OVPI and Performance Agent (3.x) Communication across firewall OVPI communicate with Performance Agent (3.x) across the firewall using following TCP ports.
88
HP SGBU Services
All 1.0 final
HP Software Support
Firewall
dced/rpcd/rpcss/135 OVPI/ any unused port >1023 Rep_server/ [REP_SERVER_PORT_RANGE]
Edit the /etc/services [UNIX] file or Edit the \%windir%\system32\drivers\etc\Services [Windows] file to configure the port numbers for AGDBSERVER_PORT and REP_SERVER_PORT_RANGE. agdbserver cccc/tcp rep_server dddd/tcp where cccc and dddd specify unused port numbers. agdbserver and rep_server will register themselves at the specified port numbers. If there are multiple data sources configured in the perflbd.rc file, then the first rep_server will use the "dddd" port number specified above. All other rep_server services will add one to the last used port number. Port numbers used for OVPI and Performance Agent (4.x) Communication across firewall OVPI communicate with Performance Agent (4.x) across the firewall using following HTTP/HTTPs ports.
89
HP SGBU Services
All 1.0 final
HP Software Support
Firewall
BBC/383 OVPI/ any unused port >1023 Optional Coda Port/ Any Unused Port >1023
19 Appendix C: Advanced Baseline Procedures

The following Information was provided by HP CPE and is intended to be used for additional base lining on advanced installations: Source: Majumdar, Shamik - System Performance analysis using Glance.ppt
90
HP SGBU Services
All 1.0 final
HP Software Support
[OVPI] System Performance Analysis using Glance
-Shamik Majumdar
2003 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice
Objectives
Glance
Overview Demo on Glance Best Practices in System Performance Analysis Bottleneck Identification using Glance
September 07
page 2
91
HP SGBU Services
All 1.0 final
HP Software Support
Performance Tools Product Overview

Glance Intro
Performance Manager
central management system on-demand performance graphs & tables for usage trend analysis and correlation uses near real-time data
Reporter
scheduled summarized performance reports for management analysis stores collected data
GlancePlus
managed node real-time, single-system performance monitoring intelligent rules-based advisor for bottleneck analysis
Performance Agent
data collection, repository & alarm generation performance, resource, and end-to-end transaction response time measurements measures any source on the system
September 07
page 4
92
HP SGBU Services
All 1.0 final
HP Software Support
Glance overview
Glance Intro
Glance answers better than any tool: "what's going on right now? Best-in-industry unix performance diagnostic! Drill-down approach to troubleshooting from bottleneck analysis to detailed visualization of over a thousand performance metrics Both Glance and OVPA share common measurement software, common parm file for application definitions, have similar alarm syntax and default bottleneck alarms Application Response Measurement (ARM) SDK included On Linux and future version 4 releases, Motif mode Glance: gpm renamed xglance
September 07
page 5
Glance and Glance Pak available on most on unix platforms: now including Linux! See Glance product documentation, especially gpms Online Help Guided Tour. Glance 2 user interfaces: charmode glance uses curses: advantage of lower overhead and "adviser-only" capability motif mode gpm (xglance): offers more detail screens, more customizable
93
HP SGBU Services
All 1.0 final
HP Software Support
Glance and OVPA dataflow
OVPM
Reporter/OVPI
OVO
NNM
Glance Motif mode Glance charmode Measurement Interface
Datacomm
Alarming
Data Collection and Management
Local export
Glance
ARM
DSI
OVPA
Instrumented applications
OS performance instrumentation
September 07
page 6
The Measurement Interface (proprietary) feeds both glance user interfaces as well as "scope" data collection of OVPA. Note that OVPA only logs a subset of the full metric set available to glance in order to optimize overhead. See the "Making Your Glance Pak Perform" whitepaper (reference slide) for more information.
94
HP SGBU Services
All 1.0 final
HP Software Support
Global, Application, and Process data

Metric Classes
Global metrics reflect system-wide activity (sum of all applications). Process metrics reflect specific per-process (including thread) activity. Application metrics sum activity for a set of processes. They keep track of activity for all processes, however short lived, even if they are not reported individually. Glance updates all metric values at the same time. MeasureWare summarizes Global, Application, and other class data over 5-minute intervals and summarizes Process data over 1-minute intervals. MultiProcessor effects: Global and Application CPU percentages reflect normalization over the number of processors (percentage of availability for entire system). Process and Thread-level CPU percentages are not normalized by the number of processors.
September 07
page 7
Important to understand interrelationships between metric classes.
95
HP SGBU Services
All 1.0 final
HP Software Support
Glance adviser
Adviser
MeasureWare
logs a subset of the available performance metrics based on the ones most relevant to longer-term analysis. adviser syntax can be used to access and log any metric. can make glance running in -adviser_only mode to log data for historical analysis.
Glance
You
September 07
page 8
Specific example for getting BYCPU metrics logged is in the paper. Several fairly complex examples are in the /opt/perf/examples/adviser directory in C.02.30 and later releases.
96
HP SGBU Services
All 1.0 final
HP Software Support
Demo
on Glance
September 07
page 9
Best Practices followed in System Performance
Understand the baseline [ what is normal for your application?] Understand the structure of your application
If using RAW logical volumes why bother Buffer Cache Size ?
Define the symptoms. [Do not use subjective complaints, use quantifiable items, like response time is slow or throughput is less etc] High utilization does not mean bottleneck. Check the Recommended Patch List Change one thing at a time Use top down approach while using glance.
September 07
page 10
97
HP SGBU Services
All 1.0 final
HP Software Support
Command line tool

Very
useful is sars report if you do not have Glance

Run sar -o sarfile 5 60 command. This takes 60 samples, each one five seconds apart (for a total of five minutes), saving the binary output to sarfile. sar puts very little load on the system, so taking these samples is not disruptive for the users. sar prints the CPU utilization at each sample, while saving the system data in sarfile. Run sar -Af sarfile >sar.report command to create a Report from sar output. Check for all the system related information from these report.
September 07
page 12
98
HP SGBU Services
All 1.0 final
HP Software Support
System Resources
CPU
Memory
Disk
Network
September 07
page 11
99
HP SGBU Services
All 1.0 final
HP Software Support
CPU Bottleneck
CPU
Symptoms from Glance

Consistent high CPU utilization (GBL_CPU_TOTAL_UTIL > 90%) AND High Run Queue (GBL_RUN_QUEUE > 3) OR Processes blocked on Priority (PROC_STOP_REASON = PRI) Sar u 5 5, if %idle is equal to 0 and runq-sz (sar -q 5 5) is frequently high around 5, system is CPU bound. Use top or ps to see top cpu usage Use uptime command and check the Load Average values. These value represents average run queue over last 1, 5 and 15 min. Do you have the baseline knowledge to identify the offending processes ? Check where CPU is spending most part of the time [sys or user mode?]
page 13
Symptoms from Command Line

Actions

September 07
Consistent high cpu utilization and there is always some processes waiting to RUN. Run Queue represents average of how many processes are runnable in each processor. PRI_QUEUE represents average number of processes blocked on CPU Resource top -d 1 -s 1 -n 5 -f /tmp/top.$$ ; tail -n 5 /tmp/top.$$ | awk '{if ($(NF-1) > 40) print $(NF-1)," ", $NF} ; rm /tmp/top.$$
100
HP SGBU Services
All 1.0 final
HP Software Support
System CPU Bottleneck

CPU
CPU bottleneck symptoms AND Most of time spent in Kernel (GBL_CPU_SYS_MODE_UTIL > 50%) Execute sar command and see the % of time spent in User/System mode [sar u 5 5] Is it due to heavy context switching? Is GBL_CPU_INTERRUPT_UTIL > 30% [IO bottleneck] Is Memory bottleneck showing up as CPU bottleneck ? Memory is fully utilized with high page outs. If none of the above, check the processes which are spending most of its time in Kernel Mode. Select the top process and run tusc./truss on that process to list the system calls that are made If third party application, can not do much else fine tune your appl.
page 14
Symptoms from command line
Actions

September 07
If GBL_CPU_INTERRUPT_UTIL > 30% then you have a IO bottleneck Choose PROC_CPU_SYS_MODE_UTIL metric to sort on Processes spending most of the time in SYS Mode will be listed. Select the top process from the list and pull down the process system call report and see the syscalls that the process is using O/p of sar u 5 5 05/11/05
SunOS ovpisunt7 5.8 Generic_108528-22 sun4u
19:18:30 19:18:35 19:18:40
%usr 100 100
%sys 0 0
%wio %idle 0 0 0 0 101
HP SGBU Services
All 1.0 final
HP Software Support
19:18:45 19:18:50 19:18:55 Average
48 0 0 50
4 0 1 1
43 0 0 9
5 100 99 41
Context Switching Bottlenecks

CPU

System CPU bottleneck AND Lots of CPU time spent in Context Switching (GBL_CSWITCH_UTIL > 30%)
Symptoms from command

sar w 5 5 and check for large context switch rates [If switch rate > syscall rate, you have trouble ]
Actions
Check if it has a very high fork rate. Monitor GBL_STARTED_PROC over time ~ 1000 [sar c 5 5 and check for large fork rate] Check whether Semaphore contention is happening [sar m/PROC_STOP_REASON=SEM] Change Kernel time slice parameter value to 20 ms from 10ms.
September 07
page 15
Context Switch can occur if currently executing process goes on for sleep Or, CPU is scheduled to process some more priority process. Often by lengthening the amount of time each process can hold a CPU, we can decrease scheduler thrashing
102
HP SGBU Services
All 1.0 final
HP Software Support
User CPU Bottlenecks

CPU
Symptoms

CPU bottleneck symptoms GBL_CPU_USER_MODE_UTIL > 50%
Execute sar command and see the % of time spent in User/System mode [sar u 5 5]
Actions

Check for infinite looping
If process is spending lot of time in user mode making no system call
Check the priority of your processes [use nice command to tune the priorities] Add more processing power
September 07
page 16
103
HP SGBU Services
All 1.0 final
HP Software Support
Disk Bottlenecks
DISK

Consistent high utilization on at least one disk device (GBL_DISK_UTIL_PEAK or highest BYDSK_UTIL > 50%) Significant queuing lengths (GBL_DISK_SUBSYSTEM_QUEUE > 3 or any BYDSK_REQUEST_QUEUE > 1). Processes or threads blocked on I/O wait reasons (PROC_STOP_REASON = CACHE, DISK, IO). Sar d 5 5, if %busy for any disk is > 50 % or avque >=2, disk is a problem. If the busiest disk is a swap device, you have a memory bottleneck [configure swap properly] sort the process list on PROC_DISK_PHYS_IO_RATE, select the process that is doing the most IO check open files. Watch for IO abuse by applications, if BYDSK_SYSTEM_IO_RATE is high. Move some filesystems to the less busy disks.
page 17
Actions

September 07
I/O Abuse applications that do open and close many number of files. If the busy disk is doing system io most of the time, that is a problem.
104
HP SGBU Services
All 1.0 final
HP Software Support
Buffer Cache Bottlenecks

Buffer

Moderate utilization on at least one disk device (GBL_DISK_UTIL_PEAK or highest BYDSK_UTIL > 25) GBL_MEM_CACHE_HIT_PCT < 90% Processes or threads blocked on Cache (PROC_STOP_REASON = CACHE)
sar -b 5 5 if %rcache is frequently less than 90% - the buffer cache is too small.
Actions
Fine tune the buffer cache size [probably use Dynamic Buffer Cache using dbc_min_pct and dbc_max_pct]
September 07
page 18
105
HP SGBU Services
All 1.0 final
HP Software Support
Memory Bottlenecks
MEM
High physical memory utilization (GBL_MEM_UTIL > 95%) AND Significant page out rate (GBL_MEM_PAGEOUT_RATE > 1) OR Any deactivations (GBL_MEM_SWAPOUT_RATE > 0) Processes or threads blocked on virtual memory (GBL_MEM_QUEUE > 0 or PROC_STOP_REASON = VM) sar q 5 5, if swpocc (percent swap queue is occupied) is greater than 5 the system is memory bound. sar w 5 5, if swpot/s (number of processes swap out per sec) is frequently greater than 1, system is memory bound Ignore Pageins, watch out page outs and Deactivations Check for overly large filesystem buffer cache size (>400 MB) Check memory leaks by application [ DATA region size grows up]
page 19

Actions

September 07
When a program touches a virtual address on a page that is not in physical memory, the result will be a "page in. When the HP-UX needs to make room in physical memory, or when a memory mapped file is posted, the result will be a "page out. Process deactivation occurs when the system is thrashing or when the amount of free memory falls below a critical level. use Glance and sort on PROC_MEM_VIRT to see the processes using most memory
106
HP SGBU Services
All 1.0 final
HP Software Support
Why Deactivation?
process priority:
the lower the process priority (meaning the higher the nice value), the more likely it will be deactivated.
process state:
a process that has been sleeping or has been in memory for some time is likely to be deactivated. A process deactivated for a while and now ready to run is likely to be reactivated. A daemon process (one that works continuously) or one marked for serialization is more likely than an interactive process (one that works in spurts) to be deactivated. Interactive processes are more likely to be reactivated than batch or serialized processes.
process type:
September 07
page 20
Swap

Gets used when Page Outs Occur. Reserved whenever Virtual memory is allocated. Need to have at least as much swap as appl will ever need Best Practice Setup two fixed Device Swap of same size on different physical disks with same Swap Priority. Use swapmem_on kernel parameter to use memory as swap also. Swapping becomes fast but memory is utilized for swapping !!
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME dev 1048576 0 1048576 0% 0 - 1 /dev/vg00/lvol2 reserve - 690724 -690724 memory 1564888 1183884 381004 76%
September 07
page 21
107
HP SGBU Services
All 1.0 final
HP Software Support
A temporary form of data storage is swap space. It should be noted that HP-UX does not "swap" any more, it pages and, as a "last resort" deactivates processes. The process of deactivation replaces what was formerly known as swapping entire processes out.
Networking Bottlenecks
Symptoms from Glance High (dependent on configuration) network packet or byte rates (GBL_NET_PACKET_RATE or specific BYNETIF_IN_BYTE_RATE or BYNETIF_OUT_BYTE_RATE > 2*average). AND
Any Output Queuing (GBL_NET_OUTQUEUE > 0).
OR
Higher than normal number of processes or threads blocked networking (PROC_STOP_REASON = NFS, LAN, RPC, Socket (if not idle)). One CPU with a high System mode or Interrupt CPU utilization while other CPUs are mostly Idle (BYCPU_CPU_INTERRUPT_UTIL > 30).
September 07
page 22
108
HP SGBU Services
All 1.0 final
HP Software Support
Networking Bottleneck
Symptoms from Command line Run the command netstat s twice,spaced 30 seconds apart. Look at the change in tcp sent data packet retransmissions, and any udp socket overflows.
Actions

If using, NFS, use nfsstat command to monitor traffic on client/server check for retransmits and timeouts. Can run glance on NFS clients to check which processes are causing problems. Get NNM Reports for more details
September 07
page 23
109

Ovpi Master 51 BP v10

Hochgeladen von

Dokumentinformationen

Originalbeschreibung:

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Ovpi Master 51 BP v10

Hochgeladen von

Copyright:

Verfügbare Formate

HP SGBU Services

Title Description File Owner

OVPI_Master_51_BP_v10.doc Michael P. Davies

Category Version Status

All 1.0 Final

HP Software OVPI Project Deployment Best Practices

Authors Editor/ Primary Contributor Version Date Status

Multiple Michael Davies 1.0 10/31/2007 Final

OVPI_Master_51_BP_v10.doc Michael P. Davies

Category Version Status

All 1.0 Final

OVPI_Master_51_BP_v02.doc Michael P. Davies

Category Version Status

All 1.0 final

Database configurations ..........................................................................................................18

OVPI Installation ................................................................................................................................23

OVPI_Master_51_BP_v02.doc Michael P. Davies

Category Version Status

All 1.0 final

Installation Process ..................................................................................................................23

OVPI_Master_51_BP_v02.doc Michael P. Davies

Category Version Status

All 1.0 final

14.1 14.2 15 15.1 15.2 16 16.1 16.2 16.3

17 17.1 17.2 17.3 17.4 17.5 17.6

OVPI_Master_51_BP_v02.doc Michael P. Davies

Category Version Status

All 1.0 final

OVPI_Master_51_BP_v02.doc Michael P. Davies

Category Version Status

All 1.0 final

3.2 Intended Audience

3.3 Document Layout

3.4 Outline of Document Topics including Intended audience

OVPI_Master_51_BP_v02.doc Michael P. Davies

Category Version Status

All 1.0 final

OVPI_Master_51_BP_v02.doc Michael P. Davies

Category Version Status

All 1.0 final

4.2 Technical Summary of Limiting Factors

OVPI_Master_51_BP_v02.doc Michael P. Davies

Category Version Status

All 1.0 final

4.3 Performance and System Utilization

OVPI_Master_51_BP_v02.doc Michael P. Davies

Category Version Status

All 1.0 final

5.2 Project Scheduling

OVPI_Master_51_BP_v02.doc Michael P. Davies

Category Version Status

All 1.0 final

OVPI_Master_51_BP_v02.doc Michael P. Davies

Category Version Status

All 1.0 final

OVPI Project Schedule Case study

Satellite Satellite1 Satellite2 Satellite3 Satellite4 Satellite5 Total Nodes

# Managed Elements 30000 30000 30000 30000 30000 150000

Architecture Decision Process

OVPI_Master_51_BP_v02.doc Michael P. Davies

Category Version Status

All 1.0 final