Spotlight

Spotlight on Oracle 8.
1
Getting Started Guide
2012 Quest Software, Inc.

ALL RIGHTS RESERVED.
This guide contains proprietary information protected by copyright. The software described in
this guide is furnished under a software license or nondisclosure agreement. This software may be
used or copied only in accordance with the terms of the applicable agreement. No part of this
guide may be reproduced or transmitted in any form or by any means, electronic or mechanical,
including photocopying and recording for any purpose other than the purchasers personal use
without the written permission of Quest Software, Inc.
The information in this document is provided in connection with Quest products. No license,
express or implied, by estoppel or otherwise, to any intellectual property right is granted by this
document or in connection with the sale of Quest products. EXCEPT AS SET FORTH IN
QUEST'S TERMS AND CONDITIONS AS SPECIFIED IN THE LICENSE AGREEMENT FOR
THIS PRODUCT, QUEST ASSUMES NO LIABILITY WHATSOEVER AND DISCLAIMS
ANY EXPRESS, IMPLIED OR STATUTORY WARRANTY RELATING TO ITS PRODUCTS
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTY OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NONINFRINGEMENT. IN NO EVENT SHALL QUEST BE LIABLE FOR ANY DIRECT,
INDIRECT, CONSEQUENTIAL, PUNITIVE, SPECIAL OR INCIDENTAL DAMAGES
(INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF PROFITS, BUSINESS
INTERRUPTION OR LOSS OF INFORMATION) ARISING OUT OF THE USE OR
INABILITY TO USE THIS DOCUMENT, EVEN IF QUEST HAS BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES. Quest makes no representations or warranties with respect
to the accuracy or completeness of the contents of this document and reserves the right to make
changes to specifications and product descriptions at any time without notice. Quest does not
make any commitment to update the information contained in this document.
If you have any questions regarding your potential use of this material, contact:
Quest Software World Headquarters
LEGAL Dept
5 Polaris Way
Aliso Viejo, CA 92656
email: legal@quest.com
Refer to our Web site (www.quest.com) for regional and international office information.
Patents
Protected by U.S. Patents #6,901,582, 7,784,027 and 8,015,454. Additional patents pending.
Trademarks
Quest, Quest Software and the Quest Software logo are trademarks and registered trademarks of
Quest Software, Inc in the United States of America and other countries. For a complete list of
Quest Softwares trademarks, please see http://www.quest.com/legal/trademark-information.aspx.
Other trademarks and registered trademarks are property of their respective owners.
Spotlight on Oracle 8.1

Getting Started Guide
Friday, 16 March 2012
Table of Contents
Welcome
Oracle Architecture Overview
Spotlight on Oracle Overview
Home Page
Alarms
Drilldowns
Features of Spotlight
10
Features Common to Spotlight Applications
10
Features Specific to Spotlight on Oracle
10
Features Specific to Spotlight on Oracle RAC
10
Predictive Diagnostics
10
DTrace Analysis (Unix Solaris 10 And Later)
11
Install Spotlight
12
Requirements
12
Steps To Install
13
Multiple Concurrent Installations
14
MSI Installers for Spotlight on Oracle
14
Uninstall Spotlight
14
Start Spotlight
15
Project Lucy
15
Authorize Spotlight
16
Spotlight Users
17
Oracle User Wizard
Spotlight on Oracle
Connect to Oracle Databases
17
20
20
Spotlight on Oracle Getting Started Guide
Table of Contents
Add A New Connection
20
Toad DBA Suite for Oracle
25
Install And Configure DTrace Scripts
26
Spotlight on Oracle Home Page
26
Sessions Panel
27
Host Panel
28
Server Processes Panel
28
SGAPanel
29
Background Processes Panel
30
Disk Storage Panel
31
Spotlight on Oracle Alarms
31
Spotlight on Oracle Drilldowns
39
Spotlight Tools
42
Tools |Explain Plan
42
Tools |SQLOptimizer
42
Tools |Space Manager
43
Tune Your Database
43
Predict Database Performance
45
Open Predictive Diagnostics
45
Predictive Diagnostics for Beginners
45
Predictive Diagnostics And Spotlight
46
Requirements for Predictive Diagnostics
46
Spotlight on Oracle RAC
48
What is Spotlight on Oracle RAC?
48
What Is Oracle RAC?
48
Connect to Spotlight on Oracle RAC
49
Spotlight on Oracle RAC Home Page
54
Table of Contents
Interconnect and Global Status Panel
55
Instance Panels
55
I/O Subsystem Panel
56
Spotlight on Oracle RAC Alarms
56
Alarms on A Database Instance
57
Alarms on The Oracle RAC
57
Spotlight on Oracle RAC Drilldowns
60
Troubleshooting Spotlight on Oracle
62
Product AuthorizationErrors
62
Connection Problems
63
Cannot Connect to Database
63
Error Attempting to Connect
63
Retrieve Data from Unix
64
Cannot Connect to ASM
64
Drilldown Problems
64
Alarm Help Displayed Instead of Drilldown
64
Dead Sessions in Top Sessions Drilldown
65
What Happens to The Operating System Drilldown when Connection Is Lost to The
Server?
65
Operating System Statistics Not Reported
65
Predictive Diagnostics Unavailable
66
High Spotlight Load on Database Server
66
Appendix: Contact Quest
67
Contact Quest Support
67
Contact Spotlight Support
67
Contact Quest Software
68
About Quest Software
68
Index
69
1
Welcome
Spotlight on Oracle is Quest Softwares powerful diagnostic and problem-resolution tool for
Oracle databases. Its unique user interface provides you with an intuitive, visual representation of
the activity on your database and operating system.
Oracle Architecture Overview

The following diagram illustrates some of the basic components of an Oracle server at the
memory, database and disk layers. This architecture has been used to design the Spotlight on
Oracle home page.
The numbers indicate the order of the flow of information.
The numbered labels in the Oracle architecture diagram correspond to the following activities:
1. The client program (for example, SQL*PLUS, Oracle Power Objects, or some other tool)
sends a SELECT statement to the server process.
2. The server process looks in the shared pool for a matching SQL statement. If none is
found, the server process parses the SQL and inserts the SQL statement into the
shared pool.
3. The server process looks in the buffer cache for the data blocks required. If found, the
data block must be moved on to the most recently used end of the Least Recently
Used (LRU) list.
Welcome
4. If the block cannot be found in the buffer cache the server process must fetch it from the
disk file. This requires a disk I/O.
5. The server process returns the rows retrieved to the client process. This may involve some
network or communications delay.
6. When the client issues the UPDATE statement the process of parsing the SQL and
retrieving the rows to be updated must occur. The update statement then changes the
relevant blocks in shared memory and updates entries in the rollback segment buffers.
7. The update statement also makes an entry in the redo log buffer that records the
transaction details.
8. The database-writer background process copies modified blocks from the buffer cache to
the database files. The Oracle session performing the update does not have to wait for
this to occur.
9. When the COMMIT statement is issued the log writer process must copy the contents of
the redo log buffer to the redo log file. The COMMIT statement does not return control to
the Oracle session issuing the commit until this write is complete.
10. If running in ARCHIVELOG mode, the archiver process copies full redo logs to the
archive destination. A redo log is not eligible for re-use until it has been archived.
11. At regular intervals, or when a redo log switch occurs, Oracle performs a checkpoint. A
checkpoint requires all modified blocks in the buffer cache to be written to disk. A redo
log file cannot be re-used until the checkpoint completes.
Recommended Reading
The best information to help you tune and configure Oracle databases can be found online. Use
as your first point of reference web sites such as:
l
The Quest Software Oracle Solutions Web site.
The Oracle home page.
Spotlight on Oracle Overview

Spotlight on Oracle makes it possible for you to view your database at different levels of detail.
Home Page
The Spotlight home page is a visual representation of the status of the major components of the
database. The detail at this level is designed to help you locate and identify bottlenecks as they
develop. Additionally, Spotlight provides tools that allow you to identify and rectify
inefficiency in your databases configuration.
If Spotlight detects that some component of the Oracle database constitutes a performance
bottleneck, the corresponding component generates a visual or auditory alarm. The exact
appearance of the alarm depends on how you have configured alarm severities. When an alarm is
current you can press F1 to display help for the alarm. Clicking the component (on the home
page ) displays the relevant drilldown.
Welcome
See "Spotlight on Oracle Home Page" (page 26) for more information.
The home page display is based on the Oracle database architecture. See "Oracle Architecture
Overview" (page 7) for more information.
Alarms
Spotlight alarms indicate when the Oracle database exceeds acceptable performance thresholds.
See "Spotlight on Oracle Alarms" (page 31) for more information.
Drilldowns
Once a problem is isolated you can display a drilldown page with charts and tables that provide
a detailed breakdown of the underlying statistics.
For example, use the Activity drilldown to display overall efficiency and resource usage. In
particular, the Event waits chart shows the amount of time database sessions spend waiting for
various resources, as well as the amount of CPU being utilized. Efforts at tuning the database are
most likely to succeed when they concentrate on the resources being used most heavily and the
Wait activity topic contains a discussion of the meaning and implications of various wait events.
Other pages help you examine detailed activity. For instance, the Top Sessions drilldown
identifies the users who contribute most to database activity. You can display various details for
these sessions, including the SQL statements, locks and resource usage.
See "Spotlight on Oracle Drilldowns" (page 39) for more information.
10
Welcome
Features of Spotlight
Features Common to Spotlight Applications
Spotlight is designed for ease of installation and use, and offers a range of tools for problem
diagnosis. Features that are common to most Spotlight applications include:
l
Easy installation.
The display of real-time data.
Comprehensive information in charts and tables.
Visible, audible and configurable alarms.
Context-sensitive help.
Calibration.
Recording and playback of historical data.
Common Spotlight technology. Spotlight on Oracle can run within the same Spotlight
console as several other Quest Spotlight products.
Features Specific to Spotlight on Oracle

l
The ability to add new Spotlight on Oracle users, and to convert the accounts of existing
Oracle users so they can use Spotlight on Oracle.
Access to a number of other Quest applications for viewing detailed information on the
specified Oracle database.
Features Specific to Spotlight on Oracle RAC

l
The ability to verify connection details for Oracle RAC instances.
The ability to open Spotlight on Oracle from Spotlight on Oracle RAC.
Predictive Diagnostics allows Spotlight to collect and analyze performance metrics for:
l
SQL statements as they are executed on the database instance.
System bottlenecks as they affect database throughput and response time.
Database resources as they reach their full capacity and affect database throughput and
concurrency.
When enough performance data is collected, Spotlight may be able to predict the future
performance for individual SQL statements on that database, for potential bottlenecks in the
system, and for database CPU, memory, and disk I/O resources.
11
Welcome
Note: When you start using Predictive Diagnostics, Spotlight will take about 10 days to collect
enough data to make valid predictions on the future performance of the database.
DTrace Analysis (Unix Solaris 10 And Later)

DTrace is a dynamic analysis tool for Sun Microsystems operating systems. It was first
introduced in the Solaris 10 operating system, and can be used ONLY on Sun platforms from
Solaris 10 onwards.
Spotlight and DTrace
Spotlight on Oracle uses DTrace to capture details of the interaction between Oracle and the
operating system.
When Oracle sessions use CPU time, most of that elapsed time is spent executing Oracle code.
However, Oracle sometimes uses system calls to perform functions such as:
l
Allocating or deallocating private (PGA) memory.
Synchronizing activities with other processes through semaphores or other IPC

mechanisms.
Allocating shared memory (SGA).
Performing I/O operations on datafiles, redo logs, raw devices, and so on.
Capturing CPU or elapsed timing to support the time model and wait interfaces.
Spotlight uses DTrace to investigate these system calls to calculate where that elapsed time is
being spent.
Install and Configure DTrace scripts
Before you can capture the details of the interaction between Oracle and the Solaris operating
system, you need to Install And Configure DTrace Scripts (page 26).
2
Install Spotlight
Requirements
Windows 7 / Windows Vista / Windows Server 2008 :
Section
Requirement
User Privileges
Ensure the Windows user installing Spotlight has

Windows Administrator privileges.
Folder Permissions
During installation, Spotlight requires write access to

the following:
l
The Spotlight installation folder (by default,

Program Files\Quest Software\Spotlight)
Program Files\Common Files\Quest Shared
ProgramData\Quest Software\Spotlight
Users\<current user>\AppData\Roaming\Quest
Software\Spotlight
Note:For 64-bit operating systems the Program Files

directory is Program Files (x86).
Permissions to Windows
registry keys and sub-keys

the following:
l
HKEY_LOCAL_
MACHINE\SOFTWARE\Wow6432Node\Quest
Software\Products
HKEY_LOCAL_
MACHINE\SOFTWARE\Wow6432Node\Quest
Software\Spotlight
HKEY_CLASSES_ROOT
13
Install Spotlight
Windows XP / Windows Server 2003:

Section
Requirement
User Privileges
Ensure the Windows user installing Spotlight has

Windows Administrator privileges.
Folder Permissions

the following:
Permissions to Windows registry

keys and sub-keys
The Spotlight installation folder (by default,

Program Files\Quest Software\Spotlight)
Program Files\Common Files\Quest Shared
Documents and Settings\All Users\Application

Data\Quest Software\Spotlight
Documents and Settings\<current

user>\Application Data\Quest
Software\Spotlight

the following:
l
HKEY_LOCAL_MACHINE\SOFTWARE\Quest
Software\Products
HKEY_LOCAL_MACHINE\SOFTWARE\Quest
Software\Spotlight
HKEY_CLASSES_ROOT
Steps To Install
To install Spotlight from CD:
1. Close all open applications on your PC.
2. Insert the Spotlight Installer CD into your CD-ROM drive.
If the installation program does not start select Windows Start | Run
l
For Spotlight delivered as part of the Toad DBA Suite, enter the command
d:\TOAD DBA Suite for Oracle vvv.exe (where d: is the CD-ROM identifier and
vvv is the Spotlight version).
For Spotlight delivered as a standalone installer, enter the command

d:\SpotlightonOracle_vvv.exe (where d: is the CD-ROM identifier and vvv is the
Spotlight version).
3. Follow the instructions on the screen to complete the installation.
14
Install Spotlight
To install Spotlight from the downloaded Quest Spotlight installer from the Quest Web site:
1. Close all open applications on your PC.
2. Execute the downloaded installer.
3. Follow the instructions on the screen to complete the installation.
Note:To cancel installation at any time, press Esc or click Cancel.
Multiple Concurrent Installations

Many Spotlight applications can run within a single Spotlight installation and you can install
and separately upgrade multiple copies of Spotlight on your computer.
Note:If Spotlight is delivered as part of the Toad DBA Suite and you require multiple Spotlight
installations on your computer then contact Quest Software for more information.
MSI Installers for Spotlight on Oracle

MSI installers are available for Spotlight on Oracle. These installers can be run silently, and
produce an installation that can be captured and re-used via automatic deployment techniques.
Contact Quest Software for more information.
Uninstall Spotlight
To remove the application and all associated files from your machine, if required.
To uninstall Spotlight
1. Open Windows Control Panel.
2. Select Programs and Features. (On Windows XP select Add/Remove Programs.)
3. Click Spotlight on Oracle | Remove | Yes.
15
Start Spotlight
Start Spotlight
To start Spotlight:
Double click on the Spotlight desktop icon
or
On the Windows task bar, click Start | Programs |Quest Software | Spotlight |
Spotlight.
During operation
Spotlight requires write access to the following:
Windows Server 2008, Windows Server
2003, Windows Vista, Windows 7
Software\Shared
Software\Spotlight
Windows XP

user>\Application Data\Quest Software\Shared
user>\Application Data\Quest
Software\Spotlight
You should also verify that you can run the required Oracle client on the Spotlight machine.
Project Lucy
The first time you run Spotlight following install you will be prompted to upload data to Project
Lucy. Project Lucy is a free service provided by Quest Software that analyzes your performance
data, gives you reports and insight into your performance and allows you to compare your
experience to everyone else that uses the service. Performance data is uploaded to Project Lucy
once a day.
Actions
To open the Project Lucy home page, in the Spotlight
Browser click Welcome to Spotlight | Spotlight on the
Web | Project Lucy.
Description
l
Learn more about

Project Lucy and how
it can benefit you.
16
Start Spotlight
Actions
To configure uploading data from Spotlight on Oracle to

Project Lucy, click View | Options | Project Lucy.
Description
l
Login to your Project

Lucy account and use
Project Lucy services.
Create a Project Lucy

account.
See "Project Lucy" in the

online help for more
information.
Authorize Spotlight
Spotlight on Oracle is supplied with a time-limited trial key so you may test the product. When
the trial key expires Spotlight on Oracle reverts to a preview mode, which limits access to the
Spotlight on Oracle home page and one connection at a time. If you select to open a drilldown
in preview mode the following message is displayed:
You do not have a license for Spotlight on Oracle that enables
access to this functionality. Please contact Quest Software for
information about obtaining the correct license.
Contact your Quest Software representative to obtain the necessary authorization keys to ensure
Spotlight remains fully functional when the trial key expires.
To enter a new authorization key:
While Spotlight is running
1. Click Help | About Spotlight.
2. Ensure Installed Products is to the front.
3. Locate any Spotlight on Oracle in the list of installed products.
4. Select View / change product license.
5. Click Change this license.
6. Enter the Authorization key.
Notes:
l
The Windows user that updates the authorization key must have administrator access to
the Windows registry and write access to the Console\Licenses sub-folder of the
Spotlight installation folder.
Make sure you enter the authorization key exactly as stated on your Product
Authorization sheet or as provided by your Quest Software distributor. The site message
17
Start Spotlight
is case sensitive. If you enter the authorization key incorrectly, the following message is
displayed: Invalid authorization key
l
If Spotlight is in preview mode and the trial period has not expired, check your computer
system date is set correctly.
Spotlight Users
Spotlight on Oracle connects to the Oracle database through the Spotlight user.
l
You should not use any system accounts. We recommend creating a separate user for
Spotlight or using a personal account.
The Spotlight user must have sufficient privileges, views, and tables to diagnose the
Oracle database. You can view the privileges, views and tables required by Spotlight
users in the Database Objects and Privileges document in the Spotlight installation
folder: \Plug-ins\SoO\.
The Oracle User Wizard configures Spotlight users.
Oracle User Wizard

The Oracle User Wizard configures Spotlight users. Use the Oracle User Wizard to:
l
Add a new user to the Oracle database
Convert the account of an existing user on the Oracle database to use Spotlight
Reconfigure an existing Spotlight user to work with new versions of Spotlight on

Oracle and Spotlight on Oracle RAC. Use this procedure if you have upgraded your
version of Spotlight.
Notes:
l
Spotlight users must be configured for each database they are to diagnose.
You cannot configure Spotlight users for a database while Spotlight is diagnosing
that database.
You must have the user ID and password of the DBA that manages the database to which
you are adding the Spotlight user.
Select Don't show this page in future to hide the first page of the Oracle User Wizard.
To fill in the Oracle User Wizard

The Oracle User Wizard runs automatically when required or to initiate it:
1. Click File | Oracle User Wizard...
2. At the DBA connection credentials page:
18
Start Spotlight
Field
Description
Enter the
SQL*Net
connection
string
Enter the database instance on which you wish to create the user.
Enter the user

name of a
DBA
account
Enter a DBA user name for that database instance (so that the
Spotlight user and the relevant objects can be created). The user
name requires SYSDBA access.
Enter the
password
Enter the DBA user password.
3. Select Create a new user

4. At the New oracle user page:
Field
Description
User name
Enter a name for the new user.
Password
Enter a password for the new user.
Confirm password
Confirm the password.
Select any table
Select to allow the user to generate explain plans.
Alter system
Select to allow the user to trace and terminate sessions.
5. At the Select tablespace page:

Field
Description
Spotlight tables
The default tablespace is USERS, if available.
temporary segments
The default temporary tablespace is TEMP, if available.
6. At the Predictive Diagnostic Settings page:

Select Create/Upgrade Predictive Diagnostics schema to employ Predictive Diagnostics
on this database instance.
7. Select to use an existing Predictive Diagnostics schema (recommended) or install
a new schema.
8. Select to install a set of sample data (which you can use to experiment with the Predictive
Diagnostics feature).
19
Start Spotlight
9. Select the settings Spotlight will use to schedule the Oracle jobs that collect data on SQL
performance and impending bottlenecks.
3
Spotlight on Oracle
Connect to Oracle Databases
1. From the Spotlight Browser select All Connections |Spotlight on Oracle
2. Select the database. If the database is not on the list then add a new connection.
Notes: (Now the database is listed in the Spotlight Browser |Live connections.)
l
Click Set timed statistics now if you are prompted to do so. You see this prompt
when for the Spotlight user the timed_statistics parameter is set to false in the
init.ora file. Spotlight requires this parameter be true for the Spotlight user to
accurately monitor the database. Clicking Set timed statistics now corrects the
problem only if the Spotlight user has Alter System privileges. Ensure the
Spotlight user has sufficient privileges. See "Spotlight Users" (page 17) for more
information.
Spotlight operates at reduced functionality once the initial trial period has expired.
See "Authorize Spotlight" (page 16) for more information.
Add A New Connection

Notes: (Before you add a new connection)
l
Ensure you create a Spotlight user with sufficient privileges to diagnose the Oracle
database. See "Spotlight Users" (page 17) for more information.
Ensure the Unix or Windows server on which the Oracle database is installed is
accessible to Spotlight on Oracle. See the Spotlight on Unix or Spotlight on Windows
Getting Started Guide.
21
Spotlight on Oracle
1. Click File |Connect
2. Select Spotlight on Oracle on the Connections menu.
3. Double-click Add new connection.
4. Ensure Details is selected. Fill in the Properties window.

Oracle connection details
Field
Description
Connection
String
Select the name of the database that you wish to connect to.
Oracle
Username
Enter the Spotlight user name. See "Spotlight Users" (page 17) for
more information.
Note: When selected, the Connection name field resets to this value.
You can then edit the Connection name field if required.
Tip: If you enter a Username that does not exist, Spotlight will
prompt you when you complete this dialog to create a new Spotlight
user.
Oracle
Password
Enter the Spotlight user password. See "Spotlight Users" (page 17) for
more information.
Oracle
Home
Select an appropriate Oracle Home. Select a location where Oracle

client software is installed.
Note:Choose a home that contains a version of the Oracle client that
is compatible with the Oracle server.
ASM connection details

ASM is built into the Oracle database kernel. It simplifies the creation and space
management in Oracle databases.
Note: Available only in Oracle 10g and later.
22
Spotlight on Oracle
Field
Description
Monitor
ASM
Select to monitor the performance of Oracle's Automated Storage

Management (ASM) technology.
ASM
Connection
String
Enter the connection string required to link to the ASM database

instance that is used for storage management for this Oracle database.
ASM
Password
Enter the user password for Spotlight to login to the ASM database
instance.
Note:The TNSNAMES entry for the ASM instance must include the
option (UR = A) for the Spotlight connection to work properly.
The user that Spotlight uses to login to the ASM database is SYS.
This is the only user name that can provide access to ASM-specific
information.
Providing SYS access to an ASM instance does NOT create a security
issue for the Oracle database, but you should limit access to the SYS
account to trusted individuals.
Because the SYS password is stored in encrypted format, nonprivileged users can still access Spotlight on Oracle. To do so:
l
Use a pre-configured or shared Spotlight on Oracle

connection, OR
Use a Spotlight connection without ASM access. Spotlight

will display Oracle-related data, but the ASM drilldown will
contain no data.
Server connection details

Field
Description
Monitor OS
Select to monitor the operating system on the database server.
OS Server
Enter the name of the server.

Tip: Use a period . to monitor the local database.
23
Spotlight on Oracle
Field
Description
OS Type
Select the type of operating system.

l
Windows
Unix(REXEC)
Unix(SSH)
Unix(SSH using Public/Private Keys)
If you select one of the Unix systems, make sure the SSH or REXEC
daemon is running on the database server and is configured to
receive remote connections. Spotlight supports both SSH1 and SSH2
protocols. Public-key encryption is supported under SSH2 only.
SSH Port
Enter the port number for SSH (secure shell) connection to the
database server. The default value is 22.
Note: Applicable when OSType is Unix(SSH) or Unix(SSH using
Public/Private Keys).
OS
Username
Type the user name to login to the database server.

Note: When OSType is Windows:
l
Not required if your login details have been assigned remote

access to the registry of the database server.
Ensure the user has access to the registry of the server.
SSH Key
Type
Select the key type from the options:RSA and DSA.
SSH Private
Key
Filename
Locate the filename that contains the private key for the Spotlight
connection.
Note: Applicable when OSType is Unix(SSH using Public/Private

Keys).
Hint: Use the "..." button.

Keys).
SSH
Passphrase
Type the passphrase used to decrypt the private key.
OS
Password
Type the user password to login to the database server (if required).
Connect to
OS
Spotlight
Select to have the database server connection open automatically

when you open the connection to the Oracle database.

Keys).
Note: Not applicable when OSType is Unix(SSH using

24
Spotlight on Oracle
Field
Description
when
starting this
connection
ESX connection details
Note: Requires Server connection details | Monitor OS selected.
Field
Description
Monitor
ESX
Select to monitor the virtual machine that hosts the database.
ESX
Web
Service
Type or select the URL of the ESX Web Service that manages the
virtual machine.
ESX
Username
Type the username required to connect to the ESX Web Service.
ESX
Password
Type the password required to connect the specified user to the ESX
Web Service.
Virtual
Machine
Name
Click ... to select the virtual machine.
Note: The username specified must have read only permissions for both
the ESX virtual machine and the ESX host that the virtual machine
resides on, or may reside on.
Stealth Collect
StealthCollect is Quest technology designed to collect SQL statements and performance
indicators from Oracle databases and host operating systems without imposing overheads
on the target database itself.
Note: Requires StealthCollect installed. StealthCollect is installed with Performance
Analysis.
Field
Description
Use
StealthCollect
Select Use StealthCollect to collect SQL statements and

performance data using StealthCollect.
SC Port No.
Enter the port number where Spotlight can access StealthCollect

data.
If required, click the button to enter the StealthCollect username
25
Spotlight on Oracle
Field
Description
and password. (The default StealthCollect username and password
is quest.)
SC Oracle
SID
Enter the StealthCollect Oracle SID for the database.
5. Click Test to verify Spotlight can connect successfully with the details provided.
Note:For connections to virtual machines, testing the connection validates that Spotlight
can connect to the Web Service using the specified username and password combination,
and that the virtual machine exists. It does not check permissions, that the virtual machine
is turned on, or that it is the same machine as the one specified in the Server Connection
Details section.
6. Click Save password details (for this connection) to save all the entered
password details.

Note: This section is relevant for systems with the following software installed: Toad DBASuite
for Oracle with Spotlight on Oracle.
To connect to an Oracle database from within Toad
1. Click the Spotlight button where it appears in Toad.
2. Spotlight on Oracle opens at the Oracle database connection associated with the Toad for
Oracle connection.
The first time you open the Oracle database connection from within Toad
You are required to fill in the Details page with the following additions. See "Connect to Oracle
Databases" (page 20) for more information.
l
The Oracle database and username fields are pre filled in as per the Toad connection.
Security requires that you re-enter your Oracle password.
The default connection name is Toad followed by the name of the Oracle database. You
may choose to replace this with any connection name of your choice. Note that a
limitation of the integration of Toad with Spotlight on Oracle is that this is a new
Spotlight connection, ignoring all other pre-existing Spotlight connections.
You may choose a different Oracle username to connect to Spotlight. For example, in an
environment with multiple database administrators there may be a communal username for
Spotlight connections. Simply replace the filled in Oracle username with the appropriate
Oracle username and password details.
You can enter additional parameters to monitor ASM, the operating system and for
virtualization.
26
Spotlight on Oracle
Using Spotlight on Oracle

l
Spotlight opens at the Home Page or Drilldown relevant to the location in Toad from
where Spotlight was opened.
You are prompted to create a new Spotlight user if the Oracle username has not been
created in Spotlight. See "Spotlight Users" (page 17) for more information.
This Spotlight connection can be found in the Spotlight Connection Manager. It can be
managed the same as any other Spotlight connection. For example, you can close the
connection, edit the connection details and re-open the connection in the Spotlight
Connection Manager.
Caution:When you click on the Spotlight button in Toad to connect to the Oracle database,
a new Spotlight connection to the Oracle database is created, regardless of any pre-existing
Spotlight connections to the Oracle database. To avoid the annoyance of setting up Spotlight
connections in Spotlight that will be set up identically through Toad later, you may choose
to create all connections through Toad if Toad integration is desired.
Install And Configure DTrace Scripts

DTrace is a dynamic analysis tool for Sun platforms. Spotlight on Oracle uses DTrace to capture
details of the interaction between Oracle and the operating system (Solaris 10 onwards).
To use DTrace within Spotlight, you need to install and configure a number of DTrace scripts on
the server. Once installed, use the DTrace drilldown to view the DTrace analysis. See "Spotlight
on Oracle Drilldowns" (page 39) for more information.
To install and configure DTrace scripts
1. Locate in the Spotlight installation folder: Plug-ins\SoO\SoO_DTrace_scripts.tar
2. Unpack SoO_DTrace_scripts.tar to a directory on the database server.
3. Ensure the unpacked files have execute permission.
4. For the account that will be used to run operating system commands from Spotlight:
l
Edit .profile to define a QUEST_SOO_HOME variable that points to the directory on

the database server where the unpacked files are.
Provide DTrace authority. Add this line to /etc/user_attr:

<<username>>::::defaultpriv=basic,dtrace_proc,dtrace_kernel
Spotlight on Oracle Home Page

The Spotlight home page shows the flow of information and commands between various subcomponents and the size and status of internal resources such as processes, disk files and memory
structures.
27
Spotlight on Oracle
Related operating system statistics are grouped together on panels that are connected by a series
of graphical flows and icons. Spotlight updates these flows in real time so you can see how
quickly data is moving through the system. The icons change color as Spotlight alarms are
raised, upgraded, downgraded and canceled.
The panels on the Spotlight on Oracle home page are described in more detail next. For a full
discussion of the panels, see the Spotlight online help.
Tip: Hover the mouse pointer over a panel component for more information.
Sessions Panel
The Sessions panel indicates the status of the Oracle communications
between client and server. Information displayed here includes:
l
The time (in milliseconds) that it takes for the query select user from
dual to be submitted, executed and returned.
The total number of users connected to the database.
The number of users who are currently active.
The sum of DB time over all sessions, divided by elapsed time.
Note: The Average Active Sessions gauge is not available for Oracle 9i.
The flows between the Sessions panel and the Server Processes panel represent the rate of data
transfer in both directions across the SQL*NET interface.
28
Spotlight on Oracle
Host Panel
The Host panel shows metrics for the operating system:
l
The total amount of CPU being used.
The number of tasks waiting for CPU resources.
The percentage of CPU that is unavailable to the virtual machine (if

applicable).
The available physical RAM.
The amount of memory that has been reclaimed from the virtual
machine by ESX (if applicable).
Server Processes Panel

The Server Processes panel indicates the status of Oracle server processes
that perform database activities on behalf of end users, and mediate database
connections:
l
Statistics for the PGA memory allocated by server processes.
Dedicated server processes that perform work on behalf of a single

client process.
Shared or multi-threaded servers (MTS) that perform work on behalf

of more than one client process.
MTS dispatchers that coordinate the allocation of shared servers to

client tasks.
Server processes that support parallel execution of queries.
The current length of the Parallel Execution Queue.

Note: The Parallel Exec Queue gauge applies only to Oracle 11gR2.
Job queue server processes that run PL/SQL commands submitted to

the Oracle job queue via the DBMS_JOB package.
The state of the EM agent connection between Oracle Enterprise

Manager (if installed) and the database.
The number of server processes that are currently active.
The flow to the Server Processes panel from the Disk Storage panel represents the rate of direct
physical reads from disk.
The flows between the Server Processes panel and the SGA panel represent the rates of:
29
Spotlight on Oracle
Blocks reads from the SGA by all server processes.
Block changes in the SGA by all server processes.
Redo buffer entries made by all server processes.
SQL parse requests per second by all server processes.
SQL execution requests per second by all server processes.
SGAPanel
The SGA panel shows details of specific memory areas within the system
global area (SGA). The SGA is an area of shared or common process memory
that is used to cache frequently used data, SQL statements, procedures and
other structures:
l
The size of the buffer cache and its components. These hold data that
is accessed frequently.
The current size of the SGA.
The size of the Java Pool, which caches class definitions, Java
methods and Java objects.
The size of the Large Pool allocation heap, which is used in MTS for
session memory. It can be used by parallel execution and backup
processes.
The size of the redo buffer. This contains redo entries that are
eventually written to the redo log.
The size of the Shared Pool, which caches SQL statements, PL/SQL
programs, object definitions and session memory for MTS sessions.
The size and activity of the result cache (Oracle 11g or later).
Whether the database is in manual or auto memory management

mode (Oracle 10g or later).
The flow between the SGA panel and the Disk Storage panel represents the rate of block
reads from disk.
The flows between the SGA panel and the Background Processes panel represent the rates of:
l
Block writes by the Database writer (DBWR) process.
I/O writes by the Recovery writer (DBWR) process.
Redo log block writes by the Redo Log Writer (LGWR) process.
30
Spotlight on Oracle
Background Processes Panel

The Background Processes panel displays the following key Oracle
background processes:
l
Database Writer (DBWRn).
The Recovery Writer (RVWRn).
The Log Writer (LGWRn).
The Archiver (ARCHn).
Data Guard Overhead and Apply Lag
Predictive Diagnostics.
Note: The n value in each process icon indicates the number of processes
there are of that type.
The flows between the Background Processes panel and the Disk Storage
panel represent:
l
The rate at which modified blocks are written from the SGA to disk
by the DBWR processes.
The rate of I/O write operations to the flash recovery area by the
RVWR processes.
The rate at which redo log entries are written to the redo log files by
the LGWR processes.
The rate at which the archiver copies from redo log files to archived
logs.
The Background Processes panel also contains a status indicator for

Predictive Diagnostics (if installed). See "Predict Database Performance"
(page 45) for more information.
31
Spotlight on Oracle
Disk Storage Panel

The Disk Storage panel represents Oracle database and log files on disk,
including:
l
Oracle database files.
Size of the flash cache.
Oracle flashback recovery files.
Oracle redo logs.
The archive log destinations.
Right-click the Disk Storage icon to open Space Manager. See "Tools
|Space Manager" (page 43) for more information.
Spotlight on Oracle Alarms

Spotlight alerts you to problems with your system by issuing an alarm. You can configure
Spotlight in the level of severity that constitutes an alarm, to disable an alarm, and the actions
Spotlight takes on raising the alarm.
When an alarm is raised Spotlight performs one or more of the following actions:
l
Changes the color or intensity of relevant components.
Gives audible warnings.
Sends email notifications.
Actions you can take on an alarm being raised include:

l
View details of the statistics that caused the alarm in a Spotlight drilldown page. See
"Spotlight on Oracle Drilldowns" (page 39) for more information.
View the Spotlight online help.
View details of the alarm in the Spotlight Alarm Log drilldown.
Snooze the alarm.
32
Spotlight on Oracle
Save the alarm details.
Filter the list of alarms.
Following are the alarms specific to Spotlight on Oracle. For possible solutions to the problems
indicated by these alarms or for information on how to diagnose problems further, see Spotlight
on Oracle alarms in the online help.
Alarm
Description
Agent Status
alarm
Spotlight raises the Agent Status alarm when the status of the EM agent is
down.
OEM is a Web-based database administration and monitoring tool that
manages Oracle components and administers Oracle databases. When
installed and active, OEM communicates with the databases it administers
via a set of EM agent processes under the automated DBSNMP user
account.
Archive
Critical
Failure alarm
An Archive Critical Failure alarm indicates that the number of working

archive destinations for the database instance is about to fall below the
minimum number needed to archive Oracle redo logs successfully. When
you see this alarm, you should make space in the archive log destination
by purging or by backing up to an offline medium any archived files
that are not immediately needed.
Archive
Destination
Failed alarm
Spotlight raises an Archive Destination Failed alarm when at least one of

the archive destinations specified for redo log files has become full. See
Archive Destination Failure alarm below.
Archive
Destination
Failure alarm
An Archive Destination Failure alarm when at least one of the archive

destinations specified for redo log files is about to become full. When you
see this alarm, you should make space in the archive log destination by
purging it, or by backing it up to an offline medium.
Array Fetch
Size alarm
When more than one row is retrieved from an Oracle database in a single
SELECT (an array fetch), the amount of network traffic associated with
SELECT operations is radically reduced. The Array Fetch Size alarm is
raised when the array fetch size for the database may be too low.
ASM Balance
alarm
This alarm indicates that there is unbalanced access to one or more of the
disks in the ASM (Automatic Storage Management) disk group that has
been allocated to the Oracle instance.
ASM Service
Time alarm
This alarm indicates that the average time taken to perform I/O on the
ASM instance has exceeded a specified threshold value. (Service time
represents the response time of a disk plus any wait time experienced
during disk operations.)
ASM Status
alarm
When a Spotlight connection is lost, Spotlight automatically tries to reestablish the connection. The ASM Status alarm indicates that Spotlight is
33
Spotlight on Oracle
Alarm
Description
re-establishing its connection to the ASM diskgroup for the Oracle
instance.
Average
Time to Sync
a Redo Log
Entry alarm
The Average Time to Sync a Redo Log Entry alarm (formerly the Average
Redo Write Time alarm) is activated when the time taken to write redo log
entries exceeds a threshold. Unlike most other Oracle write I/Os, Oracle
sessions must wait for redo log writes to complete before they can
continue processing.
Buffer Busy
Wait alarm
The Buffer Busy Wait alarm occurs when a session cannot access a block
because it is in use by another session. The two most common causes are
insufficient free lists for a table or insufficient rollback segments.
Cache Buffer
Chains Latch
Wait alarm
The Cache Buffer Chains Latch Wait alarm occurs because contention for
the cache buffer chains latch exceeds a threshold.
Cache Buffer
LRU Chains
Latch Wait
alarm
The Cache Buffer LRU Chains Latch Wait alarm occurs because
contention for the cache buffer chains latch exceeds a threshold.
Collection
Status alarm
Spotlight raises the Collection Status alarm when collection of data for the
OEM (Oracle Enterprise Manager) stops. Perhaps the server is running low
on disk space.
The Collection Status alarm often occurs with the Files Pending Upload
Alarm.
Continued
Fetch Rate
alarm
Oracle sometimes needs to migrate a row in a database from its original

block to a new block, or needs to chain an updated row across multiple
blocks. These migrated and chained rows cause additional I/O, and the
Continued Fetch alarm occurs when the number of these "continued row"
fetches exceeds a threshold.
CPU Busy
alarm
The CPU Busy alarm occurs when the total CPU utilization of the system
exceeds a threshold.
CPU
Virtualization
Overhead
alarm
The CPU Virtualization Overhead alarm is activated when Virtualization

Overhead is greater than a threshold and CPU Usage is 95% or higher.
Datafile Read
Time alarm
The Datafile Read Time alarm occurs if the average time for a random
datafile read exceeds a given threshold.
Data Guard
Configuration
The Data Guard Configuration Alarm occurs on a Primary database in an

Oracle Data Guard environment. Please supply details of Standby
34
Spotlight on Oracle
Alarm
Description
Alarm
connections to Spotlight so Spotlight can monitor the Oracle Data Guard

environment. In monitoring the Oracle Data Guard environment Spotlight
can alert you if key elements of the Oracle Data Guard environment are
not working, or if Standby instances are lagging behind.
Dispatchers
Busy alarm
The Dispatchers Busy alarm occurs when all or most dispatcher processes
are busy. In an MTS environment, multiple sessions share a smaller number
of shared server processes. The servers are usually allocated by a smaller
number of dispatcher processes.
Estimated
Shared Pool
Size alarm
This alarm occurs when the Shared Pool is set too small. A possible cause
is the use of literals rather than bind variables in SQL statements. You can
also increase the size of the Shared Pool.
Excessive
RBS
Activities
alarm
The Excessive RBS Activities alarm occurs when there are high rollback
extent/shrink/wrap activities.
Files Pending
Upload alarm
Spotlight raises the Files Pending Upload alarm when a significant number
of XML files are pending upload indicating a problem with the upload
process. The strategy for dealing with this alarm is dependent on the OEM
control. The OEMcontrol is defined in the Spotlight configuration for
OEM.
This alarm may occur in conjunction with the Collection Status Alarm.
Flash Cache
Insert Skip:
DBWR
Overloaded
alarm
The Flash Cache Insert Skip: DBWR Overloaded alarm is raised when the
ratio of insert skips from the DBWR being overloaded exceeds a threshold.
Flash Cache
Read Latency
The Flash Cache Read Latency alarm is raised when the time taken to read
a single block from flash cache exceeds a threshold. This may be caused
by a low performance flash device.
Flash Cache
Write
Complete
Waits
The Flash Cache Write Complete Waits alarm is raised when the ratio of
'write complete waits: flash cache' to DB time exceeds a threshold.
Flashback
Buffer Wait
alarm
The Flashback Buffer Wait alarm indicates that the percentage of available
time spent waiting for the Recovery Writer (RVWR) to write data to the
flashback logs has exceeded a specified threshold.
Flashback
Space Usage
This alarm is raised when the amount of used space in the Flashback
Recovery Area exceeds a threshold.
35
Spotlight on Oracle
Alarm
Description
alarm
Free Buffer
Waits alarm
The Free Buffer Waits alarm occurs when free buffer waits (as a proportion
of total waits) exceeds a threshold.
In Memory
Undo Latch
Wait alarm
The in memory undo latch is associated with Oracle's relatively new in

memory undo (IMU) structures. These structures hold in memory
information formerly maintained in rollback (undo) segments.
Instance
Failed alarm
Spotlight raises an Instance Failed alarm when a specified instance has

failed because of an Archiver error. See Archive Critical Failure alarm
above.
Insufficient
Flashback
Logs alarm
This alarm fires when the flashback logs in the flashback area do not cover
the amount of time specified in the parameter DB_FLASHBACK_
RETENTION_TARGET.
Job Processes
Busy alarm
The Job Processes Busy alarm occurs when all or most job queue processes
are busy.
Large Pool
Full alarm
This alarm fires when the Large Pool allocation heap is full. The Large
Pool is used in MTS for session memory, and can be used by parallel
execution and backup processes.
Latch Free
Waits alarm
If a session needs to acquire a latch that is held by another session, a latch

free wait may occur. The presence of latch free waits may indicate a
bottleneck within the SGA. (The specific action depends on the latch.)
Library
Cache
Latches
alarms
These alarm fire when the time spent waiting on library cache latches or
library cache pin latches exceeds a threshold. Library cache latches (or
mutexes in 10g R2 and above) protect the cached SQL statements and
object definitions held in the library cache within the Shared Pool.
Library
Cache Miss
Rate alarm
The Library Cache Miss Rate alarm occurs when the library cache hit ratio
falls below a given threshold.
Lock Wait
alarm
The Lock Wait alarm occurs when the proportion of time sessions spend
waiting for locks exceeds a threshold.
Log Buffer
Space Wait
alarm
The Log Buffer Space Wait alarm occurs if the amount of time spent
waiting for space in the redo log buffer exceeds a threshold.
Log Switch
Time alarm
The Log Switch Time alarm occurs when the amount of time spent waiting
for log switch events exceeds a threshold.
Low Free
The Low Free Physical RAM alarm occurs when the servers available
36
Spotlight on Oracle
Alarm
Description
Physical
RAM alarm
RAM is low.
Maximum
Queued Time
alarm
The Maximum Queued Time alarm becomes active when a session has
been in the parallel execution queue for longer than the specified
threshold.
Note: Applies only to Oracle 11gR2.
Multithreaded
Server alarm
The Multi-threaded Server alarm occurs when all or most multi-threaded

server (MTS) or dispatcher processes are busy.
Nonreclaimable
Flashback
Space alarm
This alarm is raised when the amount of non-reclaimable used space in the
Recovery Area (flashback area) exceeds a threshold. Non-reclaimable files
include archived redo logs, which must be backed up before being deleted.
Oracle
Memory on
Disk alarm
The Oracle Memory on Disk alarm is activated when the amount of guest
physical memory mapped to machine memory is less than the current
combined size of Oracle's SGA and PGA.
Oracle
Parameters for
Predictive
Diagnostics
alarm
The Predictive Diagnostics feature cannot predict the behavior of Oracle

databases without the access to Oracle data that is granted by several
Oracle parameters.
Oracle Status
alarm
When a Spotlight connection is lost, Spotlight automatically tries to reestablish the connection. The Oracle Status alarm indicates that Spotlight
is re-establishing its connection to the Oracle instance.
Parallel
Query Server
alarms
The Parallel Query Server alarm occurs when all or most parallel servers
are busy. Parallel server processes perform work to resolve SQL statements
that are being executed in parallel. If all parallel server processes are busy,
parallel SQL may be downgraded, or even executed serially, thus
increasing its execution time. Spotlight can raise these Parallel Query
Server alarms: Parallel Query Servers Busy, Parallel Queries Downgraded,
and Parallel Queries Serialized.
Parse Ratio
alarm
The Parse Ratio alarm is raised when the ratio of parse calls to execute
calls exceeds a threshold.
Note This alarm is initially disabled; see Total Parse Wait Time Ratio
alarm.
37
Spotlight on Oracle
Alarm
Description
Predictive
Diagnostics
alarms
Predictive Diagnostics alarm (SQL Collector or Bottleneck Collector

alarm) is raised when the appropriate collection of predictive data has not
been scheduled.
Predictive
Diagnostics
Custom SQL
Error alarm
This alarm indicates that the customized SQL query use to represent
database concurrency within Predictive Diagnostics has returned an error.
Redo
Allocation
and Redo
Copy Latch
Wait alarms
The Redo Allocation and Redo Copy Latch Wait alarms are raised when
there is contention for the redo allocation latch and redo copy latch.
Result Cache
Find/Create
Ratio alarm
When Oracle executes SQL queries, the result cache stores the result sets
of specified queries in shared memory for later use. The Find/Create Ratio
alarm occurs when the rate of find operations on the result cache drops
below the rate of create operations, and indicates that the contents of the
result cache are under-used.
Result Cache
Latch Sleep
Ratio alarm
Spotlight raises the Result Cache Latch Sleep Ratio alarm when the rate of
latch sleeps for the Result Cache latch indicates that there is contention on
the Result Cache latch, where multiple processes are competing to use the
latch and are causing a significant degradation in response time.
Row Cache
objects Latch
Wait
The row cache objects latch protects the contents of the row cache; the
section of the shared pool to cache dictionary information.
Sequential
Read Time
alarm
This alarm fires when the average time that Oracle takes to read a single
block of data typically an index-based lookup is significantly higher
than the average time recorded during calibration.
Sessions
Queued alarm
The Sessions Queued alarm becomes active when the number of sessions
in the parallel execution queue exceeds a threshold.
Note Applies only to Oracle 11gR2.
Shared Pool
Latch Wait
alarm
Shared pool latches control access to the shared pool memory map.
Shared Pool
Lock
Percentage
alarm
This alarm is raised when one user session holds a lock on object(s) in the
Shared Pool for a prolonged period, and causes other sessions to wait on
the same resource. This "blocking lock" happens rarely. When it does
occur, use a SQL query to find the affected sessions.
38
Spotlight on Oracle
Alarm
Description
Shared Server
Converted
into
Dedicated
Server alarm
Spotlight raises an alarm when many shared servers are converted, leaving
too few shared servers to meet the demand. Converted shared servers will
have a status of WAIT(RECEIVE) in v$shared_server.
Status Error
alarm
This alarm is raised when there is a problem with the way the OEM
configuration has been defined in Spotlight. Correct the values entered in
the dialog View | Options | OEM Monitoring.
Temporary IO
Wait Ratio
alarm
Spotlight raises the Temporary IO Wait Ratio alarm when the time spent in
temporary IO waits is too high as a percentage of the time spent
performing user calls on the Oracle database.
Total Parse
Wait Time
Ratio alarm
Spotlight raises this alarm when the amount of time spent parsing SQL
statements as a percentage of the active database time exceeds a level that
may cause latch contention on the shared pool and library cache latches.
Note: The Total Parse Wait Time Ratio alarm is available for Oracle 10.2
and later. When monitoring earlier Oracle systems, consider reactivating
the (disabled) Parse Ratio alarm.
Unarchived
Logs alarm
The Unarchived Logs alarm occurs when the number of unarchived redo
logs exceeds a threshold. If all online logs need archiving, database update
activity can be suspended while the redo log writer waits for the next log
to be archived.
Unix Status
alarm
When a Spotlight connection is lost, Spotlight automatically attempts to

re-establish the connection. Where the Oracle database is hosted on a Unix
server, this alarm indicates that Spotlight is re-establishing its connection
to the Unix server.
WMI Status
alarm
When a Spotlight connection is lost, Spotlight automatically attempts to

re-establish the connection. Where the Oracle database is hosted on a
Windows server, this alarm indicates that Spotlight is re-establishing its
WMI connection to the Windows server.
Write
Complete
Wait alarm
Spotlight raises the a Write Complete Wait alarm when write complete
waits (as a proportion of total waits) exceed a threshold. Write complete
waits occur when a session tries to modify a block that is currently being
written to disk by the database writer process.
Note: Spotlight on Oracle Data Guard alarms occur in the Alarm Log of the Primary Database
in the Oracle Data Guard environment. See Spotlight on Oracle Data Guard Alarms in the online
help for more information.
39
Spotlight on Oracle
Spotlight on Oracle Drilldowns

When you have isolated a problem, you can display a drilldown page, whose charts and tables
provide a detailed breakdown of the underlying statistics.
Following are Spotlight on Oracle drilldowns.
Drilldown
Top Sessions
SQL&Application
Workload
Click
to
open
Description
The Sessions drilldown displays information about the

users connected to the Oracle database. The pages and
sub-pages in the drilldown display information on:
l
All users connected to the Oracle database.
Session details for those users, including waits,

SQL statements executed, locks, resource usage,
and Oracle statistics.
Parallel queries executed for all users connected to

the Oracle database.
Oracle jobs currently scheduled for the database.
The SQL & Application Workload drilldown allows you

to select, sort and display SQL statements executed
according to specified criteria. The drilldown includes:
l
The Top SQL page, which displays the SQL

statements that consume the most system resources
for the current connection.
The Array Fetch page, which displays information

on array processing for the Oracle database under
investigation.
The Parse Activity page, which displays

information on SQL statements associated with
high parse rates (Oracle 10.2 and later).
The Sort Activity page, which displays

information on SQL statements associated with
high sort activity (Oracle 10.2 and later).
The Analyze Trace page (formerly a separate

drilldown), which displays the contents of trace
files that store information on how Oracle
processes the execution of SQL statements, and on
the resources it uses to do so (Oracle 10.2 and
later).
40
Spotlight on Oracle
Drilldown
Click
to
open
Description
The Result Cache page (Oracle 11g and later)

shows information about the result cache, which
stores the result sets of specified queries so they
can be re-used.
Activity
The Activity drilldown displays the activity on the

database. The pages and sub-pages in the drilldown
display information on waits, waiting events, blocking
locks, lock activity, latches, query servers, transactions,
rollbacks, and buffer busy waits.
I/O
The I/O drilldown displays the input and output activity

for the database. The pages and sub-pages in the
drilldown display information on I/O by category, session,
SQL, tablespaces, datafiles, segments and row chaining.
Configuration &
Memory
The Configuration and Memory drilldown shows the

memory utilization for the Shared Pool and the SGA, as
well as shared pool and buffer cache hit ratios, Oracle
alert log and Oracle parameters.
Operating System
The Operating System drilldown provides information

about the performance, processes, and storage on the
database machine under investigation.
Disk Storage
The Disk Storage drilldown displays information about

online redo logs, tablespace utilization, archive
destinations, and Flashback recovery.
Right-click the tables in the Disk Storage drilldown to
open Space Manager. See "Tools |Space Manager" (page
43) for more information.
Tuning
You can use the Tuning drilldown to view information

about Oracle memory resources in the Oracle SGA and
PGA (System and Program Global Areas), to tune memory
resources where feasible, and to view and tune latch
activity for systems with multiple CPUs.
ASM
If your system uses Automatic Storage Management

(ASM) to manage disk storage, you can use the ASM
drilldown within Spotlight on Oracle to diagnose ASM
performance. ASM is a storage management solution from
Oracle Corporation that automatically reorganizes data
based on statistics collected from usage and user access
patterns over time.
41
Spotlight on Oracle
Drilldown
Analyze Trace
Click
to
open
Description
To understand how well SQL is being processed by the

system under investigation, and how to improve that
processing, you must trace the execution of SQL
statements for:
l
How Oracle is processing the statements.
The resources that are required to execute those

statements.
Oracle stores that information in trace files. The pages in

Spotlight's Analyze Trace page allow you to view the
contents of those trace files and save the contents to the
client's computer.
Note:Oracle 9.2.0.4 or later
DTrace
DTrace is a dynamic analysis tool for Sun platforms.

Spotlight on Oracle uses DTrace to capture details of the
interaction between Oracle and the operating system
(Solaris 10 onwards). See "Install And Configure DTrace
Scripts" (page 26) for more information.
Predictive
Diagnostics (if
enabled)
Use Predictive Diagnostics to identify:

l
SQL statements ("degrading SQL") whose

performance may not scale adequately in the future
as data volumes and SQL execution rates increase.
Waiting events ("bottlenecks") that may in the

future affect database throughput and response
time.
Database resources (CPU, memory, and disk I/O)

whose limitations may in the future affect database
performance.
See "Predict Database Performance" (page 45) for more

information.
Virtualization
Use the Virtualization drilldown to view detailed

information about the CPU and memory use of a virtual
machine.
Data Guard
Monitor an Oracle Data Guard environment with

Spotlight on Oracle Data Guard.
Note:The Data Guard icon is enabled for Primary
databases configured in the Oracle Data Guard
42
Spotlight on Oracle
Drilldown
Click
to
open
Description
environment.
Reports
The Report wizard generates printable reports of four

types of monitoring data: alarms by percentage of time,
waits, Top SQL and IO. The reports are designed and
delivered through SAP Crystal Reports.
Alarm Log
The Alarm Log drilldown displays information on the

alarms associated with the Oracle database.
You can filter the Alarm Log to display only the alarms
that meet specified criteria.
Spotlight Tools
Tools |Explain Plan
Tools |Explain Plan allows you to determine the execution plan that Oracle applies to a
particular SQL statement. Spotlight on Oracle allows you to view graphical representations of
the execution plan for SQL statements being executed by a user, or for those identified by the
Top SQL Page.
To open the Explain Plan
1. Open a connection to an Oracle Database.
See "Connect to Oracle Databases" (page 20) for more information.
2. Click Tools |Explain Plan.
Tools |SQLOptimizer
SQL Optimizer provides context-sensitive tuning advice for SQL statements based on the Oracle
execution plan and the database structure.
Before you open SQL Optimizer
l
Ensure SQL Optimizer is installed on your machine.
Ensure you have DBA privileges for the database you are diagnosing.
To open SQLOptimizer
43
Spotlight on Oracle
2. Click Tools |SQLOptimizer.
Tools |Space Manager

Space Manager provides a comprehensive solution for space management and reorganization in
Oracle databases. Space Manager offers preventive maintenance, problem detection and
resolution, and capacity planning across any number of databases.
Before you open Space Manager
l
Ensure Space Manager (version 4.5 or later) is installed on your machine.
Ensure you have DBA privileges for the database you are diagnosing.
To open the Space Manager

2. Click Tools |Space Manager.
Tune Your Database

Spotlight supports an approach to Oracle performance tuning that could be described as "tuning
by bottleneck". Spotlight alerts you if any component of the Oracle architecture is forming a
bottleneck. Additionally, Spotlight provides tools that allow you to identify and rectify any
inefficiency in your databases configuration.
In general, you tune your Oracle instance by reiteratively identifying bottlenecks, contention and
critical resources, and by using Spotlight's online advice to remove the bottleneck or contention
to improve the performance of the resource.
The following is a summary of steps that you can undertake to tune Oracle using Spotlight.
l
Use the Spotlight home page to alert you to any obvious bottlenecks. If Spotlight detects
that some component of Oracle constitutes a performance bottleneck, the corresponding
component generates a visual or auditory alarm. The exact appearance of the alarm
depends on how you have configured alarm severities. When an alarm is current you can
press F1 to display help for the alarm. Clicking the object (in the main window) displays
the relevant drilldown.
See "Spotlight on Oracle Home Page" (page 26) for more information.
Use the Activity drilldown to display overall efficiency and resource usage. In particular,
the Event waits chart shows the amount of time database sessions are spending waiting
for various resources, as well as the amount of CPU being utilized. Tuning efforts are
most likely to succeed if they are concentrated on the resources being used most heavily.
The Wait activity topic contains a discussion of the meaning and implications of various
44
Spotlight on Oracle
wait events.
See "Spotlight on Oracle Drilldowns" (page 39) for more information.
l
Other pages help you examine detailed activity. For instance, the Top Sessions drilldown
shows the users who contribute most to database activity. You can display various details
for these sessions, including the SQL statements, locks and resource usage.
For further information on tuning your database, see the online help.
45

Use Predictive Diagnostics to predict the future performance of a database (and identify how to
improve that performance):
l
Identify SQL statements whose performance may not scale adequately as data volumes
and SQL execution rates increase.
Identify bottlenecks that may affect database throughput and response time.
Predict when database resources such as CPU, memory and disk I/O are expected to reach
their full capacity, and recommend strategies to enable the database to handle high loads.
Note: It takes about 10 days for Spotlight to collect enough data to make valid predictions on
the future performance of the database.
Open Predictive Diagnostics

1. Open a Spotlight on Oracle connection. See "Connect to Oracle Databases" (page 20) for
more information.
2. Click Predictive Diagnostics or CTRL+G
Predictive Diagnostics for Beginners

Although you can use any feature in Predictive Diagnostics at any time, we recommend the
following procedure for first-time users.
Follow the suggestions in
What To Do Next.
The quality of the predictions depends on the quality of the

data. What To Do Next tracks the current data and indicates
when it is incomplete or outdated.
Wait until enough real

data is available.
This usually occurs about 10 days after you begin monitoring

the database. Before that time, become familiar with how
Predictive Diagnostics works by using the supplied sample
data.
Improve how the

database is used now.
Employ the Identify Degrading SQL and Identify Database

Bottlenecks features to optimize data processing.
Note: Any changes you make MAY cause the data collected
earlier to become outdated. If so, you may need to collect new
data for another 10 days.
Use the Configuration

pages to identify the
Set parameters for the CPU, memory and disk I/O resources
available to the database.
46
resources available to the

database.
In particular, use Peak Processing Periods to identify WHEN

database resources are in greatest demand. This will improve
the precision of your predictions.
Note: The default settings we provide for these parameters may
not be appropriate for your database.
Use the Performance

Modeling pages.
Predict when and why database resources will run out, what
will cause performance to suffer, and what you need to do to
counter those problems.
Predictive Diagnostics And Spotlight

For detailed information on Predictive Diagnostics, see the Spotlight online help.
When...
Do this...
identifies a SQL statement that
may cause problems
Open the Top SQL drilldown to view further details

about the SQL statement.
Open Explain Plan to view further information about
the statement's execution plan.
Open SQL Optimizer to view context-sensitive tuning
advice for the SQL statement based on the Oracle
execution plan and the structure of the database.
identifies a bottleneck that may
cause problems
Open the Activity |Waiting Events drilldown to view

further details about the current behavior of potential
bottlenecks.
identifies a database resource
whose limitations may cause
problems
Open the I/O, Configuration & Memory, and Tuning

drilldowns to view further details about the current
behavior of CPU, memory, and disk I/O resources.
Requirements for Predictive Diagnostics

The disk storage requirements for Predictive Diagnostics schemas comprise:
Situation
Requirement
Sample data
Approximately 70 MB of disk space.
SQL
statements
A fixed overhead of approximately 10 MB per schema.

An additional requirement of approximately 1 KB per snapshot for each
47
Situation
Requirement
SQL statement.
The number of snapshots collected depends on the application workload and
size of the shared pool, the collection interval for snapshots, and the length
of time each snapshot is retained.
Example
If there are 500 eligible SQL statements per snapshot, and snapshots are
collected twice a day, and each snapshot is kept for one year, the disk
storage requirement for SQL statements is:
10MB + (500 x .001MB x 2 x 365) = 375 MB
Impending
bottlenecks
Disk storage requirements depend on how frequently data is collected, and

the length of time that snapshots are retained.
Examples
If data is collected every 10 minutes and retained for six months:
approximately 200 MB.
Performance
modeling
Disk storage requirements depend on how frequently data is collected, and

the length of time that performance data is retained.
Example
If data is collected every 10 minutes and retained for one year:
Note: Parameters set in View | Options | Predictive Diagnostics and the SQL Statement Trend
Analysis wizard affect the storage requirements for Predictive Diagnostics data. See the online
help more for more information.
48

What is Spotlight on Oracle RAC?
Spotlight on Oracle RAC is an optional add-on to Spotlight on Oracle. Spotlight on Oracle
RAC is installed automatically on your system via the Spotlight on Oracle or TOAD DBA
Suite installer.
Use Spotlight on Oracle RAC to diagnose Oracle databases on multiple host machines under the
Real Application Clusters (RAC) architecture.
When monitoring the behavior of an Oracle RAC cluster, a database administrator needs
to know:
1. How is each of the instances in my cluster performing?
2. How is the database as a whole performing?
3. Is my database experiencing cluster-related contention?
To further investigate the behavior of individual database instances within the cluster, the
database administrator can use Spotlight on Oracle RAC to launch Spotlight on Oracle on any
particular instance within a cluster. See "Instance Panels" (page 55) for more information.
What Is Oracle RAC?

Real Application Clusters (RAC) is a clustering technology introduced by Oracle in release 9i. It
supersedes Oracles previous clustering technology Oracle Parallel Server (OPS).
Oracle RAC is a shared disk clustering solution that is significantly different from the
architectures offered by Microsoft SQL Server and IBM DB2. Each node in an Oracle RAC
cluster has equal access to all of the database data through a shared disk subsystem. Data is
neither partitioned to specific nodes, nor replicated across nodes. A high-speed network
interconnect allows each node to keep its in-memory view of data consistent.
An Oracle RAC configuration has the following architectural characteristics:
1. Many instances of Oracle run across several nodes.
2. Many instances share a single physical copy of a shared Oracle database.
3. All instances have common data and control files.
4. Every instance has individual redo logs and undo segments.
5. Every instance can simultaneously execute transactions against the same database.
The following diagram illustrates the basic components of an Oracle RAC cluster.
49
A user should find an Oracle RAC database operationally identical to a database hosted on a
single server. From top to bottom, the important features represented are:
l
Cluster interconnect - A high-speed, high-bandwidth communication facility that

connects the nodes in an Oracle RAC cluster.
Multiple nodes/instances - Oracle instances run on the nodes (host machines) in the
Oracle RAC cluster. Each instance comprises an Oracle System Global Area (SGA)
plus the corresponding Oracle background processes that retain and process Oracle
database requests.
Shared disk subsystem - Database files in Oracle RAC systems are stored on multiple
disks that are shared by all the nodes in the cluster, and all nodes must be able read and
write to those disks.
Connect to Spotlight on Oracle RAC

Use this procedure to connect to an Oracle database running on an Oracle RAC cluster.
1. From the Spotlight Browser select All Connections |Spotlight on Oracle RAC
2. Select the RAC to connect to. If the RAC is not on the list then Add a new connection
50
(page 50).
3. The Connection Details Verification window is displayed if:

l
Display this dialog for the current connection is selected in the connection
properties or Verification window, OR
Spotlight encounters an error during the connection process.
You can use the Connection Details Verification window to view the available
connection data and provide or modify any data (port number, for example) that is
missing or faulty. Refer to the online help for more information.
Add a new connection
1. Ensure you create a Spotlight user with sufficient privileges to diagnose the Oracle RAC.
See "Spotlight Users" in the online help for more information.
2. Click File |Connect
3. Select Spotlight on Oracle RAC on the Connections menu.
4. Double-click Add new connection.
5. Ensure Details is selected. Fill in the Properties window.

Oracle Connection Details
Field
Description
Connection
String
Select the name (the SQL*Net alias) of the Oracle RAC database that
you wish to connect to.
51
Field
Description
Notes:
l
The name is defined within SQL*Net configuration utilities.
While this can be the name of any instance in the Oracle RAC
cluster, we recommend using the name of the load-balanced
instance the instance whose TNSNAMES entry includes the
option (LOAD_BALANCE = yes)
When selected, the Connection name field resets to this value.

You can then edit the Connection name field if required.
Oracle
Username
Enter the Spotlight user name.
Oracle
Password
Enter the Spotlight password.
Oracle
Home
Select an appropriate Oracle Home a location where Oracle client

software is installed.
Notes:
l
Choose a home that contains a version of the Oracle client

that is compatible with the Oracle RAC server.
You CANNOT change this value while an active Spotlight

connection is already using an Oracle Home.
When you specify an Oracle Home for a particular Spotlight

connection, the connection-based Oracle Home overrides any
default values.
The first active Spotlight connection in a session forces all

subsequent connections to use its specified Oracle Home.
ASM connection details

ASM is built into the Oracle database kernel. It simplifies the creation and space
management in Oracle databases.
Note: Available only in Oracle 10g and later.
Field
Description
Monitor
ASM
Select to monitor the performance of Oracle's Automated Storage

Management (ASM) technology.
ASM
Connection
String
Enter the connection string required to link to the ASM database

instance that is used for storage management for this Oracle database.
Note:The TNSNAMES entry for the ASM instance must include
52
Field
Description
option (UR = A) for the Spotlight connection to work properly.
ASM
Password
Enter the password to login to the ASM database instance.

The user that Spotlight uses to login to the ASM database is SYS.
This is the only user name that can provide access to ASM-specific
information.
Providing SYS access to an ASM instance does NOT create a security
issue for the Oracle RAC, but you should limit access to the SYS
account to trusted individuals.
Because the SYS password is stored in encrypted format, nonprivileged users can still access Spotlight on Oracle RAC. To do so:
l
Use a pre-configured or shared Spotlight on Oracle

connection, OR
Use a Spotlight connection without ASM access. Spotlight

will display Oracle-related data, but the ASM drilldown will
contain no data.
Server connection details

Field
Description
Monitor
OS
Select to monitor the operating system on the database server.
OS Server
Enter the name of the server.
Note: Clear this option if the database is running on an operating

system NOT supported by Spotlight on Oracle RAC. If you leave this
option checked and Spotlight is unable to collect statistics from the
server operating system, you may experience errors.
Tip: Use a period . to monitor the local database.

OS Type
Select the type of operating system.

l
Windows
Unix(REXEC)
Unix(SSH)
Unix(SSH using Public/Private Keys)
If you select one of the Unix systems, make sure the SSH or REXEC
daemon is running on the database server and is configured to receive
remote connections.
Note:Spotlight supports both SSH1 and SSH2 protocols; public-key
encryption is supported under SSH2 only.
53
Field
Description
SSH Port
Enter the port number for SSH (secure shell) connection to the
database servercluster. The default value is 22.
Note: Applicable when OSType is Unix (SSH) or Unix(SSH using
OS
Username
SSH Key
Type
Type the user name to login to the database servercluster.

Notes:
l
Spotlight will connect successfully to all nodes in the cluster

only if every node employs the SAME username and password.
If the username and password if required are not

completed, you will not be able to view the operating system
performance statistics for the database servercluster.
When OSType is Windows:

l
Not required if your login details have been assigned

remote access to the registry of the database
servercluster.
If your login details have NOT been set up on the

database .servercluster, you must enter a username and
password that has access to the registry of each node.
Choose the type of key to use when making the Spotlight

connection:RSA or DSA.
Public-key encryption is supported under SSH2 only. For more
information, see Public/Private keypairs in the online help.
Note: Applicable when OSType is Unix (SSH using Public/Private
Keys).
SSH
Private
Key
Filename
Locate the filename that contains the private key for the Spotlight
connection.
SSH
Passphrase
Type the passphrase used to decrypt the private key.
OS
Password
Type the user password to login to the database servercluster (if

required).
Hint: Use the "..." button.

Keys).

Keys).
Note: Not applicable when OSType is Unix (SSH using

6. Select Launch Spotlight on Oracle to open a Spotlight on Oracle connection to the
54
Oracle RAC database whenever you open the Spotlight on Oracle RAC connection. In
most cases, this should not be necessary.
7. Select Display Connection Details Verification Dialog to display the Connection
Details Verification dialog when you connect to a specified Oracle RAC cluster. The
dialog allows you to specify and test connections to ALL instances in the cluster.
8. Select Save password details to save all the password details that have been entered.
Spotlight on Oracle RAC Home Page

The Spotlight home page shows an overview of the activity in an Oracle RAC database cluster.
Related operating system statistics are grouped together on panels that are connected by a series
of graphical flows and icons. Spotlight updates these flows in real time so you can see how
quickly data is moving through the system. The icons change color as Spotlight alarms are
raised, upgraded, downgraded and canceled.
The panels on the Spotlight on Oracle home page are described in more detail next. For a full
discussion of the panels, see the Spotlight online help.
Tip: Hover the mouse pointer over a panel component for more information.
55
Interconnect and Global Status Panel
This panel shows a combination of Interconnect and Global Cache Service information, and
information about the Oracle RAC database overall. It consists of information aggregated from
queries sent out to the individual instances. This provides information on how well data is being
shared and transferred between nodes in the cluster.
Some information displayed here is easily calculated: for example, the Total Logical Reads
spinner is the sum of the individual Logical Reads values. Other components, such as Cluster
Latency and Cluster Overhead, have a more complex calculation.
Spotlight uses the Global Database label to indicate alarms in one or more Oracle instances that
may affect the performance of the Oracle RAC database as a whole.
The flows between the Interconnect panel and the Instances panel represent the flow of data
from one instance to another across the interconnect.
Instance Panels
This panels show the participating instances of a cluster. Together with separate panels for all
participating instances, this panel provides information on the performance of individual
instances in the cluster.
56
Spotlight on Oracle RAC supports a theoretical maximum of 36 instances, with up to 6 instances

shown on the home page. When there are more than 6 instances (not shown here), you can use a
navigation control in the Instances panel to navigate between groups of instances.
The color of the border and label on each instance represents the Spotlight on Oracle severity of
the instance. You can move the mouse over a named instance to display a list of all the alarms
currently raised against that instance.
Note: You can launch Spotlight on Oracle from any Instance panel on the Spotlight on Oracle
RAC home page.
The overall status of the cluster does not necessarily reflect the status of individual instances,
as problems with individual instances do not necessarily translate to a problem with the
cluster as a whole.
Two further metrics Balance and Availability are shown in the instances panel. Balance
indicates the comparative load on the instances. Availability shows the percentage of instances
for which a SQL*Net connection is currently open and working.
I/O Subsystem Panel
This panel shows information relating to the shared storage subsystem for the Oracle
RAC cluster.
Some of this information is aggregated from queries sent to the individual instances; other
information is obtained from a single query to one of the instances.
The ASM spinner is populated only when ASM (Automatic Storage Management) is being used.
See "Connect to Spotlight on Oracle RAC" (page 49) for more information.
The flows between the I/O Subsystem panel and the individual instances represent the flow of
physical reads and writes between the instances and the physical database.
Spotlight on Oracle RAC Alarms

Spotlight alerts you to problems with your system by issuing an alarm. You can configure
Spotlight in the level of severity that constitutes an alarm, to disable an alarm, and the actions
Spotlight takes on raising the alarm.
When an alarm is raised Spotlight performs one or more of the following actions:
57
Changes the color or intensity of relevant components.
Gives audible warnings.
Sends email notifications.
Actions you can take on an alarm being raised include:

l
View details of the statistics that caused the alarm in a Spotlight drilldown page. See
"Spotlight on Oracle RAC Drilldowns" (page 60) for more information.
View the Spotlight online help.
View details of the alarm in the Spotlight Alarm Log drilldown.
Snooze the alarm.
Save the alarm details.
Filter the list of alarms.
Alarms on A Database Instance

Problems in a single instance may not always affect the operation of the cluster as a whole. In
such cases, Spotlight on Oracle RAC can display an alarm in the relevant instance panel without
reporting a specific Spotlight on Oracle RAC alarm.
However, if a problem in a single instance DOES affect the operation of the whole cluster or is
repeated in multiple instances so that the combined effect is significant Spotlight on Oracle
RAC reports the problem as an aggregated alarm.
These alarms are displayed on the Spotlight on Oracle RAC home page usually in the Global
Database heading of the Interconnect and Global Status panel, or on the individual components
of the IO Subsystem panel.
Notes:
l
Aggregated alarms within Spotlight on Oracle RAC are the same as the corresponding
alarms in Spotlight on Oracle.
Not all Spotlight on Oracle alarms are aggregated in Spotlight on Oracle RAC.
Alarms on The Oracle RAC

Following are the alarms specific to Spotlight on Oracle. For possible solutions to the problems
indicated by these alarms or for information on how to diagnose problems further, see Spotlight
on Oracle RAC alarms in the online help.
Alarm
Description
Archive
Critical
Failure alarm
An Archive Critical Failure alarm indicates that the number of working

archive destinations for the database cluster is about to fall below the
minimum number needed to archive Oracle redo logs successfully. When
you see this alarm, you should make space in the archive log destination
58
Alarm
Description
by purging or by backing up to an offline medium any archived files
that are not immediately needed.
Archive
Destination
Failed alarm
Spotlight raises an Archive Destination Failed alarm when at least one of

the archive destinations specified for redo log files has become full. See
Archive Destination Failure alarm below.
Archive
Destination
Failure alarm
An Archive Destination Failure alarm when at least one of the archive

destinations specified for redo log files is about to become full. When you
see this alarm, you should make space in the archive log destination by
purging it, or by backing it up to an offline medium.
ASM Status
alarm
When a Spotlight connection is lost, Spotlight automatically tries to reestablish the connection. The ASM Status alarm indicates that Spotlight is
re-establishing its connection to the ASM diskgroup for the Oracle RAC
cluster.
Availability
alarm
This alarm indicates that the instances participating in the cluster are NOT
all currently available for access. The database administrator should
determine why the instances are unavailable, and restart them after taking
the required actions.
(Global)
Cache Miss
Rate alarm
See Miss Rate alarm below.
Cluster
Latency
alarm
Spotlight raises the Cluster Latency alarm when cluster latency from all
causes (interconnect, GES, LMS overhead) exceeds a specified threshold.
Cluster latency is defined as the average elapsed time from when a
requesting instance requests a block of data to the time when the
requesting instance receives the block.
Cluster
Overhead
alarm
Spotlight raises the Cluster Overhead alarm when the percentage of time
dedicated to cluster co-ordination activities exceeds a specified threshold.
Cluster overhead is the amount of time spent waiting for cluster related
activities as a percentage of the total time spent waiting for all activities.
Corrupt or
Lost Blocks
alarms
Spotlight raises separate alarms for corrupt and lost blocks when a
specified percentage of the blocks transferred across the interconnect are
corrupted or lost during global cache consistent read requests.
High levels of lost or corrupt blocks should not happen in normal
circumstances. If these alarms fire at all, it indicates either very serious
overloading of the Oracle RAC cluster, or software or hardware faults in
the cluster.
Data Guard
Configuration
The Data Guard Configuration Alarm occurs on a Primary database in an

Oracle Data Guard environment. Please supply details of Standby
59
Alarm
Description
Alarm
connections to Spotlight so Spotlight can monitor the Oracle Data Guard

environment. In monitoring the Oracle Data Guard environment Spotlight
can alert you if key elements of the Oracle Data Guard environment are
not working, or if Standby instances are lagging behind.
Instance
Failed alarm
Spotlight raises an Instance Failed alarm when a specified instance has

failed because of an Archiver error. See Archive Critical Failure alarm
above.
Load
Imbalance
alarm
Spotlight raises the Load Imbalance alarm when:

l
The load across instances in the cluster is significantly unbalanced,

and
The imbalance is proven to be statistically significant, beyond a

95% probability.
Cluster balance is a fundamental indication of the health and productivity

of an Oracle RAC environment. If load is not reasonably evenly
distributed across the instances in a cluster, then the DBA is likely to
regret having chosen a clustered environment.
Spotlight on Oracle RAC calculates balance by recording a user-definable
metric of load over time on each instance (the default is Logical Reads,
but you can switch this to CPU usage or Physical Writes via the
Spotlight on Oracle RAC Options window) and calculating the relative
balance of load across the cluster.
In the normal course of events, there are always minor variations in load
between instances within the cluster, so Spotlight uses a statistical
technique to determine if a variation is significant.
Note:The activation of specific Oracle Services can unbalance the
workload across the Oracle RAC cluster. In these cases, Spotlight triggers
the Load Imbalance alarm only when a more significant threshold is
reached.
Miss Rate
alarm
This alarm, more specifically known as the Cache Miss Rate alarm or
Global Cache Miss Rate alarm, is raised when the global cache local miss
rate exceeds a specified threshold. This rate is the percentage of time that
a block required for a logical read is located in an inconsistent mode on
ANOTHER instance in the cluster and must be transferred.
No Nodes
Available
alarm
This alarm indicates that Spotlight has lost its connection to the Oracle
RAC cluster, and cannot re-connect to any of the nodes in the cluster.
Check the status of your network connection, your host machines, and
your database instances.
60
Note: Spotlight on Oracle Data Guard alarms occur in the Alarm Log of the Primary Database
in the Oracle Data Guard environment. See Spotlight on Oracle Data Guard Alarms in the online
help for more information.
Spotlight on Oracle RAC Drilldowns

When you have isolated a problem, you can display a drilldown page, whose charts and tables
provide a detailed breakdown of the underlying statistics.
Following are Spotlight on Oracle drilldowns.
Drilldown
Click
to
open
Description
Top Sessions
As per Spotlight on Oracle.
SQL&Application
Workload
As per Spotlight on Oracle. The SQL & Application

Workload drilldown in Spotlight on Oracle RAC does
NOT contain the following pages:
Activity
SQL Analyze Trace
Result Cache
As per Spotlight on Oracle. The Activity drilldown in

Spotlight on Oracle RAC does NOT contain the following
pages:
l
Lock Activity
Server
Rollback
I/O
Configuration &
Memory
As per Spotlight on Oracle. The Configuration & Memory

drilldown in Spotlight on Oracle RAC does NOT contain
the following pages:
Disk Storage
Shared Pool Utilization
Buffer Cache
As per Spotlight on Oracle. The Disk Storage drilldown in

Spotlight on Oracle RAC does NOT contain the following
pages:
l
Online Redo
Recovery Area
61
Drilldown
Click
to
open
Description
Cluster
The pages in the Cluster drilldown display information

about the performance of the Oracle RAC cluster, including
details on cluster latency, balance, overhead, miss ratios,
and corrupt or lost blocks. For further information see
Cluster drilldown in the online help.
ASM
Services
The Services drilldown displays details of the services

defined and deployed for the Oracle RAC cluster under
investigation.
Note:Available for Oracle 10.2 and later.
Predictive
Diagnostics (if
enabled)
Data Guard
If the system is a Primary database in the Oracle Data

Guard environment, you can use Spotlight on Oracle Data
Guard to monitor the system's Standby databases. Standby
databases are alternative/supplementary repositories to the
production Primary database.
Note:The Data Guard icon is enabled once the system is
configured as a Primary database in the Oracle Data Guard
environment.
Alarm Log
4
This section identifies general problems that you may encounter when using Spotlight on Oracle,
and details how to address those problems. If you do not find a solution in this section then
check the Spotlight on Oracle Release Notes.
Product AuthorizationErrors
When first installed, Spotlight can be used with full functionality for a specified trial period.
When the trial period expires, contact your Quest Software representative to obtain an
authorization key to continue. See "Authorize Spotlight" (page 16) for more information.
Notes:
l
Make sure that you enter the authorization key and site message exactly as stated on your
Product Authorization sheet or as provided by your Quest Software distributor. The site
message is case sensitive. If you enter the authorization key incorrectly, the following
message is displayed:
Invalid authorization key
Once the trial period has expired, if an authorization key is not supplied, only the
Spotlight on Oracle home page will be visible. If you click any of the drilldown buttons,
the following message is displayed:
You do not have a license for Spotlight on Oracle that enables access
to this functionality. Please contact Quest Software for information
about obtaining the correct license.
If the trial date has not passed, check that the date on your system is set correctly.
63
Connection Problems
Cannot Connect to Database
Possible cause
Solution
Oracle is not running.
Verify that the Oracle instance is running

and can be accessed from a host or client
workstation.
The connect string and alias of the database

are incorrect or have changed.
Check your SQL* Net configuration and

verify the connect string and alias you are
using are correct.
The user ID you are using to log into

Spotlight does not exist in the database.
Run the User Wizard to establish the user

ID in the database.
The Spotlight user has the wrong

permissions in the database.
Contact your DBA to add relevant

permissions to the user.
Remote commands do not work when

monitoring Unix OS with a user
ID/password that exists on the machine.
Remote commands may not work unless the

user ID is added to the /etc/hosts.equiv file
on that Unix machine.
Spotlight takes too long to open the Oracle

connection so it appears to be unable to
connect.
Turn on Fast Initialize.

More information on Fast Initialize is
available in the online help.
1. Click View | Options.
2. Select Data Collection |Oracle
Initialization | Turn Fast Initialize
on.
3. Restart Spotlight.
Error Attempting to Connect

The following error message is displayed:
Connection has been refused by the server
This means that Spotlight cannot access the operating statistics of the (Windows) database
machine. Users must have access to the registry so that Spotlight can retrieve the operating
system statistics. A user account with administrative rights to the target machine will allow this.
64
Retrieve Data from Unix

If the database is running on Unix, Spotlight on Oracle may be using the REXEC or SSH
protocols to submit various Unix commands (for example, netstat, vmstat, iostat, sar) to analyze
system activity.
Example
If Spotlight uses REXEC, these commands must be accessible to an REXEC session for
Spotlight on Oracle to be able to analyze Unix activity. If these commands are not located in the
standard search path available to REXEC, Spotlight on Oracle displays an error.
Ensure that all Unix commands are located in the standard search path available to REXEC, or
one of the following paths:
/usr/bin
/usr/sbin
/usr/ucb
If the command is located elsewhere in the file system, you must create a symbolic link in one of
the areas listed above.
When observing a Unix operating system with a user ID/password that exists on the machine, it
is possible that remote commands will not work on the machine unless the user ID is added to
the /etc/hosts.equiv file on that Unix machine.
Cannot Connect to ASM

When installed and available, Automatic Storage Management (ASM) stores Oracle data in a
separate ASM database instance. Spotlight on Oracle displays performance metrics for ASM on
the home page and in the ASM drilldown.
If the database under investigation uses ASM but Spotlight cannot connect to it to retrieve the
required information, you may need to edit the TNSNAMES entry for the ASM instance.
1. Locate the tnsnames.ora file in Oracles network/admin sub-folder of the Oracle
installation folder on your Spotlight client.
2. Edit tnsnames.ora so that the TNSNAMES entry for the target ASM instance contains the
option (UR = A).
Drilldown Problems
Alarm Help Displayed Instead of Drilldown
Spotlight can be configured to show the help or jump directly to the related drilldown on
clicking a home-page control.
65
To show the drilldown

1. Click View |Options.
2. Select Alarms and notifications | Balloon help.
3. Select Always jump directly to the related drilldown.
Dead Sessions in Top Sessions Drilldown

The solution is to reset the Top Sessions drilldown.
To reset the Top Sessions drilldown
From the Top Sessions drilldown
1. Click Filter.
2. Select Reset
3. Select the appropriate frequency.
What Happens to The Operating System Drilldown when

Connection Is Lost to The Server?
The Operating System drilldown may report inaccurate data if Spotlight looses access to the
database server.
1. Reconnect to the database. When the Connect dialog is displayed ensure that the OS
server and OS type fields (and if necessary the OS Username and OS Password) are
completed correctly.
2. Ensure the Unix or Windows server on which the Oracle database is installed is
accessible to Spotlight on Oracle. See the Spotlight on Unix or Spotlight on Windows
Getting Started Guide.
Operating System Statistics Not Reported

Differences between versions of operating systems mean that Spotlight is unable to collect all
O/S statistics in all environments. Refer to the Spotlight on Oracle Release Notes for more
information on the Operating System Drilldown.
Where the operating system is Unix refer to the Spotlight on Unix Release Notes and Spotlight
on Unix online help.
Where the operating system is Windows refer to the Spotlight on Windows Release Notes and
Spotlight on Windows online help.
66
Predictive Diagnostics Unavailable

You can use Predictive Diagnostics to collect and analyze performance metrics for SQL
statements as they are executed on the database instance, system bottlenecks as they affect
database throughput and response time, and database resources whose limitations affect database
performance.
If Predictive Diagnostics is unavailable for your Spotlight connections, re-run the Oracle user
wizard to install Predictive Diagnostics.
1. Close and re-open Spotlight on Oracle.
2. Click File | Oracle User Wizard...
3. Create a spotlight user. Make sure that you install Predictive Diagnostics as part of the
procedure. See "Spotlight Users" (page 17) for more information.
Note: When you start using Predictive Diagnostics, Spotlight will take about 10 days to collect
enough data to make valid predictions on the future performance of the database. During that
time, you can use the supplied sample data to view Predictive Diagnostics at work.
High Spotlight Load on Database Server

The queries that Spotlight uses to collect data can place an additional load on the system
under diagnosis.
Most Spotlight queries consume a measurable but not significant amount of system resources.
However, queries that are complex or collect large amounts of data may put a significant load on
the system.
If you find that the load that Spotlight places on your server is too high, you can adjust
collection rates for different categories of Spotlight data, and so limit the load that Spotlight
places on the system under analysis.
To choose a standard Spotlight setting or create your own Custom settings
1. Close and re-open Spotlight on Oracle.
2. Select File | Connect |Spotlight on Oracle.
3. Right click the Spotlight on Oracle connection experiencing the high load and select
Properties.
4. Select Overhead.
5. Adjust the Spotlight load for the connection:
l
Use the Overhead Management slider to adjust the overall Spotlight load.
Alternatively, click Custom to adjust specific load categories.
6. Open the Spotlight connection.

Note:The procedure for adjusting the Spotlight load is covered in more detail in the online help.

Contact Quest Support
Quest Support is available to customers who have a trial version of a Quest product or who have
purchased a Quest product and have a valid maintenance contract. Quest Support provides
unlimited 24x7 access to our Support Portal at www.quest.com/support.
From our Support Portal, you can do the following:
l
Retrieve thousands of solutions from our Knowledge Base
Download the latest releases and service packs
Create, update and review Support cases
View the Global Support Guide for a detailed explanation of support programs, online services,
contact information, policies and procedures. The guide is available at: www.quest.com/support.
Contact Spotlight Support

Whenever you have a question about Spotlight, click Support Bundle or Contact Support on the
Help menu. This creates a file called spotlightsupport.zip in one of the following folders:
Windows XP
In a sub-folder under the Documents and Settings folder.
Windows Vista / Windows 7
In a sub-folder under the Users folder.
This file contains a snapshot of your Spotlight installation. Send this file and your request for
assistance to Quest Software (support@quest.com).
Notes:
l
Support bundles contain information for the active instance only. If you are using
Spotlight to diagnose multiple instances ensure that the correct instance is displayed
before creating the support bundle.
Support bundles cannot be created when the History Browser is open. For further
information on the History Browser, see the Spotlight online help under Concepts
and Features.
68
Contact Quest Software

Email:
info@quest.com
Mail:
Quest Software, Inc.

World Headquarters
5 Polaris Way
Aliso Viejo, CA 92656
USA
Web site:
www.quest.com
Refer to our Web site for regional and international office information.
About Quest Software

Now more than ever, organizations need to work smart and improve efficiency. Quest Software
creates and supports smart systems management productshelping our customers solve everyday
IT challenges easier and faster. Learn more at www.quest.com.
drilldowns
Index
Spotlight on Windows
activity drilldown
9, 40
aggregated alarms
57
alarm log drilldown
43
DSA
31, 42, 57
alarms
9, 31, 56
analyze trace
39
architecture diagram, Oracle
49
array fetch
39
21, 32, 40, 51, 56, 58, 64
23
DTrace
11, 26, 41
E
ESX
24, 28
event waits
explain plan
46
F
architecture diagram, Oracle RAC
ASM
9, 39, 60, 64
fast initialize
63
H
home page
8, 26
ASM drilldown
no data
52
authorize Spotlight
host panel
21, 32, 40,

51, 56, 58,
64
B
background processes panel flows
29-30
C
configuration and memory drilldown
40,
46
28
I
16
Automated Storage Management
I/O drilldown
40, 46
I/O subsystem panel
56
install
12
instance panels
55
instance panels flows
55
interconnect and global status panel
55
interconnect panel flows
55
connect to Oracle database
20
connect to Oracle RAC
49
operating system drilldown
contact support
67
Oracle architecture
7-8
28
48
40
Oracle RAC architecture
disk storage panel
31
Oracle user wizard
28-30
40
Oracle Enterprise Manager
disk storage drilldown
disk storage panel flows
54
17, 66
70
Index
P
parse activity
39
permissions
predictive diagnostics
SQL optimizer
42, 46
SSH
23, 52
12-13, 15-16
start Spotlight
15
10, 18, 30, 37, 41,

45, 66
StealthCollect
24
T
open
45
preview mode
13-14, 25
16
top sessions drilldown
privileges
9, 65
12-13
top SQL drilldown
Spotlight user
39, 46
17
trial key
project lucy
16
15
tuning drilldown
40, 46
R
U
result cache
40
REXEC
uninstall
14
user, Spotlight
17
23, 52
RSA
23
V
S
virtual system
server processes panel
24, 28, 41
28
W
server processes panel flows
27-28
wait activity
sessions drilldown
39
sessions panel
27
sessions panel flows
27
Set timed statistics now
20
sga panel
29
waiting events
Windows
sga panel flows

sort activity
space manager
28-29
39
31, 43
Spotlight on Windows
drilldowns
43
Spotlight users
17
SQL & application workload drilldown 39
9
46
23, 52

Spotlight

Hochgeladen von

Dokumentinformationen

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Spotlight

Hochgeladen von

Copyright:

Verfügbare Formate

Spotlight on Oracle 8.

2012 Quest Software, Inc.

Spotlight on Oracle 8.1

Oracle Architecture Overview

Spotlight on Oracle Overview

Features Common to Spotlight Applications

Features Specific to Spotlight on Oracle

Features Specific to Spotlight on Oracle RAC

DTrace Analysis (Unix Solaris 10 And Later)

Multiple Concurrent Installations

MSI Installers for Spotlight on Oracle

Oracle User Wizard

Spotlight on Oracle Getting Started Guide

Add A New Connection

Toad DBA Suite for Oracle

Install And Configure DTrace Scripts

Spotlight on Oracle Home Page

Server Processes Panel

Background Processes Panel

Disk Storage Panel

Spotlight on Oracle Alarms

Spotlight on Oracle Drilldowns

Tools |Explain Plan

Tools |Space Manager

Tune Your Database

Predict Database Performance

Open Predictive Diagnostics

Predictive Diagnostics for Beginners

Predictive Diagnostics And Spotlight

Requirements for Predictive Diagnostics

Spotlight on Oracle RAC

What is Spotlight on Oracle RAC?

What Is Oracle RAC?

Connect to Spotlight on Oracle RAC

Spotlight on Oracle RAC Home Page

Spotlight on Oracle Getting Started Guide

Interconnect and Global Status Panel

I/O Subsystem Panel

Spotlight on Oracle RAC Alarms

Alarms on A Database Instance

Alarms on The Oracle RAC

Spotlight on Oracle RAC Drilldowns

Troubleshooting Spotlight on Oracle

Cannot Connect to Database

Error Attempting to Connect

Retrieve Data from Unix

Cannot Connect to ASM

Alarm Help Displayed Instead of Drilldown

Dead Sessions in Top Sessions Drilldown

Predictive Diagnostics Unavailable

High Spotlight Load on Database Server

Appendix: Contact Quest

Contact Quest Support

Contact Spotlight Support

Contact Quest Software

About Quest Software

Oracle Architecture Overview

Spotlight on Oracle Getting Started Guide

The Quest Software Oracle Solutions Web site.

The Oracle home page.

Spotlight on Oracle Overview

Spotlight on Oracle Getting Started Guide

Spotlight on Oracle Getting Started Guide

The display of real-time data.