Beruflich Dokumente
Kultur Dokumente
1
Screening Room
User Guide
Document:
Created by:
Status:
MM-0016-D
EWJP
Released
Page 1 of 129
03 June 2013
Copyright
Ascom Network Testing Ltd, 2000-2013. All rights reserved
Unauthorized copying, distribution and use of the document or its content is strictly prohibited.
Note: In this guide the term "Monitor Master" refers to the Ascom product "TEMS Monitor Master".
All registered trademarks, quotes, products or service names are acknowledged.
Ascom Network Testing Ltd
Ash House, Shackleford Road
Elstead, Surrey GU8 6LB
United Kingdom
Ascom Network Testing Ltd is an Ascom company, part of the Network Testing division.
www.ascom.com
Version Overview
Version
Date
Author
Modification
1.0
25/04/2013
EWJP
1.1
03/06/2013
EWJP
Minor updates
Information in this document is subject to change without notice. For the latest version, please visit
www.ascom.com/temsmonitormaster/support
Document Management
File name:
screening_room.pdf
Pages:
129
Released on:
03/06/2013
Approved by:
AC
Page 2 of 129
03 June 2013
Table of Contents
CHAPTER 1 - INTRODUCTION .................................................................................................................................... 5
SCREENING ROOM OVERVIEW .......................................................................................................................................... 5
LOCAL MODE AND CENTRAL MODE ................................................................................................................................. 6
MAIN MENU OPTIONS ........................................................................................................................................................ 6
OTHER SCHEDULING FACILITIES ........................................................................................................................................ 7
OTHER REPORTING FACILITIES .......................................................................................................................................... 7
THIS GUIDE ....................................................................................................................................................................... 7
RELATED GUIDES .............................................................................................................................................................. 8
CHAPTER 2 - USING SCREENING ROOM ................................................................................................................. 9
LAUNCHING SCREENING ROOM ........................................................................................................................................ 9
USING SCREENING ROOM ............................................................................................................................................... 11
CHAPTER 3 - THE SCREENING ROOM OPTION .................................................................................................. 16
SCREENING ROOM FUNCTIONAL VIEW ........................................................................................................................... 17
SCREENING ROOM MONITORING VIEW ........................................................................................................................... 33
SCREENING ROOM REGRESSION VIEW ............................................................................................................................ 36
CHAPTER 4 - THE PLAN DASHBOARD OPTION ................................................................................................... 41
CREATING THE PLAN DASHBOARD ................................................................................................................................. 41
INTERPRETING THE PLAN DASHBOARD ........................................................................................................................... 44
CUSTOMISING THE PLAN DASHBOARD ............................................................................................................................ 45
MANIPULATING THE PLAN DASHBOARD ......................................................................................................................... 47
COMPARING PLAN DASHBOARD GRAPHS ........................................................................................................................ 47
CUMULATIVE PLAN DASHBOARD GRAPHS ...................................................................................................................... 50
WATCHING PLAN DASHBOARD DATA ............................................................................................................................. 51
CHAPTER 5 - RECENT TESTS & AVAILABILITY ................................................................................................. 53
USING THE RECENT SUCCESS FACTS SCREEN.................................................................................................................. 53
THE RECENT SUCCESS FACTS DISPLAY OPTIONS ............................................................................................................ 53
THE RECENT SUCCESS FACTS FILTERS ........................................................................................................................... 55
THE RECENT SUCCESS FACTS LINKS............................................................................................................................... 55
THE RECENT SUCCESS FACTS REPORT ............................................................................................................................ 55
CHAPTER 6 - METRICS, MESSAGES & TIMINGS ................................................................................................. 57
USING THE RECENT METRICS, MESSAGES OR TIMING FACTS SCREEN ............................................................................ 57
THE RECENT METRIC/TIMING/MESSAGE FACTS DISPLAY OPTIONS ................................................................................ 58
THE RECENT METRIC FACTS SCREEN .............................................................................................................................. 59
THE RECENT TIMING FACTS SCREEN .............................................................................................................................. 61
THE RECENT MESSAGE FACTS SCREEN ........................................................................................................................... 63
CHAPTER 7 - DIMENSIONS ........................................................................................................................................ 67
PLANS ............................................................................................................................................................................. 67
TESTS .............................................................................................................................................................................. 67
TRANSACTIONS ............................................................................................................................................................... 68
METRICS ......................................................................................................................................................................... 68
TIMINGS .......................................................................................................................................................................... 69
MESSAGE TYPES.............................................................................................................................................................. 69
CHAPTER 8 - LOCAL SCHEDULES ........................................................................................................................... 70
LOCAL SCHEDULES ......................................................................................................................................................... 70
ADDING A LOCAL SCHEDULE .......................................................................................................................................... 72
EDITING A LOCAL SCHEDULE.......................................................................................................................................... 75
DELETING A LOCAL SCHEDULE....................................................................................................................................... 76
CHAPTER 9 - DEVICE UPDATE ................................................................................................................................. 77
WHY USE DEVICE UPDATE? ............................................................................................................................................ 77
USING THE DEVICE UPDATE OPTION ................................................................................................................................ 77
Page 3 of 129
03 June 2013
Page 4 of 129
03 June 2013
Chapter 1 - Introduction
Note: Screening Room is a legacy tool, retained mainly to support users running scripts in legacy (local) mode;
most Monitor Master QM users should not need to use Screening Room - or read this guide!
In this Chapter:
Screening Room
Local Mode and Central Mode
Main menu options
Other reporting facilities
Other scheduling facilities
This guide
Related guides
Test run results and other data stored on this Agent. The wide range of facilities allows users to view,
explore and produce reports on data to whatever level of detail they wish. Data can also be exported in a
variety of formats for use in other applications.
Device profile updates for this Agent, to keep the device details up to date with the Device Master Live
Server or other trusted authority
Resource management for this Agent, to help avoid conflicts between Local Mode test runs
Page 5 of 129
03 June 2013
Chapter 1 - Introduction
Screening Room provides reports, resource management and scheduling for the Agent's Local Mode
scripts. (The reports can also be used for Central Mode resuilts.)
Quality Manager (QM) provides superior central reports, resource management and scheduling for all
Central Mode scripts. (The reports can also use Local Mode results.)
In a Monitor Master QM system, we strongly recommend the use of Central Mode for all scripts. You should only
use Local Mode (and Screening Room) if QM is not available.
WARNING: Do not schedule both Local Mode and Central Mode tests on the same Agent; the resource
management strategies used are incompatible.
Screening Room - review summaries and details of recent test runs, with drilldown to transaction and
content level. Also provides access to Scheduling facilities for local mode scripts
Page 6 of 129
03 June 2013
Chapter 1 - Introduction
Recent tests & availability - a tabular summary of recent tests indicating any problems
Dimensions - view your test data along different dimensions: by plan, test, transaction, metric, timing, or
message type
Device update - download the latest device profiles from the Device Master Live Server
Agent resource management - managing the resources on this Agent to avoid conflicts
Each of these main options has its own chapter later in this guide.
Quality Manager schedules Central Mode scripts to run on single or multiple Agents. These Central
Schedules support both single-party and multi-party scripts
Screening Room schedules Local Mode scripts to run on a single Agent. These Local Schedules do not
support mutli-party Scripts
Box Office provides a range of standard Dashboards (reports) for both Agent and Projector results, as well
as development and integration facilities for custom Dashboards. Dashboards can be accessed via the Box
Office GUI, or from the Quality Manager Reporting menu.
Screening Room provides access to all recent test run results (and other data) stored on an Agent.
Story Boarder provides access to summary results for the current test run only. This is mainly intended for
test script developers to test their scripts, but can also be used for ad hoc testing and monitoring in small
installations.
Projection Room provides access to test run results collected from multiple Agents by a Projector, usually
over a much long time period than Screening Room.
Box Office is the recommended reporting interface for all Monitor Master installations.
Note that the results from Local Mode and Central Mode test runs are stored in the same way (on the Agent that
ran the test, and associated Projector).
This guide
This guide consists of the following chapters:
Chapter 1 - Introduction
Page 7 of 129
03 June 2013
Chapter 1 - Introduction
Related guides
Read the QM User Guide for information on using Central Mode resource management, scheduling and reporting.
Read the Story Boarder User Guide for information on creating and running test scripts, including how to view and
interpret test results.
Read the Projection Room User Guide for information on using legacy enterprise-wide reporting.
Page 8 of 129
03 June 2013
Subsequent chapters describe each of the Home screen (main menu) options in turn.
Note: While a standard Monitor Master installation allows all Users to access all the reports and all the data on an
Agent, your system administrator may implement measures to control access to certain types of report and test
data.
From a Shortcut
From an Agent
Page 9 of 129
03 June 2013
Select the View Agent option from the Story Boarder Tools menu or click the View Agent button, to launch
Screening Room for your most recently-used Agent.
Or:
Select the Agent and click OK to launch the Screening Room for that Agent.
(If the Agent is not listed, click Add, type its network address or (routable) name and click OK to add it to
the list.)
Now use View Agent to launch Screening Room for the selected Agent.
In the address bar, type the URL of the Agent you want to connect to, or its (routable) name, followed by
:9090 and press Enter
For example, to use an Agent whose address is 10.1.1.85 and URL is agent3.xyz.com, you could type:
http://10.1.1.85:9090/
or
http://agent3.xyz.com:9090/
Or, if the name agent3 is a routable name resolving to this address, you could simply type:
http://agent3:9090/
Connect to Screening Room on the Agent you want to use by typing its URL (example)
Page 10 of 129
03 June 2013
Once the Screening Room Home page is displayed, select the Add to Favorites option from your Browser's
Favorites menu.
Edit the name if you wish, then click OK to add it your main Favorites menu. (Refer to your Browser Help
for the other options.)
You can now launch the Screening Room for this Agent by selecting this option from the Favorites menu.
To add the Shortcut to your desktop:
Select the option from the Browser's Favorites menu and drag it to the desktop.
You can now launch the Screening Room for that Agent by double-clicking the desktop shortcut.
Click the Start button and select the option Ascom/TEMS Products / TEMS Monitor Master Servers /
Server Reports / Script Server Database Reports
To print the current screen, use the Browser's Print button or Print command (in the File menu) - see
Printing
Page 11 of 129
03 June 2013
To use more of your screen for results, you can hide toolbars by selecting options from the View menu
If a panel has scrollbars, use them to scroll the content in that panel
And so on.
Screening Room uses a few controls that you might not know:
Many report windows have a refresh control in the top right corner. You can use this to choose how often
the displayed results will be updated (by fetching new data from the Agent) - see Refresh control
A button means there's more information hidden below this item; click the button to reveal it; a
means you can hide the information below this item; click the button to hide it.
Many window elements are "hot" i.e. hyperlinked. If the mouse pointer changes from an arrow to a hand
while over an element, you can click it to link to related information.
Column header buttons are displayed along the top of most tables, like this:
button
These have two main functions: to show what data is in that column, and to control how the table is sorted.
Click a button to sort the table on that column; click it again to reverse the sort order. A triangle shows the
sort column and sort direction
Windows are divided up into panels by narrow borders. You can drag these borders to change the panel's
size.
Each panel has its own right-click (popup) menu, providing access to a range of useful facilities for that
panel and its contents.
When you select an option from a dropdown list, its name is also displayed on the Browser status bar - to
help you choose names that are too long to fit in the list.
Each Screening Room screen is implemented as a frameset i.e. a group of frames (which we call panels here), so the
print options tab provides useful options to:
(Some menus have additional options, but they all have this row of buttons along the bottom.)
Page 12 of 129
03 June 2013
(This facility is provided by the Screening Room screens: Functional, Regression, and Monitoring views.)
Right-click within the panel to display a popup menu ending with these buttons:
The other screens (Recent facts, Dimensions, Dashboard and Metrics/ Messages/ Timings) present a single data
stream, so you can output the whole screen. In the top right corner of the screen is a menu of output formats.
Choose the format you want:
Rich XML will output semantically richer XML to the Browser window
Excel will output an Excel format spreadsheet. You will be asked whether to Open or Save the data. Open
will display it as an embedded spreadsheet, in the current screen; Save will output the data to the Excel
format file you specify.
Refresh control
Most reporting screens have a Refresh control in the top right corner:
Page 13 of 129
03 June 2013
You can use this to choose how often to update the screen, to reflect new results logged by the Agent. The options
are:
Warning: The shorter refresh times should only be used for short periods, when you really them, as each refresh
requires significant additional processing by the Agent. Refreshing every 3 seconds may disrupt the Agent's main
task of running Tests and collecting Testrun data.
However, note that certain reports specify a fixed time period to report on. Once you're past the period end, the
report on the screen will no longer change when refreshed, because there will be no more new data for the selected
period! To avoid this problem, either choose an end period in the future, edit the end date at regular intervals, or
use a report with a moving time window such as the Watch facility or Recent test & availability report.
Screening Room - use to review details of previous test runs, see Chapter 3, Screening Room option. Also
provides access to the test scheduling facilities - see Chapter 8, Local Schedules
Plan dashboard - use to summarize results for specific plans or tests. Also provides direct links to recentlyadded plans. See Chapter 4, Plan Dashboard option
Page 14 of 129
03 June 2013
Recent tests & availability - a tabular summary of recent tests. Also provides direct links to recent failed
tests. See Chapter 5, Recent Tests & Availability option
Metrics, messages and timings - detailed results from multiple tests. See Chapter 6, Metrics, Messages and
Timings option
Dimensions - view your test data along different dimensions: by plan, test, transaction, metric, timing, or
message type. See Chapter 7, Dimensions options
Device update - download the latest device profiles from the main Device Master Live Server. See Chapter
9, Device Update option
Agent resource management - managing the resources on this Agent to avoid conflicts. See Chapter 10,
Agent Resource Management option
You can return to the Home screen from most other Screening Room screens by clicking the Home link, in the top
right corner of the screen.
Page 15 of 129
03 June 2013
Once you've selected a view, you can switch easily to another view:
Right-click anywhere in the Monitor Master banner near the top of the window:
Page 16 of 129
03 June 2013
Choose the view you want (Functional, Monitoring, or Regression), or click Agent home to return to the
Home (main menu) screen.
Each view will now be described in detail in its own section. They have a lot in common so to avoid repetition, we
will describe Functional View in detail, then describe how Monitoring View and Regression View differ.
Select the Functional link from the Screening Room Home screen
Or
To switch from Regression View or Monitoring View: right-click near the top of the Screening Room
window (on the Monitor Master banner) to display a popup menu and select the Functional view option
Page 17 of 129
03 June 2013
Panels 3-7 are empty initially. They display details of a testrun once you've selected one in the Test Runs panel.
You can use this view to list and manage the Plans, Tests and Schedules on this Agent. You can also use it to help
select testruns to report on.
To list the Test Scripts within a Plan:
button.)
Click the
button on its Schedules folder. (If there is no Schedules folder, the Plan has no schedules.)
Page 18 of 129
03 June 2013
Click the
button on the Agent schedules folder (this folder is listed after all the Plan folders).
Click the This agent folder. Details will be displayed in the Test Runs panel.
Click its name. Details will be displayed in the Test Runs panel.
In a Plan folder, scheduled Tests are indicated by a small clock in the icon:
- Test is not scheduled (no clock)
- Test is scheduled but schedule is disabled
- Test is scheduled and schedule is enabled
In the Plans' Schedules folders and Agent schedules folder Schedule status is shown by icon color:
- Schedule is enabled
- Schedule is disabled
For information on creating and maintaining Schedules, see Chapter 8, Local Schedules.
Popup menus
If you right-click anything within this panel you will see a popup menu with four buttons along the bottom:
Refresh panel - fetch new data from the Agent and redisplay the panel using it
Print panel - print the panel as a report. Displays the standard Print options dialog
Open as Excel - output the data in Excel (spreadsheet) format. You can choose to Open the data
(display it as an embedded spreadsheet, in the same panel) or Save the data (output to a Excel format file
you specify)
Open as XML - output the data in XML format, in another Browser window. You can then view, print,
or save it using the standard Browser facilities
The other options, at the start of the menu, are used to manipulate the selected item, as follows.
Agent popup menu
If you right-click This agent, the popup menu also offers:
Agent reports - displays the Report on Agent dialog. Select the type of report required, then click Launch:
Plan and resource usage - show resource usage by all schedules on this Agent, useful for seeing
detecting conflicts
Applied compliance test cases - lists the compliance tests applied to devices in the Agent's device
database
Current licence violations - lists any attempted violations of your Monitor Master license
Recent logs - display a log for this Agent, showing all recent requests (from clients) and responses (from the
Agent). This log is mainly intended for diagnostic purposes
Page 19 of 129
03 June 2013
Plan dashboard - display the Dashboard for this Plan (see Using Plan Dashboard)
Delete this plan - from this Agent. Use with caution! Deleting a Plan will remove all its Test Scripts and
Testrun results from the Agent!
Test dashboard - display the Dashboard for this Test (this is the same as the Plan Dashboard, but with the
Plan and Test pre-selected - see Plan Dashboard reports).
Run test - displays the Run Test dialog for you to confirm (or change) the test selected, supply runtime
parameters, and enable or disable routing of results to Projector. Click OK to run the Test Script - results will
be displayed in the Testrun pane.
Edit test - displays the Edit Test dialog for you to confirm (or change) the test selected. Click OK to edit the
Test Script.
Move test - displays the Move Test dialog for you to confirm (or change) the test selected and choose the
Plan to move it to. Click OK to move the Test Script and associated results.
Delete test - from this Plan. Use with caution! Deleting a Test will delete its script and all its testrun results
from this Agent.
Use the first dropdown list to chose how to view your results:
Tabular summary of - view a tabular summary of all results from the selected tests, and grand totals.
Provides a detailed numerical summary of results in table form - see The tabular summary view
Page 20 of 129
03 June 2013
Extended summary of - view a graphical summary of all results from the selected tests, and grand totals.
Provides an "at a glance" summary of results in pie chart form - see The extended summary view
Each testrun from - (default) view a list of individual testruns, showing detailed results for each. Allows
you to quickly select a testrun of interest and drill down into its results, content, metrics, diagnostics, etc. see The each testrun view
Use the second dropdown list to select the results (testruns) to view. The options are:
All tests and plans on this agent - (default) all runs of all Tests on this Agent
(You can also select the results to view by selecting a Plan or Test from the Plans and Tests panel.)
Use the third dropdown list to select the time period to report on. The options, which are self-explanatory, are:
Click the
The table starts with grand totals (for all selected tests), followed by a row for each selected test, displaying:
To sort by a column:
Page 21 of 129
03 June 2013
Right-click anywhere in the panel to display a popup menu with these options:
Tear-off - view the panel in another browser window
Refresh panel - fetch new data from the Agent and redisplay the panel using it
Print panel - print the panel as a report. Displays the standard Print options dialog
Open as Excel - output the data in Excel (spreadsheet) format. You can choose to Open the data
(display it as an embedded spreadsheet, in the same panel) or Save the data (output to a Excel format file
you specify)
Open as XML - output the data in XML format, in another Browser window. You can then view, print,
or save it using the standard Browser facilities
The "extended summary" view
The extended summary view provides a graphical summary of results for the selected tests, like this:
The first row is a grand total for all the selected testruns. Each subsequent row summarises results for a different
selected test.
The information shown is:
Click a row to display that test's runs in the "each testrun" view.
Right-click anywhere in the panel to display a popup menu with these options:
Tear-off - view the panel in another browser window
Page 22 of 129
03 June 2013
Refresh panel - fetch new data from the Agent and redisplay the panel using it
Print panel - print the panel as a report. Displays the standard Print options dialog
Open as Excel - output the data in Excel (spreadsheet) format. You can choose to Open the data
(display it as an embedded spreadsheet, in the same panel) or Save the data (output to a Excel format file
you specify)
Open as XML - output the data in XML format, in another Browser window. You can then view, print,
or save it using the standard Browser facilities
The "each testrun" view
The each testrun view lists individual testruns in the selected sort order and provides facilities to drill down to
content, metrics, etc. for a selected testrun:
To change the sort order, simply click the column button to sort by.
To drill down, select a testrun by clicking it - this will display details about the run in the Testrun Summary and
Contents panels, described below.
The data shown is as follows:
Plan - the name of the Plan this Script was run from
Cmp - (completion) an icon indicating whether the run completed (green) or failed (red). A red icon
suggests a problem with the test script
Srv - (service) an icon indicating whether the service was available for the entire run (green) or not (red) or
unknown (grey). A red icon suggests a problem with the service under test
Sys - (system) an icon indicating whether the test system was available for the complete run (green) or not
(red) or unknown (grey). A red icon suggests a problem with the test system
Duration - how long the test ran for. A red background indicates a long run, while a green background
indicates a short run
State - whether the run succeeded, failed, was canceled or terminated due to a scripting error
Owner - the user ID and IP address of whoever ran the test. Tests that were run manually (from Story
Boarder) or by a Local Schedule will be listed as owned by SYSTEM (the IP address should help identify the
owner)
GPS - if available (via a GPS system), the co-ordinates where the test was run. This information is mainly
used by mobile test systems
Page 23 of 129
03 June 2013
The panel will normally display up to 100 test runs (rows). To change the number:
Popup menu
Right-click any testrun in the Test Runs Panel to display this popup menu:
Use the first three options (Mark all, Mark incompletes, and Unmark all) to mark testruns. You can also
mark or unmark runs by clicking the checkbox at the start of each line
Use Report marked to create a report combining the marked runs. This displays the Report Testrun dialog,
for you to select the report type, from:
Note: To see an example of each of the above reports, see Appendix B: Testrun Reports.
Page 24 of 129
03 June 2013
The small buttons along the bottom of the menu are as follows:
View detail - view testrun details in the Testrun Summary and Content panels
View detail tear-off - view testrun details in another browser window
Refresh panel - fetch new data from the Agent and redisplay the panel using it
Print panel - print the panel as a report. Displays the standard Print options dialog
Open as Excel - output the data in Excel (spreadsheet) format. You can choose to Open the data
(display it as an embedded spreadsheet, in the same panel) or Save the data (output to a Excel format file
you specify)
Open as XML - output the data in XML format, in another Browser window. You can then view, print,
or save it using the standard Browser facilities
You can also display details of a testrun (in the Testrun Summary panel and Content panel) by double-clicking it.
The Completion/service/system icons indicate whether the run completed and whether the service and
system were available for the entire run. Red icons indicate possible problems with the script, service, and
test system.
A Meta tags table, listing the meta tags for the testrun (information about the testrun rather than the results
it produced):
Page 25 of 129
03 June 2013
A Transactions Graph, showing the time taken by each Transaction (e.g. Scene) in the script. Green bars are
successful Transactions, red bars are failed Transactions.
A Transactions Table, showing data for the Transaction selected in the graph. Initially, the main
Transaction (the entire testrun) is selected; to choose another, click its bar on the graph
A Content timings graph, showing the time taken to transfer each content item.
Page 26 of 129
03 June 2013
Popup menu
Right-click anywhere in the Testrun Summary panel to display a popup menu with these buttons:
Report testrun - display a menu of all the other reports available for this testrun. Select the report you
want and click Launch to run it.
New Screening Room - open the current report on another Screening Room window
Tear-off - view panel in another browser window
Refresh panel - fetch new data from the Agent and redisplay the panel using it
Print panel - print the panel as a report. Displays the standard Print options dialog
Open as Excel - output the data in Excel (spreadsheet) format. You can choose to Open the data
(display it as an embedded spreadsheet, in the same panel) or Save the data (output to a Excel format file
you specify)
Open as XML - output the data in XML format, in another Browser window. You can then view, print,
or save it using the standard Browser facilities
The Content panel lists the content items downloaded for the selected Test Run, showing the following for each
item:
ID: Its ID
Page 27 of 129
03 June 2013
If the content has been analysed by using the Basic Content Tests Scene or MobileOK Scene, you'll see 5
additional columns:
Note: If any content pages includes a redirection command, Monitor Master turns it off (by replacing the command
with a null command), so you stay on the intended page when you view it. You can turn redirection for the entire
web site on or off by right-clicking any content from it and selecting the appropriate option from the popup menu.
Popup menu
Right-click a content item to display a popup menu with these buttons:
Launch raw content - view content in the appropriate viewer, if there is one installed on this computer
Start replay - view details of each content item in turn (as if you had clicked each one in turn)
View detail - view details of this item in the Content Summary, Content Detail and Preview panels
View detail tear-off - view details of this item in another browser window
Refresh panel - fetch new data from the Agent and redisplay the panel using it
Print panel - print the panel as a report. Displays the standard Print options dialog
Open in Excel - output the data in Excel (spreadsheet) format. You can choose to Open the data
(display it as an embedded spreadsheet, in the same panel) or Save the data (output to a Excel format file
you specify)
Open as XML - output the data in XML format, in another Browser window. You can then view, print,
or save it using the standard Browser facilities
Page 28 of 129
03 June 2013
You can also view details of a content item (in the Content Summary, Content Detail and Preview panels) by
clicking it.
Content Summary - time sent, character set, delay (time taken), size and rate
Timings - times (clock times) and delays (time taken) for the associated transaction(s)
Metrics - bytes sent/received, service and system availability, etc. for the associated transaction(s)
For example:
Page 29 of 129
03 June 2013
Popup menu
Right-click anywhere in the pane to display a popup menu with these buttons:
Tear-off - view panel in another browser window
Refresh panel - fetch new data from the Agent and redisplay the panel using it
Print panel - print the panel as a report. Displays the standard Print options dialog
Open in Excel - output the data in Excel (spreadsheet) format. You can choose to Open the data
(display it as an embedded spreadsheet, in the same panel) or Save the data (output to a Excel format file
you specify)
Open as XML - output the data in XML format, in another Browser window. You can then view, print,
or save it using the standard Browser facilities
Page 30 of 129
03 June 2013
For example:
Page 31 of 129
03 June 2013
Popup menu
Right-click anywhere in the panel to display a popup menu with these buttons:
Tear-off - view panel in another browser window
Refresh panel - fetch new data from the Agent and redisplay the panel using it
Print panel - print the panel as a report. Displays the standard Print options dialog
Open in Excel - output the data in Excel (spreadsheet) format. You can choose to Open the data
(display it as an embedded spreadsheet, in the same panel) or Save the data (output to a Excel format file
you specify)
Open as XML - output the data in XML format, in another Browser window. You can then view, print,
or save it using the standard Browser facilities
Page 32 of 129
03 June 2013
Popup menu
This panel does not have its own popup menu. If you right-click it you will see the standard Browser popup menu.
Select the Monitoring link from the Screening Room Home screen
Or
To switch from Regression View or Functional View: right-click near the top of the Screening Room window
(on the Monitor Master banner) to display a popup menu and select the Monitoring view option
Page 33 of 129
03 June 2013
button
Click the
button on its Schedules folder. (If there is no Schedules folder, the Plan has no schedules.)
Click the
button on the Agent schedules folder. (This folder is listed after all the Plan folders).
Click the This agent folder. Details will be displayed in the Test Runs panel.
Click its name. Details will be displayed in the Test Runs panel.
Page 34 of 129
03 June 2013
For details of the Plans and Tests panel, refer to its description in Functional View.
Across the top of the panel are three dropdown lists and a button, which are used to select the information to
display in the remainder of the panel:
Use the first dropdown list to chose how to view your results:
Tabular summary of - view a tabular summary of all results from the selected tests, and grand totals.
Provides a detailed numerical summary of results in table form
Extended summary of - view a graphical summary of all results from the selected tests, and grand totals.
Provides an "at a glance" summary of results in pie chart form
Each testrun from - view a list of individual testruns, showing detailed results for each. Allows you to
quickly select a testrun of interest and drill down into its results, content, metrics, diagnostics, etc.
All tests and plans on this agent - all runs of all Tests on this Agent
(You can also select the results to view by selecting a Plan or Test from the Plans and Tests panel.)
Use the third dropdown list to select the time period to report on. The options, which are self-explanatory, are:
Click the
Key facilities
Use the three dropdown lists across the top of the pane to select the view type (numerical summary,
graphical summary, or each testrun), the tests to view results for, and the period to view results for - then
click the
button. The each testrun view is best for monitoring.
Click any testrun (in "each testrun" view) to display details in the Testrun Summary panel - described next.
Page 35 of 129
03 June 2013
For details of the Test Runs panel, refer to its description in Functional View.
A Testrun summary for this run. The Completion/service/system icons indicate whether the run completed
and whether the service and system were available for the entire run. Red icons indicate possible problems
with the script, service, or test system.
A Meta tags table, listing the meta tags for the testrun (information about the testrun rather than the results
it produced).
A Transactions graph showing the time taken by each transaction (e.g. Scene) in the script.
A Transactions table showing metrics and messages for the selected transaction (e.g. Scene). (Click a
transaction's bar in the Transactions graph to select it.)
A Content timings graph, showing the time taken to transfer each content item.
For details of the Testrun Summary panel, refer to its description in Functional View.
Select the Regression link from the Screening Room Home screen
Or
To switch from Monitoring View or Functional View: right-click near the top of the Screening Room
window (on the Monitor Master banner) to display a popup menu and select the Regression view option
Page 36 of 129
03 June 2013
2.
3.
4.
5.
6.
Test Runs
Testrun Summary (lhs)
Content (lhs)
Testrun Summary (rhs)
Content (rhs)
button
Click the
button on its Schedules folder. (If there is no Schedules folder, the Plan has no schedules.)
Click the
button on the Agent schedules folder (this folder is listed after all the Plan folders).
Click the This agent folder. Details will be displayed in the Test Runs panel.
Click its name. Details will be displayed in the Test Runs panel.
For details of the Plans and Tests panel, refer to its description in Functional View.
Across the top are three dropdown lists and a button, used to select the information to display in the remainder of
the panel:
Use the first dropdown list to choose how to view your results:
Page 37 of 129
03 June 2013
Tabular summary of - view a tabular summary of all results from the selected tests, and grand totals.
Provides a detailed numerical summary of results in table form
Extended summary of - view a graphical summary of all results from the selected tests, and grand totals.
Provides an "at a glance" summary of results in pie chart form
Each testrun from - view a list of individual testruns, showing detailed results for each. Allows you to
quickly select a testrun of interest and drill down into its results, content, metrics, diagnostics, etc.
The each testrun view is the one most often used for regression testing.
Use the second dropdown list to select the results (testruns) to view. The options are:
All tests and plans on this agent - all runs of all Tests on this Agent
(You can also select the results to view by selecting a Plan or Test from the Plans and Tests panel.)
Use the third dropdown list to select the time period to report on. The options, which are self-explanatory, are:
Click the
Key facilities
Use the three dropdown lists across the top of the pane to select the view type (numerical summary,
graphical summary, or each testrun), the tests to view results for, and the period to view results for - then
click the
button. The each testrun view is best for regression testing.
Click any testrun (in the each testrun view) OR right-click it and click the
View detail (LHS) button in
the popup menu to display the results and content from a Test Run in the Testrun Summary and Content
panels on the left-hand side
Right-click any testrun (in the each testrun view) and click the
View detail (RHS) button in the popup
menu to display the results and content for another Test Run in the Testrun Summary and Content panels
on the right-hand side - producing a display like this
Page 38 of 129
03 June 2013
For details of the Test Runs panel, refer to its description in Functional View.
Popup menu
Right-click a testrun to display this popup menu for it:
Use the first three options (Mark all, Mark incompletes, and Unmark all) to mark testruns. You can also
mark or unmark a testrun by clicking the checkbox at the start of its row
Use Report marked to create a report combining the marked runs. This displays the Report Testrun dialog,
for you to select the report type.
The small buttons along the bottom of the menu are as follows:
View details (LHS) - view the testrun details in the left-hand Testrun Summary and Content panels
View detail (RHS) - view the testrun details in the left-hand Testrun Summary and Content panels
Refresh panel - fetch new data from the Agent and redisplay the panel using it
Print panel - print the panel as a report. Displays the standard Print options dialog
Open as Excel - output the data in Excel (spreadsheet) format. You can choose to Open the data
(display it as an embedded spreadsheet, in the same panel) or Save the data (output to a Excel format file
you specify)
Page 39 of 129
03 June 2013
Open as XML - output the data in XML format, in another Browser window. You can then view, print,
or save it using the standard Browser facilities
You can also view details of a testrun (in the left-hand Testrun Summary and Content panel) by double-clicking it.
A Testrun summary for this run. The Completion/service/system icons indicate whether the run completed
and whether the service and system were available for the entire run. Red icons indicate possible problems
with the script, service, or test system.
A Meta tags table, listing the meta tags for the testrun (information about the testrun rather than the results
it produced)
A Transactions graph showing the time taken by each transaction (e.g. Scene) in the script.
A Transactions table showing metrics and messages for the selected transaction (e.g. Scene). (Click a
transaction's bar in the Transactions graph to select it.)
A Content timings graph, showing the time taken to transfer each content item.
For details of the Testrun Summary panel, refer to its description in Functional View.
ID: Its ID
For details of the Content panel, refer to its description in Functional View.
Page 40 of 129
03 June 2013
Comparing graphs
Cumulative graphs
Watching data
Click the Plan dashboard option on the Screening Room Home screen:
You will now be taken through a sequence of steps to select the Plan and information you're interested in.
Hint: If you want to go back to change your mind, use your Browser's Back button or click the underlined
link for the step to return to - for example, click the Select run link to return to Step 2 from a later step.
Step 1: Choose the Plan containing the results to review from the list (e.g. Standard Scripts). Click Next>
Page 41 of 129
03 June 2013
Page 42 of 129
03 June 2013
Step 5: Choose the time period to graph, by choosing the start and end dates and times from the lists. The
dates offered correspond to test run dates for the selected tests.
Step 6: Choose up to 5 metrics to graph, from the lists. You can choose any of the metrics recorded for these
test runs in this time period.
Step 7: Customise the graph appearance. This step is optional, as the defaults are appropriate for most users
and you can quickly adjust any that aren't, once you've seen the default graphs. (See Customising graph
appearance, for details.)
Click Next> to create and display a dashboard view of the selected results:
Page 43 of 129
03 June 2013
A column of graphs for each test script (the graphs selected in step 4)
Page 44 of 129
03 June 2013
The horizontal axis of each graph shows the entire time period selected in step 5, so all graphs have the same X
axis. The vertical axis of each graph shows the values of the chosen metric.
But you can do a lot more, including:
Customising the appearance
Manipulating the graphs
Comparing graphs
Cumulative graphs
Watching data
Page 45 of 129
03 June 2013
Success: choose Not smoothed to plot each success result for each run as a separate point, or Averaged per
minute/hour/day/year to plot averages over time. Averaging makes it easier to spot trends, but might hide
fluctuations or patterns that are important to you
Metrics: choose Not smoothed to plot each metric for each run as a separate point, or Averaged per
minute/hour/day/year to plot averages over time. Averaging makes it easier to spot trends, but might hide
fluctuations or patterns that are important to you
Metric range: if using smoothed results, choose one of these options to add or remove curves showing the
upper and/or lower values:
not smoothed
Size: Choose the graph width (Narrow/Normal/Wide) from the first control, and graph height
(Short/Normal/Tall) from the second
Style: Choose the line style (Thin line or Thick line) from the first control, and the point style (No points,
Small points or Large points,) from the second
Override Y-max: The Y-axis of each graph is chosen automatically so that the highest value for each metric
plotted just fits. If you wish, you can override this process by typing a maximum Y axis value in this box.
You might use this, for example, to normalize all graphs so they are directly comparable, or to remove
spurious data from a graph.
For example, if plotting timings that are all in the range 15-20 seconds, with just one rogue value of 200
seconds, the default Y axis will be 0-200 seconds, compressing the details in the rest of the graph. Type 20 in
the Override Y-max box to ignore the rogue value and reveal the variations in the other values, if they are of
more interest:
Page 46 of 129
03 June 2013
default graph
showing rogue data
Override Y-Max = 20
hiding rogue data
Any data points that are not plotted because they don't fit are indicated by an upwards arrow (as shown in
the right-hand example, above).
Click Next> to redraw the dashboard with the new display options.
Double-click any graph to display the testrun results in another Browser window.
Select an cell, cells, graph or graphs by clicking them. Select an entire row by clicking the Select link in its
left-hand border. You can then analyse the selected metric graphs by using the three buttons in the top left
of the dashboard. to:
The graphs will be combined and displayed as a single graph in another browser window, like this:
Page 47 of 129
03 June 2013
Use the Time period options to change the period to graph, as described above.
Use the Thresholding options to display a visual warning if the data falls outside the thresholds you
specify, as described below.
Use the Customise graph appearance options to change the graph appearance, as described above.
After making any changes, click Next to apply the revised options to the graph, and re-plot it.
Using Thresholds
You can define Thresholds to help you quickly recognise when results fall outside the expected range. They are
normally used in combination with a regular refresh rate (see Refresh control), to monitor key metrics.
To display a visual warning if the most recent data point is above a threshold value:
Select the graph or graphs to check, from the Plan Dashboard screen.
When the comparison graph is displayed, change the thresholding type from None to Upward and two
additional Level controls are displayed:
Type the upper (error) threshold level in the first box, and the lower (warning) threshold level in the second
box.
Now, if the most recent data point (on all the curves) is below both thresholds, the graph will have a green
background:
Page 48 of 129
03 June 2013
If the most recent data point (on one or more curves) is above the lower threshold but below the upper threshold,
the graph will have a yellow (warning) background:
And if the most recent data point (on one or more curves) is above the upper threshold, the graph will have a red
(error) background:
Page 49 of 129
03 June 2013
To display a visual warning if the most recent data point is below a threshold value:
When the comparison graph is displayed, select a thresholding type of Downward and two additional
Level controls are displayed:
Type the upper threshold level (warning) in the first box, and the lower threshold level (error) in the second
box.
Now, if the most recent data point is above both thresholds, the graph will have a green background; if the most
recent data point is below the upper threshold, the graph will have a yellow (warning) background; and if the most
recent data point is below the lower threshold, the graph will have a red (error) background.
Select the metric graph(s) to cumulate by clicking them on the Plan Dashboard. (It only makes sense to
cumulate graphs for the same or similar metrics.)
The selected graphs will be combined and displayed together in another browser window, like this:
Page 50 of 129
03 June 2013
This new window has its own options for configuring the graph:
Use Thresholding to display a visual warning if the data falls outside the thresholds you specify. See Using
Thresholding, above.
Use Cumulative range to change the range plotted along the X-axis.
Use Customise graph appearance to change the graph appearance, as described above.
Select the graph(s) to watch by clicking them on the Plan Dashboard. (It only makes sense to watch graphs
together if they are for the same or similar metrics.)
The graphs will be combined and displayed as a single graph in another Browser window, like this:
Page 51 of 129
03 June 2013
This new window has its own options for configuring the Watch graph:
Use the Time period controls to change the period to graph. Notice that this is defined as a period relative
to now. You can choose to graph the previous:
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 20 or 30 minutes
1, 2, 3, 4, 5 or 6 hours
Use the Thresholding controls to display a visual warning if the data falls outside the thresholds you
specify. See Using Thresholding, above.
Use the Customise graph appearance controls to change the graph appearance, as described above.
Note: Remember to set a suitable refresh period, using the Refresh control, in the top right of the window. The
default (Never refresh) is not suitable.
Page 52 of 129
03 June 2013
Other links
Note: This screen includes options to report on other recent facts (metrics, messages and timings). These reports
can also be accessed directly from the Screening Room Home screen and are described fully in Chapter 6, Metrics,
Messages and Timings.
This screen initially displays the ten most recent success facts for this Agent, but is highly configurable. You can
select two different layouts (tabular and block), report on different facts (success, metrics, timings or messages),
and choose different information sources (plans or tests) to report on.
This chapter describes the Recent Success Facts screen. The other three fact types are described in the next chapter:
Recent Metric Facts
Recent Timing Facts
Recent Message Facts
Note: The Display options are the same for all fact types.
Page 53 of 129
03 June 2013
To change the facts displayed, choose from the first dropdown list:
Success facts - whether the run succeeded or failed. (See The Recent Success Facts report, below.)
To change the report layout, select the layout from the second dropdown list:
Select In rows to produce a wide tabular report, with one row per fact, like this:
Select In blocks to produce a longer, narrower, report, with one block per fact, like this:
The block report is best for printing reports; the tabular report is best for comparing results on the screen.
However, the two formats present exactly the same information and provide exactly the same facilities.
Throughout the rest of this chapter we will use the block report format, as it's more compact.
Page 54 of 129
03 June 2013
Note: The Filter options are different for each fact type.
To report on a single Plan:
Select its name from the below Plan dropdown list. Choose All again to report on all Plans running on this
Agent.
Select its name from the Test dropdown list. Choose All again to report on all Tests.
Type its id (number) in the Testrun box. Delete the contents of this box to report on all Testruns.
Select Yes from the Success filter; to just list failed runs, select No from the Success filter. To list all results,
select All.
(Go)
Click Plan to switch to a Dimensions report on the Plans on this Agent (described in Chapter 7).
Click Test to switch to a Dimensions report on the Tests on this Agent (described in Chapter 7).
Click a testrun ID in the Testrun column to view its summary results (described in Chapter 3).
Finally, you can use the Refresh control to keep the results up-to-date (see Refresh control in Chapter 2).
Page 55 of 129
03 June 2013
To display a detailed report for any testrun, click its ID in the Testrun field.
Page 56 of 129
03 June 2013
Metrics - data measurements for each Scene and the entire test run, such as system availability, total bytes
received, and payload tx/rx rates. May include metrics for Custom Transactions
Messages - messages output during the run, including outcomes (the route taken through the test), error
messages and diagnostic messages (more detailed explanations for errors)
Timings - times for each Scene and the entire test run, including start and end times (clock times) and
interval times (durations); may include Custom Timings
All three options display the Recent Facts screen, but with a different fact type displayed. To avoid repetition, we'll
start with the general principles of how to use the Recent Facts screen, and then describe each of the screens in
turn.
This screen can also display a fourth fact type: recent success facts. This report can be directly accessed from the
Screening Room Home screen (via the Recent tests & availability option) and will not be described in detail here.
For details, see Chapter 5, Recent tests & availability.
This screen initially displays the ten most recent facts of the type you clicked, but is highly configurable. You can
select two different layouts (tabular and block), report on different facts (success, metrics, timings or messages),
and choose different information sources (plans or tests) to report on.
Recent Success Facts are described in Chapter 5.
The screen layout, filters and links for each fact type are described separately after a description of the Display
options, which are the same for all types:
Display options
Recent Metrics Facts
Recent Timing Facts
Recent Message Facts
Page 57 of 129
03 June 2013
Note: The Display options are the same for all fact types.
To change the number of facts displayed:
To change the facts displayed, choose from the first dropdown list:
Success facts - whether the run succeeded or failed. (See Chapter 5.)
To change the report layout, select the layout from the second dropdown list:
Select In rows to produce a wide tabular report, with one row per fact, like this:
Select In blocks to produce a longer, narrower, report, with one block per fact, like this:
***
Page 58 of 129
03 June 2013
The block report is best for printing reports; the tabular report is best for comparing results on the screen.
However, the two formats present exactly the same information and provide exactly the same facilities.
Throughout the rest of this chapter we will use the block report format, as it's more compact.
Click the Metrics option on the Screening Room Home screen, or select the Metrics option from the Display
options on any other Recent Facts screen.
This displays the ten most recent metric facts for this Agent, like this:
Where:
(Not all columns are shown in the example above, as row format is rather wide.)
Page 59 of 129
03 June 2013
Select its name from the below Plan dropdown list. Choose All again to report on all Plans running on this
Agent.
Select its name from the Test dropdown list. Choose All again to report on all Tests.
Type its id (number) in the Testrun box. Delete the contents of this box to report on all Testruns.
Select its name from the Transaction dropdown list. Choose All again to report all Transactions.
Select its name from the Metric dropdown list. Choose All again to report all Metrics.
(Go)
Note: Additional filtering options are available for certain fact types, and are described below.
For example:
To report on a particular metric, select its name from the Metric filter and click its
button.
Page 60 of 129
03 June 2013
Click Plan to switch to a Dimensions report on the Plans on this Agent (described in Chapter 7).
Click Test to switch to a Dimensions report on the Tests on this Agent (described in Chapter 7).
Click a testrun ID (e.g. 248375) to view its summary results (described in Chapter 3).
Click Transaction to switch to a Dimensions report on the Transactions on this Agent (described in Chapter
7)
Click Metric to switch to a Dimensions report on the Metrics on this Agent (described in Chapter 7)
You can also use the Refresh control to keep the results up-to-date (see Refresh control in Chapter 2).
Click the Timings option on the Screening Room Home screen, or select the Timings option from the
Display options on any other Recent Facts screen.
This displays the ten most recent timing facts for this Agent, like this:
Where:
Page 61 of 129
03 June 2013
(Not all columns are shown in the example above, as row format is rather wide.)
Select its name from the below Plan dropdown list. Choose All again to report on all Plans running on this
Agent.
Select its name from the Test dropdown list. Choose All again to report on all Tests.
Type its id (number) in the Testrun box. Delete the contents of this box to report on all Testruns.
Select its name from the Transaction dropdown list. Choose All again to report all Transactions.
Select its name from the Timing dropdown list. Choose All again to report all Timings.
(Go)
Note: Additional filtering options are available for certain fact types, and are described below.
For example:
To report on a particular timing, select its name from the Timing filter and click its
Page 62 of 129
button.
03 June 2013
Click Plan to switch to a Dimensions report on the Plans on this Agent (described in Chapter 7).
Click Test to switch to a Dimensions report on the Tests on this Agent (described in Chapter 7).
Click a testrun ID (e.g. 1117) to view its summary results (described in Chapter 3).
Click Transaction to switch to a Dimensions report on the Transactions on this Agent (described in Chapter
7)
Click Timing to switch to a Dimensions report on the Timings on this Agent (described in Chapter 7)
You can also use the Refresh control to keep the results up-to-date (see Refresh control in Chapter 2).
Click the Messages option on the Screening Room Home screen, or select the Messages option from the
Display options on any other Recent Facts screen.
This displays the ten most recent message facts for this Agent, like this:
Page 63 of 129
03 June 2013
Where:
Type is the message type; there are many different types, including error, warning, and outcome
(Not all columns are shown in the example above, as row format is rather wide.)
Select its name from the below Plan dropdown list. Choose All again to report on all Plans running on this
Agent.
Select its name from the Test dropdown list. Choose All again to report on all Tests.
Type its id (number) in the Testrun box. Delete the contents of this box to report on all Testruns.
Page 64 of 129
03 June 2013
Select its name from the Transaction dropdown list. Choose All again to report all Transactions.
Select its name from the Type dropdown list. Choose All again to report all Timings.
To report on messages that include or do not include specific text, use the Message controls. Each line allows you
to specify a separate condition, which you can then combine with an AND or an OR operation:
Select anywhere or at start from the third element, to produce the first condition.
Select and or or from the fourth element to specify how to combine the two conditions to create the filter..
Select anywhere or at start from the seventh element, to produce the second condition.
For example, to choose messages which start with the text "Service Failure" or include the text "Failed" anywhere
within the message, make the following selections:
(Go)
Page 65 of 129
03 June 2013
Click Plan to switch to a Dimensions report on the Plans on this Agent (described in Chapter 7).
Click Test to switch to a Dimensions report on the Tests on this Agent (described in Chapter 7).
Click a testrun ID (e.g. 1130) to view its summary results (described in Chapter 3).
Click Transaction to switch to a Dimensions report on the Transactions on this Agent (described in Chapter
7)
Click Type to switch to a Dimensions report on the Message Types on this Agent (described in Chapter 7)
You can also use the Refresh control to keep the results up-to-date (see Refresh control in Chapter 2).
Page 66 of 129
03 June 2013
Chapter 7 - Dimensions
The Screening Room Home screen Dimensions option provides six different links:
Plans
Tests
Transactions
Metrics
Timings
Message types
Each option displays a different screen listing all the objects of that type known to the Agent, providing links to
recent results for each object.
Plans
The Screening Room's Dimensions/Plans option lists all the Plans stored on this Agent, in a table like this:
Click a Plan's Dashboard link to display the Dashboard for that Plan. See Chapter 4, Plan Dashboard for
details of this report.
Click a Plan's Successes, Failures, Metrics or Messages link to display the 30 most recent facts of that type
for that Plan. See Chapter 5, Recent tests & availability and Chapter 6, Metrics, messages and timings, for details
of these reports.
Tests
The Screening Room's Dimensions/Tests option lists all the Tests stored on this Agent, in a table like this:
Page 67 of 129
03 June 2013
Chapter 7 - Dimensions
Click a Test's Successes, Failures, Metrics or Messages link to display the 30 most recent facts of that type
for that Test. See Chapter 5, Recent tests & availability and Chapter 6, Metrics, messages and timings, for details
of these reports.
Transactions
The Screening Room's Dimensions/Transaction option lists all the Transactions (Scenes and Custom Transactions)
reported in the results stored on this Agent, in a table like this:
Click a Transaction's Metrics or Messages link to display the 30 most recent facts of that type for that
Transaction. See Chapter 6, Metrics, messages and timings, for details of these reports.
Metrics
The Screening Room's Dimensions/Metrics option lists all the Metrics reported in the results stored on this Agent,
in a table like this:
Page 68 of 129
03 June 2013
Chapter 7 - Dimensions
Click a Metric's Metrics link to display the 30 most recent values for that Metric. See The Recent Metric Facts
screen in Chapter 6 for details of this report.
Timings
The Screening Room's Dimensions/Timings option lists all the Timing types reported in the results stored on this
Agent, in a table like this:
Click a Timing's Timings link to display the 30 most recent values for that Timing. See The Recent Timing
Facts screen in Chapter 6 for details of this report.
Message types
The Screening Room's Dimensions/Types option lists all the Message types reported in the results stored on this
Agent, in a table like this:
Click a Message type's Messages link to display the 30 most recent Messages of that type. See The Recent
Message Facts screen in Chapter 6 for details of these reports.
Page 69 of 129
03 June 2013
Local Schedules
Local Schedules
An Agent's Local Schedules are managed from within Screening Room. To get started:
Run Screening Room for the Agent to manage. (For example, select the View Agent option from the Story
Boarder Tools menu and choose the Agent.)
Choose any of the Screening Room options from the Home screen - Monitoring is probably the most
convenient:
Page 70 of 129
03 June 2013
You use the Tests and Plans panel, top left, to manage Local Schedules. You can enlarge the panel by
dragging its lower and right-hand borders if it's too small.
Open a Plan's Schedules folder to see the Local Schedules for that Plan:
Page 71 of 129
03 June 2013
There is also a Schedules folder (in the This agent folder) that lists all the Schedules on this Agent.
Schedule icons
To help you quickly recognise which Tests are Scheduled and which Schedules are active, Monitor Master uses the
following conventions in this panel:
In lists of Tests (within the Plan folders):
A Scheduled Test has a small clock added to its icon, like this:
If the Schedule is disabled, the description has a grey background, like this:
To display information about a Schedule, hover over it (move the mouse pointer to it and wait). A popup box will
be displayed like this:
We'll now describe in detail how to manage Local Schedules i.e. how to:
Run Screening Room for the Agent and choose one of three Screening Room options (e.g. monitoring) from
the Screening Room Home page.
Right-click the Schedules folder in the Plans and Tests panel, to display this popup menu:
Note: To schedule a Test from a Plan, right-click the Plan's Schedules folder; to schedule any Test on this
Agent, right-click the Schedules folder in the This agent folder.
Select the Add Schedule option to display the Edit Schedule dialog:
Page 72 of 129
03 June 2013
Type a name for the new schedule in the Name box, such as Hourly SMS check.
Select the Test to schedule from the Test list. (See Note above.)
To enable the Schedule (so it will run once saved) leave Status set to Enabled; to disable the Schedule (so it
won't run until enabled), change Status to Disabled.
To limit the time the Test will be allowed to run for, type a time limit (in minutes) in the Duration/mins box.
If you do not want to enforce a time limit, leave this set to 0.
This feature can be used to stop multiple scheduled tests from interfering with each other. This can become
important when sharing resources such as a dial up modem, to avoid conflicts
Choose how often you want to run the test from the Strategy dropdown list. The standard options are:
Every minute
Every 5 / 10 / 15 / 30 minutes
Click Add to add the new Schedule to the Agent Schedules folder in the Plans and Tests panel.
Note: To pass parameters to the Test Run, see Test Parameters, below.
Page 73 of 129
03 June 2013
The new entry in the Agent schedules folder will show the following information:
A list of the times (minutes past the hour) the Test will run
Custom Schedules
If none of the standard scheduling strategies fits your needs:
Select [Custom] as the Strategy then click the More button to expand the Edit Schedule dialog:
You can now specify a wider variety of scheduling patterns by typing in the Minutes, Hours, Days, Months and
Weekdays fields:
Months: type the month(s) of the year to test (1-12). 1 is January, 2 is February, etc.
Weekdays: type the day(s) of the week to test (1-6). 0 is Sunday, 1 is Monday, etc.
Type the * wildcard to test at every matching period. For example, type * in the Hours field to schedule a
test for every hour; type * in the Days field to test every day - provided the other settings match.
Type a numeric value in a field to test only when that value is matched. For example, type 15 in the Hour
field to test at 3 pm - provided the other settings match.
Type the - symbol to specify a range. For example, type 8-12 in the Hour field to test at 8, 9, 10, 11 and 12 provided the other settings match.
Use the , symbol to separate values in a list. For example, type 1,15 in the Days field to test on the 1st and
15th days of the month - provided the other settings match.
Page 74 of 129
03 June 2013
Type 15 in Hours
Test parameters
To pass parameters to the test run:
Type the parameters to pass in the Test parameters field, separated by commas.
Run Screening Room for the Agent and choose one of three Screening Room options (e.g. monitoring) from
the Screening Room Home page.
Open the Schedules folder in the Plans and Tests panel, top left.
Note: To edit a Schedule from a Plan, right-click the Plan's Schedules folder; to edit any schedule on this
Agent, right-click the Schedules folder in the This agent folder.
Right-click the Schedule to edit. This will display this popup menu:
Page 75 of 129
03 June 2013
This operates just like when adding a schedule. See Adding Schedules (above) for details.
Make any changes you require and click Apply to save them.
The changes you make will take effect immediately. However, if a scheduled test run has already started, it will be
unaffected.
Run Screening Room for the Agent and choose one of three Screening Room options (e.g. monitoring) from
the Screening Room Home page.
Open the Schedules folder in the Plans and Tests panel, top left
Note: To edit a Schedule from a Plan, right-click the Plan's Schedules folder; to edit any schedule on this
Agent, right-click the Schedules folder in the This agent folder.
Right-click the Schedule to delete. This will display this popup menu:
The deletion will take effect immediately, cancelling any test runs that were scheduled but have not yet started.
However, if a scheduled test run has already started, it will be unaffected.
Hint: If you just want to stop a scheduled test for a while e.g. while on holiday, instead of deleting it you can edit it
to change its status to Disabled.
Page 76 of 129
03 June 2013
Run Screening Room for the Agent. (For example, select the View Agent option from the Story Boarder
Tools menu and choose the Agent.)
Page 77 of 129
03 June 2013
This will display the Device Management screen, which looks like this:
Page 78 of 129
03 June 2013
Add/update from file - used to take updates provided as individual files. Typically used for small
updates or where network access is not available. Supports .dmdp, .dmda, and .zip files.
Update from authority - used to update by comparing your current data against a trusted source.
Typically used to update automatically from the Device Master Live Server on a regular basis
To add a new device profile or update an existing device profile from a local file:
Either type the file pathname or click the Browse button and browse to it
Check all devices from an authority by clicking the Mark all devices button at the top of their table
Check all devices from all authorities by clicking the Mark all authorities button at the top of the
section
WARNING: Updating large numbers of profiles can take a long time! Updating 500 profiles over a typical
internet connection might take 5-6 hours.
Either click the Check for device updates of marked authorities button at the top of the section, to perform
a device update
Or click the Update analysis data of marked authorities button at the top of the section, to update analysis
data
When the update has finished, a summary dialog will be displayed - click OK to close it.
Updating a Legacy Profile converts it to a Next Generation Profile; updating a Next Generation Profile downloads
the latest newer version.
Page 79 of 129
03 June 2013
Select the Agent Resource Management option from the Screening Room Home screen (main menu) to
display the Agent Resource Management screen:
This provides access to resource management setup and reporting facilities for the Agent's resources. Setting up
Resource Management also allows you to use the Resource Management functions in test scripts so they can share
resources without conflict.
Management options
The Agent Resource Management screen provides five main options:
Resource items - define the resources on this Agent that you want to manage
Resource users - define the users of this Agent that you want to manage
Resource allocations - allocate (reserve) resources for specific users (or the system), for a specific time
Resource usage - display a report that shows when each resource was actually in use (booked by a script)
Resource graphing - create a graph of resource allocations and usage to help planning
Page 80 of 129
03 June 2013
For details, see the ResourceAcquire and ResourceRelease functions in Chapter 3 of the Advanced Scripting Guide
and Resource Management in Chapter 4 of the Advanced Scripting Guide.
it is not already acquired by another test run (using the ResourceAcquire function)
it is not already allocated to another user at this time (using the Screening Room resource management
options)
If resource acquisition fails, the code in the Advanced Script Scene might attempt to acquire a different resource, or
try to acquire the same resource again after a delay, or exit with a System error which subsequent Scenes should
handle accordingly. (For details, see Resource Management in the Advanced Scripting Guide.)
Key concepts:
resource item (resource) - any item that you want to manage, but specifically an item that's needed for a test
run but can only be used by one run at a time
resource allocation - requesting exclusive use of a resource for a specific user, for a specific period
resource management functions - script functions that can be used to acquire (claim) a free resource before
using it, then release it again once no longer needed
dynamic booking - claiming a free resource "now" from a running script by using the ResourceAcquire
resource management function (described in the Advanced Scripting Guide)
static booking - claiming a resource for a specific time period by using the Screening Room resource
allocation facility (described in this section)
allocation vs booking - you allocate a resource from Screening Room, for future use by a user; a script books
a resource for its own use when it runs by using the ResourceAcquire function. A script can book a free
resource even if that resource is not allocated to the user running it - allocating simply helps make sure the
resource is free for that user (and their scripts) to use
Notice that you don't have to define resource users or allocate resources to use Resource Management - once you
have defined the resources you want to manage by creating resource items for them (as described below), your test
scripts can use the resource management functions to book and release resources, to share them. However, the
other resource management facilities described below will allow you to set up a more efficient system of resource
sharing. For example, by manually allocating resources you can taken control of which scripts can use which
resources, when, so users know when they can run their scripts without resourcing problems.
Resource items
To define the resources on an Agent so you can manage their use, you must first add them as Resource Items:
Page 81 of 129
03 June 2013
This will list the current resources on this Agent (initially none), on the Resource items screen:
Once you have added some items, additional options are displayed for each:
button or the Add new item link - see Adding a resource item.
Page 82 of 129
03 June 2013
Type a short, unique name for the item in the Name field. This can include spaces and punctuation, for
example Modem on COM3. Use consistent meaningful names, as they will be used by the resource
management options and functions to refer to this resource.
Page 83 of 129
03 June 2013
Type any notes for this item in the Notes field. This information is optional, and can include spaces and
punctuation, for example MC35 modem with SIM #3333333333
Click the Add button to add the new resource item and return to the Resource items screen:
If you edit a Resource Item which is already allocated, all its allocations will be updated accordingly.
Page 84 of 129
03 June 2013
Click OK to delete it
If you delete a Resource Item which is already allocated, all those allocations will be deleted, too.
Resource Users
To define the users on an Agent so you can allocate resources to them, you must first add them as Resource Users:
This will list the current Resource Users of this Agent (initially just SYSTEM), on the Resource users screen:
Page 85 of 129
03 June 2013
Once you have added some users, additional options are displayed for each:
button or the Add new user link - see Adding a Resource User
Page 86 of 129
03 June 2013
Name: Type the name to use for this user throughout resource management. This can include spaces and
punctuation and will typically be the user's full name
(Domain) username: Type the user's Windows login name i.e. the username that they type to login. (Check
with the user or system administrator if unsure.)
Click the Add button to add the new user and return to the Resource Users screen:
Page 87 of 129
03 June 2013
Click Apply to make the changes and return to the Resource Users screen
If you edit a user who is already allocated resources, all their allocations will be updated accordingly.
Click OK to delete it
If you delete a Resource User who is already allocated resources, all their allocations will be deleted too.
Resource Allocations
Hint: Remember that scripts can also be designed to claim resources dynamically, by using the ResourceAcquire
and ResourceRelease functions - see the Advanced Scripting Guide for details.
To book (allocate) resources for specific users (or the system) for a specific time, you must first define them as
Resource Items and Resource Users, then:
Page 88 of 129
03 June 2013
This will list the current allocations on this Agent (initially none), on the Resource allocations screen:
Once you have added some allocations, additional options are displayed for each:
Click the
Page 89 of 129
03 June 2013
Item: Select the resource to allocate. (If you can't see the resource, you need to add a new Resource Item)
User: Select the user to allocate the resource to. Use SYSTEM to allocate resources for system use. (If you
can't see the user, you need to add a new Resource User)
Start time:
Click the
Select the start month and year from the dropdown lists at the top.
Click the start day to choose it and return to the main dialog.
Select the start hour and minute from the lists on the right-hand side of the main dialog
Note that you cannot select a start day or time that's in the past!
Hint: Screening Room will warn that the start time is after the end time, but set it as requested. To
avoid this warning, set the end time first, then the start time.
End time: select the end date and time in the same way as the start date and time.
Duration: This changes to show how long you are allocating the resource for. You cannot edit it directly.
Click the Add button to add the Resource Allocation. The system will check that the resource you've
requested is free for the requested period and:
If the resource isn't free, the request will be refused with this error message:
Page 90 of 129
03 June 2013
If the resource is free for the requested period, you will return to the Resource allocations screen
with the new allocation listed like this:
Notice the Granted column on the Resource Allocations screen.. Yes with a green background means the Resource
Allocation was granted (the requested allocation has been made). No with a red background, means the allocation
was not granted, because it clashes with a dynamic allocation. (Failed allocations occur when a script attempts to
acquire a resource dynamically i.e. without a previous allocation to the script's owner.)
Click the
Page 91 of 129
03 June 2013
Edit the Item, User, Start time and End time fields as required (for details see Adding a Resource Allocation)
Click Apply to make the changes and the system will check that the resource is free for the revised period:
If the resource isn't free, an error message will be displayed as described above. Click OK to return to
the main window - the change will be rejected.
If the resource is free, the change will be made and you will return direct to the main window.
Click the
Click OK to delete it
Deleting a Resource Allocation will free that resource for the previously allocated time.
Resource Usage
To check resource usage - which resources were acquired (booked), when, by which testruns:
Page 92 of 129
03 June 2013
This will list current and past resource usage on the Resource usage screen:
Notice the Granted column. This shows which resource usage attempts succeeded (Yes with a green
background) and which failed (No with a red background), typically because of clashes with previous
allocations or bookings. Use the Resource graphing option to check which (its block graph is easier to
interpret).
You can change the report format (number of usages shown, block or row format) and filter by item, user,
testrun, status, or grant by using the controls in the table header.
Note: This is a read-only report of historical (past) and current information - you cannot edit any of the information
shown.
Hint: Use the Resource graphing option to compare resource allocations with resource usage.
Resource graphing
Note: The resource graphs plot times using GMT (Greenwich Mean Time), but when you create resource
allocations you specify the start times using local (system) time.
To display resource allocations and resource bookings over time, in graph form:
Page 93 of 129
03 June 2013
Period starts: select the start date and time from the three dropdown lists. The earlier start you can
select is for the start of the earliest resource allocation or booking
Period ends: select the end date and time from the three dropdown lists. The latest end you can select
is for the end of the latest resource allocation or booking
Use 2. Filter on resource item to choose the Resource Item to plot usage for:
Item: to plot bookings for a particular resource item, select its name from the dropdown list. To plot
bookings for all items, leave set to All
Use 3. Filter on resource user to choose the Resource User to plot usage for:
Page 94 of 129
03 June 2013
User: to plot allocations and bookings for a particular user, select their name from the dropdown list.
To plot bookings for all users, leave set to All
Allocation: choose Not smoothed to plot each booking length as a separate point, or Averaged per
minute/hour/day/year to plot averages over time. Averaging makes it easier to spot trends, but might
hide fluctuations or patterns that are important to you
Usage: choose Not smoothed to plot each usage period as a separate point, or Averaged per
minute/hour/day/year to plot averages over time. Averaging makes it easier to spot trends, but might
hide fluctuations or patterns that are important to you
Metric range: if using smoothed results, choose one of these options to add or remove curves
showing the upper and/or lower values:
Maximum limit
Minimum limit
Neither limit
Size: Choose the graph width (Narrow/Normal/Wide) from the first control, and graph height
(Short/Normal/Tall) from the second
Style: Choose the line style (Thin line or Thick line) from the first control, and the point style (No
points, Small points or Large points) from the second
Override Y-max: The Y-axes are chosen automatically so that the longest allocated and booked
periods plotted just fit. If you wish, you can override this process by typing a maximum Y axis value
in this box. You might use this, for example, to normalize the two curves so they are directly
comparable, or to remove spurious data from the graph.
For example, if periods are mostly in the range 150-200 seconds, with just one rogue value of 2000
seconds, the default Y axis will be 0-2000 seconds, compressing the details in the rest of the graph.
Type 200 in the Override Y-max box to ignore the rogue value and reveal the variations in the other
values, if they are of more interest.
If you set the maximum, any data points which go off the top of the graph will be shown by vertical
arrowheads.
Once the options are set as you wish, click Next to plot the graphs. You can choose a line graph, a block
graphs, or both:
Page 95 of 129
03 June 2013
You can use the graphs to check whether resources are being allocated for longer than needed, or to spot
idle times.
Hover over any data point on a graph to display a description of that data.
Page 96 of 129
03 June 2013
Page 97 of 129
03 June 2013
Page 98 of 129
03 June 2013
Page 99 of 129
03 June 2013
Display one of the Screening Room views (e.g. select the functional link from the Screening Room Home
screen):
Mark the testruns to include in the report, by ticking their checkboxes in the Test Runs pane:
Right-click the pane and select Report marked from the popup menu
03 June 2013
Select the report type you want from the dropdown list:
Click Launch
An example of each report type is provided over the next few pages, for the same two testruns in each case.
03 June 2013
03 June 2013
03 June 2013
03 June 2013
Click a section's
03 June 2013
03 June 2013
Click a section's
03 June 2013
Click a section's
03 June 2013
Click a section's
03 June 2013
Click a section's
03 June 2013
Click a section's
03 June 2013
03 June 2013
Click a section's
03 June 2013
03 June 2013
Click a section's
03 June 2013
Click a section's
03 June 2013
03 June 2013
Click a section's
03 June 2013
Click a section's
03 June 2013
03 June 2013
03 June 2013
Click a section's
03 June 2013
Click a section's
03 June 2013
03 June 2013
03 June 2013
Click a section's
03 June 2013
Index
Index
A
resource users, 85
Adding
Device update, 77
Dimensions option
Message types, 69
local schedules, 72
Metrics, 68
Agent
Overview, 67
schedules, 70
Plans, 67
Tests, 67
overview, 81
Timings, 69
screen, 80
Transactions, 68
scripts, 80
Editing
Local Schedules, 75
Resource Allocations, 88
Resource Items, 81
Resource Users, 85
Central Schedules, 70
Comparing graphs, 47
Functional view, 17
Deleting
local schedules, 76
resource allocations, 88
Resource Items, 81
03 June 2013
Index
Local Schedules
Adding, 72
Display options, 58
Custom, 73
Filters, 64
Deleting, 76
Links, 65
Editing, 75
Screen layout, 63
Overview, 70
Using, 57
Passing parameters, 72
Display options, 58
Filters, 59
Links, 60
Screen layout, 59
Using, 57
Metrics option, 68
Screen layout, 53
Monitoring view, 33
Using, 53
Display options, 58
Plan Dashboard
Screen layout, 61
Creating, 41
Using, 57
Customising, 45
Refresh control, 13
Interpreting, 44
Regression view, 36
Reports
Examples, 100
Plans option, 67
Exporting, 13
Popup_menu, 20
Printing, 12
Preview panel, 32
Printing
03 June 2013
Index
Response Codes, 98
Thresholds, 48
Schedules
Timings option, 69
Agent schedules, 70
Transactions Report, 68
central schedules, 70
local schedules, 70
Using
types of, 70
Screening Room
Controls, 11
Local Schedules, 70
Exporting reports, 13
Home screen, 14
Launching, 9
Overview, 5
Printing reports, 12
Screening Room, 11
Reports, 12, 13
Using, 5, 9, 11
03 June 2013