Screening Room
-
Upload
farhan-saeed -
Category
Documents
-
view
11 -
download
4
description
Transcript of Screening Room
-
Ascom Network Testing Ltd., 2000-2013 Page 1 of 129 03 June 2013
TEMS Monitor Master v9.2.1
Screening Room
User Guide
Document: MM-0016-D
Created by: EWJP
Status: Released
Ascom Network Testing Ltd, 2000-2013
All rights reserved
-
Ascom Network Testing Ltd., 2000-2013 Page 2 of 129 03 June 2013
Copyright Ascom Network Testing Ltd, 2000-2013. All rights reserved
Unauthorized copying, distribution and use of the document or its content is strictly prohibited.
Note: In this guide the term "Monitor Master" refers to the Ascom product "TEMS Monitor Master".
All registered trademarks, quotes, products or service names are acknowledged.
Ascom Network Testing Ltd
Ash House, Shackleford Road
Elstead, Surrey GU8 6LB
United Kingdom
Ascom Network Testing Ltd is an Ascom company, part of the Network Testing division.
www.ascom.com
Version Overview
Version Date Author Modification
1.0 25/04/2013 EWJP Initial release for Monitor Master v9.2.1
1.1 03/06/2013 EWJP Minor updates
Information in this document is subject to change without notice. For the latest version, please visit
www.ascom.com/temsmonitormaster/support
Document Management
File name: screening_room.pdf
Pages: 129
Released on: 03/06/2013
Approved by: AC
-
Ascom Network Testing Ltd., 2000-2013 Page 3 of 129 03 June 2013
Table of Contents CHAPTER 1 - INTRODUCTION .................................................................................................................................... 5
SCREENING ROOM OVERVIEW .......................................................................................................................................... 5 LOCAL MODE AND CENTRAL MODE ................................................................................................................................. 6 MAIN MENU OPTIONS ........................................................................................................................................................ 6 OTHER SCHEDULING FACILITIES ........................................................................................................................................ 7 OTHER REPORTING FACILITIES .......................................................................................................................................... 7 THIS GUIDE ....................................................................................................................................................................... 7 RELATED GUIDES .............................................................................................................................................................. 8
CHAPTER 2 - USING SCREENING ROOM ................................................................................................................. 9
LAUNCHING SCREENING ROOM ........................................................................................................................................ 9 USING SCREENING ROOM ............................................................................................................................................... 11
CHAPTER 3 - THE SCREENING ROOM OPTION .................................................................................................. 16
SCREENING ROOM FUNCTIONAL VIEW ........................................................................................................................... 17 SCREENING ROOM MONITORING VIEW ........................................................................................................................... 33 SCREENING ROOM REGRESSION VIEW ............................................................................................................................ 36
CHAPTER 4 - THE PLAN DASHBOARD OPTION ................................................................................................... 41
CREATING THE PLAN DASHBOARD ................................................................................................................................. 41 INTERPRETING THE PLAN DASHBOARD ........................................................................................................................... 44 CUSTOMISING THE PLAN DASHBOARD ............................................................................................................................ 45 MANIPULATING THE PLAN DASHBOARD ......................................................................................................................... 47 COMPARING PLAN DASHBOARD GRAPHS ........................................................................................................................ 47 CUMULATIVE PLAN DASHBOARD GRAPHS ...................................................................................................................... 50 WATCHING PLAN DASHBOARD DATA ............................................................................................................................. 51
CHAPTER 5 - RECENT TESTS & AVAILABILITY ................................................................................................. 53
USING THE RECENT SUCCESS FACTS SCREEN .................................................................................................................. 53 THE RECENT SUCCESS FACTS DISPLAY OPTIONS ............................................................................................................ 53 THE RECENT SUCCESS FACTS FILTERS ........................................................................................................................... 55 THE RECENT SUCCESS FACTS LINKS ............................................................................................................................... 55 THE RECENT SUCCESS FACTS REPORT ............................................................................................................................ 55
CHAPTER 6 - METRICS, MESSAGES & TIMINGS ................................................................................................. 57
USING THE RECENT METRICS, MESSAGES OR TIMING FACTS SCREEN ............................................................................ 57 THE RECENT METRIC/TIMING/MESSAGE FACTS DISPLAY OPTIONS ................................................................................ 58 THE RECENT METRIC FACTS SCREEN .............................................................................................................................. 59 THE RECENT TIMING FACTS SCREEN .............................................................................................................................. 61 THE RECENT MESSAGE FACTS SCREEN ........................................................................................................................... 63
CHAPTER 7 - DIMENSIONS ........................................................................................................................................ 67
PLANS ............................................................................................................................................................................. 67 TESTS .............................................................................................................................................................................. 67 TRANSACTIONS ............................................................................................................................................................... 68 METRICS ......................................................................................................................................................................... 68 TIMINGS .......................................................................................................................................................................... 69 MESSAGE TYPES .............................................................................................................................................................. 69
CHAPTER 8 - LOCAL SCHEDULES ........................................................................................................................... 70
LOCAL SCHEDULES ......................................................................................................................................................... 70 ADDING A LOCAL SCHEDULE .......................................................................................................................................... 72 EDITING A LOCAL SCHEDULE.......................................................................................................................................... 75 DELETING A LOCAL SCHEDULE ....................................................................................................................................... 76
CHAPTER 9 - DEVICE UPDATE ................................................................................................................................. 77
WHY USE DEVICE UPDATE? ............................................................................................................................................ 77 USING THE DEVICE UPDATE OPTION ................................................................................................................................ 77
-
Ascom Network Testing Ltd., 2000-2013 Page 4 of 129 03 June 2013
CHAPTER 10 - AGENT RESOURCE MANAGEMENT ............................................................................................ 80
MANAGEMENT OPTIONS .................................................................................................................................................. 80 RESOURCE MANAGEMENT SCRIPTS .................................................................................................................................. 80 RESOURCE MANAGEMENT CONCEPTS .............................................................................................................................. 81 RESOURCE ITEMS ............................................................................................................................................................ 81 RESOURCE USERS............................................................................................................................................................ 85 RESOURCE ALLOCATIONS ............................................................................................................................................... 88 RESOURCE USAGE ........................................................................................................................................................... 92 RESOURCE GRAPHING ...................................................................................................................................................... 93
APPENDIX A - RESPONSE CODES ............................................................................................................................. 98
HTTP V1.X STATUS CODES .............................................................................................................................................. 98
APPENDIX B - TESTRUN REPORTS ........................................................................................................................ 100
THE HIGH DETAIL CONTENT LISTING ............................................................................................................................. 101 THE MEDIUM DETAIL CONTENT LISTING........................................................................................................................ 102 THE LOW DETAIL CONTENT LISTING .............................................................................................................................. 103 THE COMPLIANCE ERRORS AND BROKEN LINKS REPORT................................................................................................ 104 THE ALL DEVICE ERRORS AND USABILITY REPORT ........................................................................................................ 106 THE COSMETIC DEVICE ERRORS REPORT ....................................................................................................................... 107 THE FUNCTIONAL DEVICE ERRORS REPORT ................................................................................................................... 108 THE OTHER DEVICE ERRORS REPORT ............................................................................................................................. 109 THE USABILITY ERRORS REPORT ................................................................................................................................... 110 THE LOADING REPORT................................................................................................................................................... 111 THE MESSAGES REPORT ................................................................................................................................................ 112 THE USER EVENTS REPORT ............................................................................................................................................ 114 THE METRICS REPORT ................................................................................................................................................... 116 THE RESPONSE DELAY REPORT...................................................................................................................................... 117 THE DELTA TIMINGS REPORT ........................................................................................................................................ 119 THE GRAPHICAL TIMING REPORT................................................................................................................................... 120 THE TEST AND TEST RUN PERFORMANCE REPORT .......................................................................................................... 121 THE TRANSACTION TIMINGS REPORT ............................................................................................................................. 123 THE SCRIPT REPORT ...................................................................................................................................................... 124 THE SITE MAP REPORT ................................................................................................................................................... 124 THE APPLIED DEVICE TEST CASES REPORT .................................................................................................................... 125
INDEX ............................................................................................................................................................................. 127
-
Ascom Network Testing Ltd., 2000-2013 Page 5 of 129 03 June 2013
Chapter 1 - Introduction
Note: Screening Room is a legacy tool, retained mainly to support users running scripts in legacy (local) mode;
most Monitor Master QM users should not need to use Screening Room - or read this guide!
In this Chapter:
Screening Room
Local Mode and Central Mode
Main menu options
Other reporting facilities
Other scheduling facilities
This guide
Related guides
Screening Room Overview Monitor Master Screening Room (also known as the Agent Menu) provides a reporting and management
interface for Monitor Master Agents. It consists of a browser-based menu system that provides convenient access
to the following, for the selected Agent:
Test run results and other data stored on this Agent. The wide range of facilities allows users to view,
explore and produce reports on data to whatever level of detail they wish. Data can also be exported in a
variety of formats for use in other applications.
Scheduling of Local Mode scripts on this Agent.
Device profile updates for this Agent, to keep the device details up to date with the Device Master Live
Server or other trusted authority
Resource management for this Agent, to help avoid conflicts between Local Mode test runs
-
Screening Room User Guide Chapter 1 - Introduction
Ascom Network Testing Ltd., 2000-2013 Page 6 of 129 03 June 2013
The Screening Room Home screen
Notice that each Agent has its own separate Screening Room.
Note: Throughout this guide we will use the term Agent rather than "Probe", "Agent Appliance" or" Software
Agent".
Local Mode and Central Mode In a Monitor Master QM system, scripts are usually saved in Central Mode (to the QM Server) but can also be
saved in Local Mode (to an Agent):
Screening Room provides reports, resource management and scheduling for the Agent's Local Mode
scripts. (The reports can also be used for Central Mode resuilts.)
Quality Manager (QM) provides superior central reports, resource management and scheduling for all
Central Mode scripts. (The reports can also use Local Mode results.)
In a Monitor Master QM system, we strongly recommend the use of Central Mode for all scripts. You should only
use Local Mode (and Screening Room) if QM is not available.
WARNING: Do not schedule both Local Mode and Central Mode tests on the same Agent; the resource
management strategies used are incompatible.
Main menu options The Screening Room main screen (above) is a menu that provides these main options:
Screening Room - review summaries and details of recent test runs, with drilldown to transaction and
content level. Also provides access to Scheduling facilities for local mode scripts
Plan dashboard - summarise results for specific plans or tests
-
Screening Room User Guide Chapter 1 - Introduction
Ascom Network Testing Ltd., 2000-2013 Page 7 of 129 03 June 2013
Recent tests & availability - a tabular summary of recent tests indicating any problems
Metrics, messages and timings - detailed results from multiple tests
Dimensions - view your test data along different dimensions: by plan, test, transaction, metric, timing, or
message type
Device update - download the latest device profiles from the Device Master Live Server
Agent resource management - managing the resources on this Agent to avoid conflicts
Each of these main options has its own chapter later in this guide.
Other scheduling facilities Monitor Master provides two ways to schedule test runs:
Quality Manager schedules Central Mode scripts to run on single or multiple Agents. These Central
Schedules support both single-party and multi-party scripts
Screening Room schedules Local Mode scripts to run on a single Agent. These Local Schedules do not
support mutli-party Scripts
Other reporting facilities Monitor Master provides four main reporting interfaces: Box Office, Screening Room, Story Boarder and Projection
Room:
Box Office provides a range of standard Dashboards (reports) for both Agent and Projector results, as well
as development and integration facilities for custom Dashboards. Dashboards can be accessed via the Box
Office GUI, or from the Quality Manager Reporting menu.
Screening Room provides access to all recent test run results (and other data) stored on an Agent.
Story Boarder provides access to summary results for the current test run only. This is mainly intended for
test script developers to test their scripts, but can also be used for ad hoc testing and monitoring in small
installations.
Projection Room provides access to test run results collected from multiple Agents by a Projector, usually
over a much long time period than Screening Room.
Box Office is the recommended reporting interface for all Monitor Master installations.
Note that the results from Local Mode and Central Mode test runs are stored in the same way (on the Agent that
ran the test, and associated Projector).
This guide This guide consists of the following chapters:
Chapter 1 - Introduction
Chapter 2 - Using Screening Room
Chapter 3 - The Screening Room option
Chapter 4 - The Plan Dashboard option
Chapter 5 - The Recent Tests & Availability option
Chapter 6 - The Metrics, Messages & Timings option
Chapter 7 - The Dimensions option
-
Screening Room User Guide Chapter 1 - Introduction
Ascom Network Testing Ltd., 2000-2013 Page 8 of 129 03 June 2013
Chapter 8 - Local Schedules
Chapter 9 - The Device Update option
Chapter 10 - The Agent Resource Management option
It concludes with an Appendix on HTTP return codes and testrun reports.
Related guides Read the QM User Guide for information on using Central Mode resource management, scheduling and reporting.
Read the Story Boarder User Guide for information on creating and running test scripts, including how to view and
interpret test results.
Read the Projection Room User Guide for information on using legacy enterprise-wide reporting.
-
Ascom Network Testing Ltd., 2000-2013 Page 9 of 129 03 June 2013
Chapter 2 - Using Screening Room
This chapter describes:
How to launch Screening Room
How to use Screening Room - the general principles
The Home screen menu
Subsequent chapters describe each of the Home screen (main menu) options in turn.
Note: While a standard Monitor Master installation allows all Users to access all the reports and all the data on an
Agent, your system administrator may implement measures to control access to certain types of report and test
data.
Launching Screening Room You can launch Screening Room:
From Story Boarder
From Internet Explorer
From a Shortcut
From an Agent
-
Screening Room User Guide Chapter 2 - Using Screening Room
Ascom Network Testing Ltd., 2000-2013 Page 10 of 129 03 June 2013
Launching Screening Room from Story Boarder
If you are developing Test Scripts, it can be convenient to launch Screening Room from within Story Boarder.
To launch Screening Room from Story Boarder:
Select the View Agent option from the Story Boarder Tools menu or click the View Agent button, to launch
Screening Room for your most recently-used Agent.
Or:
Choose the Select Agent option/button to list the known Agents:
Select the Agent and click OK to launch the Screening Room for that Agent.
(If the Agent is not listed, click Add, type its network address or (routable) name and click OK to add it to
the list.)
Now use View Agent to launch Screening Room for the selected Agent.
Launching Screening Room from Internet Explorer
You can launch Screening Room on any machine on your network (even one without Monitor Master installed) by
typing the appropriate URL in your web browser.
To launch Screening Room from Internet Explorer:
Launch your web browser
In the address bar, type the URL of the Agent you want to connect to, or its (routable) name, followed by
:9090 and press Enter
This will display Screening Room for that Agent
For example, to use an Agent whose address is 10.1.1.85 and URL is agent3.xyz.com, you could type:
http://10.1.1.85:9090/
or
http://agent3.xyz.com:9090/
Or, if the name agent3 is a routable name resolving to this address, you could simply type:
http://agent3:9090/
Launching Screening Room from a Shortcut
Once you have connected to Screening Room (by typing the URL in Internet Explorer), you can create a Shortcut to
make connecting easier in the future.
To create a Screening Room Shortcut:
Connect to Screening Room on the Agent you want to use by typing its URL (example)
-
Screening Room User Guide Chapter 2 - Using Screening Room
Ascom Network Testing Ltd., 2000-2013 Page 11 of 129 03 June 2013
Once the Screening Room Home page is displayed, select the Add to Favorites option from your Browser's
Favorites menu.
Edit the name if you wish, then click OK to add it your main Favorites menu. (Refer to your Browser Help
for the other options.)
You can now launch the Screening Room for this Agent by selecting this option from the Favorites menu.
To add the Shortcut to your desktop:
Select the option from the Browser's Favorites menu and drag it to the desktop.
You can now launch the Screening Room for that Agent by double-clicking the desktop shortcut.
Launching Screening Room from an Agent
If you are logged in on an Agent, you can use an installed Shortcut to launch its Screening Room.
To launch Screening Room from the Agent:
Click the Start button and select the option Ascom/TEMS Products / TEMS Monitor Master Servers /
Server Reports / Script Server Database Reports
This will launch the Screening Room for that Agent
Using Screening Room Once you have launched an Agent's Screening Room , you can use it to produce a wide range of reports, schedule
tests, update device profiles, and manage resources for this Agent. These functions are described in detail in
subsequent chapters; this section describes the basic concepts and facilities you'll need to use Screening Room:
Controls
Printing
Data export
Refresh control
The Home screen
Screening Room Controls
Screening Room is a web-based interface, so many of the facilities and controls should be very familiar already, as
they are provided by your Web Browser.
For example:
Underlined text signifies a hyperlink - click it to follow the link
To return to a previous screen, use the Browser's Back button
To print the current screen, use the Browser's Print button or Print command (in the File menu) - see
Printing
-
Screening Room User Guide Chapter 2 - Using Screening Room
Ascom Network Testing Ltd., 2000-2013 Page 12 of 129 03 June 2013
To use more of your screen for results, you can hide toolbars by selecting options from the View menu
If a panel has scrollbars, use them to scroll the content in that panel
And so on.
Screening Room uses a few controls that you might not know:
Many report windows have a refresh control in the top right corner. You can use this to choose how often
the displayed results will be updated (by fetching new data from the Agent) - see Refresh control
The refresh control
A button means there's more information hidden below this item; click the button to reveal it; a button
means you can hide the information below this item; click the button to hide it.
Many window elements are "hot" i.e. hyperlinked. If the mouse pointer changes from an arrow to a hand
while over an element, you can click it to link to related information.
Column header buttons are displayed along the top of most tables, like this:
These have two main functions: to show what data is in that column, and to control how the table is sorted.
Click a button to sort the table on that column; click it again to reverse the sort order. A triangle shows the
sort column and sort direction
Windows are divided up into panels by narrow borders. You can drag these borders to change the panel's
size.
Each panel has its own right-click (popup) menu, providing access to a range of useful facilities for that
panel and its contents.
When you select an option from a dropdown list, its name is also displayed on the Browser status bar - to
help you choose names that are too long to fit in the list.
Printing Screening Room Reports
Printing uses the standard Browser facilities:
Use File/Page Setup to configure the printer and paper
Use File/Print Preview to preview (check) the output
Use File/Print to choose print options and begin printing
Each Screening Room screen is implemented as a frameset i.e. a group of frames (which we call panels here), so the
print options tab provides useful options to:
Print as laid out on screen
Print the selected frame only
Print all the frames, individually
To print the contents of a single panel:
Right-click in the panel to display a popup menu like this:
(Some menus have additional options, but they all have this row of buttons along the bottom.)
Click the Print panel button,
-
Screening Room User Guide Chapter 2 - Using Screening Room
Ascom Network Testing Ltd., 2000-2013 Page 13 of 129 03 June 2013
(This facility is provided by the Screening Room screens: Functional, Regression, and Monitoring views.)
Exporting Screening Room Reports
Screening Room provides a range of facilities to export the current report (as shown on-screen), in different data
formats, for input to other applications.
The Screening Room screens (Functional, Regression, and Monitoring views) display multiple related data streams
in separate panels. To output the contents of a panel:
Right-click within the panel to display a popup menu ending with these buttons:
Click the Open in Excel button, to output the data in Excel (spreadsheet) format. You can choose
whether to Open or Save the data. Open will display it as an embedded spreadsheet, in the same panel;
Save will output the data to the Excel format file you specify.
Click the Open as XML button, to output the data in XML format, in another Browser window for you
to view, print, save etc using the standard Browser facilities.
The other screens (Recent facts, Dimensions, Dashboard and Metrics/ Messages/ Timings) present a single data
stream, so you can output the whole screen. In the top right corner of the screen is a menu of output formats.
Choose the format you want:
Raw XML will output simple XML to the Browser window
Rich XML will output semantically richer XML to the Browser window
CSV will output comma separated values to the Browser window
Excel will output an Excel format spreadsheet. You will be asked whether to Open or Save the data. Open
will display it as an embedded spreadsheet, in the current screen; Save will output the data to the Excel
format file you specify.
Table will output the data to a specified Table.
Refresh control
Most reporting screens have a Refresh control in the top right corner:
-
Screening Room User Guide Chapter 2 - Using Screening Room
Ascom Network Testing Ltd., 2000-2013 Page 14 of 129 03 June 2013
You can use this to choose how often to update the screen, to reflect new results logged by the Agent. The options
are:
Never refresh - the default
Refresh now
Refresh every 3 seconds - see warning below
Refresh every 15 seconds
Refresh every 1 minute
Refresh every 5 minutes
Refresh every 15 minutes
Refresh every hour
Warning: The shorter refresh times should only be used for short periods, when you really them, as each refresh
requires significant additional processing by the Agent. Refreshing every 3 seconds may disrupt the Agent's main
task of running Tests and collecting Testrun data.
However, note that certain reports specify a fixed time period to report on. Once you're past the period end, the
report on the screen will no longer change when refreshed, because there will be no more new data for the selected
period! To avoid this problem, either choose an end period in the future, edit the end date at regular intervals, or
use a report with a moving time window such as the Watch facility or Recent test & availability report.
Using the Screening Room Home screen
The Home screen is displayed when you launch Screening Room:
It offers your main menu, with seven main options:
Screening Room - use to review details of previous test runs, see Chapter 3, Screening Room option. Also
provides access to the test scheduling facilities - see Chapter 8, Local Schedules
Plan dashboard - use to summarize results for specific plans or tests. Also provides direct links to recently-
added plans. See Chapter 4, Plan Dashboard option
-
Screening Room User Guide Chapter 2 - Using Screening Room
Ascom Network Testing Ltd., 2000-2013 Page 15 of 129 03 June 2013
Recent tests & availability - a tabular summary of recent tests. Also provides direct links to recent failed
tests. See Chapter 5, Recent Tests & Availability option
Metrics, messages and timings - detailed results from multiple tests. See Chapter 6, Metrics, Messages and
Timings option
Dimensions - view your test data along different dimensions: by plan, test, transaction, metric, timing, or
message type. See Chapter 7, Dimensions options
Device update - download the latest device profiles from the main Device Master Live Server. See Chapter
9, Device Update option
Agent resource management - managing the resources on this Agent to avoid conflicts. See Chapter 10,
Agent Resource Management option
You can return to the Home screen from most other Screening Room screens by clicking the Home link, in the top
right corner of the screen.
-
Ascom Network Testing Ltd., 2000-2013 Page 16 of 129 03 June 2013
Chapter 3 - The Screening Room option
You can launch any of the three Screening Room reports by clicking their links on the Screening Room Home
Screen:
They provide detailed information on individual test runs:
Click Functional to review results from functional testing
Click Monitoring to review results from service monitoring
Click Regression to compare results from regression testing
Once you've selected a view, you can switch easily to another view:
Right-click anywhere in the Monitor Master banner near the top of the window:
-
Screening Room User Guide Chapter 3 - The Screening Room option
Ascom Network Testing Ltd., 2000-2013 Page 17 of 129 03 June 2013
This will display a popup menu:
Choose the view you want (Functional, Monitoring, or Regression), or click Agent home to return to the
Home (main menu) screen.
Each view will now be described in detail in its own section. They have a lot in common so to avoid repetition, we
will describe Functional View in detail, then describe how Monitoring View and Regression View differ.
Screening Room Functional View Functional View is designed to help you review results from functional testing, allowing you to summarize results
across all plans, focus on particular plans or tests, and drill right down into the details of an individual test run.
To launch Functional View:
Select the Functional link from the Screening Room Home screen
Or
To switch from Regression View or Monitoring View: right-click near the top of the Screening Room
window (on the Monitor Master banner) to display a popup menu and select the Functional view option
In Functional View, results are displayed in seven separate panels:
-
Screening Room User Guide Chapter 3 - The Screening Room option
Ascom Network Testing Ltd., 2000-2013 Page 18 of 129 03 June 2013
The panels are:
1. Plans and Tests
2. Test Runs
3. Testrun Summary
4. Content
5. Content Summary
6. Content Detail
7. Preview
Panels 3-7 are empty initially. They display details of a testrun once you've selected one in the Test Runs panel.
Plans and Tests panel (1)
The top left panel in Screening Room's Functional View is Plans and Tests, a hierarchical view of the Plans and
Test Scripts on this Agent:
You can use this view to list and manage the Plans, Tests and Schedules on this Agent. You can also use it to help
select testruns to report on.
To list the Test Scripts within a Plan:
Click the Plan's button. (To hide them again, click its button.)
To list the scheduled Test Runs within a Plan:
Click the button on its Schedules folder. (If there is no Schedules folder, the Plan has no schedules.)
To list all schedules on this Agent:
-
Screening Room User Guide Chapter 3 - The Screening Room option
Ascom Network Testing Ltd., 2000-2013 Page 19 of 129 03 June 2013
Click the button on the Agent schedules folder (this folder is listed after all the Plan folders).
To select all Test Runs on this Agent:
Click the This agent folder. Details will be displayed in the Test Runs panel.
To select Test Runs for a specific Plan, Test, or Schedule:
Click its name. Details will be displayed in the Test Runs panel.
In a Plan folder, scheduled Tests are indicated by a small clock in the icon:
- Test is not scheduled (no clock)
- Test is scheduled but schedule is disabled
- Test is scheduled and schedule is enabled
In the Plans' Schedules folders and Agent schedules folder Schedule status is shown by icon color:
- Schedule is enabled
- Schedule is disabled
For information on creating and maintaining Schedules, see Chapter 8, Local Schedules.
Popup menus
If you right-click anything within this panel you will see a popup menu with four buttons along the bottom:
Refresh panel - fetch new data from the Agent and redisplay the panel using it
Print panel - print the panel as a report. Displays the standard Print options dialog
Open as Excel - output the data in Excel (spreadsheet) format. You can choose to Open the data
(display it as an embedded spreadsheet, in the same panel) or Save the data (output to a Excel format file
you specify)
Open as XML - output the data in XML format, in another Browser window. You can then view, print,
or save it using the standard Browser facilities
The other options, at the start of the menu, are used to manipulate the selected item, as follows.
Agent popup menu
If you right-click This agent, the popup menu also offers:
Agent home - move to the Home screen (main menu)
Agent reports - displays the Report on Agent dialog. Select the type of report required, then click Launch:
Plan and resource usage - show resource usage by all schedules on this Agent, useful for seeing
detecting conflicts
Available devices - list of devices in the Agent's device database
Applied compliance test cases - lists the compliance tests applied to devices in the Agent's device
database
Current licence violations - lists any attempted violations of your Monitor Master license
Projector outbox - lists the contents of the Projector outbox
Recent logs - display a log for this Agent, showing all recent requests (from clients) and responses (from the
Agent). This log is mainly intended for diagnostic purposes
-
Screening Room User Guide Chapter 3 - The Screening Room option
Ascom Network Testing Ltd., 2000-2013 Page 20 of 129 03 June 2013
Plan popup menu
If you right-click a Plan, the popup menu also offers:
Plan dashboard - display the Dashboard for this Plan (see Using Plan Dashboard)
Delete this plan - from this Agent. Use with caution! Deleting a Plan will remove all its Test Scripts and
Testrun results from the Agent!
Test popup menu
If you right-click a Test, the popup menu also offers:
Test dashboard - display the Dashboard for this Test (this is the same as the Plan Dashboard, but with the
Plan and Test pre-selected - see Plan Dashboard reports).
Run test - displays the Run Test dialog for you to confirm (or change) the test selected, supply runtime
parameters, and enable or disable routing of results to Projector. Click OK to run the Test Script - results will
be displayed in the Testrun pane.
Edit test - displays the Edit Test dialog for you to confirm (or change) the test selected. Click OK to edit the
Test Script.
Move test - displays the Move Test dialog for you to confirm (or change) the test selected and choose the
Plan to move it to. Click OK to move the Test Script and associated results.
Delete test - from this Plan. Use with caution! Deleting a Test will delete its script and all its testrun results
from this Agent.
Schedules folder popup menu
If you right-click the Schedules folder (within a Plan) or the Agent schedules folder, the popup menu also offers:
Add schedule - see Chapter 8, Local Schedules for details
Schedule popup menu
If you right-click a schedule, the popup menu also offers:
Edit schedule - see Chapter 8, Local Schedules for details
Delete schedule - see Chapter 8, Local Schedules for details
Test Runs panel (2)
The top right panel in Screening Room's Functional View is Test Runs. Across the top of the panel are three
dropdown lists and a (Go) button, which are used to select the information to display in the remainder of the
panel:
Use the first dropdown list to chose how to view your results:
Tabular summary of - view a tabular summary of all results from the selected tests, and grand totals.
Provides a detailed numerical summary of results in table form - see The tabular summary view
-
Screening Room User Guide Chapter 3 - The Screening Room option
Ascom Network Testing Ltd., 2000-2013 Page 21 of 129 03 June 2013
Extended summary of - view a graphical summary of all results from the selected tests, and grand totals.
Provides an "at a glance" summary of results in pie chart form - see The extended summary view
Each testrun from - (default) view a list of individual testruns, showing detailed results for each. Allows
you to quickly select a testrun of interest and drill down into its results, content, metrics, diagnostics, etc. -
see The each testrun view
Use the second dropdown list to select the results (testruns) to view. The options are:
No plan - don't display anything
All tests and plans on this agent - (default) all runs of all Tests on this Agent
Plan names - all runs of Tests in this Plan
Test names - all runs of this Test
(You can also select the results to view by selecting a Plan or Test from the Plans and Tests panel.)
Use the third dropdown list to select the time period to report on. The options, which are self-explanatory, are:
without time filter
for the last 1 hour (default)
for the last 24 hours
for the last 7 days
for the last 31 days
for the last 365 days
Click the button to create a new report with your currently selected options.
The "tabular summary" view
The tabular summary view provides a numerical summary of all results for the selected tests, like this:
The table starts with grand totals (for all selected tests), followed by a row for each selected test, displaying:
Plan - the Plan name
Test - the Test name
Completion - percent of testruns completed
Service - percent of testruns with 100% service availability
System - percentage of testruns with 100% system availability
First - the date of the first testrun
Last - the date of the last testrun
Fastest - the shortest run time (seconds)
Slowest - the longest run time (seconds)
Average - the average run time (seconds)
Runs - the number of testruns of this test
To sort by a column:
Click the column heading
Click the column headings again to reverse the sort direction.
-
Screening Room User Guide Chapter 3 - The Screening Room option
Ascom Network Testing Ltd., 2000-2013 Page 22 of 129 03 June 2013
To display a test's test runs in the "each testrun" view:
Click that row
Right-click anywhere in the panel to display a popup menu with these options:
Tear-off - view the panel in another browser window
Refresh panel - fetch new data from the Agent and redisplay the panel using it
Print panel - print the panel as a report. Displays the standard Print options dialog
Open as Excel - output the data in Excel (spreadsheet) format. You can choose to Open the data
(display it as an embedded spreadsheet, in the same panel) or Save the data (output to a Excel format file
you specify)
Open as XML - output the data in XML format, in another Browser window. You can then view, print,
or save it using the standard Browser facilities
The "extended summary" view
The extended summary view provides a graphical summary of results for the selected tests, like this:
The first row is a grand total for all the selected testruns. Each subsequent row summarises results for a different
selected test.
The information shown is:
Plan - the Plan name
Test - the Test name
A description of the test
Completion - pie chart showing percentage of testruns completed
Service - pie chart showing percentage of testruns with 100% service availability
System - pie chart showing percentage of testruns with 100% system availability
First - the date of the first testrun
Last - the date of the last testrun
Fastest - the shortest run time (seconds)
Slowest - the longest run time (seconds)
Average - the average run time (seconds)
Runs - the number of testruns of this test
Click a row to display that test's runs in the "each testrun" view.
Right-click anywhere in the panel to display a popup menu with these options:
Tear-off - view the panel in another browser window
-
Screening Room User Guide Chapter 3 - The Screening Room option
Ascom Network Testing Ltd., 2000-2013 Page 23 of 129 03 June 2013
Refresh panel - fetch new data from the Agent and redisplay the panel using it
Print panel - print the panel as a report. Displays the standard Print options dialog
Open as Excel - output the data in Excel (spreadsheet) format. You can choose to Open the data
(display it as an embedded spreadsheet, in the same panel) or Save the data (output to a Excel format file
you specify)
Open as XML - output the data in XML format, in another Browser window. You can then view, print,
or save it using the standard Browser facilities
The "each testrun" view
The each testrun view lists individual testruns in the selected sort order and provides facilities to drill down to
content, metrics, etc. for a selected testrun:
To change the sort order, simply click the column button to sort by.
To drill down, select a testrun by clicking it - this will display details about the run in the Testrun Summary and
Contents panels, described below.
The data shown is as follows:
100 rows - the run ID
Plan - the name of the Plan this Script was run from
Test - the name of the Script
Device - the device profile used
Cmp - (completion) an icon indicating whether the run completed (green) or failed (red). A red icon
suggests a problem with the test script
Srv - (service) an icon indicating whether the service was available for the entire run (green) or not (red) or
unknown (grey). A red icon suggests a problem with the service under test
Sys - (system) an icon indicating whether the test system was available for the complete run (green) or not
(red) or unknown (grey). A red icon suggests a problem with the test system
Start - the date and time the run started
End - the time the run ended
Duration - how long the test ran for. A red background indicates a long run, while a green background
indicates a short run
State - whether the run succeeded, failed, was canceled or terminated due to a scripting error
Status - indicates if the test is still running
Content - how many content items were downloaded
Owner - the user ID and IP address of whoever ran the test. Tests that were run manually (from Story
Boarder) or by a Local Schedule will be listed as owned by SYSTEM (the IP address should help identify the
owner)
Location - if specified, a description of the location of the test run (Agent)
GPS - if available (via a GPS system), the co-ordinates where the test was run. This information is mainly
used by mobile test systems
-
Screening Room User Guide Chapter 3 - The Screening Room option
Ascom Network Testing Ltd., 2000-2013 Page 24 of 129 03 June 2013
Entrypoint - where the test started
The panel will normally display up to 100 test runs (rows). To change the number:
Click the 100 rows link
Type the number of rows to display, then click OK.
Popup menu
Right-click any testrun in the Test Runs Panel to display this popup menu:
Use the first three options (Mark all, Mark incompletes, and Unmark all) to mark testruns. You can also
mark or unmark runs by clicking the checkbox at the start of each line
Use Report marked to create a report combining the marked runs. This displays the Report Testrun dialog,
for you to select the report type, from:
High detail content listing
Medium detail content listing
Low detail content listing
Compliance errors and broken links
All device errors and usability
Cosmetic device errors
Functional device errors
Other device errors
Usability errors
Loading
Messages
User events
Metrics
Response delay
Delta timings
Graphical timing
Test and test run performance
Transaction timings
Script
Site map
Applied device test cases
Note: To see an example of each of the above reports, see Appendix B: Testrun Reports.
-
Screening Room User Guide Chapter 3 - The Screening Room option
Ascom Network Testing Ltd., 2000-2013 Page 25 of 129 03 June 2013
Use Stop marked to stop the marked testruns.
Use Delete marked to delete the marked testruns.
The small buttons along the bottom of the menu are as follows:
View detail - view testrun details in the Testrun Summary and Content panels
View detail tear-off - view testrun details in another browser window
Refresh panel - fetch new data from the Agent and redisplay the panel using it
Print panel - print the panel as a report. Displays the standard Print options dialog
Open as Excel - output the data in Excel (spreadsheet) format. You can choose to Open the data
(display it as an embedded spreadsheet, in the same panel) or Save the data (output to a Excel format file
you specify)
Open as XML - output the data in XML format, in another Browser window. You can then view, print,
or save it using the standard Browser facilities
You can also display details of a testrun (in the Testrun Summary panel and Content panel) by double-clicking it.
Testrun Summary panel (3)
The bottom left panel in Screening Room's Functional View is Testrun Summary. This shows the following for the
Test Run that you've selected in the Test Runs panel::
A Testrun summary for this run:
The Completion/service/system icons indicate whether the run completed and whether the service and
system were available for the entire run. Red icons indicate possible problems with the script, service, and
test system.
A Meta tags table, listing the meta tags for the testrun (information about the testrun rather than the results
it produced):
-
Screening Room User Guide Chapter 3 - The Screening Room option
Ascom Network Testing Ltd., 2000-2013 Page 26 of 129 03 June 2013
A Transactions Graph, showing the time taken by each Transaction (e.g. Scene) in the script. Green bars are
successful Transactions, red bars are failed Transactions.
A Transactions Table, showing data for the Transaction selected in the graph. Initially, the main
Transaction (the entire testrun) is selected; to choose another, click its bar on the graph
A Content timings graph, showing the time taken to transfer each content item.
-
Screening Room User Guide Chapter 3 - The Screening Room option
Ascom Network Testing Ltd., 2000-2013 Page 27 of 129 03 June 2013
Popup menu
Right-click anywhere in the Testrun Summary panel to display a popup menu with these buttons:
Report testrun - display a menu of all the other reports available for this testrun. Select the report you
want and click Launch to run it.
New Screening Room - open the current report on another Screening Room window
Tear-off - view panel in another browser window
Refresh panel - fetch new data from the Agent and redisplay the panel using it
Print panel - print the panel as a report. Displays the standard Print options dialog
Open as Excel - output the data in Excel (spreadsheet) format. You can choose to Open the data
(display it as an embedded spreadsheet, in the same panel) or Save the data (output to a Excel format file
you specify)
Open as XML - output the data in XML format, in another Browser window. You can then view, print,
or save it using the standard Browser facilities
Content panel (4)
The top right panel in Screening Room's Functional View is Content.
The Content panel lists the content items downloaded for the selected Test Run, showing the following for each
item:
ID: Its ID
-
Screening Room User Guide Chapter 3 - The Screening Room option
Ascom Network Testing Ltd., 2000-2013 Page 28 of 129 03 June 2013
If the content has been analysed by using the Basic Content Tests Scene or MobileOK Scene, you'll see 5
additional columns:
OK: content analysis overview:
red means the item has failed (includes errors or failures);
green means it has passed (includes passes but no failures or errors);
yellow means the contents passed but there were device/compliance errors, OR there were content
analysis warnings but no errors or passes;
white means it has not been analysed (does not include any passes or failures) - typically because
there was no content to analyse
CAP: the number of Content Analysis Passes for this item
CAF: the number of Content Analysis Failures for this item
DE: the number of Device Errors for this item
CE: the number of Content Errors for this item
Content: A description e.g. its URL
Time: The time the download started
Device: The name of the device profile being tested
MIME: The content MIME type
Charset: Its character set
Delay: the time it took to download
Indicative Size: its approximate size
Indicative Rate: its approximate download rate
Note: If any content pages includes a redirection command, Monitor Master turns it off (by replacing the command
with a null command), so you stay on the intended page when you view it. You can turn redirection for the entire
web site on or off by right-clicking any content from it and selecting the appropriate option from the popup menu.
Popup menu
Right-click a content item to display a popup menu with these buttons:
Launch raw content - view content in the appropriate viewer, if there is one installed on this computer
Start replay - view details of each content item in turn (as if you had clicked each one in turn)
View detail - view details of this item in the Content Summary, Content Detail and Preview panels
View detail tear-off - view details of this item in another browser window
Refresh panel - fetch new data from the Agent and redisplay the panel using it
Print panel - print the panel as a report. Displays the standard Print options dialog
Open in Excel - output the data in Excel (spreadsheet) format. You can choose to Open the data
(display it as an embedded spreadsheet, in the same panel) or Save the data (output to a Excel format file
you specify)
Open as XML - output the data in XML format, in another Browser window. You can then view, print,
or save it using the standard Browser facilities
-
Screening Room User Guide Chapter 3 - The Screening Room option
Ascom Network Testing Ltd., 2000-2013 Page 29 of 129 03 June 2013
You can also view details of a content item (in the Content Summary, Content Detail and Preview panels) by
clicking it.
Content summary panel (5)
The Screening Room's Functional View Content summary panel displays a summary of the content item you've
selected in the Content panel, under these headings:
Content Summary - time sent, character set, delay (time taken), size and rate
Timings - times (clock times) and delays (time taken) for the associated transaction(s)
Metrics - bytes sent/received, service and system availability, etc. for the associated transaction(s)
Messages - messages from the associated transaction(s), if any were issued
Compliance Errors - any compliance errors reported for this content
For example:
-
Screening Room User Guide Chapter 3 - The Screening Room option
Ascom Network Testing Ltd., 2000-2013 Page 30 of 129 03 June 2013
Popup menu
Right-click anywhere in the pane to display a popup menu with these buttons:
Tear-off - view panel in another browser window
Refresh panel - fetch new data from the Agent and redisplay the panel using it
Print panel - print the panel as a report. Displays the standard Print options dialog
Open in Excel - output the data in Excel (spreadsheet) format. You can choose to Open the data
(display it as an embedded spreadsheet, in the same panel) or Save the data (output to a Excel format file
you specify)
Open as XML - output the data in XML format, in another Browser window. You can then view, print,
or save it using the standard Browser facilities
-
Screening Room User Guide Chapter 3 - The Screening Room option
Ascom Network Testing Ltd., 2000-2013 Page 31 of 129 03 June 2013
Content detail panel (6)
The Screening Room's Functional View Content detail panel displays the data in the content item you've selected
in the Content panel, under these headings:
Request headers - the headers used to request this content
Response headers - the headers used to deliver this content
Response binary - the content as hexadecimal bytes
Response XML - the content as XML
For example:
-
Screening Room User Guide Chapter 3 - The Screening Room option
Ascom Network Testing Ltd., 2000-2013 Page 32 of 129 03 June 2013
Popup menu
Right-click anywhere in the panel to display a popup menu with these buttons:
Tear-off - view panel in another browser window
Refresh panel - fetch new data from the Agent and redisplay the panel using it
Print panel - print the panel as a report. Displays the standard Print options dialog
Open in Excel - output the data in Excel (spreadsheet) format. You can choose to Open the data
(display it as an embedded spreadsheet, in the same panel) or Save the data (output to a Excel format file
you specify)
Open as XML - output the data in XML format, in another Browser window. You can then view, print,
or save it using the standard Browser facilities
Preview panel (7)
The Screening Room's Functional View Preview panel renders the content item you've selected in the Content
panel as it would be shown on the device. For example:
-
Screening Room User Guide Chapter 3 - The Screening Room option
Ascom Network Testing Ltd., 2000-2013 Page 33 of 129 03 June 2013
Popup menu
This panel does not have its own popup menu. If you right-click it you will see the standard Browser popup menu.
Screening Room Monitoring View Monitoring View is designed to help you monitor services.
To launch Monitoring View:
Select the Monitoring link from the Screening Room Home screen
Or
To switch from Regression View or Functional View: right-click near the top of the Screening Room window
(on the Monitor Master banner) to display a popup menu and select the Monitoring view option
In Monitoring View, the results are displayed in three separate panels:
-
Screening Room User Guide Chapter 3 - The Screening Room option
Ascom Network Testing Ltd., 2000-2013 Page 34 of 129 03 June 2013
The panels are:
1. Plans and Tests
2. Test Runs
3. Testrun Summary
Plans and Tests panel (1)
The top left panel in Screening Room's Monitoring View is Plans and Tests, a hierarchical view of the Plans and
Test Scripts on this Agent which you can use to list and manage the Plans, Tests and Schedules on this Agent. You
can also use it to help select the testruns to monitor:
To list the Test Scripts within a Plan:
Click the Plan's button. To hide them again, click its button
To list the scheduled Test Runs within a Plan:
Click the button on its Schedules folder. (If there is no Schedules folder, the Plan has no schedules.)
To list all Schedules on this Agent:
Click the button on the Agent schedules folder. (This folder is listed after all the Plan folders).
To select all Test Runs on this Agent:
Click the This agent folder. Details will be displayed in the Test Runs panel.
To select Test Runs for a specific Plan, Test, or Schedule:
Click its name. Details will be displayed in the Test Runs panel.
-
Screening Room User Guide Chapter 3 - The Screening Room option
Ascom Network Testing Ltd., 2000-2013 Page 35 of 129 03 June 2013
For details of the Plans and Tests panel, refer to its description in Functional View.
Test Runs panel (2)
The top right panel in Screening Room's Monitoring View is Test Runs.
Across the top of the panel are three dropdown lists and a button, which are used to select the information to
display in the remainder of the panel:
Use the first dropdown list to chose how to view your results:
Tabular summary of - view a tabular summary of all results from the selected tests, and grand totals.
Provides a detailed numerical summary of results in table form
Extended summary of - view a graphical summary of all results from the selected tests, and grand totals.
Provides an "at a glance" summary of results in pie chart form
Each testrun from - view a list of individual testruns, showing detailed results for each. Allows you to
quickly select a testrun of interest and drill down into its results, content, metrics, diagnostics, etc.
The each testrun view is most often used for monitoring.
Use the second dropdown list to select the results (testruns) to view. The options are:
No plan - don't display anything
All tests and plans on this agent - all runs of all Tests on this Agent
Plan names - all runs of Tests in this Plan
Test names - all runs of this Test
(You can also select the results to view by selecting a Plan or Test from the Plans and Tests panel.)
Use the third dropdown list to select the time period to report on. The options, which are self-explanatory, are:
without time filter
for the last 1 hour
for the last 24 hours
for the last 7 days
for the last 31 days
for the last 365 days
Click the button to create a new report with your currently selected options.
Key facilities
Use the three dropdown lists across the top of the pane to select the view type (numerical summary,
graphical summary, or each testrun), the tests to view results for, and the period to view results for - then
click the button. The each testrun view is best for monitoring.
Sort on any column by clicking the column heading
Click any testrun (in "each testrun" view) to display details in the Testrun Summary panel - described next.
-
Screening Room User Guide Chapter 3 - The Screening Room option
Ascom Network Testing Ltd., 2000-2013 Page 36 of 129 03 June 2013
For details of the Test Runs panel, refer to its description in Functional View.
Testrun Summary panel (3)
The bottom left panel in Screening Room's Monitoring View is Testrun Summary, which shows the following for
the selected Test Run:
A Testrun summary for this run. The Completion/service/system icons indicate whether the run completed
and whether the service and system were available for the entire run. Red icons indicate possible problems
with the script, service, or test system.
A Meta tags table, listing the meta tags for the testrun (information about the testrun rather than the results
it produced).
A Transactions graph showing the time taken by each transaction (e.g. Scene) in the script.
A Transactions table showing metrics and messages for the selected transaction (e.g. Scene). (Click a
transaction's bar in the Transactions graph to select it.)
A Content timings graph, showing the time taken to transfer each content item.
For details of the Testrun Summary panel, refer to its description in Functional View.
Screening Room Regression View Regression View is designed to help users with regression testing, by making it easy to compare results from two
separate test runs (e.g. before and after the change you're testing).
To launch Regression View:
Select the Regression link from the Screening Room Home screen
Or
To switch from Monitoring View or Functional View: right-click near the top of the Screening Room
window (on the Monitor Master banner) to display a popup menu and select the Regression view option
In Regression View, the results are displayed in six separate panels:
The panels are:
1. Plans and Tests
-
Screening Room User Guide Chapter 3 - The Screening Room option
Ascom Network Testing Ltd., 2000-2013 Page 37 of 129 03 June 2013
2. Test Runs
3. Testrun Summary (lhs)
4. Content (lhs)
5. Testrun Summary (rhs)
6. Content (rhs)
Plans and Tests panel (1)
The top left panel in Screening Room's Regression View is Plans and Tests, a hierarchical view of the Plans and
Test Scripts on this Agent which you can use to list and manage the Plans, Tests and Schedules on this Agent. You
can also use it to help select the testruns to compare:
To list the Test Scripts within a Plan:
Click the Plan's button. To hide them again, click its button
To list the scheduled Test Runs within a Plan:
Click the button on its Schedules folder. (If there is no Schedules folder, the Plan has no schedules.)
To list all Schedules on this Agent:
Click the button on the Agent schedules folder (this folder is listed after all the Plan folders).
To select all Test Runs on this Agent:
Click the This agent folder. Details will be displayed in the Test Runs panel.
To select Test Runs for a specific Plan, Test, or Schedule:
Click its name. Details will be displayed in the Test Runs panel.
For details of the Plans and Tests panel, refer to its description in Functional View.
Test Runs panel (2)
The top right panel in Screening Room's Regression View is Test Runs.
Across the top are three dropdown lists and a button, used to select the information to display in the remainder of
the panel:
Use the first dropdown list to choose how to view your results:
-
Screening Room User Guide Chapter 3 - The Screening Room option
Ascom Network Testing Ltd., 2000-2013 Page 38 of 129 03 June 2013
Tabular summary of - view a tabular summary of all results from the selected tests, and grand totals.
Provides a detailed numerical summary of results in table form
Extended summary of - view a graphical summary of all results from the selected tests, and grand totals.
Provides an "at a glance" summary of results in pie chart form
Each testrun from - view a list of individual testruns, showing detailed results for each. Allows you to
quickly select a testrun of interest and drill down into its results, content, metrics, diagnostics, etc.
The each testrun view is the one most often used for regression testing.
Use the second dropdown list to select the results (testruns) to view. The options are:
No plan - don't display anything
All tests and plans on this agent - all runs of all Tests on this Agent
Plan names - all runs of Tests in this Plan
Test names - all runs of this Test
(You can also select the results to view by selecting a Plan or Test from the Plans and Tests panel.)
Use the third dropdown list to select the time period to report on. The options, which are self-explanatory, are:
without time filter
for the last 1 hour
for the last 24 hours
for the last 7 days
for the last 31 days
for the last 365 days
Click the button to apply your currently-selected options. For example:
Key facilities
Use the three dropdown lists across the top of the pane to select the view type (numerical summary,
graphical summary, or each testrun), the tests to view results for, and the period to view results for - then
click the button. The each testrun view is best for regression testing.
Sort on any column by clicking the column heading
Click any testrun (in the each testrun view) OR right-click it and click the View detail (LHS) button in
the popup menu to display the results and content from a Test Run in the Testrun Summary and Content
panels on the left-hand side
Right-click any testrun (in the each testrun view) and click the View detail (RHS) button in the popup
menu to display the results and content for another Test Run in the Testrun Summary and Content panels
on the right-hand side - producing a display like this
-
Screening Room User Guide Chapter 3 - The Screening Room option
Ascom Network Testing Ltd., 2000-2013 Page 39 of 129 03 June 2013
For details of the Test Runs panel, refer to its description in Functional View.
Popup menu
Right-click a testrun to display this popup menu for it:
Use the first three options (Mark all, Mark incompletes, and Unmark all) to mark testruns. You can also
mark or unmark a testrun by clicking the checkbox at the start of its row
Use Report marked to create a report combining the marked runs. This displays the Report Testrun dialog,
for you to select the report type.
Use Stop marked to stop the marked testruns.
Use Delete marked to delete the marked testruns.
The small buttons along the bottom of the menu are as follows:
View details (LHS) - view the testrun details in the left-hand Testrun Summary and Content panels
View detail (RHS) - view the testrun details in the left-hand Testrun Summary and Content panels
Refresh panel - fetch new data from the Agent and redisplay the panel using it
Print panel - print the panel as a report. Displays the standard Print options dialog
Open as Excel - output the data in Excel (spreadsheet) format. You can choose to Open the data
(display it as an embedded spreadsheet, in the same panel) or Save the data (output to a Excel format file
you specify)
-
Screening Room User Guide Chapter 3 - The Screening Room option
Ascom Network Testing Ltd., 2000-2013 Page 40 of 129 03 June 2013
Open as XML - output the data in XML format, in another Browser window. You can then view, print,
or save it using the standard Browser facilities
You can also view details of a testrun (in the left-hand Testrun Summary and Content panel) by double-clicking it.
Testrun Summary panels (3, 5)
Screening Room's Regression View Testrun Summary panels show the following for the selected Test Runs:
A Testrun summary for this run. The Completion/service/system icons indicate whether the run completed
and whether the service and system were available for the entire run. Red icons indicate possible problems
with the script, service, or test system.
A Meta tags table, listing the meta tags for the testrun (information about the testrun rather than the results
it produced)
A Transactions graph showing the time taken by each transaction (e.g. Scene) in the script.
A Transactions table showing metrics and messages for the selected transaction (e.g. Scene). (Click a
transaction's bar in the Transactions graph to select it.)
A Content timings graph, showing the time taken to transfer each content item.
For details of the Testrun Summary panel, refer to its description in Functional View.
Content panels (4, 6)
Screening Room's Regression View Content panels list the content items downloaded for the selected Test Runs,
showing the following for each item:
ID: Its ID
Content: A description e.g. its URL
Time: The time the download started
MIME: The content MIME type
Charset: Its character set
Delay: the time it took to download
Indicative Size: its approximate size
Indicative Rate: its approximate download rate
For details of the Content panel, refer to its description in Functional View.
-
Ascom Network Testing Ltd., 2000-2013 Page 41 of 129 03 June 2013
Chapter 4 - The Plan Dashboard option
The Screening Room Plan dashboard option creates a visual dashboard consisting of graphs that summarise
results from multiple testruns from a Plan, making it easy for you get a broad view.
Creating the Plan Dashboard
Interpreting the Plan Dashboard
Customising the Plan Dashboard
Manipulating the Plan Dashboard
Comparing graphs
Cumulative graphs
Watching data
Creating the Plan Dashboard To create the Plan Dashboard:
Click the Plan dashboard option on the Screening Room Home screen:
You will now be taken through a sequence of steps to select the Plan and information you're interested in.
Hint: If you want to go back to change your mind, use your Browser's Back button or click the underlined
link for the step to return to - for example, click the Select run link to return to Step 2 from a later step.
Step 1: Choose the Plan containing the results to review from the list (e.g. Standard Scripts). Click Next>
Step 2: Choose External Script. Click Next>
-
Screening Room User Guide Chapter 4 - The Plan Dashboard option
Ascom Network Testing Ltd., 2000-2013 Page 42 of 129 03 June 2013
Step 3: Choose this agent. Click Next>
Step 4: Choose up to five Tests to plot on the dashboard. Click Next>
-
Screening Room User Guide Chapter 4 - The Plan Dashboard option
Ascom Network Testing Ltd., 2000-2013 Page 43 of 129 03 June 2013
Step 5: Choose the time period to graph, by choosing the start and end dates and times from the lists. The
dates offered correspond to test run dates for the selected tests.
Step 6: Choose up to 5 metrics to graph, from the lists. You can choose any of the metrics recorded for these
test runs in this time period.
Step 7: Customise the graph appearance. This step is optional, as the defaults are appropriate for most users
and you can quickly adjust any that aren't, once you've seen the default graphs. (See Customising graph
appearance, for details.)
Click Next> to create and display a dashboard view of the selected results:
-
Screening Room User Guide Chapter 4 - The Plan Dashboard option
Ascom Network Testing Ltd., 2000-2013 Page 44 of 129 03 June 2013
Interpreting the Plan Dashboard Once you have created the Plan Dashboard (previous section), you will see it consists of:
A separate tab for each Agent (the Agents selected in step 3)
A column of graphs for each test script (the graphs selected in step 4)
A row of graphs for each metric (the metrics selected in step 6)
Labels and other data in the row and column borders
-
Screening Room User Guide Chapter 4 - The Plan Dashboard option
Ascom Network Testing Ltd., 2000-2013 Page 45 of 129 03 June 2013
The horizontal axis of each graph shows the entire time period selected in step 5, so all graphs have the same X
axis. The vertical axis of each graph shows the values of the chosen metric.
But you can do a lot more, including:
Customising the appearance
Manipulating the graphs
Comparing graphs
Cumulative graphs
Watching data
Customising the Plan Dashboard To change the look of your Plan Dashboard:
Select different options from the Step 7 controls:
-
Screening Room User Guide Chapter 4 - The Plan Dashboard option
Ascom Network Testing Ltd., 2000-2013 Page 46 of 129 03 June 2013
Click Next> to redraw the graphs.
The default options are:
Success: choose Not smoothed to plot each success result for each run as a separate point, or Averaged per
minute/hour/day/year to plot averages over time. Averaging makes it easier to spot trends, but might hide
fluctuations or patterns that are important to you
Metrics: choose Not smoothed to plot each metric for each run as a separate point, or Averaged per
minute/hour/day/year to plot averages over time. Averaging makes it easier to spot trends, but might hide
fluctuations or patterns that are important to you
Metric range: if using smoothed results, choose one of these options to add or remove curves showing the
upper and/or lower values:
Maximum & minimum limits
Maximum limit
Minimum limit
Neither limit
not smoothed smoothed, neither limit smoothed, both limits
Size: Choose the graph width (Narrow/Normal/Wide) from the first control, and graph height
(Short/Normal/Tall) from the second
Style: Choose the line style (Thin line or Thick line) from the first control, and the point style (No points,
Small points or Large points,) from the second
Override Y-max: The Y-axis of each graph is chosen automatically so that the highest value for each metric
plotted just fits. If you wish, you can override this process by typing a maximum Y axis value in this box.
You might use this, for example, to normalize all graphs so they are directly comparable, or to remove
spurious data from a graph.
For example, if plotting timings that are all in the range 15-20 seconds, with just one rogue value of 200
seconds, the default Y axis will be 0-200 seconds, compressing the details in the rest of the graph. Type 20 in
the Override Y-max box to ignore the rogue value and reveal the variations in the other values, if they are of
more interest:
-
Screening Room User Guide Chapter 4 - The Plan Dashboard option
Ascom Network Testing Ltd., 2000-2013 Page 47 of 129 03 June 2013
default graph
showing rogue data
Override Y-Max = 20
hiding rogue data
Any data points that are not plotted because they don't fit are indicated by an upwards arrow (as shown in
the right-hand example, above).
Manipulating the Plan Dashboard You can change the Plan Dashboard display options (set in steps 5-7) at any time to change the look:
Simply make the change(s) to the options
Click Next> to redraw the dashboard with the new display options.
You can also:
Double-click any graph to display the testrun results in another Browser window.
Select an cell, cells, graph or graphs by clicking them. Select an entire row by clicking the Select link in its
left-hand border. You can then analyse the selected metric graphs by using the three buttons in the top left
of the dashboard. to:
Compare the results
Cumulate the results
Watch the results
Comparing Plan Dashboard graphs Once you've created a Plan Dashboard (details), you can overlay 2 or more its graphs to help you compare their
results. This only makes sense if the graphs use the same or similar metrics.
To overlay graphs:
Select the graphs to compare (overlay) by clicking them.
Click the Compare button.
The graphs will be combined and displayed as a single graph in another browser window, like this:
-
Screening Room User Guide Chapter 4 - The Plan Dashboard option
Ascom Network Testing Ltd., 2000-2013 Page 48 of 129 03 June 2013
This new window has its own set of configuration options:
Use the Time period options to change the period to graph, as described above.
Use the Thresholding options to display a visual warning if the data falls outside the thresholds you
specify, as described below.
Use the Customise graph appearance options to change the graph appearance, as described above.
After making any changes, click Next to apply the revised options to the graph, and re-plot it.
Using Thresholds
You can define Thresholds to help you quickly recognise when results fall outside the expected range. They are
normally used in combination with a regular refresh rate (see Refresh control), to monitor key metrics.
To display a visual warning if the most recent data point is above a threshold value:
Select the graph or graphs to check, from the Plan Dashboard screen.
Click the Compare button.
When the comparison graph is displayed, change the thresholding type from None to Upward and two
additional Level controls are displayed:
Type the upper (error) threshold level in the first box, and the lower (warning) threshold level in the second
box.
Click Next to apply the revised options to the graph.
Now, if the most recent data point (on all the curves) is below both thresholds, the graph will have a green
background:
-
Screening Room User Guide Chapter 4 - The Plan Dashboard option
Ascom Network Testing Ltd., 2000-2013 Page 49 of 129 03 June 2013
If the most recent data point (on one or more curves) is above the lower threshold but below the upper threshold,
the graph will have a yellow (warning) background:
And if the most recent data point (on one or more curves) is above the upper threshold, the graph will have a red
(error) background:
-
Screening Room User Guide Chapter 4 - The Plan Dashboard option
Ascom Network Testing Ltd., 2000-2013 Page 50 of 129 03 June 2013
To display a visual warning if the most recent data point is below a threshold value:
Select the graph or graphs to check, from the dashboard screen.
Click the Compare button.
When the comparison graph is displayed, select a thresholding type of Downward and two additional
Level controls are displayed:
Type the upper threshold level (warning) in the first box, and the lower threshold level (error) in the second
box.
Click Next to apply the revised options to the graph.
Now, if the most recent data point is above both thresholds, the graph will have a green background; if the most
recent data point is below the upper threshold, the graph will have a yellow (warning) background; and if the most
recent data point is below the lower threshold, the graph will have a red (error) background.
Cumulative Plan Dashboard graphs You can use Cumulative Plan Dashboard graphs to show the progress of an accumulating metric over time, with
the metric value on the X-axis and the percentage of samples meeting that metric on the Y-axis. Cumulative graphs
provide a good way to monitor targets such as "80% of SMS messages must be delivered with 10 seconds" and to
spot patterns and clustering of results which may warrant investigation.
To produce a cumulative Plan Dashboard graph:
Select the metric graph(s) to cumulate by clicking them on the Plan Dashboard. (It only makes sense to
cumulate graphs for the same or similar metrics.)
Click the Cumulate button.
The selected graphs will be combined and displayed together in another browser window, like this:
-
Screening Room User Guide Chapter 4 - The Plan Dashboard option
Ascom Network Testing Ltd., 2000-2013 Page 51 of 129 03 June 2013
This new window has its own options for configuring the graph:
Use Time period to choose the period to graph, as described above.
Use Thresholding to display a visual warning if the data falls outside the thresholds you specify. See Using
Thresholding, above.
Use Cumulative range to change the range plotted along the X-axis.
Use Customise graph appearance to change the graph appearance, as described above.
Click Next to apply the revised options to the graph.
Watching Plan Dashboard data You can use a Watch graph to display a continuously moving time window on the selected data on a Plan
Dashboard, e.g. to always display results for just the last hour.
To produce a Watch graph:
Select the graph(s) to watch by clicking them on the Plan Dashboard. (It only makes sense to watch graphs
together if they are for the same or similar metrics.)
Click the Watch button.
The graphs will be combined and displayed as a single graph in another Browser window, like this:
-
Screening Room User Guide Chapter 4 - The Plan Dashboard option
Ascom Network Testing Ltd., 2000-2013 Page 52 of 129 03 June 2013
This new window has its own options for configuring the Watch graph:
Use the Time period controls to change the period to graph. Notice that this is defined as a period relative
to now. You can choose to graph the previous:
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 20 or 30 minutes
1, 2, 3, 4, 5 or 6 hours
Use the Thresholding contro