Screening Room

129
© Ascom Network Testing Ltd., 2000-2013 Page 1 of 129 03 June 2013 TEMS™ Monitor Master v9.2.1 Screening Room User Guide Document: MM-0016-D Created by: EWJP Status: Released © Ascom Network Testing Ltd, 2000-2013 All rights reserved

description

Good Document for Configuration

Transcript of Screening Room

  • Ascom Network Testing Ltd., 2000-2013 Page 1 of 129 03 June 2013

    TEMS Monitor Master v9.2.1

    Screening Room

    User Guide

    Document: MM-0016-D

    Created by: EWJP

    Status: Released

    Ascom Network Testing Ltd, 2000-2013

    All rights reserved

  • Ascom Network Testing Ltd., 2000-2013 Page 2 of 129 03 June 2013

    Copyright Ascom Network Testing Ltd, 2000-2013. All rights reserved

    Unauthorized copying, distribution and use of the document or its content is strictly prohibited.

    Note: In this guide the term "Monitor Master" refers to the Ascom product "TEMS Monitor Master".

    All registered trademarks, quotes, products or service names are acknowledged.

    Ascom Network Testing Ltd

    Ash House, Shackleford Road

    Elstead, Surrey GU8 6LB

    United Kingdom

    Ascom Network Testing Ltd is an Ascom company, part of the Network Testing division.

    www.ascom.com

    Version Overview

    Version Date Author Modification

    1.0 25/04/2013 EWJP Initial release for Monitor Master v9.2.1

    1.1 03/06/2013 EWJP Minor updates

    Information in this document is subject to change without notice. For the latest version, please visit

    www.ascom.com/temsmonitormaster/support

    Document Management

    File name: screening_room.pdf

    Pages: 129

    Released on: 03/06/2013

    Approved by: AC

  • Ascom Network Testing Ltd., 2000-2013 Page 3 of 129 03 June 2013

    Table of Contents CHAPTER 1 - INTRODUCTION .................................................................................................................................... 5

    SCREENING ROOM OVERVIEW .......................................................................................................................................... 5 LOCAL MODE AND CENTRAL MODE ................................................................................................................................. 6 MAIN MENU OPTIONS ........................................................................................................................................................ 6 OTHER SCHEDULING FACILITIES ........................................................................................................................................ 7 OTHER REPORTING FACILITIES .......................................................................................................................................... 7 THIS GUIDE ....................................................................................................................................................................... 7 RELATED GUIDES .............................................................................................................................................................. 8

    CHAPTER 2 - USING SCREENING ROOM ................................................................................................................. 9

    LAUNCHING SCREENING ROOM ........................................................................................................................................ 9 USING SCREENING ROOM ............................................................................................................................................... 11

    CHAPTER 3 - THE SCREENING ROOM OPTION .................................................................................................. 16

    SCREENING ROOM FUNCTIONAL VIEW ........................................................................................................................... 17 SCREENING ROOM MONITORING VIEW ........................................................................................................................... 33 SCREENING ROOM REGRESSION VIEW ............................................................................................................................ 36

    CHAPTER 4 - THE PLAN DASHBOARD OPTION ................................................................................................... 41

    CREATING THE PLAN DASHBOARD ................................................................................................................................. 41 INTERPRETING THE PLAN DASHBOARD ........................................................................................................................... 44 CUSTOMISING THE PLAN DASHBOARD ............................................................................................................................ 45 MANIPULATING THE PLAN DASHBOARD ......................................................................................................................... 47 COMPARING PLAN DASHBOARD GRAPHS ........................................................................................................................ 47 CUMULATIVE PLAN DASHBOARD GRAPHS ...................................................................................................................... 50 WATCHING PLAN DASHBOARD DATA ............................................................................................................................. 51

    CHAPTER 5 - RECENT TESTS & AVAILABILITY ................................................................................................. 53

    USING THE RECENT SUCCESS FACTS SCREEN .................................................................................................................. 53 THE RECENT SUCCESS FACTS DISPLAY OPTIONS ............................................................................................................ 53 THE RECENT SUCCESS FACTS FILTERS ........................................................................................................................... 55 THE RECENT SUCCESS FACTS LINKS ............................................................................................................................... 55 THE RECENT SUCCESS FACTS REPORT ............................................................................................................................ 55

    CHAPTER 6 - METRICS, MESSAGES & TIMINGS ................................................................................................. 57

    USING THE RECENT METRICS, MESSAGES OR TIMING FACTS SCREEN ............................................................................ 57 THE RECENT METRIC/TIMING/MESSAGE FACTS DISPLAY OPTIONS ................................................................................ 58 THE RECENT METRIC FACTS SCREEN .............................................................................................................................. 59 THE RECENT TIMING FACTS SCREEN .............................................................................................................................. 61 THE RECENT MESSAGE FACTS SCREEN ........................................................................................................................... 63

    CHAPTER 7 - DIMENSIONS ........................................................................................................................................ 67

    PLANS ............................................................................................................................................................................. 67 TESTS .............................................................................................................................................................................. 67 TRANSACTIONS ............................................................................................................................................................... 68 METRICS ......................................................................................................................................................................... 68 TIMINGS .......................................................................................................................................................................... 69 MESSAGE TYPES .............................................................................................................................................................. 69

    CHAPTER 8 - LOCAL SCHEDULES ........................................................................................................................... 70

    LOCAL SCHEDULES ......................................................................................................................................................... 70 ADDING A LOCAL SCHEDULE .......................................................................................................................................... 72 EDITING A LOCAL SCHEDULE.......................................................................................................................................... 75 DELETING A LOCAL SCHEDULE ....................................................................................................................................... 76

    CHAPTER 9 - DEVICE UPDATE ................................................................................................................................. 77

    WHY USE DEVICE UPDATE? ............................................................................................................................................ 77 USING THE DEVICE UPDATE OPTION ................................................................................................................................ 77

  • Ascom Network Testing Ltd., 2000-2013 Page 4 of 129 03 June 2013

    CHAPTER 10 - AGENT RESOURCE MANAGEMENT ............................................................................................ 80

    MANAGEMENT OPTIONS .................................................................................................................................................. 80 RESOURCE MANAGEMENT SCRIPTS .................................................................................................................................. 80 RESOURCE MANAGEMENT CONCEPTS .............................................................................................................................. 81 RESOURCE ITEMS ............................................................................................................................................................ 81 RESOURCE USERS............................................................................................................................................................ 85 RESOURCE ALLOCATIONS ............................................................................................................................................... 88 RESOURCE USAGE ........................................................................................................................................................... 92 RESOURCE GRAPHING ...................................................................................................................................................... 93

    APPENDIX A - RESPONSE CODES ............................................................................................................................. 98

    HTTP V1.X STATUS CODES .............................................................................................................................................. 98

    APPENDIX B - TESTRUN REPORTS ........................................................................................................................ 100

    THE HIGH DETAIL CONTENT LISTING ............................................................................................................................. 101 THE MEDIUM DETAIL CONTENT LISTING........................................................................................................................ 102 THE LOW DETAIL CONTENT LISTING .............................................................................................................................. 103 THE COMPLIANCE ERRORS AND BROKEN LINKS REPORT................................................................................................ 104 THE ALL DEVICE ERRORS AND USABILITY REPORT ........................................................................................................ 106 THE COSMETIC DEVICE ERRORS REPORT ....................................................................................................................... 107 THE FUNCTIONAL DEVICE ERRORS REPORT ................................................................................................................... 108 THE OTHER DEVICE ERRORS REPORT ............................................................................................................................. 109 THE USABILITY ERRORS REPORT ................................................................................................................................... 110 THE LOADING REPORT................................................................................................................................................... 111 THE MESSAGES REPORT ................................................................................................................................................ 112 THE USER EVENTS REPORT ............................................................................................................................................ 114 THE METRICS REPORT ................................................................................................................................................... 116 THE RESPONSE DELAY REPORT...................................................................................................................................... 117 THE DELTA TIMINGS REPORT ........................................................................................................................................ 119 THE GRAPHICAL TIMING REPORT................................................................................................................................... 120 THE TEST AND TEST RUN PERFORMANCE REPORT .......................................................................................................... 121 THE TRANSACTION TIMINGS REPORT ............................................................................................................................. 123 THE SCRIPT REPORT ...................................................................................................................................................... 124 THE SITE MAP REPORT ................................................................................................................................................... 124 THE APPLIED DEVICE TEST CASES REPORT .................................................................................................................... 125

    INDEX ............................................................................................................................................................................. 127

  • Ascom Network Testing Ltd., 2000-2013 Page 5 of 129 03 June 2013

    Chapter 1 - Introduction

    Note: Screening Room is a legacy tool, retained mainly to support users running scripts in legacy (local) mode;

    most Monitor Master QM users should not need to use Screening Room - or read this guide!

    In this Chapter:

    Screening Room

    Local Mode and Central Mode

    Main menu options

    Other reporting facilities

    Other scheduling facilities

    This guide

    Related guides

    Screening Room Overview Monitor Master Screening Room (also known as the Agent Menu) provides a reporting and management

    interface for Monitor Master Agents. It consists of a browser-based menu system that provides convenient access

    to the following, for the selected Agent:

    Test run results and other data stored on this Agent. The wide range of facilities allows users to view,

    explore and produce reports on data to whatever level of detail they wish. Data can also be exported in a

    variety of formats for use in other applications.

    Scheduling of Local Mode scripts on this Agent.

    Device profile updates for this Agent, to keep the device details up to date with the Device Master Live

    Server or other trusted authority

    Resource management for this Agent, to help avoid conflicts between Local Mode test runs

  • Screening Room User Guide Chapter 1 - Introduction

    Ascom Network Testing Ltd., 2000-2013 Page 6 of 129 03 June 2013

    The Screening Room Home screen

    Notice that each Agent has its own separate Screening Room.

    Note: Throughout this guide we will use the term Agent rather than "Probe", "Agent Appliance" or" Software

    Agent".

    Local Mode and Central Mode In a Monitor Master QM system, scripts are usually saved in Central Mode (to the QM Server) but can also be

    saved in Local Mode (to an Agent):

    Screening Room provides reports, resource management and scheduling for the Agent's Local Mode

    scripts. (The reports can also be used for Central Mode resuilts.)

    Quality Manager (QM) provides superior central reports, resource management and scheduling for all

    Central Mode scripts. (The reports can also use Local Mode results.)

    In a Monitor Master QM system, we strongly recommend the use of Central Mode for all scripts. You should only

    use Local Mode (and Screening Room) if QM is not available.

    WARNING: Do not schedule both Local Mode and Central Mode tests on the same Agent; the resource

    management strategies used are incompatible.

    Main menu options The Screening Room main screen (above) is a menu that provides these main options:

    Screening Room - review summaries and details of recent test runs, with drilldown to transaction and

    content level. Also provides access to Scheduling facilities for local mode scripts

    Plan dashboard - summarise results for specific plans or tests

  • Screening Room User Guide Chapter 1 - Introduction

    Ascom Network Testing Ltd., 2000-2013 Page 7 of 129 03 June 2013

    Recent tests & availability - a tabular summary of recent tests indicating any problems

    Metrics, messages and timings - detailed results from multiple tests

    Dimensions - view your test data along different dimensions: by plan, test, transaction, metric, timing, or

    message type

    Device update - download the latest device profiles from the Device Master Live Server

    Agent resource management - managing the resources on this Agent to avoid conflicts

    Each of these main options has its own chapter later in this guide.

    Other scheduling facilities Monitor Master provides two ways to schedule test runs:

    Quality Manager schedules Central Mode scripts to run on single or multiple Agents. These Central

    Schedules support both single-party and multi-party scripts

    Screening Room schedules Local Mode scripts to run on a single Agent. These Local Schedules do not

    support mutli-party Scripts

    Other reporting facilities Monitor Master provides four main reporting interfaces: Box Office, Screening Room, Story Boarder and Projection

    Room:

    Box Office provides a range of standard Dashboards (reports) for both Agent and Projector results, as well

    as development and integration facilities for custom Dashboards. Dashboards can be accessed via the Box

    Office GUI, or from the Quality Manager Reporting menu.

    Screening Room provides access to all recent test run results (and other data) stored on an Agent.

    Story Boarder provides access to summary results for the current test run only. This is mainly intended for

    test script developers to test their scripts, but can also be used for ad hoc testing and monitoring in small

    installations.

    Projection Room provides access to test run results collected from multiple Agents by a Projector, usually

    over a much long time period than Screening Room.

    Box Office is the recommended reporting interface for all Monitor Master installations.

    Note that the results from Local Mode and Central Mode test runs are stored in the same way (on the Agent that

    ran the test, and associated Projector).

    This guide This guide consists of the following chapters:

    Chapter 1 - Introduction

    Chapter 2 - Using Screening Room

    Chapter 3 - The Screening Room option

    Chapter 4 - The Plan Dashboard option

    Chapter 5 - The Recent Tests & Availability option

    Chapter 6 - The Metrics, Messages & Timings option

    Chapter 7 - The Dimensions option

  • Screening Room User Guide Chapter 1 - Introduction

    Ascom Network Testing Ltd., 2000-2013 Page 8 of 129 03 June 2013

    Chapter 8 - Local Schedules

    Chapter 9 - The Device Update option

    Chapter 10 - The Agent Resource Management option

    It concludes with an Appendix on HTTP return codes and testrun reports.

    Related guides Read the QM User Guide for information on using Central Mode resource management, scheduling and reporting.

    Read the Story Boarder User Guide for information on creating and running test scripts, including how to view and

    interpret test results.

    Read the Projection Room User Guide for information on using legacy enterprise-wide reporting.

  • Ascom Network Testing Ltd., 2000-2013 Page 9 of 129 03 June 2013

    Chapter 2 - Using Screening Room

    This chapter describes:

    How to launch Screening Room

    How to use Screening Room - the general principles

    The Home screen menu

    Subsequent chapters describe each of the Home screen (main menu) options in turn.

    Note: While a standard Monitor Master installation allows all Users to access all the reports and all the data on an

    Agent, your system administrator may implement measures to control access to certain types of report and test

    data.

    Launching Screening Room You can launch Screening Room:

    From Story Boarder

    From Internet Explorer

    From a Shortcut

    From an Agent

  • Screening Room User Guide Chapter 2 - Using Screening Room

    Ascom Network Testing Ltd., 2000-2013 Page 10 of 129 03 June 2013

    Launching Screening Room from Story Boarder

    If you are developing Test Scripts, it can be convenient to launch Screening Room from within Story Boarder.

    To launch Screening Room from Story Boarder:

    Select the View Agent option from the Story Boarder Tools menu or click the View Agent button, to launch

    Screening Room for your most recently-used Agent.

    Or:

    Choose the Select Agent option/button to list the known Agents:

    Select the Agent and click OK to launch the Screening Room for that Agent.

    (If the Agent is not listed, click Add, type its network address or (routable) name and click OK to add it to

    the list.)

    Now use View Agent to launch Screening Room for the selected Agent.

    Launching Screening Room from Internet Explorer

    You can launch Screening Room on any machine on your network (even one without Monitor Master installed) by

    typing the appropriate URL in your web browser.

    To launch Screening Room from Internet Explorer:

    Launch your web browser

    In the address bar, type the URL of the Agent you want to connect to, or its (routable) name, followed by

    :9090 and press Enter

    This will display Screening Room for that Agent

    For example, to use an Agent whose address is 10.1.1.85 and URL is agent3.xyz.com, you could type:

    http://10.1.1.85:9090/

    or

    http://agent3.xyz.com:9090/

    Or, if the name agent3 is a routable name resolving to this address, you could simply type:

    http://agent3:9090/

    Launching Screening Room from a Shortcut

    Once you have connected to Screening Room (by typing the URL in Internet Explorer), you can create a Shortcut to

    make connecting easier in the future.

    To create a Screening Room Shortcut:

    Connect to Screening Room on the Agent you want to use by typing its URL (example)

  • Screening Room User Guide Chapter 2 - Using Screening Room

    Ascom Network Testing Ltd., 2000-2013 Page 11 of 129 03 June 2013

    Once the Screening Room Home page is displayed, select the Add to Favorites option from your Browser's

    Favorites menu.

    Edit the name if you wish, then click OK to add it your main Favorites menu. (Refer to your Browser Help

    for the other options.)

    You can now launch the Screening Room for this Agent by selecting this option from the Favorites menu.

    To add the Shortcut to your desktop:

    Select the option from the Browser's Favorites menu and drag it to the desktop.

    You can now launch the Screening Room for that Agent by double-clicking the desktop shortcut.

    Launching Screening Room from an Agent

    If you are logged in on an Agent, you can use an installed Shortcut to launch its Screening Room.

    To launch Screening Room from the Agent:

    Click the Start button and select the option Ascom/TEMS Products / TEMS Monitor Master Servers /

    Server Reports / Script Server Database Reports

    This will launch the Screening Room for that Agent

    Using Screening Room Once you have launched an Agent's Screening Room , you can use it to produce a wide range of reports, schedule

    tests, update device profiles, and manage resources for this Agent. These functions are described in detail in

    subsequent chapters; this section describes the basic concepts and facilities you'll need to use Screening Room:

    Controls

    Printing

    Data export

    Refresh control

    The Home screen

    Screening Room Controls

    Screening Room is a web-based interface, so many of the facilities and controls should be very familiar already, as

    they are provided by your Web Browser.

    For example:

    Underlined text signifies a hyperlink - click it to follow the link

    To return to a previous screen, use the Browser's Back button

    To print the current screen, use the Browser's Print button or Print command (in the File menu) - see

    Printing

  • Screening Room User Guide Chapter 2 - Using Screening Room

    Ascom Network Testing Ltd., 2000-2013 Page 12 of 129 03 June 2013

    To use more of your screen for results, you can hide toolbars by selecting options from the View menu

    If a panel has scrollbars, use them to scroll the content in that panel

    And so on.

    Screening Room uses a few controls that you might not know:

    Many report windows have a refresh control in the top right corner. You can use this to choose how often

    the displayed results will be updated (by fetching new data from the Agent) - see Refresh control

    The refresh control

    A button means there's more information hidden below this item; click the button to reveal it; a button

    means you can hide the information below this item; click the button to hide it.

    Many window elements are "hot" i.e. hyperlinked. If the mouse pointer changes from an arrow to a hand

    while over an element, you can click it to link to related information.

    Column header buttons are displayed along the top of most tables, like this:

    These have two main functions: to show what data is in that column, and to control how the table is sorted.

    Click a button to sort the table on that column; click it again to reverse the sort order. A triangle shows the

    sort column and sort direction

    Windows are divided up into panels by narrow borders. You can drag these borders to change the panel's

    size.

    Each panel has its own right-click (popup) menu, providing access to a range of useful facilities for that

    panel and its contents.

    When you select an option from a dropdown list, its name is also displayed on the Browser status bar - to

    help you choose names that are too long to fit in the list.

    Printing Screening Room Reports

    Printing uses the standard Browser facilities:

    Use File/Page Setup to configure the printer and paper

    Use File/Print Preview to preview (check) the output

    Use File/Print to choose print options and begin printing

    Each Screening Room screen is implemented as a frameset i.e. a group of frames (which we call panels here), so the

    print options tab provides useful options to:

    Print as laid out on screen

    Print the selected frame only

    Print all the frames, individually

    To print the contents of a single panel:

    Right-click in the panel to display a popup menu like this:

    (Some menus have additional options, but they all have this row of buttons along the bottom.)

    Click the Print panel button,

  • Screening Room User Guide Chapter 2 - Using Screening Room

    Ascom Network Testing Ltd., 2000-2013 Page 13 of 129 03 June 2013

    (This facility is provided by the Screening Room screens: Functional, Regression, and Monitoring views.)

    Exporting Screening Room Reports

    Screening Room provides a range of facilities to export the current report (as shown on-screen), in different data

    formats, for input to other applications.

    The Screening Room screens (Functional, Regression, and Monitoring views) display multiple related data streams

    in separate panels. To output the contents of a panel:

    Right-click within the panel to display a popup menu ending with these buttons:

    Click the Open in Excel button, to output the data in Excel (spreadsheet) format. You can choose

    whether to Open or Save the data. Open will display it as an embedded spreadsheet, in the same panel;

    Save will output the data to the Excel format file you specify.

    Click the Open as XML button, to output the data in XML format, in another Browser window for you

    to view, print, save etc using the standard Browser facilities.

    The other screens (Recent facts, Dimensions, Dashboard and Metrics/ Messages/ Timings) present a single data

    stream, so you can output the whole screen. In the top right corner of the screen is a menu of output formats.

    Choose the format you want:

    Raw XML will output simple XML to the Browser window

    Rich XML will output semantically richer XML to the Browser window

    CSV will output comma separated values to the Browser window

    Excel will output an Excel format spreadsheet. You will be asked whether to Open or Save the data. Open

    will display it as an embedded spreadsheet, in the current screen; Save will output the data to the Excel

    format file you specify.

    Table will output the data to a specified Table.

    Refresh control

    Most reporting screens have a Refresh control in the top right corner:

  • Screening Room User Guide Chapter 2 - Using Screening Room

    Ascom Network Testing Ltd., 2000-2013 Page 14 of 129 03 June 2013

    You can use this to choose how often to update the screen, to reflect new results logged by the Agent. The options

    are:

    Never refresh - the default

    Refresh now

    Refresh every 3 seconds - see warning below

    Refresh every 15 seconds

    Refresh every 1 minute

    Refresh every 5 minutes

    Refresh every 15 minutes

    Refresh every hour

    Warning: The shorter refresh times should only be used for short periods, when you really them, as each refresh

    requires significant additional processing by the Agent. Refreshing every 3 seconds may disrupt the Agent's main

    task of running Tests and collecting Testrun data.

    However, note that certain reports specify a fixed time period to report on. Once you're past the period end, the

    report on the screen will no longer change when refreshed, because there will be no more new data for the selected

    period! To avoid this problem, either choose an end period in the future, edit the end date at regular intervals, or

    use a report with a moving time window such as the Watch facility or Recent test & availability report.

    Using the Screening Room Home screen

    The Home screen is displayed when you launch Screening Room:

    It offers your main menu, with seven main options:

    Screening Room - use to review details of previous test runs, see Chapter 3, Screening Room option. Also

    provides access to the test scheduling facilities - see Chapter 8, Local Schedules

    Plan dashboard - use to summarize results for specific plans or tests. Also provides direct links to recently-

    added plans. See Chapter 4, Plan Dashboard option

  • Screening Room User Guide Chapter 2 - Using Screening Room

    Ascom Network Testing Ltd., 2000-2013 Page 15 of 129 03 June 2013

    Recent tests & availability - a tabular summary of recent tests. Also provides direct links to recent failed

    tests. See Chapter 5, Recent Tests & Availability option

    Metrics, messages and timings - detailed results from multiple tests. See Chapter 6, Metrics, Messages and

    Timings option

    Dimensions - view your test data along different dimensions: by plan, test, transaction, metric, timing, or

    message type. See Chapter 7, Dimensions options

    Device update - download the latest device profiles from the main Device Master Live Server. See Chapter

    9, Device Update option

    Agent resource management - managing the resources on this Agent to avoid conflicts. See Chapter 10,

    Agent Resource Management option

    You can return to the Home screen from most other Screening Room screens by clicking the Home link, in the top

    right corner of the screen.

  • Ascom Network Testing Ltd., 2000-2013 Page 16 of 129 03 June 2013

    Chapter 3 - The Screening Room option

    You can launch any of the three Screening Room reports by clicking their links on the Screening Room Home

    Screen:

    They provide detailed information on individual test runs:

    Click Functional to review results from functional testing

    Click Monitoring to review results from service monitoring

    Click Regression to compare results from regression testing

    Once you've selected a view, you can switch easily to another view:

    Right-click anywhere in the Monitor Master banner near the top of the window:

  • Screening Room User Guide Chapter 3 - The Screening Room option

    Ascom Network Testing Ltd., 2000-2013 Page 17 of 129 03 June 2013

    This will display a popup menu:

    Choose the view you want (Functional, Monitoring, or Regression), or click Agent home to return to the

    Home (main menu) screen.

    Each view will now be described in detail in its own section. They have a lot in common so to avoid repetition, we

    will describe Functional View in detail, then describe how Monitoring View and Regression View differ.

    Screening Room Functional View Functional View is designed to help you review results from functional testing, allowing you to summarize results

    across all plans, focus on particular plans or tests, and drill right down into the details of an individual test run.

    To launch Functional View:

    Select the Functional link from the Screening Room Home screen

    Or

    To switch from Regression View or Monitoring View: right-click near the top of the Screening Room

    window (on the Monitor Master banner) to display a popup menu and select the Functional view option

    In Functional View, results are displayed in seven separate panels:

  • Screening Room User Guide Chapter 3 - The Screening Room option

    Ascom Network Testing Ltd., 2000-2013 Page 18 of 129 03 June 2013

    The panels are:

    1. Plans and Tests

    2. Test Runs

    3. Testrun Summary

    4. Content

    5. Content Summary

    6. Content Detail

    7. Preview

    Panels 3-7 are empty initially. They display details of a testrun once you've selected one in the Test Runs panel.

    Plans and Tests panel (1)

    The top left panel in Screening Room's Functional View is Plans and Tests, a hierarchical view of the Plans and

    Test Scripts on this Agent:

    You can use this view to list and manage the Plans, Tests and Schedules on this Agent. You can also use it to help

    select testruns to report on.

    To list the Test Scripts within a Plan:

    Click the Plan's button. (To hide them again, click its button.)

    To list the scheduled Test Runs within a Plan:

    Click the button on its Schedules folder. (If there is no Schedules folder, the Plan has no schedules.)

    To list all schedules on this Agent:

  • Screening Room User Guide Chapter 3 - The Screening Room option

    Ascom Network Testing Ltd., 2000-2013 Page 19 of 129 03 June 2013

    Click the button on the Agent schedules folder (this folder is listed after all the Plan folders).

    To select all Test Runs on this Agent:

    Click the This agent folder. Details will be displayed in the Test Runs panel.

    To select Test Runs for a specific Plan, Test, or Schedule:

    Click its name. Details will be displayed in the Test Runs panel.

    In a Plan folder, scheduled Tests are indicated by a small clock in the icon:

    - Test is not scheduled (no clock)

    - Test is scheduled but schedule is disabled

    - Test is scheduled and schedule is enabled

    In the Plans' Schedules folders and Agent schedules folder Schedule status is shown by icon color:

    - Schedule is enabled

    - Schedule is disabled

    For information on creating and maintaining Schedules, see Chapter 8, Local Schedules.

    Popup menus

    If you right-click anything within this panel you will see a popup menu with four buttons along the bottom:

    Refresh panel - fetch new data from the Agent and redisplay the panel using it

    Print panel - print the panel as a report. Displays the standard Print options dialog

    Open as Excel - output the data in Excel (spreadsheet) format. You can choose to Open the data

    (display it as an embedded spreadsheet, in the same panel) or Save the data (output to a Excel format file

    you specify)

    Open as XML - output the data in XML format, in another Browser window. You can then view, print,

    or save it using the standard Browser facilities

    The other options, at the start of the menu, are used to manipulate the selected item, as follows.

    Agent popup menu

    If you right-click This agent, the popup menu also offers:

    Agent home - move to the Home screen (main menu)

    Agent reports - displays the Report on Agent dialog. Select the type of report required, then click Launch:

    Plan and resource usage - show resource usage by all schedules on this Agent, useful for seeing

    detecting conflicts

    Available devices - list of devices in the Agent's device database

    Applied compliance test cases - lists the compliance tests applied to devices in the Agent's device

    database

    Current licence violations - lists any attempted violations of your Monitor Master license

    Projector outbox - lists the contents of the Projector outbox

    Recent logs - display a log for this Agent, showing all recent requests (from clients) and responses (from the

    Agent). This log is mainly intended for diagnostic purposes

  • Screening Room User Guide Chapter 3 - The Screening Room option

    Ascom Network Testing Ltd., 2000-2013 Page 20 of 129 03 June 2013

    Plan popup menu

    If you right-click a Plan, the popup menu also offers:

    Plan dashboard - display the Dashboard for this Plan (see Using Plan Dashboard)

    Delete this plan - from this Agent. Use with caution! Deleting a Plan will remove all its Test Scripts and

    Testrun results from the Agent!

    Test popup menu

    If you right-click a Test, the popup menu also offers:

    Test dashboard - display the Dashboard for this Test (this is the same as the Plan Dashboard, but with the

    Plan and Test pre-selected - see Plan Dashboard reports).

    Run test - displays the Run Test dialog for you to confirm (or change) the test selected, supply runtime

    parameters, and enable or disable routing of results to Projector. Click OK to run the Test Script - results will

    be displayed in the Testrun pane.

    Edit test - displays the Edit Test dialog for you to confirm (or change) the test selected. Click OK to edit the

    Test Script.

    Move test - displays the Move Test dialog for you to confirm (or change) the test selected and choose the

    Plan to move it to. Click OK to move the Test Script and associated results.

    Delete test - from this Plan. Use with caution! Deleting a Test will delete its script and all its testrun results

    from this Agent.

    Schedules folder popup menu

    If you right-click the Schedules folder (within a Plan) or the Agent schedules folder, the popup menu also offers:

    Add schedule - see Chapter 8, Local Schedules for details

    Schedule popup menu

    If you right-click a schedule, the popup menu also offers:

    Edit schedule - see Chapter 8, Local Schedules for details

    Delete schedule - see Chapter 8, Local Schedules for details

    Test Runs panel (2)

    The top right panel in Screening Room's Functional View is Test Runs. Across the top of the panel are three

    dropdown lists and a (Go) button, which are used to select the information to display in the remainder of the

    panel:

    Use the first dropdown list to chose how to view your results:

    Tabular summary of - view a tabular summary of all results from the selected tests, and grand totals.

    Provides a detailed numerical summary of results in table form - see The tabular summary view

  • Screening Room User Guide Chapter 3 - The Screening Room option

    Ascom Network Testing Ltd., 2000-2013 Page 21 of 129 03 June 2013

    Extended summary of - view a graphical summary of all results from the selected tests, and grand totals.

    Provides an "at a glance" summary of results in pie chart form - see The extended summary view

    Each testrun from - (default) view a list of individual testruns, showing detailed results for each. Allows

    you to quickly select a testrun of interest and drill down into its results, content, metrics, diagnostics, etc. -

    see The each testrun view

    Use the second dropdown list to select the results (testruns) to view. The options are:

    No plan - don't display anything

    All tests and plans on this agent - (default) all runs of all Tests on this Agent

    Plan names - all runs of Tests in this Plan

    Test names - all runs of this Test

    (You can also select the results to view by selecting a Plan or Test from the Plans and Tests panel.)

    Use the third dropdown list to select the time period to report on. The options, which are self-explanatory, are:

    without time filter

    for the last 1 hour (default)

    for the last 24 hours

    for the last 7 days

    for the last 31 days

    for the last 365 days

    Click the button to create a new report with your currently selected options.

    The "tabular summary" view

    The tabular summary view provides a numerical summary of all results for the selected tests, like this:

    The table starts with grand totals (for all selected tests), followed by a row for each selected test, displaying:

    Plan - the Plan name

    Test - the Test name

    Completion - percent of testruns completed

    Service - percent of testruns with 100% service availability

    System - percentage of testruns with 100% system availability

    First - the date of the first testrun

    Last - the date of the last testrun

    Fastest - the shortest run time (seconds)

    Slowest - the longest run time (seconds)

    Average - the average run time (seconds)

    Runs - the number of testruns of this test

    To sort by a column:

    Click the column heading

    Click the column headings again to reverse the sort direction.

  • Screening Room User Guide Chapter 3 - The Screening Room option

    Ascom Network Testing Ltd., 2000-2013 Page 22 of 129 03 June 2013

    To display a test's test runs in the "each testrun" view:

    Click that row

    Right-click anywhere in the panel to display a popup menu with these options:

    Tear-off - view the panel in another browser window

    Refresh panel - fetch new data from the Agent and redisplay the panel using it

    Print panel - print the panel as a report. Displays the standard Print options dialog

    Open as Excel - output the data in Excel (spreadsheet) format. You can choose to Open the data

    (display it as an embedded spreadsheet, in the same panel) or Save the data (output to a Excel format file

    you specify)

    Open as XML - output the data in XML format, in another Browser window. You can then view, print,

    or save it using the standard Browser facilities

    The "extended summary" view

    The extended summary view provides a graphical summary of results for the selected tests, like this:

    The first row is a grand total for all the selected testruns. Each subsequent row summarises results for a different

    selected test.

    The information shown is:

    Plan - the Plan name

    Test - the Test name

    A description of the test

    Completion - pie chart showing percentage of testruns completed

    Service - pie chart showing percentage of testruns with 100% service availability

    System - pie chart showing percentage of testruns with 100% system availability

    First - the date of the first testrun

    Last - the date of the last testrun

    Fastest - the shortest run time (seconds)

    Slowest - the longest run time (seconds)

    Average - the average run time (seconds)

    Runs - the number of testruns of this test

    Click a row to display that test's runs in the "each testrun" view.

    Right-click anywhere in the panel to display a popup menu with these options:

    Tear-off - view the panel in another browser window

  • Screening Room User Guide Chapter 3 - The Screening Room option

    Ascom Network Testing Ltd., 2000-2013 Page 23 of 129 03 June 2013

    Refresh panel - fetch new data from the Agent and redisplay the panel using it

    Print panel - print the panel as a report. Displays the standard Print options dialog

    Open as Excel - output the data in Excel (spreadsheet) format. You can choose to Open the data

    (display it as an embedded spreadsheet, in the same panel) or Save the data (output to a Excel format file

    you specify)

    Open as XML - output the data in XML format, in another Browser window. You can then view, print,

    or save it using the standard Browser facilities

    The "each testrun" view

    The each testrun view lists individual testruns in the selected sort order and provides facilities to drill down to

    content, metrics, etc. for a selected testrun:

    To change the sort order, simply click the column button to sort by.

    To drill down, select a testrun by clicking it - this will display details about the run in the Testrun Summary and

    Contents panels, described below.

    The data shown is as follows:

    100 rows - the run ID

    Plan - the name of the Plan this Script was run from

    Test - the name of the Script

    Device - the device profile used

    Cmp - (completion) an icon indicating whether the run completed (green) or failed (red). A red icon

    suggests a problem with the test script

    Srv - (service) an icon indicating whether the service was available for the entire run (green) or not (red) or

    unknown (grey). A red icon suggests a problem with the service under test

    Sys - (system) an icon indicating whether the test system was available for the complete run (green) or not

    (red) or unknown (grey). A red icon suggests a problem with the test system

    Start - the date and time the run started

    End - the time the run ended

    Duration - how long the test ran for. A red background indicates a long run, while a green background

    indicates a short run

    State - whether the run succeeded, failed, was canceled or terminated due to a scripting error

    Status - indicates if the test is still running

    Content - how many content items were downloaded

    Owner - the user ID and IP address of whoever ran the test. Tests that were run manually (from Story

    Boarder) or by a Local Schedule will be listed as owned by SYSTEM (the IP address should help identify the

    owner)

    Location - if specified, a description of the location of the test run (Agent)

    GPS - if available (via a GPS system), the co-ordinates where the test was run. This information is mainly

    used by mobile test systems

  • Screening Room User Guide Chapter 3 - The Screening Room option

    Ascom Network Testing Ltd., 2000-2013 Page 24 of 129 03 June 2013

    Entrypoint - where the test started

    The panel will normally display up to 100 test runs (rows). To change the number:

    Click the 100 rows link

    Type the number of rows to display, then click OK.

    Popup menu

    Right-click any testrun in the Test Runs Panel to display this popup menu:

    Use the first three options (Mark all, Mark incompletes, and Unmark all) to mark testruns. You can also

    mark or unmark runs by clicking the checkbox at the start of each line

    Use Report marked to create a report combining the marked runs. This displays the Report Testrun dialog,

    for you to select the report type, from:

    High detail content listing

    Medium detail content listing

    Low detail content listing

    Compliance errors and broken links

    All device errors and usability

    Cosmetic device errors

    Functional device errors

    Other device errors

    Usability errors

    Loading

    Messages

    User events

    Metrics

    Response delay

    Delta timings

    Graphical timing

    Test and test run performance

    Transaction timings

    Script

    Site map

    Applied device test cases

    Note: To see an example of each of the above reports, see Appendix B: Testrun Reports.

  • Screening Room User Guide Chapter 3 - The Screening Room option

    Ascom Network Testing Ltd., 2000-2013 Page 25 of 129 03 June 2013

    Use Stop marked to stop the marked testruns.

    Use Delete marked to delete the marked testruns.

    The small buttons along the bottom of the menu are as follows:

    View detail - view testrun details in the Testrun Summary and Content panels

    View detail tear-off - view testrun details in another browser window

    Refresh panel - fetch new data from the Agent and redisplay the panel using it

    Print panel - print the panel as a report. Displays the standard Print options dialog

    Open as Excel - output the data in Excel (spreadsheet) format. You can choose to Open the data

    (display it as an embedded spreadsheet, in the same panel) or Save the data (output to a Excel format file

    you specify)

    Open as XML - output the data in XML format, in another Browser window. You can then view, print,

    or save it using the standard Browser facilities

    You can also display details of a testrun (in the Testrun Summary panel and Content panel) by double-clicking it.

    Testrun Summary panel (3)

    The bottom left panel in Screening Room's Functional View is Testrun Summary. This shows the following for the

    Test Run that you've selected in the Test Runs panel::

    A Testrun summary for this run:

    The Completion/service/system icons indicate whether the run completed and whether the service and

    system were available for the entire run. Red icons indicate possible problems with the script, service, and

    test system.

    A Meta tags table, listing the meta tags for the testrun (information about the testrun rather than the results

    it produced):

  • Screening Room User Guide Chapter 3 - The Screening Room option

    Ascom Network Testing Ltd., 2000-2013 Page 26 of 129 03 June 2013

    A Transactions Graph, showing the time taken by each Transaction (e.g. Scene) in the script. Green bars are

    successful Transactions, red bars are failed Transactions.

    A Transactions Table, showing data for the Transaction selected in the graph. Initially, the main

    Transaction (the entire testrun) is selected; to choose another, click its bar on the graph

    A Content timings graph, showing the time taken to transfer each content item.

  • Screening Room User Guide Chapter 3 - The Screening Room option

    Ascom Network Testing Ltd., 2000-2013 Page 27 of 129 03 June 2013

    Popup menu

    Right-click anywhere in the Testrun Summary panel to display a popup menu with these buttons:

    Report testrun - display a menu of all the other reports available for this testrun. Select the report you

    want and click Launch to run it.

    New Screening Room - open the current report on another Screening Room window

    Tear-off - view panel in another browser window

    Refresh panel - fetch new data from the Agent and redisplay the panel using it

    Print panel - print the panel as a report. Displays the standard Print options dialog

    Open as Excel - output the data in Excel (spreadsheet) format. You can choose to Open the data

    (display it as an embedded spreadsheet, in the same panel) or Save the data (output to a Excel format file

    you specify)

    Open as XML - output the data in XML format, in another Browser window. You can then view, print,

    or save it using the standard Browser facilities

    Content panel (4)

    The top right panel in Screening Room's Functional View is Content.

    The Content panel lists the content items downloaded for the selected Test Run, showing the following for each

    item:

    ID: Its ID

  • Screening Room User Guide Chapter 3 - The Screening Room option

    Ascom Network Testing Ltd., 2000-2013 Page 28 of 129 03 June 2013

    If the content has been analysed by using the Basic Content Tests Scene or MobileOK Scene, you'll see 5

    additional columns:

    OK: content analysis overview:

    red means the item has failed (includes errors or failures);

    green means it has passed (includes passes but no failures or errors);

    yellow means the contents passed but there were device/compliance errors, OR there were content

    analysis warnings but no errors or passes;

    white means it has not been analysed (does not include any passes or failures) - typically because

    there was no content to analyse

    CAP: the number of Content Analysis Passes for this item

    CAF: the number of Content Analysis Failures for this item

    DE: the number of Device Errors for this item

    CE: the number of Content Errors for this item

    Content: A description e.g. its URL

    Time: The time the download started

    Device: The name of the device profile being tested

    MIME: The content MIME type

    Charset: Its character set

    Delay: the time it took to download

    Indicative Size: its approximate size

    Indicative Rate: its approximate download rate

    Note: If any content pages includes a redirection command, Monitor Master turns it off (by replacing the command

    with a null command), so you stay on the intended page when you view it. You can turn redirection for the entire

    web site on or off by right-clicking any content from it and selecting the appropriate option from the popup menu.

    Popup menu

    Right-click a content item to display a popup menu with these buttons:

    Launch raw content - view content in the appropriate viewer, if there is one installed on this computer

    Start replay - view details of each content item in turn (as if you had clicked each one in turn)

    View detail - view details of this item in the Content Summary, Content Detail and Preview panels

    View detail tear-off - view details of this item in another browser window

    Refresh panel - fetch new data from the Agent and redisplay the panel using it

    Print panel - print the panel as a report. Displays the standard Print options dialog

    Open in Excel - output the data in Excel (spreadsheet) format. You can choose to Open the data

    (display it as an embedded spreadsheet, in the same panel) or Save the data (output to a Excel format file

    you specify)

    Open as XML - output the data in XML format, in another Browser window. You can then view, print,

    or save it using the standard Browser facilities

  • Screening Room User Guide Chapter 3 - The Screening Room option

    Ascom Network Testing Ltd., 2000-2013 Page 29 of 129 03 June 2013

    You can also view details of a content item (in the Content Summary, Content Detail and Preview panels) by

    clicking it.

    Content summary panel (5)

    The Screening Room's Functional View Content summary panel displays a summary of the content item you've

    selected in the Content panel, under these headings:

    Content Summary - time sent, character set, delay (time taken), size and rate

    Timings - times (clock times) and delays (time taken) for the associated transaction(s)

    Metrics - bytes sent/received, service and system availability, etc. for the associated transaction(s)

    Messages - messages from the associated transaction(s), if any were issued

    Compliance Errors - any compliance errors reported for this content

    For example:

  • Screening Room User Guide Chapter 3 - The Screening Room option

    Ascom Network Testing Ltd., 2000-2013 Page 30 of 129 03 June 2013

    Popup menu

    Right-click anywhere in the pane to display a popup menu with these buttons:

    Tear-off - view panel in another browser window

    Refresh panel - fetch new data from the Agent and redisplay the panel using it

    Print panel - print the panel as a report. Displays the standard Print options dialog

    Open in Excel - output the data in Excel (spreadsheet) format. You can choose to Open the data

    (display it as an embedded spreadsheet, in the same panel) or Save the data (output to a Excel format file

    you specify)

    Open as XML - output the data in XML format, in another Browser window. You can then view, print,

    or save it using the standard Browser facilities

  • Screening Room User Guide Chapter 3 - The Screening Room option

    Ascom Network Testing Ltd., 2000-2013 Page 31 of 129 03 June 2013

    Content detail panel (6)

    The Screening Room's Functional View Content detail panel displays the data in the content item you've selected

    in the Content panel, under these headings:

    Request headers - the headers used to request this content

    Response headers - the headers used to deliver this content

    Response binary - the content as hexadecimal bytes

    Response XML - the content as XML

    For example:

  • Screening Room User Guide Chapter 3 - The Screening Room option

    Ascom Network Testing Ltd., 2000-2013 Page 32 of 129 03 June 2013

    Popup menu

    Right-click anywhere in the panel to display a popup menu with these buttons:

    Tear-off - view panel in another browser window

    Refresh panel - fetch new data from the Agent and redisplay the panel using it

    Print panel - print the panel as a report. Displays the standard Print options dialog

    Open in Excel - output the data in Excel (spreadsheet) format. You can choose to Open the data

    (display it as an embedded spreadsheet, in the same panel) or Save the data (output to a Excel format file

    you specify)

    Open as XML - output the data in XML format, in another Browser window. You can then view, print,

    or save it using the standard Browser facilities

    Preview panel (7)

    The Screening Room's Functional View Preview panel renders the content item you've selected in the Content

    panel as it would be shown on the device. For example:

  • Screening Room User Guide Chapter 3 - The Screening Room option

    Ascom Network Testing Ltd., 2000-2013 Page 33 of 129 03 June 2013

    Popup menu

    This panel does not have its own popup menu. If you right-click it you will see the standard Browser popup menu.

    Screening Room Monitoring View Monitoring View is designed to help you monitor services.

    To launch Monitoring View:

    Select the Monitoring link from the Screening Room Home screen

    Or

    To switch from Regression View or Functional View: right-click near the top of the Screening Room window

    (on the Monitor Master banner) to display a popup menu and select the Monitoring view option

    In Monitoring View, the results are displayed in three separate panels:

  • Screening Room User Guide Chapter 3 - The Screening Room option

    Ascom Network Testing Ltd., 2000-2013 Page 34 of 129 03 June 2013

    The panels are:

    1. Plans and Tests

    2. Test Runs

    3. Testrun Summary

    Plans and Tests panel (1)

    The top left panel in Screening Room's Monitoring View is Plans and Tests, a hierarchical view of the Plans and

    Test Scripts on this Agent which you can use to list and manage the Plans, Tests and Schedules on this Agent. You

    can also use it to help select the testruns to monitor:

    To list the Test Scripts within a Plan:

    Click the Plan's button. To hide them again, click its button

    To list the scheduled Test Runs within a Plan:

    Click the button on its Schedules folder. (If there is no Schedules folder, the Plan has no schedules.)

    To list all Schedules on this Agent:

    Click the button on the Agent schedules folder. (This folder is listed after all the Plan folders).

    To select all Test Runs on this Agent:

    Click the This agent folder. Details will be displayed in the Test Runs panel.

    To select Test Runs for a specific Plan, Test, or Schedule:

    Click its name. Details will be displayed in the Test Runs panel.

  • Screening Room User Guide Chapter 3 - The Screening Room option

    Ascom Network Testing Ltd., 2000-2013 Page 35 of 129 03 June 2013

    For details of the Plans and Tests panel, refer to its description in Functional View.

    Test Runs panel (2)

    The top right panel in Screening Room's Monitoring View is Test Runs.

    Across the top of the panel are three dropdown lists and a button, which are used to select the information to

    display in the remainder of the panel:

    Use the first dropdown list to chose how to view your results:

    Tabular summary of - view a tabular summary of all results from the selected tests, and grand totals.

    Provides a detailed numerical summary of results in table form

    Extended summary of - view a graphical summary of all results from the selected tests, and grand totals.

    Provides an "at a glance" summary of results in pie chart form

    Each testrun from - view a list of individual testruns, showing detailed results for each. Allows you to

    quickly select a testrun of interest and drill down into its results, content, metrics, diagnostics, etc.

    The each testrun view is most often used for monitoring.

    Use the second dropdown list to select the results (testruns) to view. The options are:

    No plan - don't display anything

    All tests and plans on this agent - all runs of all Tests on this Agent

    Plan names - all runs of Tests in this Plan

    Test names - all runs of this Test

    (You can also select the results to view by selecting a Plan or Test from the Plans and Tests panel.)

    Use the third dropdown list to select the time period to report on. The options, which are self-explanatory, are:

    without time filter

    for the last 1 hour

    for the last 24 hours

    for the last 7 days

    for the last 31 days

    for the last 365 days

    Click the button to create a new report with your currently selected options.

    Key facilities

    Use the three dropdown lists across the top of the pane to select the view type (numerical summary,

    graphical summary, or each testrun), the tests to view results for, and the period to view results for - then

    click the button. The each testrun view is best for monitoring.

    Sort on any column by clicking the column heading

    Click any testrun (in "each testrun" view) to display details in the Testrun Summary panel - described next.

  • Screening Room User Guide Chapter 3 - The Screening Room option

    Ascom Network Testing Ltd., 2000-2013 Page 36 of 129 03 June 2013

    For details of the Test Runs panel, refer to its description in Functional View.

    Testrun Summary panel (3)

    The bottom left panel in Screening Room's Monitoring View is Testrun Summary, which shows the following for

    the selected Test Run:

    A Testrun summary for this run. The Completion/service/system icons indicate whether the run completed

    and whether the service and system were available for the entire run. Red icons indicate possible problems

    with the script, service, or test system.

    A Meta tags table, listing the meta tags for the testrun (information about the testrun rather than the results

    it produced).

    A Transactions graph showing the time taken by each transaction (e.g. Scene) in the script.

    A Transactions table showing metrics and messages for the selected transaction (e.g. Scene). (Click a

    transaction's bar in the Transactions graph to select it.)

    A Content timings graph, showing the time taken to transfer each content item.

    For details of the Testrun Summary panel, refer to its description in Functional View.

    Screening Room Regression View Regression View is designed to help users with regression testing, by making it easy to compare results from two

    separate test runs (e.g. before and after the change you're testing).

    To launch Regression View:

    Select the Regression link from the Screening Room Home screen

    Or

    To switch from Monitoring View or Functional View: right-click near the top of the Screening Room

    window (on the Monitor Master banner) to display a popup menu and select the Regression view option

    In Regression View, the results are displayed in six separate panels:

    The panels are:

    1. Plans and Tests

  • Screening Room User Guide Chapter 3 - The Screening Room option

    Ascom Network Testing Ltd., 2000-2013 Page 37 of 129 03 June 2013

    2. Test Runs

    3. Testrun Summary (lhs)

    4. Content (lhs)

    5. Testrun Summary (rhs)

    6. Content (rhs)

    Plans and Tests panel (1)

    The top left panel in Screening Room's Regression View is Plans and Tests, a hierarchical view of the Plans and

    Test Scripts on this Agent which you can use to list and manage the Plans, Tests and Schedules on this Agent. You

    can also use it to help select the testruns to compare:

    To list the Test Scripts within a Plan:

    Click the Plan's button. To hide them again, click its button

    To list the scheduled Test Runs within a Plan:

    Click the button on its Schedules folder. (If there is no Schedules folder, the Plan has no schedules.)

    To list all Schedules on this Agent:

    Click the button on the Agent schedules folder (this folder is listed after all the Plan folders).

    To select all Test Runs on this Agent:

    Click the This agent folder. Details will be displayed in the Test Runs panel.

    To select Test Runs for a specific Plan, Test, or Schedule:

    Click its name. Details will be displayed in the Test Runs panel.

    For details of the Plans and Tests panel, refer to its description in Functional View.

    Test Runs panel (2)

    The top right panel in Screening Room's Regression View is Test Runs.

    Across the top are three dropdown lists and a button, used to select the information to display in the remainder of

    the panel:

    Use the first dropdown list to choose how to view your results:

  • Screening Room User Guide Chapter 3 - The Screening Room option

    Ascom Network Testing Ltd., 2000-2013 Page 38 of 129 03 June 2013

    Tabular summary of - view a tabular summary of all results from the selected tests, and grand totals.

    Provides a detailed numerical summary of results in table form

    Extended summary of - view a graphical summary of all results from the selected tests, and grand totals.

    Provides an "at a glance" summary of results in pie chart form

    Each testrun from - view a list of individual testruns, showing detailed results for each. Allows you to

    quickly select a testrun of interest and drill down into its results, content, metrics, diagnostics, etc.

    The each testrun view is the one most often used for regression testing.

    Use the second dropdown list to select the results (testruns) to view. The options are:

    No plan - don't display anything

    All tests and plans on this agent - all runs of all Tests on this Agent

    Plan names - all runs of Tests in this Plan

    Test names - all runs of this Test

    (You can also select the results to view by selecting a Plan or Test from the Plans and Tests panel.)

    Use the third dropdown list to select the time period to report on. The options, which are self-explanatory, are:

    without time filter

    for the last 1 hour

    for the last 24 hours

    for the last 7 days

    for the last 31 days

    for the last 365 days

    Click the button to apply your currently-selected options. For example:

    Key facilities

    Use the three dropdown lists across the top of the pane to select the view type (numerical summary,

    graphical summary, or each testrun), the tests to view results for, and the period to view results for - then

    click the button. The each testrun view is best for regression testing.

    Sort on any column by clicking the column heading

    Click any testrun (in the each testrun view) OR right-click it and click the View detail (LHS) button in

    the popup menu to display the results and content from a Test Run in the Testrun Summary and Content

    panels on the left-hand side

    Right-click any testrun (in the each testrun view) and click the View detail (RHS) button in the popup

    menu to display the results and content for another Test Run in the Testrun Summary and Content panels

    on the right-hand side - producing a display like this

  • Screening Room User Guide Chapter 3 - The Screening Room option

    Ascom Network Testing Ltd., 2000-2013 Page 39 of 129 03 June 2013

    For details of the Test Runs panel, refer to its description in Functional View.

    Popup menu

    Right-click a testrun to display this popup menu for it:

    Use the first three options (Mark all, Mark incompletes, and Unmark all) to mark testruns. You can also

    mark or unmark a testrun by clicking the checkbox at the start of its row

    Use Report marked to create a report combining the marked runs. This displays the Report Testrun dialog,

    for you to select the report type.

    Use Stop marked to stop the marked testruns.

    Use Delete marked to delete the marked testruns.

    The small buttons along the bottom of the menu are as follows:

    View details (LHS) - view the testrun details in the left-hand Testrun Summary and Content panels

    View detail (RHS) - view the testrun details in the left-hand Testrun Summary and Content panels

    Refresh panel - fetch new data from the Agent and redisplay the panel using it

    Print panel - print the panel as a report. Displays the standard Print options dialog

    Open as Excel - output the data in Excel (spreadsheet) format. You can choose to Open the data

    (display it as an embedded spreadsheet, in the same panel) or Save the data (output to a Excel format file

    you specify)

  • Screening Room User Guide Chapter 3 - The Screening Room option

    Ascom Network Testing Ltd., 2000-2013 Page 40 of 129 03 June 2013

    Open as XML - output the data in XML format, in another Browser window. You can then view, print,

    or save it using the standard Browser facilities

    You can also view details of a testrun (in the left-hand Testrun Summary and Content panel) by double-clicking it.

    Testrun Summary panels (3, 5)

    Screening Room's Regression View Testrun Summary panels show the following for the selected Test Runs:

    A Testrun summary for this run. The Completion/service/system icons indicate whether the run completed

    and whether the service and system were available for the entire run. Red icons indicate possible problems

    with the script, service, or test system.

    A Meta tags table, listing the meta tags for the testrun (information about the testrun rather than the results

    it produced)

    A Transactions graph showing the time taken by each transaction (e.g. Scene) in the script.

    A Transactions table showing metrics and messages for the selected transaction (e.g. Scene). (Click a

    transaction's bar in the Transactions graph to select it.)

    A Content timings graph, showing the time taken to transfer each content item.

    For details of the Testrun Summary panel, refer to its description in Functional View.

    Content panels (4, 6)

    Screening Room's Regression View Content panels list the content items downloaded for the selected Test Runs,

    showing the following for each item:

    ID: Its ID

    Content: A description e.g. its URL

    Time: The time the download started

    MIME: The content MIME type

    Charset: Its character set

    Delay: the time it took to download

    Indicative Size: its approximate size

    Indicative Rate: its approximate download rate

    For details of the Content panel, refer to its description in Functional View.

  • Ascom Network Testing Ltd., 2000-2013 Page 41 of 129 03 June 2013

    Chapter 4 - The Plan Dashboard option

    The Screening Room Plan dashboard option creates a visual dashboard consisting of graphs that summarise

    results from multiple testruns from a Plan, making it easy for you get a broad view.

    Creating the Plan Dashboard

    Interpreting the Plan Dashboard

    Customising the Plan Dashboard

    Manipulating the Plan Dashboard

    Comparing graphs

    Cumulative graphs

    Watching data

    Creating the Plan Dashboard To create the Plan Dashboard:

    Click the Plan dashboard option on the Screening Room Home screen:

    You will now be taken through a sequence of steps to select the Plan and information you're interested in.

    Hint: If you want to go back to change your mind, use your Browser's Back button or click the underlined

    link for the step to return to - for example, click the Select run link to return to Step 2 from a later step.

    Step 1: Choose the Plan containing the results to review from the list (e.g. Standard Scripts). Click Next>

    Step 2: Choose External Script. Click Next>

  • Screening Room User Guide Chapter 4 - The Plan Dashboard option

    Ascom Network Testing Ltd., 2000-2013 Page 42 of 129 03 June 2013

    Step 3: Choose this agent. Click Next>

    Step 4: Choose up to five Tests to plot on the dashboard. Click Next>

  • Screening Room User Guide Chapter 4 - The Plan Dashboard option

    Ascom Network Testing Ltd., 2000-2013 Page 43 of 129 03 June 2013

    Step 5: Choose the time period to graph, by choosing the start and end dates and times from the lists. The

    dates offered correspond to test run dates for the selected tests.

    Step 6: Choose up to 5 metrics to graph, from the lists. You can choose any of the metrics recorded for these

    test runs in this time period.

    Step 7: Customise the graph appearance. This step is optional, as the defaults are appropriate for most users

    and you can quickly adjust any that aren't, once you've seen the default graphs. (See Customising graph

    appearance, for details.)

    Click Next> to create and display a dashboard view of the selected results:

  • Screening Room User Guide Chapter 4 - The Plan Dashboard option

    Ascom Network Testing Ltd., 2000-2013 Page 44 of 129 03 June 2013

    Interpreting the Plan Dashboard Once you have created the Plan Dashboard (previous section), you will see it consists of:

    A separate tab for each Agent (the Agents selected in step 3)

    A column of graphs for each test script (the graphs selected in step 4)

    A row of graphs for each metric (the metrics selected in step 6)

    Labels and other data in the row and column borders

  • Screening Room User Guide Chapter 4 - The Plan Dashboard option

    Ascom Network Testing Ltd., 2000-2013 Page 45 of 129 03 June 2013

    The horizontal axis of each graph shows the entire time period selected in step 5, so all graphs have the same X

    axis. The vertical axis of each graph shows the values of the chosen metric.

    But you can do a lot more, including:

    Customising the appearance

    Manipulating the graphs

    Comparing graphs

    Cumulative graphs

    Watching data

    Customising the Plan Dashboard To change the look of your Plan Dashboard:

    Select different options from the Step 7 controls:

  • Screening Room User Guide Chapter 4 - The Plan Dashboard option

    Ascom Network Testing Ltd., 2000-2013 Page 46 of 129 03 June 2013

    Click Next> to redraw the graphs.

    The default options are:

    Success: choose Not smoothed to plot each success result for each run as a separate point, or Averaged per

    minute/hour/day/year to plot averages over time. Averaging makes it easier to spot trends, but might hide

    fluctuations or patterns that are important to you

    Metrics: choose Not smoothed to plot each metric for each run as a separate point, or Averaged per

    minute/hour/day/year to plot averages over time. Averaging makes it easier to spot trends, but might hide

    fluctuations or patterns that are important to you

    Metric range: if using smoothed results, choose one of these options to add or remove curves showing the

    upper and/or lower values:

    Maximum & minimum limits

    Maximum limit

    Minimum limit

    Neither limit

    not smoothed smoothed, neither limit smoothed, both limits

    Size: Choose the graph width (Narrow/Normal/Wide) from the first control, and graph height

    (Short/Normal/Tall) from the second

    Style: Choose the line style (Thin line or Thick line) from the first control, and the point style (No points,

    Small points or Large points,) from the second

    Override Y-max: The Y-axis of each graph is chosen automatically so that the highest value for each metric

    plotted just fits. If you wish, you can override this process by typing a maximum Y axis value in this box.

    You might use this, for example, to normalize all graphs so they are directly comparable, or to remove

    spurious data from a graph.

    For example, if plotting timings that are all in the range 15-20 seconds, with just one rogue value of 200

    seconds, the default Y axis will be 0-200 seconds, compressing the details in the rest of the graph. Type 20 in

    the Override Y-max box to ignore the rogue value and reveal the variations in the other values, if they are of

    more interest:

  • Screening Room User Guide Chapter 4 - The Plan Dashboard option

    Ascom Network Testing Ltd., 2000-2013 Page 47 of 129 03 June 2013

    default graph

    showing rogue data

    Override Y-Max = 20

    hiding rogue data

    Any data points that are not plotted because they don't fit are indicated by an upwards arrow (as shown in

    the right-hand example, above).

    Manipulating the Plan Dashboard You can change the Plan Dashboard display options (set in steps 5-7) at any time to change the look:

    Simply make the change(s) to the options

    Click Next> to redraw the dashboard with the new display options.

    You can also:

    Double-click any graph to display the testrun results in another Browser window.

    Select an cell, cells, graph or graphs by clicking them. Select an entire row by clicking the Select link in its

    left-hand border. You can then analyse the selected metric graphs by using the three buttons in the top left

    of the dashboard. to:

    Compare the results

    Cumulate the results

    Watch the results

    Comparing Plan Dashboard graphs Once you've created a Plan Dashboard (details), you can overlay 2 or more its graphs to help you compare their

    results. This only makes sense if the graphs use the same or similar metrics.

    To overlay graphs:

    Select the graphs to compare (overlay) by clicking them.

    Click the Compare button.

    The graphs will be combined and displayed as a single graph in another browser window, like this:

  • Screening Room User Guide Chapter 4 - The Plan Dashboard option

    Ascom Network Testing Ltd., 2000-2013 Page 48 of 129 03 June 2013

    This new window has its own set of configuration options:

    Use the Time period options to change the period to graph, as described above.

    Use the Thresholding options to display a visual warning if the data falls outside the thresholds you

    specify, as described below.

    Use the Customise graph appearance options to change the graph appearance, as described above.

    After making any changes, click Next to apply the revised options to the graph, and re-plot it.

    Using Thresholds

    You can define Thresholds to help you quickly recognise when results fall outside the expected range. They are

    normally used in combination with a regular refresh rate (see Refresh control), to monitor key metrics.

    To display a visual warning if the most recent data point is above a threshold value:

    Select the graph or graphs to check, from the Plan Dashboard screen.

    Click the Compare button.

    When the comparison graph is displayed, change the thresholding type from None to Upward and two

    additional Level controls are displayed:

    Type the upper (error) threshold level in the first box, and the lower (warning) threshold level in the second

    box.

    Click Next to apply the revised options to the graph.

    Now, if the most recent data point (on all the curves) is below both thresholds, the graph will have a green

    background:

  • Screening Room User Guide Chapter 4 - The Plan Dashboard option

    Ascom Network Testing Ltd., 2000-2013 Page 49 of 129 03 June 2013

    If the most recent data point (on one or more curves) is above the lower threshold but below the upper threshold,

    the graph will have a yellow (warning) background:

    And if the most recent data point (on one or more curves) is above the upper threshold, the graph will have a red

    (error) background:

  • Screening Room User Guide Chapter 4 - The Plan Dashboard option

    Ascom Network Testing Ltd., 2000-2013 Page 50 of 129 03 June 2013

    To display a visual warning if the most recent data point is below a threshold value:

    Select the graph or graphs to check, from the dashboard screen.

    Click the Compare button.

    When the comparison graph is displayed, select a thresholding type of Downward and two additional

    Level controls are displayed:

    Type the upper threshold level (warning) in the first box, and the lower threshold level (error) in the second

    box.

    Click Next to apply the revised options to the graph.

    Now, if the most recent data point is above both thresholds, the graph will have a green background; if the most

    recent data point is below the upper threshold, the graph will have a yellow (warning) background; and if the most

    recent data point is below the lower threshold, the graph will have a red (error) background.

    Cumulative Plan Dashboard graphs You can use Cumulative Plan Dashboard graphs to show the progress of an accumulating metric over time, with

    the metric value on the X-axis and the percentage of samples meeting that metric on the Y-axis. Cumulative graphs

    provide a good way to monitor targets such as "80% of SMS messages must be delivered with 10 seconds" and to

    spot patterns and clustering of results which may warrant investigation.

    To produce a cumulative Plan Dashboard graph:

    Select the metric graph(s) to cumulate by clicking them on the Plan Dashboard. (It only makes sense to

    cumulate graphs for the same or similar metrics.)

    Click the Cumulate button.

    The selected graphs will be combined and displayed together in another browser window, like this:

  • Screening Room User Guide Chapter 4 - The Plan Dashboard option

    Ascom Network Testing Ltd., 2000-2013 Page 51 of 129 03 June 2013

    This new window has its own options for configuring the graph:

    Use Time period to choose the period to graph, as described above.

    Use Thresholding to display a visual warning if the data falls outside the thresholds you specify. See Using

    Thresholding, above.

    Use Cumulative range to change the range plotted along the X-axis.

    Use Customise graph appearance to change the graph appearance, as described above.

    Click Next to apply the revised options to the graph.

    Watching Plan Dashboard data You can use a Watch graph to display a continuously moving time window on the selected data on a Plan

    Dashboard, e.g. to always display results for just the last hour.

    To produce a Watch graph:

    Select the graph(s) to watch by clicking them on the Plan Dashboard. (It only makes sense to watch graphs

    together if they are for the same or similar metrics.)

    Click the Watch button.

    The graphs will be combined and displayed as a single graph in another Browser window, like this:

  • Screening Room User Guide Chapter 4 - The Plan Dashboard option

    Ascom Network Testing Ltd., 2000-2013 Page 52 of 129 03 June 2013

    This new window has its own options for configuring the Watch graph:

    Use the Time period controls to change the period to graph. Notice that this is defined as a period relative

    to now. You can choose to graph the previous:

    1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 20 or 30 minutes

    1, 2, 3, 4, 5 or 6 hours

    Use the Thresholding contro