ALM 11 Reporting - c.ymcdn.comc.ymcdn.com/.../ALM11Reporting.pdf · ALM 11 Reporting Features ......

Company Confidential

Reporting in ALM 11.0 & Assure TotalView Presenter: Shir Goldberg

Assure


Assure - Release Quality on Time

We know HP ALM & Quality Center

Since 2006, offices in USA, Finland and Israel

HP AllianceOne Technology Partner of the year 2011

We provide innovative products, solutions and services

for HP ALM & Quality Center customers

With Assure’s offering, customers are able to streamline

processes, boost efficiency and decrease overall QA and

administration costs

Assure works with some of the world’s largest and most

demanding HP ALM & Quality Center customers


New Reporting Possibilities in ALM 11.0

Enhanced monitoring of release performance vs. plan

Track planned release scope and objectives and their

expected progress throughout release duration

Graphs are available for publishing on web pages as

HTML (SharePoint, Wiki, etc)


New Reporting Features in ALM 11.0

Project Reports

Single configuration for mutiple output formats (doc, docx, html)

Rich style and layout options using MS Word templates

Centralized template management

Release Analysis

Milestones are placed on the release timeline to define a

lifecycle of a product

Key Performance Indicator (KPI)

• Measure your release objectives and analyze bottlenecks

• Set on Milestones to track performance


New Reporting Features in ALM 11.0

In Release Scorecard you can easily follow the

performance of a release based on KPI`s

KPI Analysis Graphs gives possibility to drill down to find

root cause of possible problems on release


New ALM11 Features: Project Reports

Only in ALM 11

Rich style and layout options using MS Word templates

Reports can be created with custom layout and for example

company logo

Single configuration for mutiple output formats (doc,

docx, html)

Centralized template management

Enhanced performance

Project reports can be created based on Baseline


New ALM11 Features: Release Scope

Deliverables for your release can be defined by creating

Scope Items

Scope Items can be for example

New feature

Change for existing feature

New theme

Scope Items will be linked to Requirements, Tests, Test

Sets and Defects


Define milestones

Define KPI’s for milestone

Define releases

Project Planning & Tracking

Define & Preview Thresholds

Define Scope


New ALM11 Features: Milestones

Milestone is a point in release timeline that signifies the

completion of a deliverable

It enables users to track and validate the progress of the

release

Milestones can be for example

Integration between subsystems

Code freeze

Release to production

Scope Items will be associated to Milestones to follow

release deliverables


New ALM11 Features: Key Point Indicator (KPI)

KPI`s will give release stakeholders a clear picture of

Where are we now?

Are we reaching the goals set for the Release?

Predefined KPI`s cover for example

Passed Test %

Test Instances Executed %

Severe Defects

Measured entities can be filtered by UDF`s and TDF`s


New ALM11 Features: KPI Thresholds

Thresholds will give clear indication of what is expected

on release performance for current milestone

Thresholds curve is set per KPI to follow the releases progress

KPI thresholds reflect set objectives. • A threshold specifies an acceptable value for a KPI measurument

on specific time

• For example Requirement Coverage KPI is set close to 100% and

Severe Defects KPI is set close to 0%


New ALM11 Features: Release Scorecard

Release Scorecard view provides the release status at

one glance

How KPI Tresholds are met will be indicated on different

colours: green (OK), yellow (warning) or red (critical)


New ALM11 Features: Release Analysis Graphs

Individual Scorecard cells can be opened and it will show

drill down of a chosen KPI over time

KPI Type Analysis will give a clear picture of release

performance


Best Practices of using PP&T

Use Release Templates in the Template project to

standardize Releases across projects!

Decide on terminology, attributes, KPI’s that will be used

throughout the organization

Limit the number of Scope Items in order to keep

tracking the release manageable for all stakeholders

Maximum of 30 Scope Items is recommended

Maximum amount can be set with a Site Admin parameter

When rescheduling Milestones, remember to re-adjust

KPI thresholds as they are not automatically adjusted!

Resources:

PP&T Best Practices book from HP


Known Limitations of new ALM 11 Reporting Features

KPIs can not be copied between projects (can be shared

from Templates)

KPIs are taken once/twice per day

Historical PPT data will be purged and purged data

cannot be restored

PPT data calculations effects performance

KPIs are calculated once they are defined – no past data


Administrative Aspects of new ALM 11 Features

Scheduling of Project Planning and Tracking calculations

should be done in site`s off-peak hours

Calculation duration can be limited but Release

calculations that have yet to start are considered failed

Calculations can be performed every 24 or 12 hours.

Business demands, database strength and number

milestones and KPI`s should be considered

”Run Now” –button allows PPT calculations to be

triggered manually


ALM/QC Reporting in General

ALM/ Quality Center offers four tools for reporting purposes

Dashboard module

• For creating textual reports or graphs based on field values

• With ’Graph Wizard’ you can create an own graph

• Excel-reports (SQL queries to QC database)

Document Generator for generating project level Word documents

• You can get all project data into a single document

• Can contain pre-defined Graphs and Attachments

– Snapshot of the current situation

Analysis menu in each module

• Standard Reports and Graphs that can be customized and saved to Dashboard module for future use

Live Analysis in Test Plan and Test Lab

• Two configurable graphs side by side showing statistics of the selected folder


Features of the Dashboard module

Fully integrated Dashboard module to centralize HP

Quality Center reporting tools including standard reports,

graphs, Excel reports and dashboard

Standardized cross-project or cross-module reporting

and drill-down graphs

Customizable pages

Including two tabs:

Analysis View for generating graphs / reports and viewing them

one at the time

Dashboard View for viewing several graphs at the same time

Possible to create private and public graphs / reports

Access rights to Dashboard are granted based on the

usergroups, everybody can view, but selected groups can create

graphs and reports


Known Limitations of Dashboard Module

Different results might be shown in Analysis items and

Dashboard pages in public folders depending on the

data hiding definitions for the user group

By default it is possible to perform any kind of SQL

statements with the Query Builder of the Excel reports

(also deleting whole table in database)

Restrict the allowed SQL commands in the Site Admin!


Benefits of the Dashboard Module

Simplified installation, configuration and maintenance

Dashboard Page lets you compare several reports side

by side in one view

Project Reports:

Rich style and layout option using MS Word templates

Enhanced performance

Single configuration for multiple output formats (doc, docx, html)

Centralized template management (only in ALM 11)


Live Analysis

Provides a dynamic quick visual overview of all the tests

in a folder or test sets in a test set folder

Live Analysis tab is visible in tree view when folder is

selected

Graphs are updated on a fly when different folder is

selected


Document Generator

A Microsoft Word based solution for reporting that

enables you to create a hard copy of the data contained

in a Quality Center project

Possibility to export data from following modules • Requirements

• Test Plan

• Test Lab

• Defects

Useful for off-line work

Created document is always a snap shot of the current

situation


Excel Reports

Data from ALM can be exported to Excel with Excel

Reporting

This allows users to analyze the data with Excel

capabilities

Set of data will defined by SQL queries on the project

database

Excel reports can be exported from one project at a time


Best Practises: Excel Reports

It’s recommended to allow creation of new Excel reports only to selected user groups

Complex or bad SQL queries can affect QC performance

Remove the creation rights from most people

Review new Excel reports before they are deployed to production environment


Best Practices: Project Planning

Define all the needed reports already in the early phase of the QC Deployment

Needed reports affects to the data model of the QC Project

Needed fields

Mandatory fields

Definition and enforcement of list values

• Verify value

• Use one lookup list for same field in different entities (e.g. Product , Feature etc)


Best Practices: Product Maturity Reports

Requirement point of view:

How much a certain Requirement is tested?

How many Defects has been found?

Number of critical Defects found?

Requirement types point of view?

Functionality maturity

Performance maturity

Usability maturity

Release point of view:

How many tests of the release has been executed?

• Pass / Fail ratio of those tests


Reporting related Site Admin Parameters

DASHBOARD_PAGE_ITEM_LIMIT Maximum amount of graphs on a dashboard page

Default value is 4

SQL_QUERY_VALIDATION_ENABLED If set as ”N” QC doesn’t check that SQL query of an Excel report is valid

SQL_QUERY_VALIDATION_BLACK_LIST To make sure that certain SQL commands (DELETE, DROP, UPDATE etc) are not allowed

Ensures that you don’t harm QC database

REPORT_QUERY_RECORDS_LIMIT Maximum number of records that can be retreived from database for an Excel report

If value is set to ”-1” the number is unlimited


Summary

ALM 11 is a flexible tool for all projects reporting

Different reporting cababilities give detailed information

about project performance for various stakeholders

Multi-Project analytics and Reporting for HP Quality Center and ALM Shir Goldberg


INNOVATIVE PRODUCTS FOR HP ALM & QUALITY CENTER

ASSURE PROJECT HARMONIZER

ASSURE TOTALVIEW

ASSURE ADMIN CONSOLE

ASSURE TRASHBIN

ASSURE BUGZILLA ADAPTER

ASSURE TEST DATA INTEGRITY


Next generation reporting for HP Quality Center & HP ALM

ASSURE TOTALVIEW


Your Vision

A complete picture of your QC/ALM data

Unlimited access to QC/ALM data

Status of requirement coverage and product maturity

Reporting without performance impact on the production

environment

Visualize trends to improve efficiency and output

Multi-Project analysis and reporting

A reliable and accurate reporting solution

A product that is user friendly and quick to deploy

A solution that supports business processes


Our vision

Provide a Business Intelligence solution...

Where data consumers can access HP Quality Center

data without boundaries or limitations

That provides trend and history analysis

That supplies compehensive multi-project analytics

Which is a user friendly data source for reporting and

data analysis interfaces

That empowers business decision makers

A Data Mart is the intelligent data solution for QC reporting!


Why BI?

“You can't beat a BI solution for reporting. The source

system will never be able to compete. It is a proven fact

by leading companies all over the world.”

“If a company wants to beat competition it must be one

step ahead-BI is the key”

“All leading software providers supply a separate

platform for BI that is not part of any legacy system”

“The source system will never be able to

incorporate BI best practices as it is always limited in its

resources and capabilities”


Assure TotalView


What is TotalView?

TotalView is a BI solution for HP Quality Center & HP

ALM

Revolutionary approach to QC/ALM reporting and data analysis

Supports decision makers and business processes

Leverages BI Best Practices

Product Components

Data Mart

Management Console

ETL (Extract-Transfer-Load) processes

Ready to use dashboard (using QlikView)

Report library


Key Features

Gives access to all HP Quality Center & HP ALM data

Multi-Project analytics

Reliable historic data and trend analysis

QC/ALM version upgrades are supported

A complete picture of QC/ALM data

A ”ready to go” dashboard

Management Console

Define project groups for reporting

Select the User Defined (custom) Fields required for reporting

Manage the Data Mart


Benefits of a Data Mart

A separate database dedicated to data analysis

Minimal impact on the production database

Intuitive data structure and easily accessible

Accurate and trustworthy data

Recording of data changes enables trend analysis

Based on BI methodology and best practices


The make up of a Data Mart


A ”Star Schema”

A star schema consists of a Fact table and multiple Dimension

tables and represents a single business subject for analysis.

For example – “Execution Progress”


A data model

A logical representation of the Data Mart and all its star

schemas.

Execution Progress

Test Readiness

Defect Detection

Priority Bug Status

Severity

Plan Status

REQ

TEST

Release Cycle

Release

Test Set

Person

Time Exec

Status

Test Instance


TotalView Management Console


Why TotalView?

Without TotalView

Complex table structure

Limited Depth

No QC/ALM version support

Complex maintenance

Short-term solution

Analysis on Prod. DB

Complex data management

Complex report build

Limited “Cross-Project”

With TotalView

Intuitive table structure

Trend and Progress

QC/ALM version support

Minimal maintenance

Long-term solution

Dedicated analysis DB

Management console

Simple report build

Unlimited “Multi Project Analytics”


Other Considerations

Multi-Project Analytics require a uniform data structure

QC10 Premier / ALM11

Assure Project Harmonizer

Strict Policies for project creation and changes

Multi-Project Analytics require a uniform data values

QC10 Premier / ALM11

Assure Admin Console

Assure Data Scrub

Strict Policies for project creation and changes


Contact Assure

Web: www.assure.net

Email: [email protected]

Twitter: http://twitter.com/assureqc

Me: [email protected]

http://www.assure.net/

mailto:[email protected]

http://twitter.com/assureqc

mailto:[email protected]

ALM 11 Reporting - c.ymcdn.comc.ymcdn.com/.../ALM11Reporting.pdf · ALM 11 Reporting Features ......

Documents

Transcript of ALM 11 Reporting - c.ymcdn.comc.ymcdn.com/.../ALM11Reporting.pdf · ALM 11 Reporting Features ......