Zuzena demo orientation slides v03 2u
-
Upload
lee-hughes -
Category
Software
-
view
113 -
download
0
Transcript of Zuzena demo orientation slides v03 2u
10/14/2015
1
Automated TestingQUICK ORIENTATION & PRODUCT DEMONSTRATION
Zuzena automated test engine –
5) Customer Benefits
4) Scenarios & Repositories
3) Demo Core Capabilities
2) Anatomy of a Test Case
1) Customer Goals
• Data-aware
• Scenario-driven
• Repository-based
2
10/14/2015
2
Customer Goals for the QA Process
Simplicity
• A process that developers, management, & sponsors understand
Accessibility
• Developers can drive unit test cases
• System testers can add integration test cases
• SMEs can contribute acceptance test case
Labor Efficiency
• Reduce QA’s percent of budget while increasing test coverage & quality of results
Visibility
• Translate complex process into a few, intelligible graphs
3
Zuzena automated test engine –
5) Customer Benefits
4) Scenarios & Repositories
3) Demo Core Capabilities
2) Anatomy of a Test Case
1) Customer Goals
• Data-aware
• Scenario-driven
• Repository-based
4
10/14/2015
3
Anatomy of a Test Case
Container
•The Test Case
•How granular?
•Reusable components?
•Runtime parms?
Source Data
•From where?
•How much at once?
•Clean, dirty, incomplete, misorderd?
ETL Call
•EDW layer?
•Initial or incremental?
•Mid month, end of month or EOY?
Actual Results
•Collisions between testers?
•Text, DBMS?
Expected Results
•Subsets to match source data?
Assertions
•Formulas?
•Statistics?
•Comparisons?
•Further ETL?
Outcomes
•Tagged by versions, build, iterations, run dates?
•Drill down
5
Development Integration System Acceptance
Test Case Workflow
Ceregenics proprietary information. 6
Test DataActual
Results
EDW
Test
Scripts
Outcomes
Expected
Results
Assertions
10/14/2015
4
SSH &
ODBC
Engine-Driven Test Case
Ceregenics proprietary information. 7
Test DataActual
Results
EDW
Test
Scripts
Hyper Cube of
Test Scenarios
Data Set
Outcomes
Expected
Results
Assertions
Zuzena automated test engine –
5) Customer Benefits
4) Scenarios & Repositories
3) Demo Core Capabilities
2) Anatomy of a Test Case
1) Customer Goals
• Data-aware
• Scenario-driven
• Repository-based
8
Build a test case
10/14/2015
5
On-Demand Demo on YouTube
Web addresses for the demo:
• http://youtu.be/YX8lKSACELA
• https://vimeo.com/141469720
Locations of major narrative points:
• Create the test case container 0:19
• Add a input data source 2:17
• Add an ETL action step 5:56
• Add an validation assertion 9:10
• Reusable test widgets 9:50
• Run an assertion that fails 12:10
• Fix assertion 13:43
• Re-run to see it pass 14:25
9
Zuzena’s Run Modes
Via the GUI
• Developing test cases
Via the Command Line
• preparing an automated run
• https://youtu.be/Bn27KkHyCCY
Via the scheduler
• nightly regression test of customized test suite
10
Major narrative points
0:20 a command to run a dynamically defined test suite
0:45 the same command with parms annotated
1:05 viewing the results
10/14/2015
6
Zuzena automated test engine –
5) Customer Benefits
4) Scenarios & Repositories
3) Demo Core Capabilities
2) Anatomy of a Test Case
1) Customer Goals
• Data-aware
• Scenario-driven
• Repository-based
11
User-Defined Metadata Tags
Structure: <tag type>, <tag value>
12
Tag Type Sample Values
Error Class Nominal Data, Dirty Data, Missing Data
Environment Unit, Integration, System, Acceptance
Time Point Day 0, Day 1, End-of-Month, EOY
Project Master Customer, Sales, Finance
App Layer Landing, Integration, Presentation, Dashboards
Column Type Replicated, Derived, Source Metadata, EDW Metadata
10/14/2015
7
RepeatableTesting: Scenario Cycle
Ceregenics proprietary information. 13
Test DataActual
Results
EDW
Test
Scripts
Hyper Cube of
Test Scenarios
Error Class
Outcomes
Expected
Results
Assertions
Zuzena automated test engine –
5) Customer Benefits
4) Scenarios & Repositories
3) Demo Core Capabilities
2) Anatomy of a Test Case
1) Customer Goals
• Data-aware
• Scenario-driven
• Repository-based
14
10/14/2015
8
Benefit: Single Repository = Structure + Organization
15
From this...
To this...
Repository Benefits
Remember where you left it
• Far less time hunting and gathering
Know what it should look like
• Understand it when you find it
Fast find + fast recognition
• faster overall testing
Lower risk
• Less project management stress
16
10/14/2015
9
Benefit #2
With a repository,
you can count what
you should have
17
Benefit #1: Visible Progress to Goal
18
10/14/2015
10
Benefit #3
Repository + User-
Defined Tags
Categorize,
visualize,
and understand
19
Benefit #1 – Drill Down on Problem Areas
20
Project: Master Customer
(Part 1)
Huh?
10/14/2015
11
Benefit #1 – Drill Down on Problem Areas
21
Project: Master Customer, Landing Zone
(Part 2)
OK
Benefit #1 – Drill Down on Problem Areas
22
Project: Master Customer, Integration Layer
(Part 3)
Hmmm....
10/14/2015
12
Benefit #1 – Drill Down on Problem Areas
23
Project: Master Customer, Integration Layer, Derived Columns
(Part 4)
Hmmm....
Benefit #1 – Drill Down on Problem Areas
24
Project: Master Customer, Integration Layer, Derived Columns, Global Accounts Subject Area
(Part 5)
There’s the
problem
10/14/2015
13
Zuzena automated test engine –
• Data-aware = Elements for source and expected data
• Repository-based = Structured & organized
• Scenario-driven = User tags allow subsetting
25
Customer Goals for the QA Process
Simplicity
• A process that developers, management, & sponsors understand
Accessibility
• Developers can drive unit test cases
• System testers can add integration test cases
• SMEs can contribute acceptance test case
Labor Efficiency
• Reduce QA’s percent of budget while increasing test coverage & quality of results
Visibility
• Translate complex process into a few, intelligible graphs
26
Structured test definition with reuable components
GUI makes defining test cases intelligible, trainable, & repeatable
More tests defined in less time, far more runs within each project
Repository, subset reporting yields trend lines for team, PMs, and sponsors
10/14/2015
14
Acquiring Zuzena
First repository: usually a DEV host for programmers
Second repository: usually an SIT host for system testers
Discounts for UAT and PROD hosts
Procurement Patterns
• Host licenses alone
• Hosts + installation & training
• Comprehensive QA method & plan + hosts + installation & training
27
Comprehensive QA Planning
Identify the best of 40 test types to use
Negotiate a unit test pattern with programmers
Define a funnel & promotion process to SIT, UAT, & PROD
Establish reusable test widgets
Document who, when, where
Enumerate tests by requirement & component project targets
Derive user metadata tags, create compliant repository
Prioritize and implement repository reporting for dev, PM, sponsors
28
10/14/2015
15
999 18th Street, Suite 3000
Denver CO 80033303.274.9101
WWW.ZUZENA.COM
29