OOW16 - Planning Your Upgrade to Oracle E-Business Suite 12.2 [CON1423]
OOW16 - Testing Oracle E-Business Suite Best Practices [CON6713]
-
Upload
vasuballa -
Category
Technology
-
view
78 -
download
12
Transcript of OOW16 - Testing Oracle E-Business Suite Best Practices [CON6713]
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Testing Oracle E-Business Suite Best Practices Keeping Up with Change…
Confidential – Oracle Internal/Restricted/Highly Restricted
Presented with
Mazen Arar Senior Director, Oracle Sarvesh Tiwari Vice President, Genpact September 21, 2016
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
Confidential – Oracle Internal/Restricted/Highly Restricted 3
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
EBS Evolution
Testing Readiness
Recommendations & Tools
Customer Story
1
2
3
4
4 Confidential – Oracle Internal/Restricted/Highly Restricted
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
EBS Evolution Change is Constant
• Change happens every day
• In today’s business world, it is the only thing that is consistent
• Long-term plans may become outdated and possibly obsolete within months
• Business plans must be reviewed, updated, and revised constantly
5 Confidential – Oracle Internal/Restricted/Highly Restricted
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
EBS Evolution Oracle E-Business Suite in the Digital World
Cloud Social Mobile
IoT Big Data
Talent Mgmt
Revenue Mgmt
Recruiting
Innovation Mgt Commerce
Financial Reporting
Global Trade Mgmt
Enterprise Planning
Service
Transportation Mgmt
Sales
Marketing
6 Confidential – Oracle Internal/Restricted/Highly Restricted
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
EBS Evolution
Testing Readiness
Recommendations & Tools
Customer Story
1
2
3
4
7 Confidential – Oracle Internal/Restricted/Highly Restricted
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Testing Readiness Challenging Facts
• Continuous development calls for continuous testing
• With ongoing EBS innovation, all teams need to be aligned
• Different types of activities require different strategy for testing
• QA teams have to be pro-active not re-active
• Focusing on the present and planning for the future
8 Confidential – Oracle Internal/Restricted/Highly Restricted
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Testing Readiness Ongoing EBS Innovation
• New features / Products
– EBS 12.1
– EBS 12.2
– Modern / Simplified UI
• Mobile
• In Memory Applications
• Big Data - Information Discovery
• Cloud
• IoT
9 Confidential – Oracle Internal/Restricted/Highly Restricted
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Testing Readiness Strategy
10
Smartphone Applications
In Memory Applications
Information Discovery
Cloud
IoT
Confidential – Oracle Internal/Restricted/Highly Restricted
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
EBS Evolution
Testing Readiness
Recommendations & Tools
Customer Story
1
2
3
4
11 Confidential – Oracle Internal/Restricted/Highly Restricted
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Recommendations & Tools Where Do You Start?
12
Upgrades
Confidential – Oracle Internal/Restricted/Highly Restricted
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Recommendations & Tools
Automate
Execute
Plan
13 Confidential – Oracle Internal/Restricted/Highly Restricted
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Recommendations & Tools
• What is the task on hand
• Available resources
– Release Content Document • Information about new or changed functionality
• List of bugs included
– Functional Impact Assessment document • Functional changes
• Functional impacts
• Testing recommendations
14
Plan
Confidential – Oracle Internal/Restricted/Highly Restricted
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Recommendations & Tools
15
Evaluate
• Task on hand
Review
• Documentation
Scope
• Key business flows
• Impacted Flows
• Usage patterns
Plan
Confidential – Oracle Internal/Restricted/Highly Restricted
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Recommendations & Tools
Automate
Execute
Plan
16 Confidential – Oracle Internal/Restricted/Highly Restricted
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Recommendations & Tools Basics …
17
Execute
Entry / Exit Criteria
Follow A Testing Methodology Based on Task
Theme Based Exploratory Role based End to end flow
based
Mark Test Flows With A Priority
Basic Sanity check Detailed
Confidential – Oracle Internal/Restricted/Highly Restricted
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 18
Shorter Release Cycle
Automation coverage
Rapid Addition of Features
Faster validation
Security
Testing data security and data leaks
Integration
Test integrations for Cloud & On Premise
Multi Tenancy
Simulated customers and roles Testing
Recommendations & Tools Cloud Testing …
Execute
Accelerate testing with automation tools
Test in short sprints
Authentication, Authorization, Input Validation, PWD, etc ..
End to end flows between the cloud apps and integrating apps including on premise ERP
Ensure isolation of individual customer applications and data
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 19
Complex Architecture
Component specific & integration
testing
Complex Data
Controlled statistical data
with pre-defined patterns
Security
Data security, leaks, and
authentication
Validation
Insights & predictions VS. Pre-validated
data
Performance and Scalability
Performance and stress
testing
Recommendations & Tools IoT Testing …
Execute
Test different services separately and run end to end application flows
Feed data into the IOT apps to see if system can decode the time series data
Authentication, Authorization, Input Validation, PWD, etc ..
Feed same data to IoT apps & data mining tool to compare results
Test with volume data & multiple users
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Recommendations & Tools Good Practice
• Be creative / Innovative
– Review results
– Learn from results • Hot spots
• Safe spots
– Plan next cycle with a new flavor • Shift focus to hot spots
• Certify safe spots
– Improve scoping
20
Execute
Confidential – Oracle Internal/Restricted/Highly Restricted
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Recommendations & Tools
Automate
Execute
Plan
21 Confidential – Oracle Internal/Restricted/Highly Restricted
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Recommendations & Tools
• Manual Vs. Automation
• Reduce testing time & effort
• Repeatable!
22
Automate
Confidential – Oracle Internal/Restricted/Highly Restricted
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Recommendations & Tools
23
Anyone
Critical + Repeatable Flows
It is 2016
Automate
Confidential – Oracle Internal/Restricted/Highly Restricted
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
• Load and Performance Style Testing • Web and Oracle Application Testing Accelerators • Smart Integration with EM Diagnostics
• Test Planning, Requirements and Test Cases • Defects Tracking • Reporting
• Functional and Regression Testing • Web and Oracle Application Testing Accelerators • Oracle Flow Builder and Starter Pack
Oracle Application Testing Suite Overview
Oracle Functional Testing Suite for Oracle Applications
Oracle Load Testing Suite for Oracle Applications
Oracle Test Manager
25
Automate
Confidential – Oracle Internal/Restricted/Highly Restricted
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Oracle Functional Testing Suite for Oracle Applications Oracle Flow Builder and EBS Pre-built Test Content
• Use Flow Builder to Generate Test Scripts
• Start with EBS Pre-Built Test Content
• Create and Maintain Your Test Scripts
– Add Your Flows & Data
– Extend with More Components
– Support Customizations
– Non technical resources
Record Manually
Generate
Test Automation
Scripts
OpenScript
Record & Playback + Advanced Scripting
Flow Builder
Framework to combine flows and data to generate
test scripts
EBS Pre-Built Test Content
Your Data
Your
Flows
Ora
cle
Fu
nct
ion
al T
est
ing
Suit
e
for
Ora
cle
Ap
plic
atio
ns
EBS Quality Assurance Team Reduced Automation Time by 60%
26
Automate
Confidential – Oracle Internal/Restricted/Highly Restricted
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Oracle Flow Builder: EBS Pre-Built Test Content Starter Pack
• Commonly used base functionality
• For creating high level certification flows
• Broad coverage to jump start the testing
For Both 12.1.3 and 12.2
2000+ Components and 200+ Flows across EBS
Procurement Financials
Manufacturing Logistics Service Master Data Management
Order Management
27
Asset Lifecycle
Management
Projects HCM
3600+ components and 230+ flows across EBS
Logistics Manufacturing Financials
Projects Procurement
Product Information Management Service
Human Capital Management
Order Management
Asset Lifecycle
Management
Automate
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Oracle Flow Builder: EBS Pre-Built Test Content Advanced Pack
• Niche products which stand alone and deep functionality
• For creating detailed and complex test flows
• Significant number of components above Starter Pack (95% coverage)
For Both 12.1.3 and 12.2
28
10 product families / products across EBS
Incentive Compensation
Discrete Manufacturing
Channel Revenue
Management
Project Management Procurement
Process Manufacturing
Property Manager Service
Lease /Finance Management
Asset Lifecycle
Management
Automate
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Oracle Flow Builder: EBS Web Services & APIs Support
• What are Web Services & APIs?
– Stored procedures that enable you to insert and update data into Oracle Applications
• Automate Web Services & APIs using OFB
– No need for an in-depth knowledge of PL/SQL
• Advantage – Automated mass upload into the application
– Scripts created are more stable and robust than UI
29
Automate
Confidential – Oracle Internal/Restricted/Highly Restricted
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Automate
Oracle Flow Builder: EBS Pre-Built Test Content Advanced Pack includes content for API/Web Services
• Drastic reduction in test execution time
• Hybrid flows, where you can execute a flow with API and UI scenarios
• Verification of data in UI, Database, or output data after interface execution
Enhanced Content for 12.1.3
30
875 components and 48 flows across EBS
Logistics Manufacturing Financials
Projects Procurement
Product Information Management Service
Trade Management
Order Management
Asset Lifecycle
Management
API Content for EBS 12.1.3
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Oracle Flow Builder: EBS Web Services & APIs Support
• UI VS. API
– Value added through execution • In Oracle Project Management:
– Creating a Project takes 1 - 1.5 hrs using UI mode VS. 26 seconds using OFB API
– Create Task Assignments takes 15 min using UI mode VS. 8 seconds using OFB API
31
Automate
Confidential – Oracle Internal/Restricted/Highly Restricted
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Demo Oracle Flow Builder - APIs
Oracle Confidential – Internal/Restricted/Highly Restricted 32
33 Qualcomm Proprietary and Confidential
Testimonial from Qualcomm
• Created ~115 custom components in Oracle Flow Builder which saved huge effort during design phase.
• Created ~160 automated scripts which cover ~1470 business process scenarios in past ~2years
• Metrics against manual testing that is qualified for automation: ◦ Time Savings with Automation ~50%
◦ Automation Script Coverage ~50%
• Automated scripts used in below activities which reduced a significant manual effort during major projects: ◦ Regression cycle
◦ Quick Build test
◦ Month-end activity close
◦ Pre-production checkout
◦ User Acceptance test
◦ Go-live checkout
◦ Emergency test in PRD Support
34 Qualcomm Proprietary and Confidential
Testimonial from Qualcomm
• Custom Solutions: ◦ We use QC for test management. Created a plug-in in order to interface OATS scripts directly to update results in QC.
◦ For accelerated test execution created customized framework in order to execute same OATS script in parallel for
different business scenarios
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
EBS Evolution
Testing Readiness
Recommendations & Tools
Customer Story
1
2
3
4
35 Confidential – Oracle Internal/Restricted/Highly Restricted
Testing Oracle E-Business Suite
Best Practices & Lessons Learned
Sarvesh Tiwari
Lead Solutions Architect
[email protected] @ideating
37 © 2015 Copyright Genpact. All Rights Reserved.
68,000 +
employees 800+ Clients
(One Third of Fortune
500)
30 languages
17 delivery
countries
70 delivery
centers
50 + Oracle Go Lives /
year
BPM Analytics Digital Technolog
y Reengineer Consulting
Outcome driven, Process-
led ERP
Analytic Solutions & Insights to support
executive decisions
Using Lean Six Sigma for process
improvements & Transformations Robotic and Digital technologies While running some of the most
complex business processes for
our clients
Deep domain consulting for business
blueprints
AT-A-GLANCE Founded by GE in 1997
NYSE (“G”)
$2.5 Billion (FY2015)
38 © 2015 Copyright Genpact. All Rights Reserved.
FUTURE IS HERE, NOW
Computing Everywhere … EBS on wearables?
Internet of Things … sensor to boardroom
Advanced Pervasive Invisible Analytics … In-memory
Context Rich Applications UI … tried Endeca?
Cloud-Client Architecture … mobile native
Infinitely Elastic Infra … cloud compute…
39 © 2015 Copyright Genpact. All Rights Reserved.
IT Practices
Business
Technology
17% CIOs say they are fast enough for the pace of their business
“IT Speed: The Crisis and the Savior of the Enterprise” Forrester Research
CRISIS
40 © 2015 Copyright Genpact. All Rights Reserved.
ORGANIZATIONS ARE LEARNING FROM SILICON VALLEY
LEAN START UP | AGILE DEV + OPS
TEST DRIVEN DEVELOPMENT | CONTINUOUS TESTING
41 © 2015 Copyright Genpact. All Rights Reserved.
HOWEVER, MOST FACE SPEED BUMPS IN TESTING
45% of effort is spent in creation of prerequisites and
documenting, rather than testing the outcomes
53% ERP Programs execute EBS testing as “part
time” function of the business end users
62% report having no automation in EBS Functional
Testing
43% CRITICAL PATH OF A TYPICAL EBS PROJECT IS IN TESTING
42 © 2015 Copyright Genpact. All Rights Reserved.
LEARNING FROM A JOURNEY
US headquartered global leader in data center solutions, cloud
management, performance management, services management,
workload automation and virtualization lifecycle management products
& services
• Complex global EBS footprint (Fin, SCM, CRM)
• 58 SOBs, 40 countries, 13 localizations
• 20+ modules, 30+ integrated systems
• Heavily customized (3,500+ CEMLI)
• 5 custom-built bespoke bolt-on modules
• Products shifting to SaaS
• Failed attempts to upgrade to R12.2
• Heavily manual-dependent testing effort
• Painful release cycle time: want to establish monthly
releases
CONTEXT CHALLENGE
CLIENT
43 © 2015 Copyright Genpact. All Rights Reserved.
5 TOP CHALLENGES THEY FACED IN AUTOMATING EBS TESTING
• Domain Knowledge and Testing knowledge mutually exclusive impacting speed, quality
and effectiveness of testing
• Too much cycle time it takes to develop automated test scripts
• Always Obsolete: Scripting based test cases /data become easily obsolete therefore
require huge ongoing effort in keeping scripts up to date.
• Difficult to discover / reuse the test data, scripts, scenarios from release to release, project
to project
• Difficult to scope and tailor (Test some / Test whole) for individual releases
44 © 2015 Copyright Genpact. All Rights Reserved.
JOURNEY
Oct 2014
Dec 2014
Mar 2015
June 2015
Sep 2015
Nov 2015
Jan 2016
R12 Upgrade Kick
Off
OATS Pilot
1st OATS MVP
Ready – Used in
Unit Test
Progressive
Additions in
OATS
R122.2 Gone
Live
Monthly
Regression
Pack
Test Iterations Ongoing Optimization
45 © 2015 Copyright Genpact. All Rights Reserved.
RESULTS
•50% faster test iterations
•40% Less business effort in test cycles: Focus on value-added
test
•60+% reduction in regression cycle times
•Production releases: 8 /year (from earlier 4 /year)
•Adoption of Agile and Test-driven development methods
•100% Sanity, 80% Regression, 60% “projects” EBS testing
automated
46 © 2015 Copyright Genpact. All Rights Reserved.
LESSONS LEARNED: BALANCE OF MANUAL & AUTOMATED
Low Testing
Effort
Repeatable / Rule Based
High Testing
Effort
High Intuitive
1 2
• Automate the pre-requisites, support SMEs to focus
on “real” validation
• “Auto-gathering” of test evidence / progress: End to
End scenarios
• Role Based Testing
3
Limited value: diminishing returns of automation
Low Hanging Fruits
Automate to sustainable extent
X
47 © 2015 Copyright Genpact. All Rights Reserved.
BUSINESS-DRIVEN DOMAIN KNOWLEDGE LED
Script-free;
Drag & Drop;
Keywords;
Tags
48 © 2015 Copyright Genpact. All Rights Reserved.
PRE-BUILT TEST CONTENT: AGILE JUMP START
Business Scenarios
• End to end test scenarios covering
detailed critical business
processes, use cases and industry
nuances
• Context specific functional
scenarios
• COA change
• R12.2 upgrade
49 © 2015 Copyright Genpact. All Rights Reserved.
DESIGN FOR MAINTAINABILITY: MINIMIZE OBSOLESCENCE
•EBS Configuration Half Life:
•Data / Set Ups: 3 - 6 months
•Functionality/Logic: 6 - 12 months
•UI / major patch: 9 – 18 months
•Continuously updated library of test content
•Not “Screen Placement” but Object-Action driven
• “Keywords > Component > Flow” | upward bubbling of changes
•Changes in scripts for EBS customizations
50 © 2015 Copyright Genpact. All Rights Reserved.
HOT SPOTS VS SAFE SPOTS
•Test Clustering: L1 … L5 end to end scripts
• Iterative Learning: Progressive adaptability – Hot Spots vs Safe Spots
• Impact Traceability to Test Scenarios
•Test Driven Enhancement Management
•Continuous Optimization: “Regression Packs”, “Sanity Packs”
•Test Data
•Role based Focal Data Points (PO data vs Accounting data)
•Data Randomization, Variability
•Purpose-driven Data Generator scripts
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Questions
52 Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted