Paraso& Proprietary and Confiden1al 1
5/25/16
Evolving from Automated to Continuous Testing
for Agile and DevOps
[email protected] Solution Architect
Paraso& Proprietary and Confiden1al 2 Paraso& Proprietary and Confiden1al 2
To start with…
§ Created: BC 196
§ Found: 1799
§ London: 1801
§ Bri1sh Museum: 1802
§ Transla1on: 1822 § Egyp1an hieroglyphs § Demo1c script § Ancient Greek
Paraso& Proprietary and Confiden1al 3 Paraso& Proprietary and Confiden1al 3
Anno 2016, you cannot keep things for yourself…
Paraso& Proprietary and Confiden1al 4 Paraso& Proprietary and Confiden1al 4
Impact of Faulty So&ware
-‐5.70% -‐1.9B
Paraso& Proprietary and Confiden1al 5 Paraso& Proprietary and Confiden1al 5
Anno 2016: So&ware Failures = Headlines
-$2.55 Bn -4.08%
The day of the announcement companies lost an average of shareholder value
So&ware failures make headline news— eroding customer confidence, shareholder value and brand equity
Paraso& Proprietary and Confiden1al 6 Paraso& Proprietary and Confiden1al 6
Escala1ng Cost of Failure: Public companies
Paraso& Proprietary and Confiden1al 7 Paraso& Proprietary and Confiden1al 7
Re-‐Inven1ng so&ware development
The evolu1on of the so#ware development lifecycle (SDLC) over the past 3 years is forcing the transforma1on of so&ware tes1ng
Transforma)on Business Impact • Business risk associated with
so1ware failure at an all 4me high • Public companies lose an average of
$2.5 Bn in market cap on the day of the announcement
• “Digital Industrial Revolu4on” makes all a so1ware company
• Trend toward Dev/Test insourcing as compe44ve advantage is via so1ware
• Strong adop4on of “agile” development prac4ces
• Forces more technical tes4ng, early. Agile is forcing “shi1-‐le1”
• DevOps is applying lean prac4ces to SDLC
• DevOps is exposing 20 years of process barriers protected by silos and culture
• Cloud has disrupted tradi4onal SDLC and is driving down costs
• Switching costs associated with applica4ons at an all 4me low
Paraso& Proprietary and Confiden1al 8 Paraso& Proprietary and Confiden1al 8
How to manage the Disrup1on ?
• Security • Safety Cri1cal • FDA • Automo1ve • Aerospace
Compliance • DevOps • Agile • Lean • Bi-‐Modal
Ini1a1ves • Microservices • Containers • Cloud • IoT
Technology • Produc1on Outage • Applica1on Slow • User Experience • Connec1vity
Failure
Does the release candidate have an acceptable level of risk?
Mitigate Business Risk
Paraso& Proprietary and Confiden1al 9
Automa1on expands through whole SDLC
Con1nuous Delivery
Con1nuous Integra1on
Delivery Team
Version Control
Build & Unit Test
Automated Acceptance
Tests
User Acceptance
Tests Release
Check In Trigger
Trigger Trigger
Trigger Trigger Approval Approval
Check In
Check In
Feedback
Feedback
Con1nuous Deployment
Produc1on
Business
Paraso& Proprietary and Confiden1al 10 Paraso& Proprietary and Confiden1al 10
Fixed Fixed
Impact of Test in the Evolving SDLC (1)
Plan Develop Build Test Release
Test as a 1me-‐boxed event is the prevailing method
“Are we done tes1ng?”
Paraso& Proprietary and Confiden1al 11 Paraso& Proprietary and Confiden1al 11
Impact of Test in the Evolving SDLC (2)
Plan Develop
Build Test
Release
Build Test
Build Test
Build Test
Build Test
Build Test
Build Test
Build Test
Build Test
Build Test
Build Test
“Does the release candidate have an acceptable level of risk?”
Paraso& Proprietary and Confiden1al 12 Paraso& Proprietary and Confiden1al 12
Impact of Test in the Evolving SDLC (3)
Speeding up the conveyor belt does not yield be>er results…
… you need Quality @ Speed
The new so&ware paradigm requires the business to define and establish automated quality gates that con1nuously mi1gate the risk of business failure
Paraso& Proprietary and Confiden1al 13 Paraso& Proprietary and Confiden1al 13
How to achieve this Quality @ Speed ?
Take the top-‐down view: § Business risks drive quality ac1vi1es § Team collaborates on the defini1on of risk § Baseline and threshold criteria measured and
monitored § Results of quality ac1vi1es are visible and
translated for all levels of business
This approach will allow you to go from “automated” to “con1nuous”…
Paraso& Proprietary and Confiden1al 14 Paraso& Proprietary and Confiden1al 14
From Automated to Con1nuous (1)
Paraso& Proprietary and Confiden1al 15 Paraso& Proprietary and Confiden1al 15
From Automated to Con1nuous (2)
The steps required for moving from “Automated” to “Con1nuous”
§ Business expecta1ons clearly defined § Business risks iden1fied § Per applica1on, team, release
§ Defects automa1cally priori1zed versus business drivers § No defect le& behind § Mi1gate business risks
§ Tes1ng in ‘complete’ test environments con1nuously § Leveraging simula1on – Service Virtualiza1on § Protec1ng the real-‐user experience business process
§ Feedback loop for defect preven1on § Learn by capturing in paoerns § Defect preven1on prac1ces enhanced
Paraso& Proprietary and Confiden1al 16 Paraso& Proprietary and Confiden1al 16
Con1nuous Tes1ng: Quality Product & Process
Automated Tes)ng provides the “sensors” (detec)on processes) § Fixing defects is the expected outcome § Iden1fies opportuni1es for defect preven1on § Prevents a release candidate from going to the next stage of the release cycle
Con)nuous Tes)ng addresses the broader challenge § Fix the product § Op1mize the process (improve the
effec1veness of the “sensors”)
Con1nuously refine the process in light of business risks as well as so&ware quality
Con)nuous Tes)ng addresses the broader challenge
§ Fix the product § Op1mize the process (improve the effec1veness of the “sensors”) § Con1nuously refine the process in light of business risks as well as so&ware
quality
Paraso& Proprietary and Confiden1al 17 Paraso& Proprietary and Confiden1al 17
Con1nuous Tes1ng
Con)nuous Tes)ng
Paraso& Proprietary and Confiden1al 18 Paraso& Proprietary and Confiden1al 18
Transforming the Iden1fica1on of Risk
From To
Causal Observa1ons Probabilis1c
Defect Documenta1on Most Likely Causes
Only Structured Data Structured and Unstructured
Filters Business Policies
Ad Hoc Automated
Tool Dependent Tool Independent
Business Objectives
Paraso& Proprietary and Confiden1al 19 Paraso& Proprietary and Confiden1al 19
Con1nuous Tes1ng mi1gates business risk
Why Con)nuous Tes)ng?
Systema1c, centralized decision making to factor
business risk into SDLC
Safety net to allow developers to bring innova1ons to market faster
A feedback system for beoer trade-‐off decisions between release scope,
1me and quality
Paraso& Proprietary and Confiden1al 20 Paraso& Proprietary and Confiden1al 20
Con1nuous Tes1ng builds on automa1on
Con)nuous Tes)ng
Not more automa1on, beoer automa1on
Business and development leaders agree upon and automate the mi1ga1on of risks throughout the SDLC
Both business and technical managers make beoer trade-‐off decisions between
release scope, 1me and quality
Organiza1ons evolve from just measuring product quality to measuring the process effec1veness
Paraso& Proprietary and Confiden1al 21 Paraso& Proprietary and Confiden1al 21
Con1nuous Tes1ng monitors the process
Con)nuous Tes)ng
Con)nuous Deployment
Con)nuous Delivery
Con)nuous Release
Con)nuous Integra)on
§ Con1nuous Tes1ng focuses on a comprehensive view of business risk
§ Con1nuous Tes1ng provides a workflow for priori1zing fixes and quality improvements, and preven1ng recurrence, according to business risk and business priori1es
Con)nuous Tes)ng enables con)nuous improvement
Paraso& Proprietary and Confiden1al 22 Paraso& Proprietary and Confiden1al 22
Feature Release
Business Stakeholder Developer Customer Support
Con1nuous Tes1ng in ac1on: Use Quality Gates
SoKware Feature
Con1nuous measurements mean con1nuous refinement of the process
Real-‐1me feedback from
Objec1ve assessment and go/no go Defects are eliminated at the point that they are easiest to fix
Business Stakeholder Developer Customer Support
Quality gates: Organiza1ons can automa1cally promote so&ware through quality gates when business expecta1ons have been met
Paraso& Proprietary and Confiden1al 23 Paraso& Proprietary and Confiden1al 23
Summary: The elements of Con1nuous Tes1ng
Con)nuous Tes)ng re-‐posi)ons the ques)on from “are you done tes)ng?” to “is the level of risk understood and accepted?”
Con)nuous Tes)ng
Risk Assessment
Policy Analysis
Requirements Traceability
Test Environment
Access
Test Op)miza)on
Advanced Analysis
Ensure access to complete test environments
Automate defect preven1on and Policy measurement
Expand test coverage and measure test effec1veness
Define ac1onable prac1ces
Connect func1onal with non-‐func1onal requirements
Process improvement opportuni1es
Paraso& Proprietary and Confiden1al 24 Paraso& Proprietary and Confiden1al 24
How we can help ?
Does the release candidate have an acceptable level of risk?
Mitigate Business Risk
Con1nuous Tes1ng • Test Configura1on and Simula1on • Test Environment Management • Defect Detec1on • Change Control
Development Tes1ng • Risk Defini1on • Process Control • Defect Preven1on • Visibility
§ Methodology: § Research:
§ SW to support development & con1nuous tes1ng:
Paraso& Proprietary and Confiden1al 25 Paraso& Proprietary and Confiden1al 25
Let us help you to decipher your SDLC…
Paraso& Proprietary and Confiden1al 26 Paraso& Proprietary and Confiden1al 26
Paraso& Con1nuous Tes1ng
Paraso& develops automated so&ware quality solu1ons that prevent and detect risks associated with applica1on failure
Emphasizes business expecta1ons to mi1gate
business risks
Enables development
tes1ng
Promotes early detec1on and remedia1on
Facilitates integra1on tes1ng
Provides ubiquitous test environment access with Service Virtualiza1on
Assists in end-‐to-‐end
system tes1ng
The ParasoK Con)nuous Tes)ng
Top Related