Reality shock j aracil
-
Upload
globe-testing -
Category
Software
-
view
290 -
download
1
description
Transcript of Reality shock j aracil
Madrid, 26th to 28th May 2014
Reality Shock:
Performance Testing
Jose Aracil - Operations Manager
@pep0 - @globetesting
Madrid, 26th to 28th May 2014
A radically different kind of user
System-centric User-centric
Users born after 1980
Mainframe DevicesWebClient/Server
Time
Madrid, 26th to 28th May 2014
2014
A tipping point for app. delivery
“New application
projects surge in 2014
and beyond.”
“Social, mobile, analytics,
cloud have reached the
tipping point in 2014.”
201536 per app
2020120 per app
2010 4 per app
Number of releases
per app per year
• Apps everywhere
• Accessed through multiple devices
• 30x increase in application releases
• Closed feedback loop with customers
• Focus on time-to-value and ROI
What we expect:
Madrid, 26th to 28th May 2014
Changes are coming faster
Cloud and composite applications are increasingly complex
Customers demand mobile access services
The pace of change is only increasing
Project and delivery teams are both agile and global
Businesses need applications to connect with:
• Customers
• Partners
• Suppliers…
Madrid, 26th to 28th May 2014
Dev/test lab
Developer
And the need for speed changes everything
Tester
Agile Development
“The new QA is extensively
automated and executed
by a continuous
integration process”--
Forrester Research
Provision
environments at
“agile speed”
Shift left, test
continuously
Agile
Development,
weekly drops
Madrid, 26th to 28th May 2014
Continuous Testing for
faster app delivery“Shift-Left”
With Confidence
Now Continuous Testing is more critical than ever
• Improve development
efficiency
• Reduce the test burden
• Accelerate application
delivery
Design Develop
Provision
Env.
Deploy
Test App.
Run test
Test results
Madrid, 26th to 28th May 2014
Madrid, 26th to 28th May 2014
Application usage has changed
Performance testing must change
Application development has changed
Madrid, 26th to 28th May 2014
You must
leave the black
box
Madrid, 26th to 28th May 2014
Sorry… I changed my mind!
Madrid, 26th to 28th May 2014
Sorry… I changed my mind!updated
Madrid, 26th to 28th May 2014
Planning Test Strategy
Reporting Final report
Preparation
Tool setupTest Env. + Data Test case design
Monitor config. Scripting
Smoke Test
Execution
Baseline
Soak Test Load Testing Other tests
Stress
Performance Testing
An
ális
is +
Cha
nge
s+
Rep
ort
Madrid, 26th to 28th May 2014
Planning
Preparation
Execution
Reporting
Test Strategy
Tool setupTest Env. + Data Test case design
Monitor config. Scripting
Smoke Test
Baseline
Soak Test Load Testing Other tests
Stress
Final report
Performance Testing
An
ális
is +
Cha
nge
s+
Rep
ort
But what get’s in the way?
Dependencies
Madrid, 26th to 28th May 2014
Good for one project
Madrid, 26th to 28th May 2014
Good for one project
Madrid, 26th to 28th May 2014
New set of problems…
Multiple
projects
Continuous
Integration
Auditing
Multiple people
Deployment of
LoadGenerators
Dependencies
Asset sharing
Reporting
SecurityCollaboration
Best
practices
Network
conditions
Cloud
Low application
maturity
Agile (good and bad)
Madrid, 26th to 28th May 2014
Project based testing challenges
Project AHP LoadRunner 8.1
1500 Vusers
Project BHP LoadRunner 11.0
1500 Vusers
Project COther load test SW
2500 Vusers
Project DNo load test budget
“Hope for the best”
Unrealistic “code freeze”
No sharing of best practices & skills – each project is unique and isolated
Duplicated resources (HW and SW) in separate projects
Some test cases not fully working (missing components)
End-to-end behaviour not fully tester
Madrid, 26th to 28th May 2014
End to end
Search PayBuy Ship Track
EXAMPLE:
The Online Shopping
Process
Product PaymentCustomer Profiles Shipping
Track packagePublic CloudTraditional Managed CloudComposite
APIs enable integration
Each have SLAs and performance
dependencies
Components anywhere– in-house or cloud
Madrid, 26th to 28th May 2014
Virtualisation Four-legged table
User
Network
Service
Data
User
• Virtualise user conditions to simulate real world conditions
Network
• Virtualise network conditions to simulate production architecture and real user conditions
Service
• Virtualise services and service interactions to simulate E2E business processes
Data
• Virtualise production data and transaction responses to simulate application request and responses
Madrid, 26th to 28th May 2014
We are now able to truly apply a
agile approached to load testing
given the reduction in scripting and
build time.
Our scripting time is reduced by
more than 75% and resource
productivity across projects has
improved by more that 50%.
Source: TechValidate. TVID: B40-ED8-557
User
Madrid, 26th to 28th May 2014
Enable Agile performance DeliveryUser
Test
Build Load Test scripts during Dev
Run Unit Tests in Load Test Scenarios
Add Performance Tests as build steps in continuous integration tools
Madrid, 26th to 28th May 2014
Mobile is different
Analyze Mobile Performance on Network
Backend
monitoring
Mobile
performance
automation
Mobile Network
characteristics
Network
Identify bottlenecks before end users
Test in both good and BAD network connections
Understand impact of mobile on entire system
Madrid, 26th to 28th May 2014
Existing Infrastructure
Service Virtualization Concept
Mainframe
JDBC
SOAP
RFC/IDO
C
MQ/CIC
S
Third Party
Application Under Test
Mobile
App
Web
browser
Composite
Application
API
SAP System
Existing database
Web service and
Legacy application
REST
Pay-per-transaction
Under
construction
Service
Madrid, 26th to 28th May 2014
Existing Infrastructure
Service Virtualization Concept
Virtual services are created to
stand in for the real service when
developers or testers need
access
Mainframe
JDBC
SOAP
RFC/IDO
C
MQ/CIC
S
Third Party
Application Under Test
Mobile
App
Web
browser
Composite
Application
API
SAP System
Existing database
Web service and
Legacy application
REST
Pay-per-transaction
Under
construction
Service
Virtualization
SOAP
REST
JDBC
MQ
RFC
CICS
Simulatio
n
Data
Perf.
Config
Service
Madrid, 26th to 28th May 2014
Service Virtualization is the answer
Functional test
Performance Test
Regression test
Dependencies
Madrid, 26th to 28th May 2014
Service Virtualization is the answer
Functional test
Performance Test
Regression test
DependenciesVirtualization
Allow frequent access to app components, APIs and services
Remove constraints with virtual version
No access to a component? Use virtual component
Enable performance test teams to set up simulated services
Madrid, 26th to 28th May 2014
Bring production data into performance
testing as baseline
• Collect, analyze and compare
performance data from production
• Updated log file analysis of production
data for IIS
• Integrate with Google Analytics
DataPerformance application lifecycle
Value:
Performance testing closer to reality with accurate,
realistic, reliable test: reduce risk and improve test results
Madrid, 26th to 28th May 2014
Data
Virtualise production data and
transaction responses to simulate
application request and responses
Data
Madrid, 26th to 28th May 2014
3 Keys to better
performance
• Test early
• Test often
• Monitor and leverage
data from monitoring
as baseline for test
The NEW Reality
User Network Service Data
Madrid, 26th to 28th May 2014
Q&A
Madrid, 26th to 28th May 2014
IF YOU LIKE IT
PLEASE RATE MEVisit our blog to get all the slides
WWW.GLOBETESTING.COM