Post on 18-Dec-2014
description
Kees Blokland and Jeroen Mengerink, Polteq
Testing of Cloud Services
The Approach: From Risks to Test
Measures
www.eurostarconferences.com
@esconfs #esconfs
Testing of Cloud Services
The Approach: From Risks to Test
Measures
Kees Blokland
Jeroen Mengerink
Polteq Test Services BV
The Netherlands
searching, recording, accounting, paying, writing,
reviewing, tracking, calculating, developing, listening,
analyzing, transmitting, learning, controll ing,
purchasing, testing, alarming, changing, updating,
deleting, accessing, rejecting, correcting, studying,
booking, receiving, tracing, protecting, deciding,
managing, teaching, facilitating, identifying, copying,
removing, demonstrating, checking, showing,
selecting, subscribing, unsubscribing, sharing,
mailing, communicating, reading, playing, working,
meeting, gambl ing, shopping, storing, cross
checking, retrieving, configuring, sketching, saving,
accelerating, enhancing, creating, growing, checking
in, checking out, finding out, reaching, denying,
talking, designing, making, verifying, measuring
5
Surf
Transfer
Develop and Test
Operate and Manage Store
6
Surf
Transfer
Develop and Test
Operate and Manage Store
storage claim
80% unused
redundancy, proliferation limitations
environmentally unfriendly
7
Surf
Transfer
Develop and Test
Operate and Manage Store
standard software bandwidth
internet technology
virtualization
SOA
8
searching, recording, accounting, paying, writing,
reviewing, tracking, calculating, developing, listening,
analyzing, transmitting, learning, controll ing,
purchasing, testing, alarming, changing, updating,
deleting, accessing, rejecting, correcting, studying,
booking, receiving, tracing, protecting, deciding,
managing, teaching, facilitating, identifying, copying,
removing, demonstrating, checking, showing,
selecting, subscribing, unsubscribing, sharing,
mailing, communicating, reading, playing, working,
meeting, gambl ing, shopping, storing, cross
checking, retrieving, configuring, sketching, saving,
accelerating, enhancing, creating, growing, checking
in, checking out, finding out, reaching, denying,
talking, designing, making, verifying, measuring
Surf
Transfer
Develop and Test
Operate and Manage Store
US: National Institute of Standards and Technology http://www.nist.gov
Essential characteristics
On-demand service
Self service provisioning, pay-per-use
No human interaction
US: National Institute of Standards and Technology http://www.nist.gov
Essential characteristics
On-demand service
Broad network access
Standard mechanisms over networks
“Any” client
US: National Institute of Standards and Technology http://www.nist.gov
Essential characteristics
On-demand service
Broad network access
Resource pooling
Multi-tenant
Storage, processing, memory, virtual machines, …
Location independent
US: National Institute of Standards and Technology http://www.nist.gov
Essential characteristics
On-demand service
Broad network access
Resource pooling
Rapid elasticity
Rapid scale in and out
“Any quantity” at any time
US: National Institute of Standards and Technology http://www.nist.gov
Essential characteristics
On-demand service
Broad network access
Resource pooling
Rapid elasticity
Measured service
Controlled resource use
Transparency, pay-per-use
US: National Institute of Standards and Technology http://www.nist.gov
Essential characteristics
On-demand service
Broad network access
Resource pooling
Rapid elasticity
Measured service
Deployment models
– private cloud
– community cloud
– public cloud
– hybrid cloud
Service Models
Software as a Service
Platform as a Service
Infrastructure as a Service
16
What is “done” in the cloud?
>500 Private Hybride Community IaaS, PaaS, DaaS, SaaS
Taas
*aaS
Data Centre Data Management Business processes
Consumer Public SaaS
Surf and mail Apps Social media Dropbox Google services Spotify Picasa Games ……………
<500, SME Public *aaS
Mail Storage
Infrastructure
CRM
Finance
Business processes
Continuity
Privacy
Multi platform
Legislation
Cyber crime
Impact organisation
Standards
143
Continuïty
Privacy
Multi platform
Legislation
Cyber crime
Impact organisation
Standards Performance
Security
Availability & Continuity
Functionality
Manageability
Legislation & Regulations
Suppliers & Outsourcing Ris
ks
19
Other customers
YOUR
Operational Profile
YOUR
Operational Profile
PLUS
Performance
Security
Availability & Continuity
Functionality
Manageability
Legislation & Regulations
Suppliers & Outsourcing Ris
ks
20
Everything over the web
The idea:
“it’s safe”
Home ground for
hackers
Performance
Security
Availability & Continuity
Functionality
Manageability
Legislation & Regulations
Suppliers & Outsourcing Ris
ks
21
Bring Your Own Device
No free choice of
device.
Endless
possibilities.
Performance
Security
Availability & Continuity
Functionality
Manageability
Legislation & Regulations
Suppliers & Outsourcing Ris
ks
22
Backup and recovery
Taken care of.
Who will support
me?
Performance
Security
Availability & Continuity
Functionality
Manageability
Legislation & Regulations
Suppliers & Outsourcing Ris
ks
23
Updates, patches, fixes, …
Planned and
controlled
Do I have a
choice?
Performance
Security
Availability & Continuity
Functionality
Manageability
Legislation & Regulations
Suppliers & Outsourcing Ris
ks
24
Where is my data?
And is that OK?
In house.
Somewhere…
Performance
Security
Availability & Continuity
Functionality
Manageability
Legislation & Regulations
Suppliers & Outsourcing Ris
ks
25
Performance
Security
Availability & Continuity
Functionality
Manageability
Legislation & Regulations
Suppliers & Outsourcing Ris
ks
26
Testing?
Check
Intake
Trial
Interview
Proof of concept
27
Testing!
Check
Intake
Trial
Interview
Proof of concept Testen Proef Intake
Interview Proof of concept
28
Performance Testing
Security Testing
Manageability Testing
Availability & Continuity
Testing
Functional Testing
Migration Testing
Testing caused by
Legislation & Regulations
Testing in Production
Testing during Selection T
es
t Me
as
ure
s
Testen Proef Intake
Interview Proof of concept
29
Performance
Security
Availability & Continuity
Functionality
Manageability
Legislation & Regulations
Suppliers & Outsourcing Ris
ks
Performance Testing
Security Testing
Manageability Testing
Availability & Continuity
Testing
Functional Testing
Migration Testing
Testing caused by
Legislation & Regulations
Testing in Production
Testing during Selection T
es
t Me
as
ure
s
30
Performance
Security
Availability & Continuity
Functionality
Manageability
Legislation & Regulations
Suppliers & Outsourcing Ris
ks
Performance Testing
Security Testing
Manageability Testing
Availability & Continuity
Testing
Functional Testing
Migration Testing
Testing caused by
Legislation & Regulations
Testing in Production
Testing during Selection T
es
t Me
as
ure
s
31
Performance
Security
Availability & Continuity
Functionality
Manageability
Legislation & Regulations
Suppliers & Outsourcing Ris
ks
Performance Testing
Security Testing
Manageability Testing
Availability & Continuity
Testing
Functional Testing
Migration Testing
Testing caused by
Legislation & Regulations
Testing in Production
Testing during Selection T
es
t Me
as
ure
s
32
Performance Testing
Security Testing
Manageability Testing
Availability & Continuity
Testing
Functional Testing
Migration Testing
Testing caused by
Legislation & Regulations
Testing in Production
Testing during Selection T
es
t Me
as
ure
s
Performance
Security
Availability & Continuity
Functionality
Manageability
Legislation & Regulations
Suppliers & Outsourcing Ris
ks
Architecture
From “individual” risks
to
“individual” test measures
33
Selection
Implementation
Production
Performance Testing
Security Testing
Manageability Testing
Availability & Continuity
Testing
Functional Testing
Migration Testing
Testing caused by
Legislation & Regulations
Testing in Production
Testing during Selection T
es
t Me
as
ure
s
34
Performance Testing
Security Testing
Manageability Testing
Availability & Continuity
Testing
Functional Testing
Migration Testing
Testing caused by
Legislation & Regulations
Testing in Production
Testing during Selection T
es
t Me
as
ure
s
Performance
Security
Availability & Continuity
Functionality
Manageability
Legislation & Regulations
Suppliers & Outsourcing Ris
ks
35
logo van Flair
17-12-2010
Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Selection Criteria
Performance Testing
Security Testing
Manageability Testing
Availability & Continuity
Testing
Functional Testing
Migration Testing
Testing caused by
Legislation & Regulations
Testing in Production
Testing during Selection T
es
t Me
as
ure
s
36
logo van Flair
17-12-2010
Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Proof of Concept
Performance Testing
Security Testing
Manageability Testing
Availability & Continuity
Testing
Functional Testing
Migration Testing
Testing caused by
Legislation & Regulations
Testing in Production
Testing during Selection T
es
t Me
as
ure
s
37
Performance Testing
Security Testing
Manageability Testing
Availability & Continuity
Testing
Functional Testing
Migration Testing
Testing caused by
Legislation & Regulations
Testing in Production
Testing during Selection T
es
t Me
as
ure
s
Performance
Security
Availability & Continuity
Functionality
Manageability
Legislation & Regulations
Suppliers & Outsourcing Ris
ks
38
logo van Flair
17-12-2010
Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Known measures
tuned and tweaked
New measures developed
Performance Testing
Security Testing
Manageability Testing
Availability & Continuity
Testing
Functional Testing
Migration Testing
Testing caused by
Legislation & Regulations
Testing in Production
Testing during Selection T
es
t Me
as
ure
s
39
logo van Flair
17-12-2010
Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Load Testing
YOUR
Operational Profile
YOUR
Operational Profile
PLUS
ACTUAL MOMENT
Performance Testing
Security Testing
Manageability Testing
Availability & Continuity
Testing
Functional Testing
Migration Testing
Testing caused by
Legislation & Regulations
Testing in Production
Testing during Selection T
es
t Me
as
ure
s
40
logo van Flair
17-12-2010
Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Online – Offline
Use case testing.
Global testing.
Performance Testing
Security Testing
Manageability Testing
Availability & Continuity
Testing
Functional Testing
Migration Testing
Testing caused by
Legislation & Regulations
Testing in Production
Testing during Selection T
es
t Me
as
ure
s
41
logo van Flair
17-12-2010
Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Any device – any platform
Multiplatform
testing.
Multiplatform
testing.
Performance Testing
Security Testing
Manageability Testing
Availability & Continuity
Testing
Functional Testing
Migration Testing
Testing caused by
Legislation & Regulations
Testing in Production
Testing during Selection T
es
t Me
as
ure
s
42
Internet Explorer 6
Internet Explorer 7
Internet Explorer 8
Firefox 3.5
Firefox 3.6
Firefox 4
Safari 4
Safari 5
Chrome11
Opera11
Windows XP
Windows Vista
Windows 7
Windows 2003 server
Windows 8
Windows CE
Linux
Unix
Mac OS Lion
Mac OS Snowleopard
iOS
Android
Operating Systems
Browsers
Multiplatform
Devices
Computer
Mobile
Tablet
PC
Macintosh
SUN
NOKIA …
Samsung … Windows Mobile
iPhone ..
Xxx …
MOTOROLA…
Blackberry…
ASUS..
Xxx …
5-18
43
logo van Flair
17-12-2010
Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Any device – any platform
Multiplatform
testing.
Multiplatform
testing.
Performance Testing
Security Testing
Manageability Testing
Availability & Continuity
Testing
Functional Testing
Migration Testing
Testing caused by
Legislation & Regulations
Testing in Production
Testing during Selection T
es
t Me
as
ure
s
44
logo van Flair
17-12-2010
Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Legislation + Regulations
=
Test basis
Incidental testing.
Compliancy testing.
Performance Testing
Security Testing
Manageability Testing
Availability & Continuity
Testing
Functional Testing
Migration Testing
Testing caused by
Legislation & Regulations
Testing in Production
Testing during Selection T
es
t Me
as
ure
s
45
Performance Testing
Security Testing
Manageability Testing
Availability & Continuity
Testing
Functional Testing
Migration Testing
Testing caused by
Legislation & Regulations
Testing in Production
Testing during Selection T
es
t Me
as
ure
s
Performance
Security
Availability & Continuity
Functionality
Manageability
Legislation & Regulations
Suppliers & Outsourcing Ris
ks
46
logo van Flair
17-12-2010
Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Continuous
End-to-End Test
Functionals
and
non-functionals
Performance Testing
Security Testing
Manageability Testing
Availability & Continuity
Testing
Functional Testing
Migration Testing
Testing caused by
Legislation & Regulations
Testing in Production
Testing during Selection T
es
t Me
as
ure
s
Continuity
Privacy
Multi platform
Legislation
Cyber crime
Impact organisation
Standards
Check
Intake
Trial
Interview
Proof of concept
Continuity
Privacy
Multi platform
Legislation
Cyber crime
Impact organisation
Standards
Check
Intake
Trial
Interview
Proof of concept
49
Performance Testing
Security Testing
Manageability Testing
Availability & Continuity
Testing
Functional Testing
Migration Testing
Testing caused by
Legislation & Regulations
Testing in Production
Testing during Selection T
es
t Me
as
ure
s
Performance
Security
Availability & Continuity
Functionality
Manageability
Legislation & Regulations
Suppliers & Outsourcing Ris
ks
Architecture
From “individual” risks
to
“individual” test measures
Test starts earlier
Test scope is widened
Test will never stop
50
Performance Testing
Security Testing
Manageability Testing
Availability & Continuity
Testing
Functional Testing
Migration Testing
Testing caused by
Legislation & Regulations
Testing in Production
Testing during Selection T
es
t Me
as
ure
s
Performance
Security
Availability & Continuity
Functionality
Manageability
Legislation & Regulations
Suppliers & Outsourcing Ris
ks
Questions?
51
Performance Testing
Security Testing
Manageability Testing
Availability & Continuity
Testing
Functional Testing
Migration Testing
Testing caused by
Legislation & Regulations
Testing in Production
Testing during Selection T
es
t Me
as
ure
s
Performance
Security
Availability & Continuity
Functionality
Manageability
Legislation & Regulations
Suppliers & Outsourcing Ris
ks
Thank you!