COCOMA presentation, FIA 2013
-
Upload
bonfire -
Category
Technology
-
view
184 -
download
0
description
Transcript of COCOMA presentation, FIA 2013
A Framework for Modeling and Execution ofInfrastructure Contention ExperimentsCarmelo Ragusa, Philip Robinson and Sergej SvorobejMERMAT 2013, FIA, 7 May, Dublin
© 2013 SAP AG or an SAP affiliate company. All rights reserved. 2
Agenda
Introduction– Problem analysis of conducting experiments about resource sharing in multi-tenancy
systems
Proposed solution– COCOMA framework: objectives, design, benefits and stakeholders
Use case– How resource contention affect users for different physical resources partitioning
configurations
Summary and future work
© 2013 SAP AG or an SAP affiliate company. All rights reserved. 3
Motivation
Resource (memory, disk, network, cpu) contention occurs in shared multy-tenancy environments such as clouds
Cloud Infrastructure
SuT Unknown
More detailed cloud experiments can be interested to investigate how the System under Test (SuT) performs under various contention patterns that may occur in practice.
Problem:
How can we study these issues?
How can we emulate multi-tenancy behaviour?
How can we reproduce the same experiments?
© 2013 SAP AG or an SAP affiliate company. All rights reserved. 4
Requirements for supporting rigoroussoftware testing in contentious, multi-tenant environments
• Scalability
• Reproducibility
• Portability
• Extensibility
• Self-containment
• Controllability
© 2013 SAP AG or an SAP affiliate company. All rights reserved. 5
Current approaches to reproduce operational conditions such as contention
• Manual: use of a number of resource-specific tools and command-line operations in order to manipulate operational conditions.
• Client-Based: simulation of usage, using multithreaded client-side request generators in order to load target environment and raise required operational conditions.
• Ad-Hoc Scripting: use of custom-built scripts for higher-level coordination of load generation and test execution.
Each one fails in same way to meet the previous listed requirements
© 2013 SAP AG or an SAP affiliate company. All rights reserved. 6
Proposed solution
A framework for COntrolled COntentious and Malicious (COCOMA) patterns: deliberately make the platform “misbehave” – contention, faults and attacks.
Cloud Infrastructure
SuT UnknownCOCOMA
Experimenters will be able to:• study their system under real world effects conditions • control those conditions• reproduce exact conditions • correlate conditions and results of their system under test• use those findings to discover weaknesses and tune/enhance their system
© 2013 SAP AG or an SAP affiliate company. All rights reserved. 7
Design principles
• Separation of concerns
• Provide unified and coherent interface for staging experiments
• Allowing advanced workload patterns via distribution algorithms
• Abstraction from low level tools
• Easy composition of complex patterns
• Easy extensibility
Virtual Machine
COCOMA
Distribution Algorithms
Distribution Algorithms
Distribution Algorithms
Stressapptest
Lookbusy
Iperf
IperfIperfetc.
Emulators
Resource contention
Experiment XML
© 2013 SAP AG or an SAP affiliate company. All rights reserved. 8
Concepts and terminology
• An emulation is composed of pairs of distribution-emulator
• An emulator in a distribution is bound to a specific resource type
• A distribution is a workload trend
• Distributions are broken down into multiple runs to create the desired trend
• Runs are single instantiation of low level tools
• For complex scenarios users can specify multiple distribution-emulator pairs
Emulation Emulator
Distribution Resource-Type
e.g. CPU, RAM, Net, I/O
1 *
1*
1
1 11
- type- duration- granularity- parameters[] e.g. linear, trapezoidal, exp,
trace
Emulation time
Distribution-1 (CPU)
Distribution-2 (RAM)
Distribution-3 (NET)
Distribution-4 (I/O)
t1 tn
© 2013 SAP AG or an SAP affiliate company. All rights reserved. 9
Use case: How resource contention affect users for different physical resources partitioning configurations
Test environment: single machine with 4 CPU cores and 16 GB of RAM
CPU (cores) RAM (GB range)
L (Low) 1 1 - 4
ML (Medium-Low) 2 5 - 8
MH (Medium-High) 3 9 - 12
H (High) 4 13 -16
COCOMA SuT Inactive VM Free resources
Conf CPU RAM Num VMs CPU RAM CPU RAM CPU RAM
1 MH H 1 L L n/a n/a n/a n/a
2 ML ML 1 ML ML n/a n/a n/a n/a
3 L L 1 MH H n/a n/a n/a n/a
4 MH H 3 L L n/a n/a n/a n/a
5 MH L 3 L L n/a n/a n/a n/a
6 L H 1 H L n/a n/a n/a n/a
7 ML ML 1 L L n/a n/a L ML
8 ML ML 1 L L L ML n/a n/a
Each configuration represents VMs with different resources assigned, providing a specific physical resources partitioning
We abstract resources’ values with ranges from Low to High
© 2013 SAP AG or an SAP affiliate company. All rights reserved. 10
Validation use case
• sysbench CPU benchmark calculating the first 100K prime numbers
• memspeed RAM benchmark
• tiobench IO benchmark
© 2013 SAP AG or an SAP affiliate company. All rights reserved. 11
Stakeholders
• Performance testers/engineers
• Cloud Service Providers
• Cloud Application Administrators
• Application Developers and Testers
• Benchmarks and Standards Groups
© 2013 SAP AG or an SAP affiliate company. All rights reserved. 12
Summary
• Experimenters need means that recreate conditions in cloud-like multy-tenancy shared systems to enable testing their solutions
• Current approaches are limited as they fail to meet requirements for such environments
• We proposed a solution that • abstracts from low level tools• enables from simple to complex repeatable scenarios, reducing experimenters’ effort• allows investigating the system behaviour and correlate it to the specific conditions
created
• A simple validation use case was presented
© 2013 SAP AG or an SAP affiliate company. All rights reserved. 13
Future work
• Implementing malicious and faultiness modules, and relative distributions
• Real system workload traces parsing and replay capability• Distributed COCOMA solution to enable large complex distributed
scenarios• Web UI to manage the framework• Contention at CPU cache level• Tool to record system traces in a COCOMA compatible format for reply
purpose• Extend emulation analyser for distributed version to get warnings for
large scenarios' emulations
© 2013 SAP AG or an SAP affiliate company. All rights reserved. 14
Acknowledgements
This work has being carried out within the BonFIRE project, which has received research funding from the EC’s Seventh Framework Programs (EU ICT-2009-257386 IP under the Information and Communication Technologies Program)
Thank you
Contact information:
Carmelo RagusaSAP HANA Cloud Infrastructure, [email protected]
COCOMA is released as Open Source under Apache v2 license:https://github.com/cragusa/cocoma