Making Workflows Work Prof. Yike Guo Dept. of Computing Imperial College London InforSense Limited
Imperial College - Department of Computing Continuous Performance Testing in Virtual Time Nikos...
-
Upload
aileen-webb -
Category
Documents
-
view
213 -
download
0
Transcript of Imperial College - Department of Computing Continuous Performance Testing in Virtual Time Nikos...
ContinuousPerformance Testing
in Virtual Time
Nikos Baltas & Tony Field
Department of Computing
Imperial College London
Imperial College - Department of Computing
Motivation
Performance analysis:Early stages: performance modellingLate stages: profiling & performance testing Intermediate stages: extract models from code,
calibrate them and analyse...
Can we simplify this? Can we performance test incomplete applications?
Imperial College - Department of Computing
Idea
Integrate models and code
Don't model existing code – execute it in virtual time!
Use performance models that describe the expected response times of missing (or existing) methods
Combine measurements and model simulations to guide the thread progress of the application
Imperial College - Department of Computing
VEX
Virtual Execution framework (VEX)
Java applications
Components Thread control Profiling I/O handling
Imperial College - Department of Computing
VEX simulation
Model existing code by profiling it on the simulation host
Map the (possibly scaled) measurements on a virtual timeline
Control the thread progress to enforce a fair scheduling policy in virtual time
Imperial College - Department of Computing
Integration of models to VEX
Don't just scale method durations: replace them
Use time acquired by the simulation of performance models (open queuing networks)
Decouple profiled time from virtual time measurements
Use for either implemented methods or code stubs
Continuous performance testing
Imperial College - Department of Computing
Issues
How does the model-simulation time relate to the virtual time of executed code?
How do we control the model simulation?
What if the code of the model-described method interacts with the application?
Imperial College - Department of Computing
Model simulation in VEX
Thread adds job Thread state depends
on job state in queue: L-Think: Running/able R-Think: Timed-waiting Queuing: Waiting L-Serv: Running/able R-Serv: Timed-waiting
Thread and job join at a Sink node
Imperial College - Department of Computing
Model description: JMT [1]
[1] M. Bertoli, G. Casale, and G. Serazzi, “JMT: performance engineering tools for system modeling,” SIGMETRICS Perform.Eval.Rev., vol. 36, no. 4, pp. 10–15, 2009.
Imperial College - Department of Computing
Functional consistency
T1
T2
Performancemodel for M()
synchronized(obj) {resume suspend
M() { synchronized(obj) {
wait for T1 to resume
blocked on obj monitor
Deadlock!violate virtual scheduleand resume anyway
} exit monitor obj suspend
enter/exit monitor
} M()
Join with model resume
Imperial College - Department of Computing
Usage
-Xbootclasspath/p:vex.jar –javaagent:jine.jar –agentpath:libjinevex.sojava Class
void foo() {
…..
bar();
…..
}
@virtualtime.ModelPerformance(
jmtModelFilename=“file.jsimg”,
replaceMethodBody=true,
sourceNodeLabel=“Source0”)
Imperial College - Department of Computing
Evaluation
Case study to demonstrate usage and verify functionality of approach
Sample results and parameters
Validate process by comparing predicted results to real times
Study: remote DB server with local cache
Imperial College - Department of Computing
Conclusion
Integration of models and code within VEX
Models and code can be used interchangeably
Continuous performance testing without extracting models from code
Prediction errors due to disregard for background system load and I/O approximation
Imperial College - Department of Computing
Future work
Handle more behaviours (asynchronous I/O)
Lower-level VEX and/or distributed virtual execution
Application on larger scale projects
Release VEX under open source license
Extensions to other languages (C++)