Building Web Services Middleware with Predictable Service Execution
-
Upload
vidura-gamini-abhaya -
Category
Technology
-
view
841 -
download
1
Transcript of Building Web Services Middleware with Predictable Service Execution
Building Web Services Middleware with
Predictable Service Execution
Vidura Gamini AbhayaProf. Zahir Tari and Assoc. Prof. Peter Bertok
Distributed Systems and Networking GroupSchool of Computer Science and IT
RMIT UniversityMelbourne, Australia
December 13, 2010
[WISE 2010 - pages 23-37 in proceedings]
Intro. Guidelines Case Study Results Conclusion
Presentation Structure
1 Introduction
2 Guidelines for Predictability
3 Case Study
4 Empirical Results
5 Conclusion
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Motive. QoS in the Cloud ThreadPools WS Stack
1 Introduction
2 Guidelines for Predictability
3 Case Study
4 Empirical Results
5 Conclusion
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Motive. QoS in the Cloud ThreadPools WS Stack
Motivation
Web Services enable communication between DistributedHeterogeneous Systems.
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Motive. QoS in the Cloud ThreadPools WS Stack
Motivation
Web Services enable communication between DistributedHeterogeneous Systems. It is the foundation for...
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Motive. QoS in the Cloud ThreadPools WS Stack
Motivation
Web Services enable communication between DistributedHeterogeneous Systems. It is the foundation for...
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Motive. QoS in the Cloud ThreadPools WS Stack
Motivation
Web Services enable communication between DistributedHeterogeneous Systems. It is the foundation for...
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Motive. QoS in the Cloud ThreadPools WS Stack
Motivation - QoS in the Cloud
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Motive. QoS in the Cloud ThreadPools WS Stack
Motivation - QoS in the Cloud
Increased the importance of QoS
Lack of attempts in achieving Execution-time QoS
On achieving consistent and predictable Execution times
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Motive. QoS in the Cloud ThreadPools WS Stack
Web Services Middleware
WS-Middleware are optimised for throughput by design
Requests are accepted unconditionally
Uses thread-pools to execute them in a best-effort manner
Processes multiple requests in-parallel in a processor sharingmanner
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Motive. QoS in the Cloud ThreadPools WS Stack
ThreadPool Pattern
Increases throughput
At the increase of the mean execution time of a web service
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Motive. QoS in the Cloud ThreadPools WS Stack
ThreadPool Pattern
Increases throughput
At the increase of the mean execution time of a web service
No. of requests served in-parallel ∝ Mean time to service a request
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Motive. QoS in the Cloud ThreadPools WS Stack
Web Services Software Stack
The execution of a web service request goes through multiplelayers of software
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Motive. QoS in the Cloud ThreadPools WS Stack
Web Services Software Stack
Predictability at the upper layers can only be achieved if it issupported at the lower levels
Thread priority level mismatches between dev platform and OS
Service invocations can be interrupted by a process/thread with higher priority
Delays in resource reservations at OS level
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Guidelines
1 Introduction
2 Guidelines for Predictability
3 Case Study
4 Empirical Results
5 Conclusion
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Guidelines
Guidelines for Predictability in Service Execution
- Enable Web Service Middleware to achieve Predictability in Service Execution.- Can be followed in building new middleware or to enhance existing.- Applicable for both SOAP and REST based services
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Guidelines
Guidelines for predictability in service execution
G1. Dev platform and OS used must guarantee predictability of execution
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Guidelines
Guidelines for predictability in service execution
G1. Dev platform and OS used must guarantee predictability of execution
G2. Schedule requests based on their deadlines
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Guidelines
Guidelines for predictability in service execution
G1. Dev platform and OS used must guarantee predictability of execution
G2. Schedule requests based on their deadlines
G3. Select requests based on their laxity
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Guidelines
Guidelines for predictability in service execution
G1. Dev platform and OS used must guarantee predictability of execution
G2. Schedule requests based on their deadlines
G3. Select requests based on their laxity
G4. Introduce fine-grain differentiation in request processing
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Guidelines
Guidelines for predictability in service execution
G1. Dev platform and OS used must guarantee predictability of execution
G2. Schedule requests based on their deadlines
G3. Select requests based on their laxity
G4. Introduce fine-grain differentiation in request processing
G5. Minimise priority inversions
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Overview Infra. Commons Standalone Cluster PI
1 Introduction
2 Guidelines for Predictability
3 Case Study
4 Empirical Results
5 Conclusion
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Overview Infra. Commons Standalone Cluster PI
Case Study - Overview
Infrastructure
Common enhancements made
Schedulability CheckPriority ModelReal-time Scheduler
Stand-alone web service engine
Apache Axis2
Web service cluster
Dispatcher - Apache SynapseExecutors - Apache Axis2
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Overview Infra. Commons Standalone Cluster PI
Infrastructure
Dev Platform and OS
Following G1,
Solaris 10 08/05 is used as the Real-time OS
Sun Real-time Java Specification is used as the Dev Platform
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Overview Infra. Commons Standalone Cluster PI
Schedulability Check
Following G3, every incoming request is subjectto a Schedulability Check (Ref: Algorithm 1 onpaper and mathematical model in [5]).
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Overview Infra. Commons Standalone Cluster PI
Schedulability Check
Following G3, every incoming request is subjectto a Schedulability Check (Ref: Algorithm 1 onpaper and mathematical model in [5]).
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Overview Infra. Commons Standalone Cluster PI
Priority Model
Following G4, three priority levels are introduced (used by Real-time schedulerfollowing Algorithm 3 on paper)
High - Cannot be interrupted by GC
Mid - Can be interrupted by the GC
Low - Can be interrupted by the GC, Highest priority level available on StandardJava
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Overview Infra. Commons Standalone Cluster PI
Real-time Scheduler
Schedules requests based on their deadlines
Follows Earliest Deadline First (EDF) technique
Uses the priority model on threads to decide which request isexecuted at what time
..and follows G2 in doing so.
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Overview Infra. Commons Standalone Cluster PI
Enhancements made to Axis2
Details
Thread-pools have been replaced with RT-Threadpools
A Real-time scheduler has been introduced
Multiple execution lanes are used
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Overview Infra. Commons Standalone Cluster PI
Enhancements made to Synapse
Details
RT-Axis2 instances used as Executors
Enhanced version of Synapse used for dispatching
Thread-pools have been replaced with RT-Thread poolsA Real-time scheduler has been introducedMultiple execution lanes are used
Request processing happens at Dispatcher, Service Invocation at Executors
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Overview Infra. Commons Standalone Cluster PI
Minimising Priority Inversions
Can be caused by, I/O activities such as reading/writing to files and sockets.
To prevent priority inversions (following G5),
Avoid outputs resulting in on-screen messages or log file writes
Use in-memory logging and delayed writes
Use offline debugging techniques/toolse.g. Oracle Thread Scheduling Visualizer (TSV) [16]
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Overview Infra. Commons Standalone Cluster PI
Minimising Priority Inversions - TSV
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Stand Alone Cluster
1 Introduction
2 Guidelines for Predictability
3 Case Study
4 Empirical Results
5 Conclusion
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Stand Alone Cluster
Stand-alone Engine performance
10
20
30
40
50
60
70
80
90
100
0.1s - 0.25s 0.1s - 0.5s 0.25s - 1s 0.25s - 2s
Per
cent
age
of d
eadl
ines
met
off
perc
enta
ge a
ccep
ted
Inter-arrival times
Comparison of Unmod. Axis2 vs RT-Axis2 - Deadline Achievement Rates
Unmod. Axis2RT-Axis2
RT-Axis2 accepts between 18.1% (fastest) and 96.7% (slowest) of the requests.
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Stand Alone Cluster
Stand-alone Engine performance
0
20000
40000
60000
80000
100000
120000
140000
160000
180000
0 500000 1e+06 1.5e+06 2e+06 2.5e+06 3e+06 3.5e+06 4e+06 4.5e+06 5e+06
Exe
cutio
n T
ime
(ms)
Task Size
1 - 5000000 (Uniform); 0.25sec - 1sec (Uniform)
Real-tme Axis2Unmodified Axis2
0
50000
100000
150000
200000
250000
0 500000 1e+06 1.5e+06 2e+06 2.5e+06 3e+06 3.5e+06 4e+06 4.5e+06 5e+06E
xecu
tion
Tim
e (m
s)
Task Size
1 - 5000000 (Uniform); 0.1sec - 0.5sec (Uniform)
Real-tme Axis2Unmodified Axis2
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Stand Alone Cluster
RT-RoundRobin vs Round-Robin
10
20
30
40
50
60
70
80
90
100
0.05s � 0.1s 0.1s � 0.25s 0.1s - 0.5s 0.25s - 1s
Per
cent
age
of d
eadl
ines
met
Inter-arrival times
Comparison of Round-robin vs. RT-RoundRobin - Deadline Achievement Rates
RR - 2 Execs.RR - 3 Execs.RR - 4 Execs.
RT-RR - 2 Execs.RT-RR - 3 Execs.RT-RR - 4 Execs.
RT-RR accepts between 20.5% (2 Exec. - fastest) and 99.9% (4 Exec. -slowest) of the requests.
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Stand Alone Cluster
RT-RoundRobin vs Round-Robin
0
20000
40000
60000
80000
100000
120000
0 500000 1e+06 1.5e+06 2e+06 2.5e+06 3e+06 3.5e+06 4e+06 4.5e+06 5e+06
Exe
cutio
n T
ime
(ms)
Task Size
Comparison of execution times (1 - 5000000 (Uniform); 0.1sec - 0.25sec (Uniform) roundrobin 3 exec)
RT-RoundRobinSimple Round-Robin
0
2000
4000
6000
8000
10000
12000
0 500000 1e+06 1.5e+06 2e+06 2.5e+06 3e+06 3.5e+06 4e+06 4.5e+06 5e+06E
xecu
tion
Tim
e (m
s)Task Size
Task Size distribution between 3 Executors (1 - 5000000 (Uniform); 0.1sec - 0.5sec (Uniform) roundrobin 3 exec)
Executor 1Executor 2Executor 3
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Future Work Recap
1 Introduction
2 Guidelines for Predictability
3 Case Study
4 Empirical Results
5 Conclusion
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Future Work Recap
Future Work
Further analysis of cluster behaviour with mix of tasks
Advanced dispatching and re-dispatching models
Predictability in the networking layer
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Future Work Recap
1 Introduction
2 Guidelines for Predictability
3 Case Study
4 Empirical Results
5 Conclusion
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings
Intro. Guidelines Case Study Results Conclusion Future Work Recap
Conclusion
With the growing popularity of Cloud Computing and it’sservice offerings, Execution level QoS has become animportant aspect
Existing WS-Middleware are optimized for throughput therebylacks predictability of execution
Presented Guidelines for WS-Middleware to achievepredictability of execution
Discussed 2 case studies as examples
Presented empirical data, on evaluating the twoimplementations
V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings