Mohammed Abouzour, Kenneth Salem, Peter Bumbulis Presentation by Mohammed Abouzour SMDB2010.
-
Upload
erick-shelton -
Category
Documents
-
view
216 -
download
0
Transcript of Mohammed Abouzour, Kenneth Salem, Peter Bumbulis Presentation by Mohammed Abouzour SMDB2010.
![Page 1: Mohammed Abouzour, Kenneth Salem, Peter Bumbulis Presentation by Mohammed Abouzour SMDB2010.](https://reader035.fdocuments.in/reader035/viewer/2022062515/56649d1f5503460f949f23db/html5/thumbnails/1.jpg)
AUTOMATIC TUNING OF THE
MULTIPROGRAMMING LEVEL IN
SYBASE SQL ANYWHERE
Mohammed Abouzour, Kenneth Salem, Peter Bumbulis
Presentation by Mohammed Abouzour
SMDB2010
![Page 2: Mohammed Abouzour, Kenneth Salem, Peter Bumbulis Presentation by Mohammed Abouzour SMDB2010.](https://reader035.fdocuments.in/reader035/viewer/2022062515/56649d1f5503460f949f23db/html5/thumbnails/2.jpg)
2
INTRODUCTION TO PROBLEM Database servers have two main
architectures:Worker-per-connection:
(Oracle, Sybase ASE) A worker is dedicated for each connection for the
life of the connectionWorker-per-request:
(Oracle, Microsoft SQL Server, Sybase ASE, Sybase SQL Anywhere) A worker pool and a request queue: each worker
picks and complete one request at a time. No guarantee that the same worker will service the same connection.
![Page 3: Mohammed Abouzour, Kenneth Salem, Peter Bumbulis Presentation by Mohammed Abouzour SMDB2010.](https://reader035.fdocuments.in/reader035/viewer/2022062515/56649d1f5503460f949f23db/html5/thumbnails/3.jpg)
3
WORKER-PER-REQUEST ARCHITECTURE: Efficient in resource use and handling of
large number of connections(1)
How to choose the size of the worker pool?A large worker pool:
Increases the concurrency level of the server Increases contention on server resources Increases working set size of server
A smaller worker pool: Under utilization of hardware resources Limit concurrency level of workload Possibility of a server hang due to no workers
available to handle outstanding requests
![Page 4: Mohammed Abouzour, Kenneth Salem, Peter Bumbulis Presentation by Mohammed Abouzour SMDB2010.](https://reader035.fdocuments.in/reader035/viewer/2022062515/56649d1f5503460f949f23db/html5/thumbnails/4.jpg)
4
OUR SOLUTION Dynamically adjust the size of the
worker pool (the multiprogramming level of the server)
Benefits of dynamic MPL:One less parameter for DBAs to worry about
One of the design goals of Sybase SQL Anywhere Improve server performance for different
workloadsBetter handling of changes in workload
transaction mix We used throughput as a way to detect
degraded server performance
![Page 5: Mohammed Abouzour, Kenneth Salem, Peter Bumbulis Presentation by Mohammed Abouzour SMDB2010.](https://reader035.fdocuments.in/reader035/viewer/2022062515/56649d1f5503460f949f23db/html5/thumbnails/5.jpg)
5
OUTLINE Introduction to Problem Controller Design and Tuning Algorithms Experimental Study Results Future Work
![Page 6: Mohammed Abouzour, Kenneth Salem, Peter Bumbulis Presentation by Mohammed Abouzour SMDB2010.](https://reader035.fdocuments.in/reader035/viewer/2022062515/56649d1f5503460f949f23db/html5/thumbnails/6.jpg)
6
CONTROLLER DESIGN
![Page 7: Mohammed Abouzour, Kenneth Salem, Peter Bumbulis Presentation by Mohammed Abouzour SMDB2010.](https://reader035.fdocuments.in/reader035/viewer/2022062515/56649d1f5503460f949f23db/html5/thumbnails/7.jpg)
7
CONTROLLER DESIGN
![Page 8: Mohammed Abouzour, Kenneth Salem, Peter Bumbulis Presentation by Mohammed Abouzour SMDB2010.](https://reader035.fdocuments.in/reader035/viewer/2022062515/56649d1f5503460f949f23db/html5/thumbnails/8.jpg)
8
TUNING ALGORITHMS Tuning algorithms are based on a paper by
Heiss and Wagner(2): Incremental Steps algorithm (IS) Parabola Approximation (PA)
Original algorithms were studied in simulation
Our contribution to the original algorithms(3): We have extended the IS algorithm We have used a commercial database system
and an industry standard benchmark (TPC-C) We have studied the behaviour of the above two
algorithms under different workloads
![Page 9: Mohammed Abouzour, Kenneth Salem, Peter Bumbulis Presentation by Mohammed Abouzour SMDB2010.](https://reader035.fdocuments.in/reader035/viewer/2022062515/56649d1f5503460f949f23db/html5/thumbnails/9.jpg)
9
NOTATION
Notation Description
ti End of current control interval and start of the next control interval ti+1
P(ti) The actual measured throughput level of the server at time ti. Throughput is the number of request completed during the previous measurement interval
n(ti) The workload concurrency level at time ti
n*(ti) The number of workers the server has available.
![Page 10: Mohammed Abouzour, Kenneth Salem, Peter Bumbulis Presentation by Mohammed Abouzour SMDB2010.](https://reader035.fdocuments.in/reader035/viewer/2022062515/56649d1f5503460f949f23db/html5/thumbnails/10.jpg)
10
HILL CLIMBING ALGORITHM Original IS algorithm did not handle a throughput
curve that flattens out Modified the IS algorithm as follows:
Follow the throughput curve up as long as percentage of change in throughput is greater than C times the percentage of change in number of threads
Follow the throughput curve down as long as throughput increases Built-in bias to keep the number of threads as low as possible
![Page 11: Mohammed Abouzour, Kenneth Salem, Peter Bumbulis Presentation by Mohammed Abouzour SMDB2010.](https://reader035.fdocuments.in/reader035/viewer/2022062515/56649d1f5503460f949f23db/html5/thumbnails/11.jpg)
11
GLOBAL PARABOLA APPROXIMATION ALGORITHM Model the throughput curve as a
parabolic function: Collect two points from the last two
measurements and use the origin Only make adjustments if the computed
parabolic curve is concave down, otherwise just make a random adjustment
![Page 12: Mohammed Abouzour, Kenneth Salem, Peter Bumbulis Presentation by Mohammed Abouzour SMDB2010.](https://reader035.fdocuments.in/reader035/viewer/2022062515/56649d1f5503460f949f23db/html5/thumbnails/12.jpg)
12
EXPERIMENTAL STUDY: Used the TPC-C workload for our benchmark Used two workloads variations:
TPCC1: Standard TPC-C transaction mix TPCC2: Modified TPC-C transaction mix
(lighter on I/O and higher CPU component) Two different buffer pool configurations:
BIGMEM: 1.0GB SMALLMEM: 0.5GB
Three different workload configurations: TPCC1-BIGMEM TPCC1-SMALLMEM TPCC2-BIGMEM
![Page 13: Mohammed Abouzour, Kenneth Salem, Peter Bumbulis Presentation by Mohammed Abouzour SMDB2010.](https://reader035.fdocuments.in/reader035/viewer/2022062515/56649d1f5503460f949f23db/html5/thumbnails/13.jpg)
13
EXPERIMENTAL STUDY:METHODOLOGY Workload characterization experiments:
Ran each workload under different fixed MPL setting and generated a throughput curve for each workload
Experiments:Auto-tuning experiments:
We ran each of the workloads and allowed the auto tuning algorithms to adjust MPL dynamically
Changing workloads experiments: We start with one workload and half way in the
experiment we switch to a different workload
![Page 14: Mohammed Abouzour, Kenneth Salem, Peter Bumbulis Presentation by Mohammed Abouzour SMDB2010.](https://reader035.fdocuments.in/reader035/viewer/2022062515/56649d1f5503460f949f23db/html5/thumbnails/14.jpg)
14
WORKLOAD CHARACTERISTICS:
TPCC1-BIGMEM TPCC1-SMALLMEM
![Page 15: Mohammed Abouzour, Kenneth Salem, Peter Bumbulis Presentation by Mohammed Abouzour SMDB2010.](https://reader035.fdocuments.in/reader035/viewer/2022062515/56649d1f5503460f949f23db/html5/thumbnails/15.jpg)
15
TPCC1-BIGMEM
Hill ClimbingGlobal Parabola Approximation
![Page 16: Mohammed Abouzour, Kenneth Salem, Peter Bumbulis Presentation by Mohammed Abouzour SMDB2010.](https://reader035.fdocuments.in/reader035/viewer/2022062515/56649d1f5503460f949f23db/html5/thumbnails/16.jpg)
16
TPCC1-SMALLMEM
Hill ClimbingGlobal Parabola Approximation
![Page 17: Mohammed Abouzour, Kenneth Salem, Peter Bumbulis Presentation by Mohammed Abouzour SMDB2010.](https://reader035.fdocuments.in/reader035/viewer/2022062515/56649d1f5503460f949f23db/html5/thumbnails/17.jpg)
17
TPCC1-BIGMEM TO TPCC1-SMALLMEM
Hill ClimbingGlobal Parabola Approximation
![Page 18: Mohammed Abouzour, Kenneth Salem, Peter Bumbulis Presentation by Mohammed Abouzour SMDB2010.](https://reader035.fdocuments.in/reader035/viewer/2022062515/56649d1f5503460f949f23db/html5/thumbnails/18.jpg)
18
LESSONS LEARNED Not all workloads in database systems have
hill-shaped throughput curve Both algorithms were able to react
appropriately to workloads changes The Hill Climbing algorithm:
Smoother in making adjustments (more desirable) Takes longer to reach optimal values Can get stuck in a local minima
The Global Parabola approximation algorithm: Takes variable step sizes and hence reacts faster
to changes Very sensitive to normal minor workload
fluctuations
![Page 19: Mohammed Abouzour, Kenneth Salem, Peter Bumbulis Presentation by Mohammed Abouzour SMDB2010.](https://reader035.fdocuments.in/reader035/viewer/2022062515/56649d1f5503460f949f23db/html5/thumbnails/19.jpg)
19
PRACTICAL EXPERIENCE WITH SQL ANYWHERE 12 A hybrid controller:
Uses the Hill Climbing algorithm for a number of steps and collects data
Uses the data collected with the Parabola Approximation algorithm
We reduced the control interval length Hill Climbing modifications:
Take dynamic step sizes to speed up algorithm C parameter reduced to make algorithm more aggressive
Parabola Approximation modifications: Use least-square method to do curve fitting on collected
data Results:
90-95% of optimal tpmC compared to a hand tuned version In other internal CPU-bound benchmark was able to
improve performance by 32%
![Page 20: Mohammed Abouzour, Kenneth Salem, Peter Bumbulis Presentation by Mohammed Abouzour SMDB2010.](https://reader035.fdocuments.in/reader035/viewer/2022062515/56649d1f5503460f949f23db/html5/thumbnails/20.jpg)
20
CONCLUSION AND FUTURE WORK Dynamically adjusting the MPL:
Improves server performance in some workloads compared to a fixed MPL
Handles changes to workload conditions effectively Brings us one step closer to a self managing database system
Algorithm parameters can affect the performance of the algorithms: Control interval length Hill Climbing:
Step sizes C parameter
The algorithms did not behave good for the following workloads: Bursty workloads: busy periods followed by idle periods
followed by busy periods Long running requests: e.g. creating an index
Use response time to help the auto tuning algorithm
![Page 21: Mohammed Abouzour, Kenneth Salem, Peter Bumbulis Presentation by Mohammed Abouzour SMDB2010.](https://reader035.fdocuments.in/reader035/viewer/2022062515/56649d1f5503460f949f23db/html5/thumbnails/21.jpg)
21
REFERENCES
1. B. Schroeder, M. Harchol-Balter, A. Iyengar, E. Nahum, and A. Wierman, “How to determine a good multi-programming level for external scheduling,” in Proceedings of the 22nd International Conference on Data Engineering. Washington, DC, USA: IEEE Computer Society, 2006, p. 60.
2. H.-U. Heiss and R. Wagner. “Adaptive load control in transaction processing systems,” in VLDB ‘91: Proceedings of the 17th International Conference on Very Large Data Bases. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 1991, pp. 47-54.
3. M. Abouzour, “Automatically tuning database server multiprogramming level,” Master’s thesis, University of Waterloo, 2007
![Page 22: Mohammed Abouzour, Kenneth Salem, Peter Bumbulis Presentation by Mohammed Abouzour SMDB2010.](https://reader035.fdocuments.in/reader035/viewer/2022062515/56649d1f5503460f949f23db/html5/thumbnails/22.jpg)
22
QUESTIONS
![Page 23: Mohammed Abouzour, Kenneth Salem, Peter Bumbulis Presentation by Mohammed Abouzour SMDB2010.](https://reader035.fdocuments.in/reader035/viewer/2022062515/56649d1f5503460f949f23db/html5/thumbnails/23.jpg)
23
TPC-C CONFIGURATION 150 Warehouses 10 clients per warehouse 500 connections C = 0.4, Delta = 10 Gamma=0.333, Delta = 10 Windows 2003 32-bit Dual 1.80Ghz Intel Xeon 48 Spindles 10K RPM SCSI drives