auto-tuning database kernels 2 - Harvard...
Transcript of auto-tuning database kernels 2 - Harvard...
![Page 1: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/1.jpg)
auto-tuning database kernels 2.0prof. Stratos Idreos
HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/
class 22
![Page 2: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/2.jpg)
Stratos Idreos
a bit more about auto-tuning db kernels
and then a couple of open research topics
![Page 3: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/3.jpg)
Stratos Idreos
storage indexing queryschema loadX X X
be able to query the data immediately & with good performance
raw data explore data and gain knowledge “immediately”
![Page 4: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/4.jpg)
Stratos Idreos
database cracking
![Page 5: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/5.jpg)
Stratos Idreos
database cracking
idle timeworkload knowledge
external tools
human control
![Page 6: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/6.jpg)
Stratos Idreos
database crackingauto-tuning database kernels
incremental, adaptive, partial indexing
idle timeworkload knowledge
external tools
human control
![Page 7: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/7.jpg)
Stratos Idreos
database crackingauto-tuning database kernels
incremental, adaptive, partial indexing
indexing
initialization querying
idle timeworkload knowledge
external tools
human control
![Page 8: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/8.jpg)
Stratos Idreos
database crackingauto-tuning database kernels
incremental, adaptive, partial indexing
indexing
initialization querying
idle timeworkload knowledge
external tools
human control
![Page 9: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/9.jpg)
Stratos Idreos
database crackingauto-tuning database kernels
incremental, adaptive, partial indexing
indexing
initialization querying
idle timeworkload knowledge
external tools
human control
every query is treated as an advice on how data should be stored
![Page 10: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/10.jpg)
Stratos Idreos
13 16 4 9 2 12 7 1 19 3 14 11 8 6
column AQ1: select R.A from R where R.A>10 and R.A<14
Database Cracking CIDR 2007
![Page 11: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/11.jpg)
Stratos Idreos
13 16 4 9 2 12 7 1 19 3 14 11 8 6
column AQ1: select R.A from R where R.A>10 and R.A<14
4 9 2 7 1 3 8 6 13 12 11 16 19 14
piece1: A<=10
Database Cracking CIDR 2007
![Page 12: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/12.jpg)
Stratos Idreos
13 16 4 9 2 12 7 1 19 3 14 11 8 6
column AQ1: select R.A from R where R.A>10 and R.A<14
4 9 2 7 1 3 8 6 13 12 11 16 19 14
piece1: A<=10
piece2: 10<A<14
Database Cracking CIDR 2007
![Page 13: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/13.jpg)
Stratos Idreos
13 16 4 9 2 12 7 1 19 3 14 11 8 6
column AQ1: select R.A from R where R.A>10 and R.A<14
4 9 2 7 1 3 8 6 13 12 11 16 19 14
piece1: A<=10
piece2: 10<A<14
piece3: A>=14
Database Cracking CIDR 2007
![Page 14: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/14.jpg)
Stratos Idreos
13 16 4 9 2 12 7 1 19 3 14 11 8 6
column AQ1: select R.A from R where R.A>10 and R.A<14
4 9 2 7 1 3 8 6 13 12 11 16 19 14
piece1: A<=10
piece2: 10<A<14
piece3: A>=14
resu
lt
Database Cracking CIDR 2007
![Page 15: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/15.jpg)
Stratos Idreos
13 16 4 9 2 12 7 1 19 3 14 11 8 6
column AQ1: select R.A from R where R.A>10 and R.A<14
4 9 2 7 1 3 8 6 13 12 11 16 19 14
piece1: A<=10
piece2: 10<A<14
piece3: A>=14
resu
lt
gain knowledge on how data is organized
Database Cracking CIDR 2007
![Page 16: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/16.jpg)
Stratos Idreos
13 16 4 9 2 12 7 1 19 3 14 11 8 6
column AQ1: select R.A from R where R.A>10 and R.A<14
4 9 2 7 1 3 8 6 13 12 11 16 19 14
piece1: A<=10
piece2: 10<A<14
piece3: A>=14
dynamically/on-the-fly within the select-operator
resu
lt
gain knowledge on how data is organized
Database Cracking CIDR 2007
![Page 17: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/17.jpg)
Stratos Idreos
13 16 4 9 2 12 7 1 19 3 14 11 8 6
column AQ1: select R.A from R where R.A>10 and R.A<14
4 9 2 7 1 3 8 6 13 12 11 16 19 14
piece1: A<=10
piece2: 10<A<14
piece3: A>=14
Q2: select R.A from R where R.A>7 and R.A<=16
dynamically/on-the-fly within the select-operatorDatabase Cracking CIDR 2007
![Page 18: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/18.jpg)
Stratos Idreos
13 16 4 9 2 12 7 1 19 3 14 11 8 6
column AQ1: select R.A from R where R.A>10 and R.A<14
4 9 2 7 1 3 8 6 13 12 11 16 19 14
piece1: A<=10
piece2: 10<A<14
piece3: A>=14
Q2: select R.A from R where R.A>7 and R.A<=16
dynamically/on-the-fly within the select-operatorDatabase Cracking CIDR 2007
![Page 19: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/19.jpg)
Stratos Idreos
13 16 4 9 2 12 7 1 19 3 14 11 8 6
column AQ1: select R.A from R where R.A>10 and R.A<14
4 9 2 7 1 3 8 6 13 12 11 16 19 14
piece1: A<=10
piece2: 10<A<14
piece3: A>=14
4 2 1 3 6 7 9 8 13 12 11 14 16 19
piece1: A<=7
piece2: 7<A<=10Q2: select R.A from R where R.A>7 and R.A<=16
dynamically/on-the-fly within the select-operatorDatabase Cracking CIDR 2007
![Page 20: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/20.jpg)
Stratos Idreos
13 16 4 9 2 12 7 1 19 3 14 11 8 6
column AQ1: select R.A from R where R.A>10 and R.A<14
4 9 2 7 1 3 8 6 13 12 11 16 19 14
piece1: A<=10
piece2: 10<A<14
piece3: A>=14
4 2 1 3 6 7 9 8 13 12 11 14 16 19
piece1: A<=7
piece2: 7<A<=10
piece3: 10<A<14Q2: select R.A from R where R.A>7 and R.A<=16
dynamically/on-the-fly within the select-operatorDatabase Cracking CIDR 2007
![Page 21: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/21.jpg)
Stratos Idreos
13 16 4 9 2 12 7 1 19 3 14 11 8 6
column AQ1: select R.A from R where R.A>10 and R.A<14
4 9 2 7 1 3 8 6 13 12 11 16 19 14
piece1: A<=10
piece2: 10<A<14
piece3: A>=14
4 2 1 3 6 7 9 8 13 12 11 14 16 19
piece1: A<=7
piece2: 7<A<=10
piece3: 10<A<14
piece4: 14<=A<=16
piece5: A>16
Q2: select R.A from R where R.A>7 and R.A<=16
dynamically/on-the-fly within the select-operatorDatabase Cracking CIDR 2007
![Page 22: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/22.jpg)
Stratos Idreos
13 16 4 9 2 12 7 1 19 3 14 11 8 6
column AQ1: select R.A from R where R.A>10 and R.A<14
4 9 2 7 1 3 8 6 13 12 11 16 19 14
piece1: A<=10
piece2: 10<A<14
piece3: A>=14
4 2 1 3 6 7 9 8 13 12 11 14 16 19
piece1: A<=7
piece2: 7<A<=10
piece3: 10<A<14
piece4: 14<=A<=16
piece5: A>16
Q2: select R.A from R where R.A>7 and R.A<=16
dynamically/on-the-fly within the select-operator
resu
lt
Database Cracking CIDR 2007
![Page 23: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/23.jpg)
Stratos Idreos
13 16 4 9 2 12 7 1 19 3 14 11 8 6
column AQ1: select R.A from R where R.A>10 and R.A<14
4 9 2 7 1 3 8 6 13 12 11 16 19 14
piece1: A<=10
piece2: 10<A<14
piece3: A>=14
4 2 1 3 6 7 9 8 13 12 11 14 16 19
piece1: A<=7
piece2: 7<A<=10
piece3: 10<A<14
piece4: 14<=A<=16
piece5: A>16
Q2: select R.A from R where R.A>7 and R.A<=16
dynamically/on-the-fly within the select-operator
resu
lt
the more we crack, the more we learn
Database Cracking CIDR 2007
![Page 24: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/24.jpg)
Stratos Idreos
100K random selections random selectivity random value ranges in a 10 million integer column
set-upScan
Full Index
Crack
0.001
0.01
0.1
1
10
100
1000
1 10 100 1000 10000 100000
Resp
onse
tim
e (
secs
)
Query sequence (x1000)
continuous adaptation
Database Cracking CIDR 2007
![Page 25: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/25.jpg)
Stratos Idreos
100K random selections random selectivity random value ranges in a 10 million integer column
almost no initialization overhead
set-upScan
Full Index
Crack
0.001
0.01
0.1
1
10
100
1000
1 10 100 1000 10000 100000
Resp
onse
tim
e (
secs
)
Query sequence (x1000)
continuous adaptation
Database Cracking CIDR 2007
![Page 26: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/26.jpg)
Stratos Idreos
100K random selections random selectivity random value ranges in a 10 million integer column
almost no initialization overhead
continuous improvement
set-upScan
Full Index
Crack
0.001
0.01
0.1
1
10
100
1000
1 10 100 1000 10000 100000
Resp
onse
tim
e (
secs
)
Query sequence (x1000)
continuous adaptation
Database Cracking CIDR 2007
![Page 27: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/27.jpg)
Stratos Idreos
100K random selections random selectivity random value ranges in a 10 million integer column
almost no initialization overhead
continuous improvement
set-upScan
Full Index
Crack
0.001
0.01
0.1
1
10
100
1000
1 10 100 1000 10000 100000
Resp
onse
tim
e (
secs
)
Query sequence (x1000)
continuous adaptation
Database Cracking CIDR 2007
![Page 28: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/28.jpg)
cracking databases
updates (SIGMOD07)
storage-restrictions (SIGMOD09)
benchmarking (TPCTC10)
concurrency control
(PVLDB12)
algorithms (PVLDB11)
basics (CIDR07)
multi-cores (SIGMOD15)
hadoop (Yale/Saarland)
b-trees (HP Labs)
>1 columns (SIGMOD09)
robustness (PVLDB12)
adaptive storage
(SIGMOD14)
time-series (SIGMOD14)
encryption (SIGMOD16)
![Page 29: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/29.jpg)
Stratos Idreos
concurrency control
Concurrency Control, PVLDB 12
problem: read queries become write queries! (?)
goal: be able to crack for multiple queries in parallel
![Page 30: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/30.jpg)
Stratos Idreos
traditional indexing adaptive indexing
Concurrency Control, PVLDB 12
![Page 31: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/31.jpg)
Stratos Idreos
traditional indexing adaptive indexingwrite queries read queries
Concurrency Control, PVLDB 12
![Page 32: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/32.jpg)
Stratos Idreos
traditional indexing adaptive indexing
change index contents and structure
only index structure changes
write queries read queries
Concurrency Control, PVLDB 12
![Page 33: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/33.jpg)
Stratos Idreos
traditional indexing adaptive indexing
change index contents and structure
only index structure changes
no need for traditional locks = too heavy
short term latches = fast and release quickly
write queries read queries
Concurrency Control, PVLDB 12
![Page 34: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/34.jpg)
Stratos Idreos
traditional indexing adaptive indexing
all or nothing incremental and optional
change index contents and structure
only index structure changes
write queries read queries
Concurrency Control, PVLDB 12
![Page 35: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/35.jpg)
Stratos Idreos
traditional indexing adaptive indexing
all or nothing incremental and optional
change index contents and structure
only index structure changes
v1 v2 v3
write queries read queries
Concurrency Control, PVLDB 12
![Page 36: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/36.jpg)
Stratos Idreos
traditional indexing adaptive indexing
all or nothing incremental and optional
change index contents and structure
only index structure changes
v1 v2 v3
a b
write queries read queries
Concurrency Control, PVLDB 12
![Page 37: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/37.jpg)
Stratos Idreos
traditional indexing adaptive indexing
all or nothing incremental and optional
change index contents and structure
only index structure changes
v1 v2 v3 v1 v2 v3
a b
a b
write queries read queries
Concurrency Control, PVLDB 12
![Page 38: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/38.jpg)
Stratos Idreos
traditional indexing adaptive indexing
all or nothing incremental and optional
change index contents and structure
only index structure changes
v1 v2 v3 v1 v2 v3
a b
a b
write queries read queries
Concurrency Control, PVLDB 12
![Page 39: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/39.jpg)
Stratos Idreos
traditional indexing adaptive indexing
all or nothing incremental and optional
change index contents and structure
only index structure changes
v1 v2 v3 v1 v2 v3
a b
a b
v1 v2 v3a
write queries read queries
Concurrency Control, PVLDB 12
![Page 40: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/40.jpg)
Stratos Idreos
traditional indexing adaptive indexing
all or nothing incremental and optional
change index contents and structure
only index structure changes
v1 v2 v3 v1 v2 v3
a b
a b
v1 v2 v3a
write queries read queries
Concurrency Control, PVLDB 12
![Page 41: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/41.jpg)
Stratos Idreos
traditional indexing adaptive indexing
all or nothing incremental and optional
impact stable storage stable storage optional
change index contents and structure
only index structure changes
write queries read queries
Concurrency Control, PVLDB 12
![Page 42: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/42.jpg)
Stratos Idreos
traditional indexing adaptive indexing
all or nothing incremental and optional
impact stable storage stable storage optional
change index contents and structure
only index structure changes
v1 v2 v3 v1 v2 v3a b
write queries read queries
a b
Concurrency Control, PVLDB 12
![Page 43: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/43.jpg)
Stratos Idreos
traditional indexing adaptive indexing
all or nothing incremental and optional
impact stable storage stable storage optional
change index contents and structure
only index structure changes
need to serialize can execute in any order
write queries read queries
Concurrency Control, PVLDB 12
![Page 44: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/44.jpg)
Stratos Idreos
select min(C) from R where A<10 & B<20
diskA B C D
Concurrency Control, PVLDB 12
![Page 45: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/45.jpg)
Stratos Idreos
select min(C) from R where A<10 & B<20
A<10disk memory
A B C D
Concurrency Control, PVLDB 12
![Page 46: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/46.jpg)
Stratos Idreos
select min(C) from R where A<10 & B<20
A<10disk memory
A B C D IDs
Concurrency Control, PVLDB 12
![Page 47: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/47.jpg)
Stratos Idreos
select min(C) from R where A<10 & B<20
A<10disk memory
A B C D IDs B
Concurrency Control, PVLDB 12
![Page 48: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/48.jpg)
Stratos Idreos
select min(C) from R where A<10 & B<20
B<20A<10disk memory
A B C D IDs B
Concurrency Control, PVLDB 12
![Page 49: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/49.jpg)
Stratos Idreos
select min(C) from R where A<10 & B<20
B<20A<10disk memory
A B C D IDs B CIDs
Concurrency Control, PVLDB 12
![Page 50: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/50.jpg)
Stratos Idreos
select min(C) from R where A<10 & B<20
B<20 minCA<10disk memory
A B C D IDs B CIDs
Concurrency Control, PVLDB 12
![Page 51: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/51.jpg)
Stratos Idreos
select min(C) from R where A<10 & B<20
B<20 minCA<10disk memory
A B C D IDs B CIDs
column lock and release as soon as an operator completes
Concurrency Control, PVLDB 12
![Page 52: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/52.jpg)
Stratos Idreos
need to latch only to be cracked pieces (max 2 per select)
Concurrency Control, PVLDB 12
select [a,b]
![Page 53: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/53.jpg)
Stratos Idreos
piece locking
avl-tree crack column
Concurrency Control, PVLDB 12
![Page 54: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/54.jpg)
Stratos Idreos
piece locking
avl-tree crack columncrack select
Concurrency Control, PVLDB 12
![Page 55: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/55.jpg)
Stratos Idreos
piece locking
avl-tree crack columncrack selectwlock
Concurrency Control, PVLDB 12
![Page 56: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/56.jpg)
Stratos Idreos
piece locking
avl-tree crack columncrack select
wlock
Concurrency Control, PVLDB 12
![Page 57: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/57.jpg)
Stratos Idreos
piece locking
avl-tree crack column
Concurrency Control, PVLDB 12
![Page 58: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/58.jpg)
Stratos Idreos
piece locking
avl-tree crack columnmax
Concurrency Control, PVLDB 12
![Page 59: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/59.jpg)
Stratos Idreos
piece locking
avl-tree crack columnmaxrlock
Concurrency Control, PVLDB 12
![Page 60: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/60.jpg)
Stratos Idreos
piece locking
avl-tree crack columnmax
rlock
Concurrency Control, PVLDB 12
![Page 61: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/61.jpg)
Stratos Idreos
piece locking
avl-tree crack columnmax
rlock
Concurrency Control, PVLDB 12
![Page 62: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/62.jpg)
Stratos Idreos
piece locking
avl-tree crack columnmax
rlock
Concurrency Control, PVLDB 12
![Page 63: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/63.jpg)
Stratos Idreos
avl-tree crack column1090
140
200
300
piece locking
![Page 64: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/64.jpg)
Stratos Idreos
avl-tree crack column crack select1090
140
200
300
65
230
piece locking
![Page 65: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/65.jpg)
Stratos Idreos
avl-tree crack column crack selectwlock
wlock
1090
140
200
300
65
230
piece locking
![Page 66: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/66.jpg)
Stratos Idreos
avl-tree crack column crack selectwlock
wlock
r/wlock
1090
140
200
300
65
230
piece locking
![Page 67: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/67.jpg)
Stratos Idreos
avl-tree crack column crack selectwlock
wlock
q1,q2,q3...qn1090
140
200
300
65
230
10
90
piece locking
![Page 68: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/68.jpg)
Stratos Idreos
avl-tree crack column crack selectwlock
wlock
q1,q2,q3...qn1090
140
200
300
65
230
10
90
2030
7080
piece locking
![Page 69: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/69.jpg)
Stratos Idreos
avl-tree crack column crack selectwlock
wlock
q1,q2,q3...qn1090
140
200
300
65
230
10
90
2030
7080
wlock
piece locking
![Page 70: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/70.jpg)
Stratos Idreos
0
10
20
30
40
50
60
70
80
1 2 4 8 16 32
Tota
l Tim
e for
1024 Q
ueries
(secs
)
Number of Concurrent Clients
scansort
crack
300
0
50
100
150
200
250
350
1 2 4 8 16 32
Thro
ughput ove
r 1024 Q
ueries
(Queries
/ se
c)
Number of Concurrent Clients
scansort
crack
4.5
4.55
4.6
4.65
4.7
4.75
4.8
Tota
l Tim
e for
1024 Q
ueries
(secs
)
(Sequential Execution)
Concurrency Control
Enabled Disabled
Rows: 100M Query: select sum(A) from R where v1 < A1 < v2 Selectivity: 0.01%, Random, # of queries: 1024 Clients: 1-32, Machine: 4 cores
adaptive indexing maintains its performance advantage
Concurrency Control, PVLDB 12
![Page 71: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/71.jpg)
Stratos Idreos
crack time and wait time
10-6
10-5
10-4
10-3
10-2
10-1
100
1 10 100 1000
Tim
e p
er
query
(se
cs)
Query sequence
index refinementwait time
Query: select sum(A) from R where v1 < A1 < v2 Selectivity: 0.01%, Random, # of queries: 1024 Clients: 8, Machine: 4 cores
adaptive indexing maintains
the adaptive behavior
Concurrency Control, PVLDB 12
![Page 72: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/72.jpg)
Stratos Idreos
stochastic cracking robustness
Stochastic Cracking, PVLDB 12
![Page 73: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/73.jpg)
Stratos Idreos
adaptive indexing
v4v5 v3 v2 v1
![Page 74: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/74.jpg)
Stratos Idreos
Response time: X
adaptive indexing
v4v5 v3 v2 v1
![Page 75: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/75.jpg)
Stratos Idreos
Response time: X
adaptive indexing
v4v5 v3v2 v1
![Page 76: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/76.jpg)
Stratos Idreos
Response time: X
adaptive indexing
Response time: 1000X
v4v5 v3v2 v1
![Page 77: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/77.jpg)
Stratos Idreos
v4v5 v3 v2 v1
Stochastic Cracking, PVLDB 12
![Page 78: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/78.jpg)
Stratos Idreos
v4v5 v3 v2 v1 v1 v2 v3 v4 v5
Stochastic Cracking, PVLDB 12
![Page 79: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/79.jpg)
Stratos Idreos
v4v5 v3 v2 v1
v1 v2 v3 v4 v5
Stochastic Cracking, PVLDB 12
![Page 80: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/80.jpg)
Stratos Idreos
v4v5 v3v2 v1
v1 v2 v3 v4 v5
Stochastic Cracking, PVLDB 12
![Page 81: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/81.jpg)
Stratos Idreos
v4v5 v3v2 v1
v1 v2 v3 v4 v5
v1 v2 v3 v4 v5
Stochastic Cracking, PVLDB 12
![Page 82: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/82.jpg)
Stratos Idreos Stochastic Cracking, PVLDB 12
![Page 83: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/83.jpg)
Stratos Idreos
select [15,55]
Stochastic Cracking, PVLDB 12
![Page 84: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/84.jpg)
Stratos Idreos
select [15,55]
Stochastic Cracking, PVLDB 12
![Page 85: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/85.jpg)
Stratos Idreos
10 20 30 40 50 60
select [15,55]
Stochastic Cracking, PVLDB 12
![Page 86: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/86.jpg)
Stratos Idreos
10 20 30 40 50 60
select [15,55]
select [15,55]
Stochastic Cracking, PVLDB 12
![Page 87: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/87.jpg)
Stratos Idreos
10 20 30 40 50 60
select [15,55]
select [15,55]
Stochastic Cracking, PVLDB 12
![Page 88: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/88.jpg)
Stratos Idreos
10 20 30 40 50 60
select [15,55]
select [15,55]
Stochastic Cracking, PVLDB 12
![Page 89: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/89.jpg)
Stratos Idreos
10 20 30 40 50 60
select [15,55]
select [15,55]
the amount of work for each query depends on the index state
the state of the index depends on past queries patterns
Stochastic Cracking, PVLDB 12
![Page 90: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/90.jpg)
Stratos Idreos
good sequence
column with 100 unique integers [1,100]
Stochastic Cracking, PVLDB 12
![Page 91: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/91.jpg)
Stratos Idreos
q1
good sequence
column with 100 unique integers [1,100]<50
Stochastic Cracking, PVLDB 12
![Page 92: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/92.jpg)
Stratos Idreos
Nq1
good sequence
column with 100 unique integers [1,100]<50
Stochastic Cracking, PVLDB 12
![Page 93: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/93.jpg)
Stratos Idreos
Nq1
good sequence
column with 100 unique integers [1,100]<50
Stochastic Cracking, PVLDB 12
![Page 94: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/94.jpg)
Stratos Idreos
Nq1q2
good sequence
column with 100 unique integers [1,100]<50<25
Stochastic Cracking, PVLDB 12
![Page 95: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/95.jpg)
Stratos Idreos
NN/2
q1q2
good sequence
column with 100 unique integers [1,100]<50<25
Stochastic Cracking, PVLDB 12
![Page 96: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/96.jpg)
Stratos Idreos
NN/2
q1q2
good sequence
column with 100 unique integers [1,100]<50<25
Stochastic Cracking, PVLDB 12
![Page 97: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/97.jpg)
Stratos Idreos
NN/2
q1q2 q3
good sequence
column with 100 unique integers [1,100]<50<25 <75
Stochastic Cracking, PVLDB 12
![Page 98: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/98.jpg)
Stratos Idreos
NN/2N/2
q1q2 q3
good sequence
column with 100 unique integers [1,100]<50<25 <75
Stochastic Cracking, PVLDB 12
![Page 99: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/99.jpg)
Stratos Idreos
NN/2N/2
q1q2 q3
good sequence
column with 100 unique integers [1,100]<50<25 <75
Stochastic Cracking, PVLDB 12
![Page 100: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/100.jpg)
Stratos Idreos
NN/2N/2
q1q2 q3
good sequence
column with 100 unique integers [1,100]<50<25 <75
Stochastic Cracking, PVLDB 12
![Page 101: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/101.jpg)
Stratos Idreos
NN/2N/2
q1q2 q3
good sequence
bad sequence
column with 100 unique integers [1,100]<50<25 <75
Stochastic Cracking, PVLDB 12
![Page 102: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/102.jpg)
Stratos Idreos
NN/2N/2
q1q2 q3
q1
good sequence
bad sequence
column with 100 unique integers [1,100]<50<25 <75
<2
Stochastic Cracking, PVLDB 12
![Page 103: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/103.jpg)
Stratos Idreos
NN/2N/2
q1q2 q3
q1N
good sequence
bad sequence
column with 100 unique integers [1,100]<50<25 <75
<2
Stochastic Cracking, PVLDB 12
![Page 104: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/104.jpg)
Stratos Idreos
NN/2N/2
q1q2 q3
q1N
good sequence
bad sequence
column with 100 unique integers [1,100]<50<25 <75
<2
Stochastic Cracking, PVLDB 12
![Page 105: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/105.jpg)
Stratos Idreos
NN/2N/2
q1q2 q3
q1 q2N
good sequence
bad sequence
column with 100 unique integers [1,100]<50<25 <75
<2 <3
Stochastic Cracking, PVLDB 12
![Page 106: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/106.jpg)
Stratos Idreos
NN/2N/2
q1q2 q3
q1 q2NN-1
good sequence
bad sequence
column with 100 unique integers [1,100]<50<25 <75
<2 <3
Stochastic Cracking, PVLDB 12
![Page 107: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/107.jpg)
Stratos Idreos
NN/2N/2
q1q2 q3
q1 q2NN-1
good sequence
bad sequence
column with 100 unique integers [1,100]<50<25 <75
<2 <3
Stochastic Cracking, PVLDB 12
![Page 108: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/108.jpg)
Stratos Idreos
NN/2N/2
q1q2 q3
q1 q2 q3NN-1
good sequence
bad sequence
column with 100 unique integers [1,100]<50<25 <75
<2 <3 <4
Stochastic Cracking, PVLDB 12
![Page 109: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/109.jpg)
Stratos Idreos
NN/2N/2
q1q2 q3
q1 q2 q3NN-1N-2
good sequence
bad sequence
column with 100 unique integers [1,100]<50<25 <75
<2 <3 <4
Stochastic Cracking, PVLDB 12
![Page 110: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/110.jpg)
Stratos Idreos
NN/2N/2
q1q2 q3
q1 q2 q3NN-1N-2
good sequence
bad sequence
column with 100 unique integers [1,100]<50<25 <75
<2 <3 <4
blindly adapting to queries is not always a good idea
Stochastic Cracking, PVLDB 12
![Page 111: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/111.jpg)
Stratos Idreos Stochastic Cracking, PVLDB 12
query driven
![Page 112: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/112.jpg)
Stratos Idreos
to be cracked
Stochastic Cracking, PVLDB 12
query driven
![Page 113: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/113.jpg)
Stratos Idreos
to be cracked
q random
Stochastic Cracking, PVLDB 12
query driven
![Page 114: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/114.jpg)
Stratos Idreos
to be cracked
q random
progressive cracking
Stochastic Cracking, PVLDB 12
query driven
![Page 115: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/115.jpg)
Stratos Idreos
to be cracked
q random
q1: <v1progressive cracking
Stochastic Cracking, PVLDB 12
query driven
![Page 116: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/116.jpg)
Stratos Idreos
to be cracked
q random
q1: <v1 crack + filter <v1progressive cracking
Stochastic Cracking, PVLDB 12
query driven
![Page 117: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/117.jpg)
Stratos Idreos
to be cracked
q random
random
swap
q1: <v1 crack + filter <v1progressive cracking
Stochastic Cracking, PVLDB 12
query driven
![Page 118: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/118.jpg)
Stratos Idreos
to be cracked
q random
random
swap
q1: <v1 crack + filter <v1
scan + filter <v1
progressive cracking
Stochastic Cracking, PVLDB 12
query driven
![Page 119: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/119.jpg)
Stratos Idreos
to be cracked
q random
randomq1: <v1progressive cracking
q2: <v2
Stochastic Cracking, PVLDB 12
query driven
![Page 120: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/120.jpg)
Stratos Idreos
to be cracked
q random
randomq1: <v1progressive cracking
scan + filter <v2q2: <v2
Stochastic Cracking, PVLDB 12
query driven
![Page 121: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/121.jpg)
Stratos Idreos
to be cracked
q random
randomq1: <v1progressive cracking
scan + filter <v2
crack + filter <v2
q2: <v2
swap
Stochastic Cracking, PVLDB 12
query driven
![Page 122: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/122.jpg)
Stratos Idreos
to be cracked
q random
randomq1: <v1progressive cracking
scan + filter <v2
crack + filter <v2
q2: <v2
swap
Stochastic Cracking, PVLDB 12
query driven
![Page 123: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/123.jpg)
Stratos Idreos
cracking on Skyserver (4TB) (Sloan Digital Sky Survey, www.sdss.org)
cracking answers 160.000 queries while full indexing is still half way creating one index
Stochastic Cracking, PVLDB 12
![Page 124: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/124.jpg)
Stratos Idreos
multi-core utilization
Multi-cores, SIGMOD 15
![Page 125: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/125.jpg)
Stratos Idreos Multi-cores, SIGMOD 15
select [a,b]
core 1 core 2
![Page 126: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/126.jpg)
Stratos Idreos
select [a,b]
Multi-cores, SIGMOD 15
![Page 127: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/127.jpg)
Stratos Idreos
select [a,b]
Multi-cores, SIGMOD 15
![Page 128: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/128.jpg)
Stratos Idreos
select [a,b]
<a >=a
Multi-cores, SIGMOD 15
![Page 129: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/129.jpg)
Stratos Idreos
select [a,b]
<a >=a
core 1 core 2 core 3 … core N
Multi-cores, SIGMOD 15
![Page 130: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/130.jpg)
Stratos Idreos Multi-cores, SIGMOD 15
![Page 131: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/131.jpg)
Stratos Idreos
…
Multi-cores, SIGMOD 15
![Page 132: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/132.jpg)
Stratos Idreos
core 1 core 2 core 3
…
core N…
Multi-cores, SIGMOD 15
![Page 133: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/133.jpg)
Stratos Idreos
core 1 core 2 core 3
…
core N…
…
Multi-cores, SIGMOD 15
![Page 134: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/134.jpg)
Stratos Idreos
<a
>=a
core 1 core 2 core 3
…
core N…
…
Multi-cores, SIGMOD 15
![Page 135: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/135.jpg)
Stratos Idreos
<a
>=a
core 1 core 2 core 3
…
core N…
…
Multi-cores, SIGMOD 15
![Page 136: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/136.jpg)
Stratos Idreos
<a
>=a
core 1 core 2 core 3
…
core N…
…
<a >=a
merge
Multi-cores, SIGMOD 15
![Page 137: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/137.jpg)
Stratos Idreos
problem: cores may be under utilized
goal: either fully utilize a core or shut it down
Holistic, SIGMOD 15
![Page 138: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/138.jpg)
Stratos Idreos
scheduler
monitoring CPU utilization
worker 1worker 2
worker 3worker 4
worker 5
worker 6worker 7
worker 8
thread pool
when there is an underutilized CPU, pin a thread to it and to do a cracking task
![Page 139: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/139.jpg)
Stratos Idreos
index space
crack
partitions size - access frequency - hit ratio
random works best
![Page 140: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/140.jpg)
Stratos Idreos
108 tuples - 10 attributes, random queries
1
10
100
1000
10000
1 10 100 1000
Cu
mu
lativ
e R
esp
on
se T
ime
(se
c)
Query sequence
no indexing
offline indexing
online indexing
adaptive indexing
holistic indexing
(a) Performance
0
50
100
150
200
AdaptiveIndexing
HolisticIndexing
To
tal R
esp
on
se T
ime
(se
c)
1
9
90
900
(b) Performance Breakdown
0
500
1000
1500
2000
2500
3000
3500
0 2 4 6 8 10
Cu
mu
lativ
e #
ind
ex
pa
rtiti
on
s
Query sequence (x100)
adaptive indexing
holistic indexing
(c) Index Partitions
0
5
10
15
20
25
30
2 4 6 8 10 12 14 0
8
16
24
32
To
tal R
esp
on
se T
ime
(se
c) o
f #
wo
rke
rs
#W
ork
ers
Holistic Worker Activation
time
#workers
(d) Idle CPU UtilizationFigure 6: Improving performance with holistic indexing.
center slice is contiguous, while the remaining n� 1 slices con-sist of two disjoint halves, each, that are arranged concentricallyaround the center slice (xi and yi indicate the first and the last el-ement of piece i respectively). n threads crack the n slices inde-pendently applying a vectorized, out-of-place cracking algorithm(Figure 5), which was proven in [44] to be the most CPU efficientsingle-threaded cracking implementation reported so far. Finally,the local data are merged into a big cracked piece (Figure 4(b)).We found that devoting all resources to perform adaptive indexingfor user queries in parallel does not lead to the absolute best per-formance. Specifically, we found that we can improve performanceeven more by assigning part of the resources to holistic indexing. Inthis way, some of the CPU resources are assigned to parallel crack-ing for user queries but the rest of the CPU resources are distributedacross several holistic workers for additional index refinements. Inthe experimental section we show why this approach is better thanassigning all available resources to user queries.
5. EXPERIMENTAL ANALYSISIn this section, we demonstrate that holistic indexing leads to a
self-organizing always-on DBMS with substantial benefits in termsof response time; with zero administration or set-up effort holisticindexing improves performance adaptively by exploiting all avail-able CPU resources to the maximum. We present a detailed exper-imental analysis using both standard benchmarks such as TPC-Hand real-life workloads such as SkyServer as well as synthetic mi-crobenchmarks for a fine-grained analysis.
We use a dual-socket machine equipped with two 2.00 GHz In-tel(R) Xeon(R) CPU E5-2650 processors and with 256 GB RAM.Each processor has 8 hyper-threading cores resulting in 32 hard-ware threads in total. The operating system is Fedora 20 (kernelversion 3.12.10). All experiments we report are based on an im-plementation of holistic indexing in MonetDB and assume a main-memory environment.
5.1 Improving over State-of-the-Art IndexingIn our first experiment we demonstrate that holistic indexing
has the potential to bring substantial performance improvementsover existing state-of-the-art indexing approaches. We test holisticindexing against parallel versions of adaptive indexing (databasecracking), offline indexing, online indexing and plain scans.
For plain scans (no indexing), we use a parallel select opera-tor implemented in MonetDB. For offline and online indexing wesort the columns using a highly parallel NUMA-aware sorting algo-rithm that was introduced in [9] (m-way, 16-byte keys) and is pub-licly available in [1]. Specifically, in offline indexing we pre-sortall the columns before query processing, while in online indexing
we assume that after processing a few queries we understand theworkload patterns and then we sort the relevant columns. MonetDBautomatically detects that a column is sorted and can use efficientbinary search actions during select operations. For adaptive index-ing we use the parallel vectorized database cracking algorithm thatwas introduced in [44] (see Section 4.2).
Here we use a synthetic benchmark. The query workload con-sists of 103 range select queries over a table of 10 attributes; eachquery touches a single attribute (we will see more complex querieslater on). Each attribute consists of 230 uniformly distributed inte-gers, while the value range requested by each query (and thus theselectivity) is random. All queries are of the following form.
select A from R where A < v
The tested scenario assumes a dynamic and ad-hoc environmentwith zero workload knowledge and zero idle time to pre-index thedata. Figure 6(a) shows the results. On the x-axis queries are rankedin execution order. The y-axis represents the cumulative responsetime as the query sequence evolves, i.e., each point (x,y) representsthe sum of the execution time y for the first x queries. In this way,the graph shows how the response times evolve as we process moreand more queries.
If there is no indexing support (plain scans), the entire column isscanned in parallel by 32 threads for every query. Because of thisstable access pattern, the cumulative response time of the query se-quence grows linearly as every query has similar cost. With offlineindexing, on the other hand, it takes 12 seconds to completely sorteach column, assuming a priori workload knowledge. This leads toa 120 seconds initialization overhead to sort all 10 columns. Sincethere is no idle time before the first query, the sorting cost is addedto the execution time of the first query in Figure 6(a). After thefirst query, all queries are answered with fast binary search actionswhich results in a rather flat cumulative curve. With online index-ing, the first 100 queries are answered without any index supportand thus the cost grows linearly. After the 100th query, assumingenough workload knowledge has been obtained via monitoring, weproceed to sort all 10 columns. The sorting cost is added to theexecution cost of the 101st query, since there is no idle time be-tween the 100th and the 101st query. As with offline indexing, allsubsequent queries are answered very fast with binary search overthe sorted column. Thus, both in offline and in online indexing,the whole query sequence is affected by the sorting costs. On theother hand, adaptive indexing continuously improves performancerequiring no workload knowledge and without penalizing individ-ual queries. This improvement is due to the fact that adaptive in-dexing builds only partial indices which it incrementally refines asqueries arrive. However, there is still room for big improvements.
![Page 141: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/141.jpg)
Stratos Idreos
NORMALIZED DATA DENORMALIZED DATA
![Page 142: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/142.jpg)
Stratos Idreos
NORMALIZED DATA DENORMALIZED DATA
good for updates, storage but we need joins
![Page 143: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/143.jpg)
Stratos Idreos
NORMALIZED DATA DENORMALIZED DATA
good for updates, storage but we need joins
only fast scans but expensive to create,
storage & updates
![Page 144: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/144.jpg)
Stratos Idreos
adaptive denormalization
normalized data
first award in ACM SIGMOD undergrad
research competition
![Page 145: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/145.jpg)
Stratos Idreos
adaptive denormalization
normalized data possible denormalized space
first award in ACM SIGMOD undergrad
research competition
![Page 146: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/146.jpg)
Stratos Idreos
adaptive denormalization
continuously physically reorganize data based on incoming query patterns (joins)
normalized data possible denormalized space
first award in ACM SIGMOD undergrad
research competition
![Page 147: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/147.jpg)
Stratos Idreos
adaptive denormalization
continuously physically reorganize data based on incoming query patterns (joins)
normalized data
denormalized fragments queries only need to fast scan
possible denormalized space
first award in ACM SIGMOD undergrad
research competition
![Page 148: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/148.jpg)
Stratos Idreos
adaptive denormalization
continuously physically reorganize data based on incoming query patterns (joins)
normalized data
denormalized fragments queries only need to fast scan
possible denormalized space
first award in ACM SIGMOD undergrad
research competition
![Page 149: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/149.jpg)
Stratos Idreos
adaptive denormalization
continuously physically reorganize data based on incoming query patterns (joins)
normalized data
denormalized fragments queries only need to fast scan
possible denormalized space
first award in ACM SIGMOD undergrad
research competition
![Page 150: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/150.jpg)
Stratos Idreos
adaptive denormalization
continuously physically reorganize data based on incoming query patterns (joins)
normalized data
denormalized fragments queries only need to fast scan
possible denormalized space
first award in ACM SIGMOD undergrad
research competition
![Page 151: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/151.jpg)
Stratos Idreos
years
daily
dat
a
[IBMbigdata]years
daily
dat
a
[StratosGuess]
data
* ski
llsdata system design,
set-up, tune, use
![Page 152: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/152.jpg)
Stratos Idreos
data systems that are easy to design(storage, data flow, algorithms, tuning, etc)
![Page 153: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/153.jpg)
Stratos Idreos
SystemDesign
FullImplemention
PrototypeImplementation
workloads, h/wexpected properties
(performance, throughput,
energy, budget, …)
1
2
3
application specific workload, h/w & expected properties
Set-up &
Tune
4 Auto-tuningduring query processing
5
Design & DevelopmentRoles: Architects & Developers
Set-up and TuningRoles: Database Administrators
![Page 154: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/154.jpg)
Stratos Idreos
say we need a system for workload X (data/access patterns): should we strip down a relational system?
should we build up a key-value store or main-memory system? should we build something from scratch?
![Page 155: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/155.jpg)
Stratos Idreos
say we need a system for workload X (data/access patterns): should we strip down a relational system?
should we build up a key-value store or main-memory system? should we build something from scratch?
say the workload (read/write ratio) shifts (e.g., due to app features):should we use a different data layout for base data - diff updates?
should we use different indexing or no indexing?
![Page 156: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/156.jpg)
Stratos Idreos
say we buy new hardware X (flash/memory): should we change the size of b-tree nodes?
should we change the merging strategy in our LSM-tree?
say we need a system for workload X (data/access patterns): should we strip down a relational system?
should we build up a key-value store or main-memory system? should we build something from scratch?
say the workload (read/write ratio) shifts (e.g., due to app features):should we use a different data layout for base data - diff updates?
should we use different indexing or no indexing?
![Page 157: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/157.jpg)
Stratos Idreos
say we buy new hardware X (flash/memory): should we change the size of b-tree nodes?
should we change the merging strategy in our LSM-tree?
say we want to improve response time:would it be beneficial if we would buy faster flash disks?
would it be beneficial if we buy more memory? or just spend the same budget on improving software?
say we need a system for workload X (data/access patterns): should we strip down a relational system?
should we build up a key-value store or main-memory system? should we build something from scratch?
say the workload (read/write ratio) shifts (e.g., due to app features):should we use a different data layout for base data - diff updates?
should we use different indexing or no indexing?
![Page 158: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/158.jpg)
Stratos Idreos
# of design options?
![Page 159: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/159.jpg)
Stratos Idreos
# of design options?
> 1 QUINTILLION(and counting…)
![Page 160: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/160.jpg)
Stratos Idreos
data systems design (and research) is kind of an art(both in the good & in the bad sense)
![Page 161: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/161.jpg)
Stratos Idreos
data systems design (and research) is kind of an art
but can we keep up?
(both in the good & in the bad sense)
![Page 162: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/162.jpg)
Stratos Idreos
disk memory flash …
1 easily utilize past concepts/designs
![Page 163: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/163.jpg)
Stratos Idreos
do not miss out on cool ideas and concepts
# of
cita
tions
0
7
14
21
28
35
1996 1999 2002 2005 2008 2011 2014
P. O’Neil, E. Cheng, D. Gawlick, E, O'Neil The log-structured merge-tree (LSM-tree) Acta Informatica 33 (4): 351–385, 1996
2
![Page 164: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/164.jpg)
Stratos Idreos
do not miss out on cool ideas and concepts
# of
cita
tions
0
7
14
21
28
35
1996 1999 2002 2005 2008 2011 2014
P. O’Neil, E. Cheng, D. Gawlick, E, O'Neil The log-structured merge-tree (LSM-tree) Acta Informatica 33 (4): 351–385, 1996
Google publishes BigTable
2
![Page 165: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/165.jpg)
Stratos Idreos
data+queries+hardware
data system
self-designing data systems
can we design new systems in weeks instead of years?
![Page 166: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/166.jpg)
Stratos Idreos
data+queries+hardware
data system
self-designing data systems
easy to design
adapt to environment
can we design new systems in weeks instead of years?
![Page 167: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/167.jpg)
Stratos Idreos
INTERACTIVE DATA SYSTEM DESIGN/TUNING/TESTING
datasystem
designer
![Page 168: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/168.jpg)
Stratos Idreos
How to model/abstract design decisions?
How much can happen automatically (auto-design)?
![Page 169: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/169.jpg)
Stratos Idreos
How to model/abstract design decisions?
How much can happen automatically (auto-design)?
move from design based on intuition & experience only to a more formal and systematic way to design systems
![Page 170: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/170.jpg)
Stratos Idreos
row-store
column-store
key-value store
hybrid store
ACM SIGMOD Blog, June’15
1. write/extend modules in a high level language
(optimizations)2. modules=
storage/execution/data flow
3. try out >>1 designs (sets of modules)
![Page 171: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/171.jpg)
Stratos Idreos
data systems that are easy to use
dbTouch
show me something interesting
Queriosity
DATA
cidr2013/icde2014
bigdata2015
can we design systems where no SQL is needed?
![Page 172: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/172.jpg)
cracking example
dbTouch CIDR 2013
![Page 173: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/173.jpg)
Stratos Idreos
data systems todayallow us to answer queries fast
data systems tomorrow should allow us to find fast which queries to ask
db
db explore
![Page 174: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/174.jpg)
Stratos Idreos
a semester of quizzes and brainstorming
option between systems project (key-value store) & research with DASlab
(only for CS165 students or otherwise advanced students)
![Page 175: auto-tuning database kernels 2 - Harvard Universitydaslab.seas.harvard.edu/.../CS165Fall2016Class20.pdf · class 22. Stratos Idreos a bit more about auto-tuning db kernels and then](https://reader033.fdocuments.in/reader033/viewer/2022050420/5f8f3ca9134c17431e12011e/html5/thumbnails/175.jpg)
the end
\*project deadline Dec 21 OH and labs continue until then*/
DASlab productions 2016
learn the concepts, not just the techniques learn to adapt
it all starts with how we store the data yes you can do research