Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking...
Transcript of Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking...
![Page 1: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/1.jpg)
‹#›
Daniel Mitterdorfer @dmitterd
Benchmarking Elasticsearch with Rally
![Page 2: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/2.jpg)
Outline
2
The Need for Benchmarking in the Elasticsearch Project1
2
3
7 Deadly Benchmarking Sins
Demo
![Page 3: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/3.jpg)
“Elasticsearch is just a search engine, isn't it?”
3
![Page 4: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/4.jpg)
4
![Page 5: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/5.jpg)
5
![Page 6: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/6.jpg)
6
![Page 7: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/7.jpg)
How do you evaluate performance for all these
use-cases?
7
![Page 8: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/8.jpg)
What we do: Measure, Measure, Measure
8
During Development
Source: https://github.com/elastic/elasticsearch/issues/7309
![Page 9: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/9.jpg)
What we do: Measure, Measure, Measure
9
During Development
![Page 10: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/10.jpg)
What we do: Measure, Measure, Measure
10
Nightly benchmarks
Source: https://elasticsearch-benchmarks.elastic.co/geonames/
![Page 11: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/11.jpg)
What we do: Measure, Measure, Measure
11
Sizing benchmarks for specific scenarios*
*) numbers on axis intentionally stripped to avoid interpretation out of context
![Page 12: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/12.jpg)
What we do: Measure, Measure, Measure
12
Performance measurement / tuning at customer site
![Page 13: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/13.jpg)
13
7 Deadly Benchmark Sins
![Page 14: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/14.jpg)
Sin #1: Not paying attention to system setup
• Bare-metal
• SSDs
• Server-class CPU
• Single socket, multi socket?
• Enough memory head-room for FS cache
14
Hardware
![Page 15: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/15.jpg)
Sin #1: Not paying attention to system setup
• Linux, Windows
• Check network configuration
• File system, LVM, etc.
• No Swap
• I/O scheduler: cfq, noop, deadline
• CPU governor: powersave, performance
15
Operating System
![Page 16: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/16.jpg)
Sin #1: Not paying attention to system setup
• Beware of unwanted caching effects (FS cache, …)
• Benchmark driver and ES on separate machines
• One node per machine (or adjust JVM parameters (GC threads))
• Low-latency, high-throughput network between benchmark driver and ES
• No other traffic on this network
16
Benchmark Setup
![Page 17: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/17.jpg)
Sin #2: No warmup
• JIT compiler needs to run first
• Creation of long-living data structures
• FS cache for Lucene segments (memory-mapped IO)
• Benchmark driver needs to reach stable state too
17
Awake before your first coffee? Elasticsearch isn’t either.
![Page 18: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/18.jpg)
Warmup Behaviour: C2 Compilation Events/s
18
![Page 19: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/19.jpg)
Warmup Behaviour: Benchmark Driver Throughput
19
![Page 20: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/20.jpg)
Sin #3: No bottleneck analysis
• Benchmark driver
• System setup: analysis of system background noise (jhiccup)
• Network
20
Are you really benchmarking what you think you’re benchmarking?
![Page 21: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/21.jpg)
First Driver Stress Tests
21
Contention all over the place
![Page 22: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/22.jpg)
Sin #4: The divine benchmarking script
• Not paying attention how metrics are gathered
• System.currentTimeMillis() vs. System.nanoTime()
• Not checking measurement overhead
• No return code checks: the fast 404
• Blind trust in tools: No cross-verification
22
“After all, it produces numbers with 6 decimal places!”
![Page 23: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/23.jpg)
Cross-Validation of Metrics
23
Metric Rally Flight Recorder GC log
Young Gen GC 79,416 ms 89,003 ms(?) 80,853 ms
Old Gen GC 23,964 ms 156,630 ms(?) 23,989 ms
![Page 24: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/24.jpg)
Sin #5: Denying Statistics
• How is run-to-run variance distributed?
• Multiple trial runs and t-test
24
Run-to-run variance
![Page 25: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/25.jpg)
Run-to-run Variance Verification
25
![Page 26: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/26.jpg)
Sin #5: Denying Statistics
• The meaningless mean: Half of the responses are worse than the mean
• Cannot calculate 99.99th percentile from 10 samples
• Don’t average percentiles
• Latency distribution is multi-modal
26
Latency Measurement
![Page 27: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/27.jpg)
Sin #6: Vague Metrics
• Latency
• Service Time
• Throughput
• Waiting Time
• Response Time
• Utilisation
• …
27
![Page 28: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/28.jpg)
28
Service Time
![Page 29: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/29.jpg)
while (!isDone()) {Request req = createRequest();long start = System.nanoTime();// block until the request has finishedsend(req);long end = System.nanoTime();long serviceTime = end - start;
}
Sin #6: Vague metricsService Time
29
![Page 30: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/30.jpg)
30
Waiting Time
![Page 31: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/31.jpg)
31
Response Time / Latency
+
![Page 32: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/32.jpg)
// generator threadwhile (!isDoneGenerating()) {
Request req = createRequest();long start = System.nanoTime();queue.put(req, start);
}
// request issuing threadwhile (!isDoneSending()) {
request, start = queue.take();send(request);long end = System.nanoTime();long latency = end - start;
}
Sin #6: Vague metricsLatency
32
![Page 33: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/33.jpg)
0% utilisation: no waiting time
33
Utilisation
![Page 34: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/34.jpg)
100% utilisation: high waiting time
34
Utilisation
![Page 35: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/35.jpg)
35
Throughput and Utilisation
![Page 36: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/36.jpg)
Sin #6: Vague metrics
36
Latency … at which throughput?
Created based on http://robharrop.github.io/maths/performance/2016/02/20/service-latency-and-utilisation.html
![Page 37: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/37.jpg)
// generator threadwhile (!isDoneGenerating()) { Request req = createRequest();
long start = System.nanoTime();queue.put(req, start);Thread.sleep(waitTime(targetThroughput));
}
// request issuing threadwhile (!isDoneSending()) {
request, start = queue.take();send(request);long end = System.nanoTime();long latency = end - start;
}
Sin #6: Vague metricsLatency at a defined throughput
37
![Page 38: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/38.jpg)
Sin #7: Treat Performance as One-Dimensional
• Bulk size
• Query parameters
• Document structure
38
Vary inputs
![Page 39: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/39.jpg)
Sin #7: Treat Performance as One-Dimensional
• Run queries in different order: Avoid caching effects
• Interfere operations: How does indexing behave with concurrent queries?
39
Vary execution order
![Page 40: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/40.jpg)
Sin #7: Treat Performance as One-Dimensional
• Hardware
• OS
• JDK
• …
40
And more
![Page 41: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/41.jpg)
How we measure
41
![Page 42: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/42.jpg)
42
![Page 43: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/43.jpg)
Rally
https://github.com/elastic/rally
43
You know … for benchmarking Elasticsearch
![Page 44: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/44.jpg)
10.000 feet view of Rally
44
Rally benchmarked cluster
provisions
attached telemetry
system metrics
applies load
Tracks (Benchmark Data)
Metrics Store
![Page 45: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/45.jpg)
45
Demo
![Page 46: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/46.jpg)
Summary
46
![Page 47: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/47.jpg)
47
![Page 48: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/48.jpg)
‹#›
Performance is easy, all you need to know is everything
Sergey Kuksenko, Oracle Performance Engineer
![Page 49: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/49.jpg)
49
Questions?
![Page 51: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/51.jpg)
Further Resources: Talks and Articles
• What is coordinated omission? https://groups.google.com/forum/#!msg/mechanical-sympathy/icNZJejUHfE/BfDekfBEs_sJ
• Example: “Fixing Coordinated Omission in Cassandra Stress”: https://psy-lob-saw.blogspot.de/2016/07/fixing-co-in-cstress.html
• Relating Service Utilisation to Latency: http://robharrop.github.io/maths/performance/2016/02/20/service-latency-and-utilisation.html
• “How not to measure latency”: http://www.youtube.com/watch?v=lJ8ydIuPFeU
• “Benchmarking Blunders and Things That Go Bump in the Night”: http://arxiv.org/pdf/cs/0404043v1.pdf
51
![Page 52: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/52.jpg)
Further Resources: Tools & Methodology
• USE method: http://www.brendangregg.com/usemethod.html
• Java Flight Recorder: http://docs.oracle.com/javacomponents/jmc-5-5/jfr-runtime-guide/index.html
• JITWatch: https://github.com/AdoptOpenJDK/jitwatch
• Rally: https://github.com/elastic/rally
52
![Page 53: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/53.jpg)
Image Sources (1/3)
• “book_stacks” by “Hung Thai”: https://www.flickr.com/photos/96055807@N02/10893926256/ (CC BY 2.0)
• “Curiosity Mastcam L sol 673” by “2di7 & titanio44”: https://www.flickr.com/photos/lunexit/14570422596/ (CC BY-NC-ND 2.0)
• “80's style Hacker Picture” by “Brian Klug”: https://www.flickr.com/photos/brianklug/6870005158/ (CC BY-NC 2.0)
• “gags9999”: https://www.flickr.com/photos/gags9999/14124313715/ (CC BY 2.0)
• “Espresso Machine” by “Joseph Morris”: https://www.flickr.com/photos/josephmorris/16961075629/ (CC BY 2.0)
53
![Page 54: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/54.jpg)
Image Sources (2/3)
• “Its about the Coffee” by “Neil Moralee”: https://www.flickr.com/photos/neilmoralee/8179963297/ (CC BY-NC-ND 2.0)
• “On an adventure” by “Dirk Dallas”: https://www.flickr.com/photos/dirkdallas/14988429720/ (CC BY-NC 2.0)
• “Traffic Jam” by “lorenz.markus97”: https://www.flickr.com/photos/lorenz_markus/17449315008/ (CC BY 2.0)
• “Swirl Me Back Home” by “Nick Fisher”: https://www.flickr.com/photos/cobrasick/5297980956/ (CC BY-ND 2.0)
• “Works Mini Cooper S DJB 93B” by “Andrew Basterfield”: https://www.flickr.com/photos/andrewbasterfield/4759364589/ (CC BY-SA 2.0)
54
![Page 55: Benchmarking Elasticsearch V4 - Daniel Mitterdorferdaniel.mitterdorfer.name/talks/2016/Benchmarking Elasticsearch.pdf · Outline 2 1 The Need for Benchmarking in the Elasticsearch](https://reader034.fdocuments.in/reader034/viewer/2022051320/5a8141797f8b9a571e8d3f51/html5/thumbnails/55.jpg)
Image Sources (3/3)
• “photo” by “Odi Kosmatos”: https://www.flickr.com/photos/kosmatos/8162850619/ (CC BY 2.0)
• “Bachelor Students - Chemistry Lab” by “NTNU”: https://www.flickr.com/photos/92416586@N05/12188423293/ (CC BY 2.0)
• “42” by “Lisa Risager”: https://www.flickr.com/photos/risager/5067595483/ (CC BY-SA 2.0)
55