How to Run Solr on Docker and Why
-
Upload
sematext-group-inc -
Category
Technology
-
view
3.989 -
download
2
Transcript of How to Run Solr on Docker and Why
![Page 1: How to Run Solr on Docker and Why](https://reader031.fdocuments.in/reader031/viewer/2022022414/5870a4341a28abcb078b5673/html5/thumbnails/1.jpg)
O C T O B E R 1 1 - 1 4 , 2 0 1 6 • B O S T O N , M A
![Page 2: How to Run Solr on Docker and Why](https://reader031.fdocuments.in/reader031/viewer/2022022414/5870a4341a28abcb078b5673/html5/thumbnails/2.jpg)
How to run Solr on Docker. And why.Rafał Kuć, Radu Gheorghe
Sematext Group, Inc.
![Page 3: How to Run Solr on Docker and Why](https://reader031.fdocuments.in/reader031/viewer/2022022414/5870a4341a28abcb078b5673/html5/thumbnails/3.jpg)
3
01Next ~40 minutes
![Page 4: How to Run Solr on Docker and Why](https://reader031.fdocuments.in/reader031/viewer/2022022414/5870a4341a28abcb078b5673/html5/thumbnails/4.jpg)
4
02Common issues we all know
overprovisionedserversresources not utilized
!= != !=
differences in environments
$$$money loss
![Page 5: How to Run Solr on Docker and Why](https://reader031.fdocuments.in/reader031/viewer/2022022414/5870a4341a28abcb078b5673/html5/thumbnails/5.jpg)
5
02One of the solutions
productionQAtestdevelopment
![Page 6: How to Run Solr on Docker and Why](https://reader031.fdocuments.in/reader031/viewer/2022022414/5870a4341a28abcb078b5673/html5/thumbnails/6.jpg)
6
03Bare metal
Hardware
Operating System
Libraries
Application Application
bare metal
![Page 7: How to Run Solr on Docker and Why](https://reader031.fdocuments.in/reader031/viewer/2022022414/5870a4341a28abcb078b5673/html5/thumbnails/7.jpg)
7
03Traditional virtual machine
Hardware
Operating System
Libraries
bare metal
Hardware
Host Operating System
Hypervisor
Guest OS Guest OS
Libraries Libraries
Application Application
Application Application
traditional VM
![Page 8: How to Run Solr on Docker and Why](https://reader031.fdocuments.in/reader031/viewer/2022022414/5870a4341a28abcb078b5673/html5/thumbnails/8.jpg)
8
03Docker container
Hardware
Operating System
Libraries
bare metal
Hardware
Host Operating System
traditional VM
Hypervisor
Guest OS Guest OS
Libraries Libraries
Application Application
Hardware
Host Operating System
Docker Engine
Libraries Libraries
Application Application
Application Application
container
![Page 9: How to Run Solr on Docker and Why](https://reader031.fdocuments.in/reader031/viewer/2022022414/5870a4341a28abcb078b5673/html5/thumbnails/9.jpg)
9
03Unikernel architecture
Hardware
Operating System
Libraries
bare metal
Hardware
Host Operating System
traditional VM
Hypervisor
Guest OS Guest OS
Libraries Libraries
Application Application
Hardware
Host Operating System
container
Docker Engine
Libraries Libraries
Application Application
Hardware
unikernel
Kernel + App + Libs Application Application
Host Operating System
Hypervisor
![Page 10: How to Run Solr on Docker and Why](https://reader031.fdocuments.in/reader031/viewer/2022022414/5870a4341a28abcb078b5673/html5/thumbnails/10.jpg)
10
01Solr + Docker how to
+
![Page 11: How to Run Solr on Docker and Why](https://reader031.fdocuments.in/reader031/viewer/2022022414/5870a4341a28abcb078b5673/html5/thumbnails/11.jpg)
11
01Docker + Solr – how to
docker run solr:latestdocker run -p 8983:8983 solr:latestdocker run --name lr_single_solr -d -p 8983:8983 -t solr:6.2.1docker run --name lr_alpine_solr -d -p 8983:8983 -t solr:6.2-alpinedocker run -it --name lr_solr_mem -p 8983:8983 -d -m 1G solr:6.2.1docker run -p 8983:8983 -d --memory-swappiness=0 solr:6.2.1docker run -it --cpuset-cpus="0,1" -p 8983:8983 -d solr:6.2.1docker run --name solr_one -d -p 8983:8983 –p 9983:9983 -t solr:6.2.1 -cdocker run --name solr_two -d --link solr_one -p 6983:6983 -t solr:6.2.1 -z solr_one:9983
![Page 12: How to Run Solr on Docker and Why](https://reader031.fdocuments.in/reader031/viewer/2022022414/5870a4341a28abcb078b5673/html5/thumbnails/12.jpg)
12
01Demo time!
https://github.com/sematext/lucene-revolution-samples/tree/master/2016/solr_docker
DEMOTIME
![Page 13: How to Run Solr on Docker and Why](https://reader031.fdocuments.in/reader031/viewer/2022022414/5870a4341a28abcb078b5673/html5/thumbnails/13.jpg)
13
01
![Page 14: How to Run Solr on Docker and Why](https://reader031.fdocuments.in/reader031/viewer/2022022414/5870a4341a28abcb078b5673/html5/thumbnails/14.jpg)
14
01Very simple test infrastructure
Ubuntu + Solr 6.2Ubuntu + Docker 1.12.2 (native)
VMs (Ubuntu) + Solr 6.2
![Page 15: How to Run Solr on Docker and Why](https://reader031.fdocuments.in/reader031/viewer/2022022414/5870a4341a28abcb078b5673/html5/thumbnails/15.jpg)
15
01Solr configuration – collection
![Page 16: How to Run Solr on Docker and Why](https://reader031.fdocuments.in/reader031/viewer/2022022414/5870a4341a28abcb078b5673/html5/thumbnails/16.jpg)
16
01Solr configuration
10 stored fields Hard commit – 15 sec10 indexed fields Soft commit - 1 sec7 doc values
![Page 17: How to Run Solr on Docker and Why](https://reader031.fdocuments.in/reader031/viewer/2022022414/5870a4341a28abcb078b5673/html5/thumbnails/17.jpg)
17
01Other parts
Completely out of the box experience!
![Page 18: How to Run Solr on Docker and Why](https://reader031.fdocuments.in/reader031/viewer/2022022414/5870a4341a28abcb078b5673/html5/thumbnails/18.jpg)
18
01Ubuntu + Solr, 1 Solr instance
Test 1Indexing throughput - 3348 docs/sec
Test 2Indexing throughput - 2754 docs/sec
Querying throughput - 209 q/secSprzedaż
CPU Mem I/O
![Page 19: How to Run Solr on Docker and Why](https://reader031.fdocuments.in/reader031/viewer/2022022414/5870a4341a28abcb078b5673/html5/thumbnails/19.jpg)
19
01Ubuntu + Solr, 4 Solr instances
Test 1Indexing throughput - 3618docs/sec
Test 2Indexing throughput - 3024 docs/sec
Querying throughput - 267,7 q/secSprzedaż
CPU Mem I/O
![Page 20: How to Run Solr on Docker and Why](https://reader031.fdocuments.in/reader031/viewer/2022022414/5870a4341a28abcb078b5673/html5/thumbnails/20.jpg)
20
01Ubuntu + 1 VM, 1 Solr instance
Test 1Indexing throughput - 2052 docs/sec
Test 2Indexing throughput - 1944 docs/sec
Querying throughput - 36,7 q/secSprzedaż
CPU Mem I/O
![Page 21: How to Run Solr on Docker and Why](https://reader031.fdocuments.in/reader031/viewer/2022022414/5870a4341a28abcb078b5673/html5/thumbnails/21.jpg)
21
01Ubuntu + Docker, 4 Solr instances
Test 1Indexing throughput - 2754 docs/sec
Test 2Indexing throughput - 2484 docs/sec
Querying throughput - 108,6 q/secSprzedaż
CPU Mem I/O
![Page 22: How to Run Solr on Docker and Why](https://reader031.fdocuments.in/reader031/viewer/2022022414/5870a4341a28abcb078b5673/html5/thumbnails/22.jpg)
22
01Performance summary
Test results on OSX != Test results on Linux
Bare metal is the fastest from those tested
Docker will be faster than traditional VM
Bare metal > Docker > VMs
![Page 23: How to Run Solr on Docker and Why](https://reader031.fdocuments.in/reader031/viewer/2022022414/5870a4341a28abcb078b5673/html5/thumbnails/23.jpg)
23
01
![Page 24: How to Run Solr on Docker and Why](https://reader031.fdocuments.in/reader031/viewer/2022022414/5870a4341a28abcb078b5673/html5/thumbnails/24.jpg)
24
01Bare metal based architecture
One per machine vs Multiple per machineMultiple per machine may be a better choiceHarder to scale -> machine provisioning neededFull control over resourcesNoisy neighbour problems can appearBest or almost best performance
![Page 25: How to Run Solr on Docker and Why](https://reader031.fdocuments.in/reader031/viewer/2022022414/5870a4341a28abcb078b5673/html5/thumbnails/25.jpg)
25
01Virtual machine based architecture
Multiple VMs per machine allowedEasy to scale, but requires preparationClear boundaries between VMGuest operating system overheadWorst performance compared to the rest
![Page 26: How to Run Solr on Docker and Why](https://reader031.fdocuments.in/reader031/viewer/2022022414/5870a4341a28abcb078b5673/html5/thumbnails/26.jpg)
26
01Docker based architecture
Multiple per host easily possibleEasy to scale until host resources allowEasily add constraints per containerLow overhead compared to traditional VMNative on Linux == best possible performance
![Page 27: How to Run Solr on Docker and Why](https://reader031.fdocuments.in/reader031/viewer/2022022414/5870a4341a28abcb078b5673/html5/thumbnails/27.jpg)
27
01
http://cliparts.co/clipart/3733708
![Page 28: How to Run Solr on Docker and Why](https://reader031.fdocuments.in/reader031/viewer/2022022414/5870a4341a28abcb078b5673/html5/thumbnails/28.jpg)
28
01Should I care about Solr?
Tuning is still needed, no matter what ENV you use
Keep in mind ENV constraintsWatch out for:
- merges – adjust or split data- memory issues- I/O subsystem- CPU utilization- garbage collection- proper Solr configuration
![Page 29: How to Run Solr on Docker and Why](https://reader031.fdocuments.in/reader031/viewer/2022022414/5870a4341a28abcb078b5673/html5/thumbnails/29.jpg)
29
01Should I care about Docker?Docker reports eventsWe can listen and react to eventsExample of events to react to
- kill- start- stop- destroy- die- restart - pause- unpause- oom
![Page 30: How to Run Solr on Docker and Why](https://reader031.fdocuments.in/reader031/viewer/2022022414/5870a4341a28abcb078b5673/html5/thumbnails/30.jpg)
30
01Should I care about Docker volumes?
Docker volumes also report eventsWe can listen and react to those as wellWe can automatically react to:
- create- mount- unmount- destroy
![Page 31: How to Run Solr on Docker and Why](https://reader031.fdocuments.in/reader031/viewer/2022022414/5870a4341a28abcb078b5673/html5/thumbnails/31.jpg)
31
01Solr + Docker, should I?
Yes, when in need of dynamic scalingYes, when in need of resources utilizationLots of nodes
-> look at swarm, mesos and kubernetes
![Page 32: How to Run Solr on Docker and Why](https://reader031.fdocuments.in/reader031/viewer/2022022414/5870a4341a28abcb078b5673/html5/thumbnails/32.jpg)
32
01Thank you! If you want to get in touch
Rafał[email protected]@ kucrafal
[email protected]@ radu0gheorghe
http://sematext.com@ sematext
Join US!
http://sematext.com/jobs/