Scylla Summit 2017: From Elasticsearch to Scylla at Zenly
-
Upload
scylladb -
Category
Technology
-
view
721 -
download
0
Transcript of Scylla Summit 2017: From Elasticsearch to Scylla at Zenly
![Page 1: Scylla Summit 2017: From Elasticsearch to Scylla at Zenly](https://reader030.fdocuments.in/reader030/viewer/2022020203/5a6512897f8b9aa2548b6a55/html5/thumbnails/1.jpg)
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
From Elasticsearch To Scyllaat Zenly
10 Month in production
Head of infrastructure, Zenly - https://zen.ly
Jean-Baptiste Dalido
![Page 2: Scylla Summit 2017: From Elasticsearch to Scylla at Zenly](https://reader030.fdocuments.in/reader030/viewer/2022020203/5a6512897f8b9aa2548b6a55/html5/thumbnails/2.jpg)
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
Jean-Baptiste Dalido
2
● Zenly’s Head of infrastructure, now a Snap Inc.
Company
● Experience with Mobile Startups, previously
Appgratis and Batch.com
● Worked with Cassandra in production for quite
some time
![Page 3: Scylla Summit 2017: From Elasticsearch to Scylla at Zenly](https://reader030.fdocuments.in/reader030/viewer/2022020203/5a6512897f8b9aa2548b6a55/html5/thumbnails/3.jpg)
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
![Page 4: Scylla Summit 2017: From Elasticsearch to Scylla at Zenly](https://reader030.fdocuments.in/reader030/viewer/2022020203/5a6512897f8b9aa2548b6a55/html5/thumbnails/4.jpg)
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
FROM ELASTICSEARCH
![Page 5: Scylla Summit 2017: From Elasticsearch to Scylla at Zenly](https://reader030.fdocuments.in/reader030/viewer/2022020203/5a6512897f8b9aa2548b6a55/html5/thumbnails/5.jpg)
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
X Use cases for X Databases
5
▪ Elasticsearch, as a main database
▪ Redis, for cache
▪ Cassandra, for gigantic datasets (chat messages)
▪ Postgres, for results of heavy computations
▪ Name it, it was in production ...
![Page 6: Scylla Summit 2017: From Elasticsearch to Scylla at Zenly](https://reader030.fdocuments.in/reader030/viewer/2022020203/5a6512897f8b9aa2548b6a55/html5/thumbnails/6.jpg)
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
ElasticSearch as a main database ?
6
▪ Easy data-modelling
▪ Search as a query language ooffsets data-model changes and deficiencies
o comes at a cost, search is not cheap
▪ KV Semantics
▪ Sharding/Replication
▪ Document versioning
![Page 7: Scylla Summit 2017: From Elasticsearch to Scylla at Zenly](https://reader030.fdocuments.in/reader030/viewer/2022020203/5a6512897f8b9aa2548b6a55/html5/thumbnails/7.jpg)
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
Why leaving ?
7
▪ Data-model became more and more stable
▪ Particular Workload, Heavy updates
▪ Unification of databaseso Less operational cost (onboarding, maintenance…)
o Simpler monitoring
▪ No real need for ElasticSearch features anymore
![Page 8: Scylla Summit 2017: From Elasticsearch to Scylla at Zenly](https://reader030.fdocuments.in/reader030/viewer/2022020203/5a6512897f8b9aa2548b6a55/html5/thumbnails/8.jpg)
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
WAIT, WHY NOT STICK WITH
CASSANDRA ?
![Page 9: Scylla Summit 2017: From Elasticsearch to Scylla at Zenly](https://reader030.fdocuments.in/reader030/viewer/2022020203/5a6512897f8b9aa2548b6a55/html5/thumbnails/9.jpg)
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
Total time for which application threads were stopped: (too many) seconds
AKA: Stop the world
9
![Page 10: Scylla Summit 2017: From Elasticsearch to Scylla at Zenly](https://reader030.fdocuments.in/reader030/viewer/2022020203/5a6512897f8b9aa2548b6a55/html5/thumbnails/10.jpg)
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
Cassandra ?
10
▪ Garbage Collection is a deal breaker for small clusters
▪ Overcome by a high number of nodes
▪ Proficiency requires knowledge and experience
▪ Cassandra without GC might be a close-to-perfect database
![Page 11: Scylla Summit 2017: From Elasticsearch to Scylla at Zenly](https://reader030.fdocuments.in/reader030/viewer/2022020203/5a6512897f8b9aa2548b6a55/html5/thumbnails/11.jpg)
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
MOVING TO SCYLLA(after days of debate, and no GC)
![Page 12: Scylla Summit 2017: From Elasticsearch to Scylla at Zenly](https://reader030.fdocuments.in/reader030/viewer/2022020203/5a6512897f8b9aa2548b6a55/html5/thumbnails/12.jpg)
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
WE WERE TOLD NOT TO USE DOCKER FOR
PRODUCTION(but we did it anyway)
![Page 13: Scylla Summit 2017: From Elasticsearch to Scylla at Zenly](https://reader030.fdocuments.in/reader030/viewer/2022020203/5a6512897f8b9aa2548b6a55/html5/thumbnails/13.jpg)
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
First migration, cluster in Kubernetes
13
▪ Easily deploy/update within our infrastructureo 12 Kubernetes clusters in production
▪ Except Kafka, we have everything in Docker
▪ Scale within minutes
▪ Networking, logs, everything is included
![Page 14: Scylla Summit 2017: From Elasticsearch to Scylla at Zenly](https://reader030.fdocuments.in/reader030/viewer/2022020203/5a6512897f8b9aa2548b6a55/html5/thumbnails/14.jpg)
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
First migration, cluster in Kubernetes
14
▪ Unexpected High latencies and load was over the roof
▪ ScyllaDB’s team to the rescue
▪ It’s all about Seastar internals
![Page 15: Scylla Summit 2017: From Elasticsearch to Scylla at Zenly](https://reader030.fdocuments.in/reader030/viewer/2022020203/5a6512897f8b9aa2548b6a55/html5/thumbnails/15.jpg)
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
Seastar and Kubernetes
15
▪ Seastar is sharding, one shard, one core
▪ Networking and data are sharded per core
▪ Pinned CPU in Kubernetes was not availableo keep an eye on Cpu-Manager
![Page 16: Scylla Summit 2017: From Elasticsearch to Scylla at Zenly](https://reader030.fdocuments.in/reader030/viewer/2022020203/5a6512897f8b9aa2548b6a55/html5/thumbnails/16.jpg)
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
Moving, part 2
![Page 17: Scylla Summit 2017: From Elasticsearch to Scylla at Zenly](https://reader030.fdocuments.in/reader030/viewer/2022020203/5a6512897f8b9aa2548b6a55/html5/thumbnails/17.jpg)
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
New infrastructure
17
▪ 3 clusters
▪ 2 single region clusters
▪ 1 double region clusteroRedundancyoRead-Heavy workload without impact on production
▪ 7 machines per cluster/regiono 10 cpuso 36Gb of RAMo x2 - 375GB local SSD
▪ No Docker
![Page 18: Scylla Summit 2017: From Elasticsearch to Scylla at Zenly](https://reader030.fdocuments.in/reader030/viewer/2022020203/5a6512897f8b9aa2548b6a55/html5/thumbnails/18.jpg)
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
GCP Deployment
18
▪ Ubuntu 16.04 LTS
▪ NVMeoCascade failure on GCP machines with kernel 4.8.0oUnexpected Chaos Monkey
▪ Resiliency theory proved to be right
▪ Failure did not provoke any downtime
![Page 19: Scylla Summit 2017: From Elasticsearch to Scylla at Zenly](https://reader030.fdocuments.in/reader030/viewer/2022020203/5a6512897f8b9aa2548b6a55/html5/thumbnails/19.jpg)
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
Clusters as feature isolation
19
▪ More nodes
▪ Better overview of workloado Feature teams can scale their needsoDevelopers can improve performances and see impactoOffer data teams a way to query production clusters
▪ Resiliency if implemented the right way
![Page 20: Scylla Summit 2017: From Elasticsearch to Scylla at Zenly](https://reader030.fdocuments.in/reader030/viewer/2022020203/5a6512897f8b9aa2548b6a55/html5/thumbnails/20.jpg)
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
Uptime
20
▪ 100% uptime for our 3 clusterso 0 cases of major failure due to ScyllaDB
▪ Few crashes that were always machine related
▪ No impact on production
![Page 21: Scylla Summit 2017: From Elasticsearch to Scylla at Zenly](https://reader030.fdocuments.in/reader030/viewer/2022020203/5a6512897f8b9aa2548b6a55/html5/thumbnails/21.jpg)
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
Everything is about performance
![Page 22: Scylla Summit 2017: From Elasticsearch to Scylla at Zenly](https://reader030.fdocuments.in/reader030/viewer/2022020203/5a6512897f8b9aa2548b6a55/html5/thumbnails/22.jpg)
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
300k requests per second on 7 nodes
22
![Page 23: Scylla Summit 2017: From Elasticsearch to Scylla at Zenly](https://reader030.fdocuments.in/reader030/viewer/2022020203/5a6512897f8b9aa2548b6a55/html5/thumbnails/23.jpg)
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
Latencies distribution over the same traffic
23
▪ p95 in under 1ms, provided by Google Cloud Platform
![Page 24: Scylla Summit 2017: From Elasticsearch to Scylla at Zenly](https://reader030.fdocuments.in/reader030/viewer/2022020203/5a6512897f8b9aa2548b6a55/html5/thumbnails/24.jpg)
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
Scylla is production ready
24
▪ It’s fast, so fastoMake mistakeso Experiments
▪ You don’t need to be a Cassandra ninja
▪ It’s a supported product (Scylla 2 is out)
▪ Keep an eye on Kubernetes
![Page 25: Scylla Summit 2017: From Elasticsearch to Scylla at Zenly](https://reader030.fdocuments.in/reader030/viewer/2022020203/5a6512897f8b9aa2548b6a55/html5/thumbnails/25.jpg)
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
Go behind usual use-cases
25
▪ Using Scylla as a cache instead of Redis ?
▪ I/O scheduler
▪ Memstore
▪ Cache
▪ Persistence beyond the Memory
▪ Amazing performances
▪ Do you still need a cache ?
![Page 26: Scylla Summit 2017: From Elasticsearch to Scylla at Zenly](https://reader030.fdocuments.in/reader030/viewer/2022020203/5a6512897f8b9aa2548b6a55/html5/thumbnails/26.jpg)
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
We don’t :)
![Page 27: Scylla Summit 2017: From Elasticsearch to Scylla at Zenly](https://reader030.fdocuments.in/reader030/viewer/2022020203/5a6512897f8b9aa2548b6a55/html5/thumbnails/27.jpg)
PRESENTATION TITLE ON ONE LINE AND ON TWO LINES
First and last namePosition, company
THANK YOU
@jbaptistedalido
Please stay in touch