Use Cases of Elasticsearch and Considerations in Multi-tenancy
-
Upload
- -
Category
Technology
-
view
252 -
download
0
Transcript of Use Cases of Elasticsearch and Considerations in Multi-tenancy
![Page 1: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/1.jpg)
Use Cases of Elasticsearch AND
Considerations in Multi-tenancy
2016-09-14
eurie Inc. Takahiro Ikeuchi
![Page 2: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/2.jpg)
AgendaAbout us and our product
Our use cases of Elasticsearch
Combination with RDB
Considerations in Multi-tenancy
How do we search private documents ?
Self-Managed vs Amazon ES
2
![Page 3: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/3.jpg)
Conclusion
3
![Page 4: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/4.jpg)
ConclusionIn the case of Multi-tenancy model, processingAuthentication and Authorization should behandled by your backend applications.
If a strict synchronous execution is not needed,an asyncronous execution using AWS Labmdais a good option.
You may want to implement wrapper logics tostandardize interfaces.
I would recommend that you adopt "self-managed" until Amazon ES supports VPC.
4
![Page 5: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/5.jpg)
About us and our product
5
![Page 6: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/6.jpg)
AuthorTakahiro Ikeuchi @iktakahiro
Company / Community
eurie Inc. Founder & CEO
SQUEEZE Inc. Tech Adviser
PyData.Tokyo Organizer
Specialties (or just a dabbler :-D
Go lang, Python, React.js, TypeScript
Cloud Infrastructure, UI Design etc...
6
![Page 7: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/7.jpg)
7
![Page 8: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/8.jpg)
A Customer Support Tool for Innovative Teams
8
![Page 9: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/9.jpg)
ArchitecturesRESTful API
Go lang (framework: echo)
Single Page Application
React.js + Typescript
Elasticsearch
Cloud Native
AWS Aurora, Lambda, CloudFront, WAF...
Microservice Architecture
9
![Page 11: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/11.jpg)
Our use cases of Elasticsearch
11
![Page 12: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/12.jpg)
Searching Message
12
![Page 13: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/13.jpg)
13
![Page 14: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/14.jpg)
Why Elasticsearch ?
14
![Page 15: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/15.jpg)
Why Elasticsearch?Full-Text Search
Morphological Analysis
High-Performance / Scalable
Powerful weighted-search, sort, �lter
Similar documents Recommendations
Pretty REST API, Client Library etc...
15
![Page 16: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/16.jpg)
Architecture
16
![Page 17: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/17.jpg)
Architecture
17
![Page 18: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/18.jpg)
GET docs from Elasticsearch
18
![Page 19: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/19.jpg)
The �ow of searching messages
1. A user enters a keyword in the client app. A client sends a GET Request to a REST API
on the backend app.
2. The REST API receives the GET Request. Authentication / Authorization
3. The backend app sends a GET Request toElasticsearch Receiving search-results.
4. The backend app builds / returns a response.
5. The client app receives the response from theREST API. Rendering User-Interfaces.
19
![Page 20: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/20.jpg)
GET docs from Elasticsearch
20
![Page 21: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/21.jpg)
Web API Design
21
![Page 22: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/22.jpg)
Web API DesignGET /meesage/search?q=word&page=1&per_page=10
Response Body
{ "took": 36, "_scroll_id": "", "hits": { "total": 1,
The above JSON is a responce of Elasticsearchitself. It was not changed.
22
![Page 23: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/23.jpg)
Web API DesignResponse Header
Link:
Createing a Response Header: Link based on total (and the others) value in a search-result.
RFC 5988 - Web Linkinghttps://tools.ietf.org/html/rfc5988
</messages/search?q=gmail&page=1&per_page=10>; rel="first",</messages/search?q=gmail&page=2&per_page=10>; rel="next", </messages/search?q=gmail&page=2&per_page=10>; rel="last"
23
![Page 24: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/24.jpg)
Combination with RDB
24
![Page 25: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/25.jpg)
Considerations in combination with RDB
Would you really need RDB?
Would you need a strict synchronous?
Is UPDATE/DELETE executed frequently?
In the case of eurie Desk :
At the �rst, all documents are inserted intoRDB (Amazon Aurora).
Just after inserting, documents are also put toElasticsearch asynchronously.
Elasticsearch is used on write-only situations.25
![Page 26: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/26.jpg)
POST docs to RDB
26
![Page 27: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/27.jpg)
PUT docs to Elasticsearch (Async)
27
![Page 28: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/28.jpg)
Considerations in Multi-tenancy
28
![Page 29: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/29.jpg)
The case of Multi-tenancy model
â Issues
A user or role based access controll system fordocuments is needed.
Documents should not be able to referred byother tenants.
How do we search private documents?
29
![Page 30: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/30.jpg)
The case of Multi-tenancy model
Solutions
You should build Elasticsearch in a internal(private) network.
A processing Authentication and Authorizationshould be handled by your backend apps.
Of couse use Index for each tenant.
30
![Page 31: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/31.jpg)
RDB and ES instances boot in the Private Network
31
![Page 32: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/32.jpg)
Amazon ES Pros/Cons
Amazon ES = Amazon Elasticsearch Service
https://aws.amazon.com/elasticsearch-service/
Pros
Very easy, even High-Available structures.
By default, Amazon ES has Kuromoji-plugin. So you can use Elasticsearch immediately.
32
![Page 33: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/33.jpg)
Amazon ES Pros/Cons
Cons
It's only booted in a global netowrk area.
IP Restrictions OK, but we can't use "Secuirty Groups".
Or there is an option to develop a systemusing "IAM"...
Is the newest version of Elasticsearch releasedimmediately?
We don't have permission to install plug-infreely. 33
![Page 34: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/34.jpg)
Self-Managed vs Amazon ES
If you want use Elasticsearch to search none-private documents, It might be that adoptingAmazon ES is a wise way.
On the other hand, in the case of privatedocuments, consider to adopt Self-Managed.
It's layer is same as RDB.
I hope Amazon ES supports soon.
34
![Page 35: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/35.jpg)
ConclusionIn the case of Multi-tenancy model, processingAuthentication and Authorization should behandled by your backend applications.
If a strict synchronous execution is not needed,an asyncronous execution using AWS Labmdais a good option.
You may want to implement wrapper logics tostandardize interfaces.
I would recommend that you adopt "self-managed" until Amazon ES supports VPC.
35
![Page 36: Use Cases of Elasticsearch and Considerations in Multi-tenancy](https://reader031.fdocuments.in/reader031/viewer/2022022203/587790be1a28ab0f778b5167/html5/thumbnails/36.jpg)
AppendixElasticsearch o�cial site
Elasticsearch client library for Go lang:olivere/elastic
eurie.io
36