Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory...
Transcript of Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory...
![Page 2: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/2.jpg)
2
About Redis
Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case.
• The open source home and commercial provider of Redis• Providing Data Analytics at the Speed of business• #1 Top Databases Popularity Rankings, #1 Leader and Challenger in Analyst
Quadrants• Multi-model, can support all popular database models and modern use cases
Founded in 2011HQ in Mountain View CA, R&D center in Tel-Aviv IL.
![Page 3: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/3.jpg)
3
A Quick Recap of Redis
Key
"I'm a Plain Text String!"
{ A: “foo”, B: “bar”, C: “baz” }
Strings / Bitmaps / BitFields
Hash Tables (objects!)
Linked Lists
Sets
Sorted Sets
Geo Sets
HyperLogLog
{ A , B , C , D , E }
[ A → B → C → D → E ]
{ A: 0.1, B: 0.3, C: 100, D: 1337 }
{ A: (51.5, 0.12), B: (32.1, 34.7) }
00110101 11001110 10101010
![Page 4: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/4.jpg)
4
Redise Platform
Redise Node
Cluster Manager
Redise Cluster• Shared nothing cluster architecture
• Fully compatible with open source
commands & data structures
Enterprise Layer
REST APIZero latency proxy
![Page 5: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/5.jpg)
5
Redise Cloud Private
Redis Labs Products
Redise Cloud Redise Pack ManagedRedise Pack
SERVICES SOFTWARE
Fully managed Redise service in VPCs within
AWS, MS Azure, GCP & IBM Softlayer
Fully managed Redise service on hosted servers
within AWS, MS Azure, GCP, IBM Softlayer,
Heroku, CF & OpenShift
Downloadable Redise software for any
enterprise datacenter or cloud environment
Fully managed Redise Pack in private data
centers
or or or
![Page 6: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/6.jpg)
6
But Can Redis Do X?
Secondary Index?
Time Series?
Full Text Search?
Graph?
Machine Learning?
AutoComplete?
SQL?
![Page 7: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/7.jpg)
7
Redis Modules
● C extensions to Redis
● Work at native speed
● Add commands
● Can implement their own native data types
● Can extend redis or just use it as a “Server
Framework”● Isolated API ● ABI Backwards Compatibility
![Page 8: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/8.jpg)
8
Adapt your database to your data
Secure way to store data in Redis via encrypt/decrypt
with various Themis primitives
Time Series Graph
Time series values aggregation in Redis
Crypto Engine Wrpper
Graph database on Redis based on Cypher language
Based on Generic Cell Rate Algorithm (GCRA)
Rate Limiter
ReJSON
Secondary Index/RQL
JSON Engine on Redis.Pre-released
Indexing + SQL -like syntax for querying indexes.
Pre-released
Neural Redis Redis-ML RediSearch
Full Text Search Engine in Redis
Machine Learning Model Serving
Simple Neural Network Native to Redis
![Page 9: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/9.jpg)
9
RediSearch=Text Search
![Page 10: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/10.jpg)
10
RediSearch V1
● From-Scratch search index over redis● Uses String DMA for holding compressed index
data● This allows efficient compression of data● Includes stemming, exact phrase match, etc.● Up to X5 faster than Elastic / Solr
![Page 11: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/11.jpg)
11
Search in Action
> FT.CREATE products SCHEMA title TEXT price NUMERICOK> FT.ADD products id1 1.0 FIELDS title "LG 42'' LCD TV" price 500OK> FT.ADD products id2 1.0 FIELDS title "Toshiba 40'' LCD TV" price 400OK> FT.SEARCH products "lcd tv" FILTER price 450 +inf1) (integer) 12) "id1"3) 1) "title" 2) "LG 42'' LCD TV" 3) "price" 4) "500"
![Page 12: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/12.jpg)
12
Where
Source code: https://github.com/RedisLabsModules/RediSearch
Documentation: http://redisearch.io/
![Page 13: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/13.jpg)
13
redis-ml +neural-redis=Machine Learning
![Page 14: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/14.jpg)
14
Redis-ML
● Database for ML models● Train wherever you want● Store the result in redis● Query models in real time● Scale your models beyond a single machine
![Page 15: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/15.jpg)
15
Redis Module
Tree Ensembles
Linear Regression
Logistic Regression
Matrix + Vector Operations
More to come...
The Redis-ML Module
![Page 16: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/16.jpg)
16
Neural Redis
● Developed by Salvatore● Training is done inside
Redis● Online continuous training
process● Builds Fully Connected NNs
![Page 17: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/17.jpg)
17
Neural Redis In Action
>NR.CREATE net REGRESSOR 2 3 -> 1 NORMALIZE DATASET 50 TEST 10
>NR.OBSERVE net 4 5 -> 9
>NR.OBSERVE net 1 1 -> 2
....
>NR.OBSERVE net 5 6 -> 11
>NR.TRAIN net AUTOSTOP
>NR.RUN net 1 1
1) "2.1406970024108887"
>NR.RUN net 3 5
1) "8.3342075347900391"
![Page 18: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/18.jpg)
18
WhereRedis MLSource code: https://github.com/redislabsmodules/redis-ml
Neural RedisSource code: https://github.com/antirez/neural-redis
![Page 19: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/19.jpg)
19
ReJSON = Redis + JSON
![Page 20: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/20.jpg)
20
ReJSON [Preview Release]
● A custom JSON data type for Redis ● Keys can contain any valid JSON value
● Scalars, objects or arrays● Nested or not
● Data is stored decoded in binary format● JSONPath-like syntax for direct access to
elements● Strongly-typed atomic commands
![Page 21: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/21.jpg)
21
ReJSON In Action
> JSON.SET doc . '{ "foo": "bar", "baz": [1,2] }'> JSON.SET doc .goo true> JSON.NUMINCRBY doc .baz[0] 9> JSON.GET doc .baz"[10,2]"> JSON.GET doc ."{\"foo\":\"bar\",\"baz\":[10,2],\"goo\":true}"
![Page 22: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/22.jpg)
22
General JSON.DEL, JSON.GET, JSON.MGET, JSON.SET & JSON.TYPE
Numbers JSON.NUMINCRBY & JSON.NUMMULTBYStrings JSON.STRAPPEND & JSON.STRLENObjects JSON.OBJKEYS & JSON.OBJLENArrays JSON.ARRAPPEND, JSON.ARRINDEX, JSON.ARRINSERT, JSON.ARRLEN, JSON.ARRPOP &
JSON.ARRTRIM
Other JSON.RESP
ReJSON commands
![Page 23: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/23.jpg)
23
Where
Source code: https://github.com/RedisLabsModules/rejsonDocumention: https://redislabsmodules.github.io/rejson
![Page 24: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/24.jpg)
24
redis-secondary=Secondary Indexes
![Page 25: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/25.jpg)
25
Redis-Secondary
● Schema aware secondary indexes● Integers, floats, booleans, strings, etc
● Either a raw data type or automatic index● Treat HASH keys as objects or table rows● SQL-like WHERE clauses as proxies to redis
commands
![Page 26: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/26.jpg)
26
Secondary Indexing In Action> IDX.CREATE users_name_age TYPE HASH SCHEMA name STRING age INT32
> IDX.INTO users_name_age HMSET user1 name "alice" age 30
> IDX.INTO users_name_age HMSET user2 name "bob" age 25
> IDX.FROM users_name_age WHERE "name LIKE 'b%'" HGET $ name
1) user1
2) "bob"
> IDX.FROM users_name_age WHERE "name >= 'alice' AND age < 31" HGETALL $
1) user1
...
![Page 27: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/27.jpg)
27
Where
Source code: https://github.com/RedisLabsModules/secondary
![Page 28: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/28.jpg)
28
Modules For Probabilistic Data Analysis
● Optimized for memory space efficiency ● Ultra-fast query response times even with millions of elements● Does not require storing of the actual streaming data elements
Counts of observations:Tracks the frequency of samples in a stream of observations
redabloom topk countminsketch
Bloom filter:Used for set membership
Top k most frequent:Counts the top most frequent elements in a stream of data
T-digest
Rank based stats estimator:Useful for quantiles, cumulative distributions for very large datasets
![Page 29: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/29.jpg)
29
Redabloom=Scalable, counting bloom filters● Membership Query. Test whether
an element is a member of a set. ● False positive matches are
possible, but false negatives are not
● Provides scalable, counting bloom filters
● Port of the dablooms library● Provides automatic ID generation
using a sequence or the server’s clock
Source:https://github.com/RedisLabsModules/redablooms
> CBF.ADD users Jeff
> CBF.ADD users Bob
> CBF.ADD users Itamar
> CBF.ADD users Dvir
> CBF.REM users Itamar
> CBF.CHECK users Dvir
(integer) 1
> CBF.CHECK users kamran
(integer) 0
> CBF.CHECK users Itamar
(integer) 0
>
![Page 30: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/30.jpg)
30
> TDIGEST.NEW latency.disk1
> TDIGEST.ADD latency.disk1 7
> TDIGEST.ADD latency.disk1 30
....
> TDIGEST.QUANTILE latency.disk1
0.95
1) "32.124999999999993"
>
t-digest=quantiles and cumulative distribution
● Accurate on-line accumulation of rank-based statistics
● Quantiles and cumulative distribution
Source: https://github.com/usmanm/redis-tdigest
![Page 31: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/31.jpg)
31
The Start of a Module Ecosystem
● Modules were launched May 2016 at RedisConf● Initial modules started to appear, mostly toy projects● We launched Modules-Hub
![Page 32: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/32.jpg)
32
Redis Modules Hack 2016
● A global, distributed hackathon● With on-site events in TLV and SF● Some very interesting projects● We gave them cash!
![Page 33: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/33.jpg)
33
First Place - Redis-Cell
● Author: Brandur Leach● Advanced rate limiting algorithm● Actually written in Rust, not C
CL.THROTTLE user123 15 30 60 1
▲ ▲ ▲ ▲ ▲
| | | | └───── apply 1 token (default if omitted)
| | └──┴─────── 30 tokens / 60 seconds
| └───────────── 15 max_burst
└─────────────────── key "user123"
![Page 34: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/34.jpg)
34
Notable Mention: Redis-Graph
● Author: Roi Lipman● A fast graph database implemented in Redis● Subset of Cypher (Neo4J QL) implementation● Uses Redis Hash keys as nodes
GRAPH.QUERY presidents
"MATCH (president)-[born]->(state:Hawaii)
RETURN president.name, president.age"
![Page 35: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/35.jpg)
35
Other Cool Modules
● Time-series aggregation engine● Pyrecks - Python function execution● Statsd reporting of Redis metrics● Zstd value compression● Time decaying popularity ranking● Shared memory support● And many more!
![Page 36: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/36.jpg)
36
When
![Page 37: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/37.jpg)
37
Redis 4.0 status
● RC2 released 2 months ago● RC3 soon to be released ● Surprisingly little serious issues reported so far ● However… a few things to fix / add before
4.0-final ● ETA: about 2 months
![Page 38: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/38.jpg)
38
Coming In Redise
● To be released soon in Redis Labs’ enterprise cluster
● Pre-bundled modules:● RediSearch● ReJSON● Secondary● Redis-ML
● Use your own modules at their own risk
![Page 39: Multiple NoSQL Use Cases with Redis Modules...2 About Redis Open source. The leading in-memory database platform, supporting any high performance OLTP or OLAP use case. • The open](https://reader035.fdocuments.in/reader035/viewer/2022071603/613e2f9c59df642846165ddb/html5/thumbnails/39.jpg)
39
Thank You!