Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed...
Transcript of Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed...
![Page 1: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/1.jpg)
Redis 7 andBeyondMeir Shpilraien, Yossi Gottlieb & Itamar Haber
![Page 2: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/2.jpg)
Itamar HaberTechnology Evangelist @ Redis Labs
I'm a Redis GeekCommunity Leader @ Redis Core Team
I love counting: one, two, three! Ah ah ah!I have an obsession with data, technology and people around it
Partner of the One, fathered the Three & adoring my dog
![Page 3: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/3.jpg)
Redis 7: Agenda
1. Last year in Open Source Redis
2. Redis Functions
3. RedisRaft
4. A Glimpse into the Future
![Page 4: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/4.jpg)
Redis 7 in a Nutshell
● To be released later this year
● To be driven and developed by the community
● An opportunity to introduce new features, major changes & break stuff
● Some features are already being designed and/or implemented
● Others are waiting to be...
![Page 5: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/5.jpg)
● Apr 30, 2020: Redis 6.0
● Jun 30, 2020: Antirez steps back
● Jul 1, 2020: New project governance model
● Jul 9, 2020: Redis 6.0.6
● Feb 22, 2021: Redis 6.2 "Community Edition"
● Feb 26, 2021: 🎂 Redis is 12 years old
Last Year in Open Source Redis Some of the changes:
● Core team made up of 5 members
● 14 Redis Contributors appointed
● A revamped release process
● Open discussions & plans
● Pull Request review process
![Page 6: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/6.jpg)
The governance model is intended to be a meritocracy, aiming to empower individuals who demonstrate a long-term commitment and make significant contributions
![Page 7: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/7.jpg)
Metric Before After Change
PRs Created 526 820 56%
PRs Closed 421 1078 156%
PRs Open 889 634 -29%
Git Authors 80 149 86%
Code Contributors
Brewed in cauldron.io from
Data from the unstable branch:
● Before: 9 months between 2019-10-01 and 2020-06-30
● After: 9 months between 2020-07-01 and 2021-03-31
Drive-byRepeat
* Higher is better
Drive-by and Repeat (5+) Contributor Counts
![Page 8: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/8.jpg)
Redis 7 and Beyond:The Content
![Page 9: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/9.jpg)
Redis Functions
Meir Shpilraien
![Page 10: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/10.jpg)
Agenda
Redis FunctionLua
Current approach to scripting in Redis
New approach to programmability in Redis
A Redis Module, more powerful than Lua and less complex than module
RedisGears
Powerful but less accessible
Modules
![Page 11: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/11.jpg)
Lua Scripting
● Added in Redis 2.8● Allows compose logic that runs server side● Considered part of the client application, As such:
○ Not replicated ○ Not persisted○ May be lost at any time
● Client application must keep the Lua scripts in the application itself, and handle versioning and compatibility across clients
![Page 12: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/12.jpg)
Redis Modules
● Added in version 4.0● Can extend Redis in an almost any way● Potentially unsafe (a bug in module will crash the Redis)● Written in C (or some other low level language), which is less
prefered by most developers● Not part of Redis core
![Page 13: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/13.jpg)
RedisGears
● A Redis module (disclaimer: author here)● Provide Python API (Java, JS, C, and more languages are
coming)● Cluster agnostic● Supports Event processing
![Page 14: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/14.jpg)
Redis Functions - Proposal
● Enhance the existing Lua scripts support. As such, functions have:○ Full interaction with Redis using commands invocation○ Atomicity
● Considered part of the server. As such, Functions are:○ Replicated○ Persisted○ Named
● Invocation is done by the function name● Agnostic to the programing language (Lua5.1 and QuickJS)
![Page 15: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/15.jpg)
Redis Functions API
● FUNCTION CREATE ENGINE_NAME [REPLACE] [ARGS_DESCRIPTOR <ARGS DESCRIPTOR>] [DESC <DESCRIPTION>] BLOB
● FUNCTION CALL NAME NUM_KEYS key1 key2 … ARGS arg1 arg2● FUNCTION INFO NAME● FUNCTION DELETE NAME
![Page 16: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/16.jpg)
Demo
![Page 17: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/17.jpg)
RedisRaft in a Nutshell
● Strong consistency (strict serializable) Redis deployment option
● 100% Redis Module
● Open-source dual licensing (AGPLv3/RSAL)
![Page 18: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/18.jpg)
Raft in a Nutshell
Leader
Follower Follower
LogFSM
● Leader Election
● Log Replication
● Commit & Apply to FSM
RPC
For more information, see:https://raft.github.io
![Page 19: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/19.jpg)
Architecture
Commands
Raft Thread
Async I/O RPC
Raft Library
Raft Log Log File
Applied Commands
Module API
Cluster Nodes
RPC
RPC
↑ Queue ↓
![Page 20: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/20.jpg)
/bin/bash$
![Page 21: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/21.jpg)
Where we stand?
● More work to do before 1.0
● Jepsen testing round #1 (Q1/2020)
● First release - H2/2021 🤞https://github.com/RedisLabs/RedisRaft
![Page 22: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/22.jpg)
RedisRaft ∈ Redis 7
● Our mission is scaling up
● Redis becomes an ecosystem
○ Clients
○ Modules
○ Dev & Ops Tools
![Page 23: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/23.jpg)
Redis 7 and Beyond:The Known Knowns...
![Page 24: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/24.jpg)
Science Fiction, a.k.a Projects/4
● Nested Element Expiry
● Reliable Keyspace Notifications
● Key Annotations
● Cluster & Sentinel: next steps
● Incremental evolutions: ACL, Streams, multi-part AOF, Module API, ...
● Place your bets?
![Page 25: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/25.jpg)
History Will Judge Us
● It is hardly only about the code
● It is a collaborative effort by a community
● There's more than one way to peel this banana, including #HelpWanted:adding your opinion, giving feedback, designing features, reviewing code, testing the implementation, documenting everything, triaging new issues, reproducing known bugs, optimizing performance, solving hard problems…
● See you on github.com/redis/redis
![Page 26: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/26.jpg)
Thank [email protected]
![Page 27: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/27.jpg)
Redis Functions
Meir Shpilraien
![Page 28: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/28.jpg)
Agenda
Redis FunctionLua
Current approach to scripting in Redis
New approach to programmability in Redis
A Redis Module, more powerful than Lua and less complex than module
RedisGears
Powerful but less accessible
Modules
![Page 29: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/29.jpg)
Lua Scripting
● Added in Redis 2.8● Allows compose logic that runs server side● Considered part of the client application, As such:
○ Not replicated ○ Not persisted○ May be lost at any time
● Client application must keep the Lua scripts in the application itself, and handle versioning and compatibility across clients
![Page 30: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/30.jpg)
Redis Modules
● Added in version 4.0● Can extend Redis in an almost any way● Potentially unsafe (a bug in module will crash the Redis)● Written in C (or some other low level language), which is less
prefered by most developers● Not part of Redis core
![Page 31: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/31.jpg)
RedisGears
● A Redis module (disclaimer: author here)● Provide Python API (Java, JS, C, and more languages are
coming)● Cluster agnostic● Supports Event processing
![Page 32: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/32.jpg)
Redis Functions - Proposal
● Enhance the existing Lua scripts support. As such, functions have:○ Full interaction with Redis using commands invocation○ Atomicity
● Considered part of the server. As such, Functions are:○ Replicated○ Persisted○ Named
● Invocation is done by the function name● Agnostic to the programing language (Lua5.1 and QuickJS)
![Page 33: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/33.jpg)
Redis Functions API
● FUNCTION CREATE ENGINE_NAME [REPLACE] [ARGS_DESCRIPTOR <ARGS DESCRIPTOR>] [DESC <DESCRIPTION>] BLOB
● FUNCTION CALL NAME NUM_KEYS key1 key2 … ARGS arg1 arg2● FUNCTION INFO NAME● FUNCTION DELETE NAME
![Page 34: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/34.jpg)
Demo
![Page 35: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/35.jpg)
Thank [email protected]
![Page 36: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/36.jpg)
RedisRaft in a Nutshell
● Strong consistency (strict serializable) Redis
deployment option
● 100% Redis Module
● Open-source dual licensing (AGPLv3/RSAL)
![Page 37: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/37.jpg)
Raft in a Nutshell
Leader
Follower Follower
LogFSM
● Leader Election
● Log Replication
● Commit & Apply to FSM RPC
For more information, see:https://raft.github.io
![Page 38: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/38.jpg)
Architecture
Commands
Raft Thread
Async I/O RPC
Raft Library
Raft Log Log File
Applied Commands
Module API
Cluster Nodes
RPC
RPC
↑ Queue ↓
![Page 39: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/39.jpg)
/bin/bash$
![Page 40: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/40.jpg)
Where we stand?
● More work to do before 1.0
● Jepsen testing round #1 (Q1/2020)
● First release - H2/2021 🤞
https://github.com/RedisLabs/RedisRaft
![Page 41: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/41.jpg)
RedisRaft ∈ Redis 7
● Our mission is scaling up
● Redis becomes an ecosystem
○ Clients
○ Modules
○ Dev & Ops Tools
![Page 42: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/42.jpg)
Redis 7 and Beyond:The Known Knowns...
![Page 43: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/43.jpg)
Science Fiction, a.k.a Projects/4
● Nested Element Expiry
● Reliable Keyspace Notifications
● Key Annotations
● Cluster & Sentinel: next steps
● Incremental evolutions: ACL, Streams, multi-part AOF, Module API, ...
● Place your bets?
![Page 44: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/44.jpg)
History Will Judge Us
● It is hardly only about the code
● It is a collaborative effort by a community
● There's more than one way to peel this banana, including #HelpWanted:adding your opinion, giving feedback, designing features, reviewing code, testing the implementation, documenting everything, triaging new issues, reproducing known bugs, optimizing performance, solving hard problems…
● See you on github.com/redis/redis
![Page 45: Redis 7 and Beyond...Redis 7 in a Nutshell To be released later this year To be driven and developed by the community An opportunity to introduce new features, major changes & break](https://reader036.fdocuments.in/reader036/viewer/2022071603/613e2f9f59df642846165ddd/html5/thumbnails/45.jpg)
Thank [email protected]