Redis Developers Day, Oct 19th 2015, London

Itamar Haber, Chief Developer Advocateitamar@redislabs.com

New Data Structure or New API?

3

A range of queries that’re non-trivial with Redis

• Sorted Sets are awesome for answering single score queries• Composite scores (53 bits space) are also very useful• With ZRANGEBYLEX you can do any range queries• Have you read the new page at http://redis.io/topics/indexes?

• But a common query “category” that’s non-trivial and inefficient to do with Redis is ranges over n-dimensions

4

Example: 2-dimensional query

Given users ages and salaries – find all that are between x1 and x2 years old and earn between y1 and y2

Sorted Sets zquery “Algorithm”:1. ZUNIONSTORE tmp 1 ages2. ZREMBYRANGE ages –inf (x13. ZREMBYRANGE ages (x2 +inf4. ZINTERSTORE tmp 2 tmp salaries WEIGHTS 0 15. reply = ZRANGEBYSCORE tmp [y1 [y26. [DEL tmp]

5

A proof of concept: Quadtree in Redis Hash

https://gist.github.com/itamarhaber/c1ffda42d86b314ea701

Idea: store nodes as hash fields, lazy loading

Benchmark results on 100K points

zquery: 2.41 requests per secondQuadtree query: 73.69 requests per second

k-d trees are similar, but for k dimensions

6

So?

1. Proposed to @antirez to add k-d trees2. Have you read the new page at http://redis.io/topics/indexes? 3. We still need, regardless the data structure, a higher level API to

do effective querying (reminder: there’s no STORE in ZRANGEBYLEX)

Not a real command but:

SELECT myzset WHERE cond1 cond2 cond3…

The State of Lua

8

We need a debugger

You can trace: redis.log, PUBLISH, ECHO…You can simulate: http://www.trikoder.net/blog/redis-and-lua-scripts-new-workflow-89But you can’t debug “in vitro” (https://github.com/RedisLabs/redis-lua-debugger/ - doesn’t work since https://github.com/antirez/redis/commit/30278061cc834b4073b004cb1a2bfb0f195734f7)Two approaches:1. Embedded debugger (https://github.com/slembcke/debugger.lua)2. Remote debugger (ZeroBrane?)

9

Other Lua Stuff

• Calling a script from a script (f_<sha1> hack? Modules?)• Persist Lua scripts and load them from disk, also on startup• Keyspace notifications/PubSub triggered Lua scripts ~= triggers

(mattsta)

10

Lua PRs

• Removes dofile() from Lua by evilpacket: antirez/redis: https://github.com/antirez/redis/pull/732

• Lua-based maxmemory eviction policies by lamby: https://github.com/antirez/redis/pull/2319

• new Lua memory limit by dovkfir: antirez/redis: https://github.com/antirez/redis/pull/2505

• Adds Lua transaction support by josiahcarlson: antirez/redis: https://github.com/antirez/redis/pull/2701

• No PR, but Redis Labs once suggested doing operation replication to avoid long running scripts on slaves

Thank you