Redis Developers Day 2014 - Redis Labs Talks
-
Upload
redis-labs -
Category
Technology
-
view
320 -
download
1
description
Transcript of Redis Developers Day 2014 - Redis Labs Talks
@Redis Developers Day 2014
1st talk: day to day challenges
● ±150 new databases every day (>60,000 total)
● A mix of very small (few MB) and extremely large (TB+)
datasets
● PAYG model that can grow to any size by changing
maxmemory
● ±250 node failure events (~10 every month) – w/o
upgrades
● 5-6 complete data-center outages so far
● Supporting/managing DBs with zero knowledge of
application
● Running a modified version of Redis from day 1
2nd talk: Extending Redis to Flash
● Redis #1 hindering factor is the cost of
deployment ➔ RAM costs x10 SSD and x250 HDD
● Advanced flash technologies:➔ 1M IOPS (4KB)
➔ Latency: DIMM – 1-2µsec; Plugable – 100µsec
Why do we modify Redis?
● Dense multi-tenant architecture and still
maintaining SLAs (single-tenant, dedicated
instances supported)
● Efficient large-scale system administration
● Clustering (in production since 06/2012)
How do we manage changes?
● Many custom modifications are of no use for anyone else,
so we don’t bother
● Fixes, minor changes to upstream ASAP
● Periodic batches of more significant features
o Low hit rate, community view vs. enterprise/service
provider view
● Our “playground” fork
https://github.com/redislabs/redis
Skin-of-our-teeth replication
● DB is growing, sync to a bigger instance
● Lots of writes, output buffers and COW bloat
● Redis has a point of no return
⇓
Off-load some of this to the slave
Flush output buffer to slave, Mux RDB
Improved Replication Performance
Master Memory
Master Disk Network Slave Disk Slave Memory
● Eliminate disk impact on replication performance (pain point)● Current Redis scheme can still be better sometimes
Lua Replication
● Old debate: SCRIPT vs Commandso In some cases performance is dramatic
o We think both options are needed (user-controlled)
External RDB processing
● We do it often, and not only us:
https://github.com/sripathikrishnan/redis-rdb-tools
● Python is too slow for serious work
● We want to avoid code duplication
● Built-in rdb.c was too hard-coded
● Alternative SAX-like (callback-based) parser
● Maybe even librdb.a?
Resource Management
● maxmemory can be confusing (think eviction/OOM
because of output buffers)
● How big can we expect the actual process? Master
switches are missing (e.g. max allocated network buffers)
● Graceful handling of output buffer limits
● Fragmentation issues
o jemalloc and the 12% magic
o active defrag and bigger ideas for bigger problems
More Resource Management
● Automatic process oom_score_adj settings to
guard data in low memory events
● Automatically reject BGSAVE and
BGREWRITEAOF or terminate child on low
memory
Cluster features
● Lua key ranges / patterns
● Global keys
Extending Redis to Flash - goals
● More (cheap) memory not better persistence
● 100% compatibility with Redis
● Pluggable storage back-ends / non-volatile DIMM
● Utilize storage concurrency
● Use case: limited set of hot objects over large
keyspace
● Different from other approaches: Ledis, ARDB,
SSDB
Extending Redis to Flash - blocks
Main thread
Clients
I/O jobqueues
I/O Threads
K/V storage backend API
Storage backend
Extending Redis to Flash - moar
● Still being prepared for prime time
● Planned to be open sourced once ready
Thank You