History of ZFSMatt Ahrens

Delphix

History of ZFSBroad StrokesPrehistory: before 2001

Early days: 2001-2005

Maturation: 2005-2010

Production: 2010-2016

Future: 2016+

Feature StoriesSend/Receive

Scrub/Resilver

Deduplication

Prehistory1995-2001

Prehistory1995-2001

End the suffering

Early Days2001-2005

Maturation2006-2010

Maturation2006-2010

With the help of Dtrace

Production2011-2016

Production2011-2016

With the help of Dtrace

OpenZFS Write Throttle: write latency histogram

One line per txg:https://github.com/ahrens/dtrace/blob/master/txg.xd

txg 15332333 is nexttime since last sync | written by sync | | syncing time (% pass 1) | | | | write rate while syncing | | | | | highest dirty (% of limit) | | | | | | highest throttle delay | | | | | | | | v v v v v v v v 0ms 23MB in 1045ms ( 7% p1) 22MB/s 999MB (24%) 0ms 786MB in 7977ms (59% p1) 98MB/s 1079MB (26%) 0ms 1174MB in 8311ms (79% p1) 141MB/s 1303MB (31%) 0ms 912MB in 6998ms (49% p1) 130MB/s 1287MB (31%) 0ms 912MB in 5083ms (69% p1) 179MB/s 1175MB (28%) 0ms 771MB in 7372ms (69% p1) 104MB/s 1644MB (40%) 73us 1259us

One line per 100ms: https://github.com/ahrens/dtrace/blob/master/writethrottle.xd

spa_sync(txg=15332324) startingwr=766 (32127KB) undirtied=55720KB dirty=42% q/a/m: aw=4139/80/80wr=684 (34096KB) undirtied=58492KB dirty=37% q/a/m: aw=10569/60/60wr=549 (27962KB) undirtied=36188KB dirty=35% q/a/m: aw=13443/40/40wr=568 (32864KB) undirtied=39060KB dirty=34% q/a/m: aw=17476/40/40...wr= 69 ( 831KB) undirtied= 2460KB dirty=37% q/a/m: aw=805/6/60wr= 52 ( 3236KB) undirtied= 3032KB dirty=39% q/a/m: aw=549/6/60wr= 85 ( 1509KB) undirtied= 1800KB dirty=40% q/a/m: aw=316/8/80wr= 57 ( 0KB) undirtied= 0KB dirty=41% q/a/m: aw=968/8/80wr= 79 ( 0KB) undirtied= 0KB dirty=44% q/a/m: aw=257/10/100wr=250 ( 6094KB) undirtied= 0KB dirty=46% q/a/m: aw=1184/86/100 dsl_pool_sync(pass=1) took 13647ms

Send / Receive

2005: Matt goes to China, gets frustrated with network latency2007: send -R [for Fishworks]2015: receive prefetching2015: resumable send/receive

Scrub / Resilver

Initially: restart whole resilver when create snapshot2008: complete rewrite to fix that

https://blogs.oracle.com/ahrens/entry/new_scrub_code(note mention of “bp rewrite” for device removal)

2015: don’t redo last part of resilver when create snapshot2017? Mostly-in-order resilver

Deduplication

2009: One of the last major ZFS features from Sun2010: Lots of performance work

Single-threaded free!Future? Still ripe for future work

Future2017+

Performance&

Integration