Rohonc Codex downloaded from : … · Rohonc Codex downloaded from :
[JSDC 2016] Codex: Conditional Modules Strike Back
Click here to load reader
Transcript of [JSDC 2016] Codex: Conditional Modules Strike Back
Alex Liu
A N E T F L I X E N G I N E E R I N G O R I G I N A L
codex
conditionaL ModuleS Strike bacK
#netflixeverywhere
contentsI. The Problem
II. Codex: Conditional Bundling
III. Scaling for Netflix
IV. Looking to the Future
i. the Problem
It's all about building JS bundles.
// bundle.js var a = require('a'); var b = require('b');
bundle.js
// bundle.js var a = require('a'); var b = require('b');
bundle.js
var bundles = [ 'bundle.js' ];
bundle.js
home.jsprofile.jsfeed.js album.js signup.js
home.js profile.js feed.js
signup.js account.js setting.js
album.js photo.js login.js
var bundles = [ 'home.js', 'profile.js', 'feed.js', 'signup.js' 'account.js', 'settings.js', 'album.js', 'photo.js', 'login.js' ];
var bundles = [ 'home.js', 'profile.js', 'feed.js', 'signup.js' 'account.js', 'settings.js', 'album.js', 'photo.js', 'login.js' ];
/home /profile /feed
var bundles = [ 'home.js', 'profile.js', 'feed.js', 'signup.js' 'account.js', 'settings.js', 'album.js', 'photo.js', 'login.js' ];
/home /profile /feed
home.js profile.js feed.js
html5shiv es5-shim
var bundles = [ 'home.js', 'homeIE.js', 'profile', 'profileIE', 'feed.js', 'feedIE.js' 'signup.js', 'signupIE.js', ... ];
var bundles = [ 'home.js', 'homeIE.js', 'profile', 'profileIE', 'feed.js', 'feedIE.js' 'signup.js', 'signupIE.js', ... ];
var bundles = [ 'home.js', 'homeIE.js', 'profile', 'profileIE', 'feed.js', 'feedIE.js' 'signup.js', 'signupIE.js', ... ];
/home
var bundles = [ 'home.js', 'homeIE.js', 'profile', 'profileIE', 'feed.js', 'feedIE.js' 'signup.js', 'signupIE.js', ... ];
/home
var bundles = [ 'home.js', 'homeIE.js', 'profile', 'profileIE', 'feed.js', 'feedIE.js' 'signup.js', 'signupIE.js', ... ];
/home
home.js
NO
var bundles = [ 'home.js', 'homeIE.js', 'profile', 'profileIE', 'feed.js', 'feedIE.js' 'signup.js', 'signupIE.js', ... ];
/home
home.js homeIE.js
YESNO
Netflix AB testing!
AB Test w/ multiple cells
Cells Control (Cell 1) Cell 2 Cell 3
Movie Cover Art
AB Test w/ multiple cells
Cells Control (Cell 1) Cell 2 Cell 3
Movie Cover Art
14% 6%
Netflix AB testing!
Netflix AB testing!
home.js
home.js
newSearch.jsoldSearch.js
home.js
oldSearch.js
home.js
oldSearch.js
home.js
newSearch.js
home.js
newSearch.jsoldSearch.js
home.js
newSearch.jsoldSearch.js
jQuery React~80KB ~120KB
larger bundle size: • file sizes • time to download • memory usage • time to interactive (TTI)
old school Lego bricks were generic
new Lego is about specialization
hard to reuse specialized bricks
home.js
newSearch.jsoldSearch.js
home.js
newSearch.jsoldSearch.js
home.js
newSearch.jsoldSearch.js
// starting to look like a // lot of bundles... var bundles = [ 'homeNewSearch.js', 'homeNewSearchIE.js', 'homeOldSearch.js', 'homeOldSearchIE.js', ... ];
// starting to look like a // lot of bundles... var bundles = [ 'homeNewSearch.js', 'homeNewSearchIE.js', 'homeOldSearch.js', 'homeOldSearchIE.js', ... ];
4x variations already!
Netflix runs hundreds of AB tests
Netflix runs hundreds of AB tests
but we personalize on many other dimensions too
|S1| ⋅ |S2| ⋅ ⋅ ⋅ |Sn| = |S1 × S2 × ⋅ ⋅ ⋅ × Sn|
|S1| ⋅ |S2| ⋅ ⋅ ⋅ |Sn| = |S1 × S2 × ⋅ ⋅ ⋅ × Sn|
3100 = 5.1537752e47
40,000,000,000 bricks to reach the
40,000,000,000 bricks to reach the
7,600,000,000,000,000 bricks to reach
40,000,000,000 bricks to reach the
7,600,000,000,000,000 bricks to reach
Enough bricks to reach 6.7812832e32 times.
that’s a #$*%^ ton of bundles!
https://xkcd.com/303/
Website's full bundle is 10MB+
how do we deal with conditional modules?
ii. codex conditional Bundling
:
what if we generate on-demand?
what if we generate on-demand?
1. identify the UI variation 2. generate the bundle
how do we identify the UI variation?
AB Tests
AB Tests
AB Tests
truthsnoun, plural [trooth z, trooths]
a bucket of boolean flags used to build a personalized Netflix experience
{ "webfonts": false, "instantSearch": true, "socialFeatures": false, "motionBanner": true, "html5video": true, "customScrollbar": true }
{ "webfonts": false, "instantSearch": true, "socialFeatures": false, "motionBanner": true, "html5video": true, "customScrollbar": true }
inputs and outputs are NOT 1:1
how do we generate the bundle?
home.js
newSearch.jsoldSearch.js
// home.js if (truths.isNewSearch === true) { require('./newSearch'); } else { require('./oldSearch'); }
home.js
newSearch.jsoldSearch.js
home.js
newSearch.jsoldSearch.js
conditioncondition
home.js
newSearch.jsoldSearch.js
conditioncondition
conditioncondition
isNewSearch!isNewSearch
// home.js if (truths.isNewSearch === true) { require('./newSearch'); } else { require('./oldSearch'); }
home.js
newSearch.jsoldSearch.js
isNewSearch!isNewSearch
home.js
newSearch.js
newEntryPoint.js
oldSearch.js
!isNewSearch isNewSearch
git
git
git
codex(node.js module)
git
codexartifact
(node.js module)
artifact
artifact
artifact
{ "home.js": { "deps": [ "dep1.js", "dep2.js", "dep3.js", ], "conditionalDeps": { "newSearch.js": { "name": "isNewSearch", "value": true },
"dep3.js", ], "conditionalDeps": { "newSearch.js": { "name": "isNewSearch", "value": true }, "oldSearch.js": { "name": "isNewSearch", "value": false } } } }
"dep3.js", ], "conditionalDeps": { "newSearch.js": { "name": "isNewSearch", "value": true }, "oldSearch.js": { "name": "isNewSearch", "value": false } } } }
it's a conditional map!
web/v1 web/v2 web/v3 web/v4
artifacttruths
<html/>
http://codex.nflxext.com/web/v1/83af
http://codex.nflxext.com/web/v1/83af
<script/>
http://codex.nflxext.com/web/v1/83af
<script/>
http://codex.nflxext.com/web/v1/83af
<script/> <script/>
codex
? i got this!
http://codex.nflxext.com/web/v1/83af
codex
http://codex.nflxext.com/web/v1/83af http://codex.nflxext.com/{team}/{version}/{truths}
codexweb/v1
http://codex.nflxext.com/web/v1/83af http://codex.nflxext.com/{team}/{version}/{truths}
codex { 83: newSearchTest, af: isChrome }
web/v1
http://codex.nflxext.com/web/v1/83af http://codex.nflxext.com/{team}/{version}/{truths}
home.js
oldSearch.js
home.js
oldSearch.js
home.js
oldSearch.js
home.js
newSearch.jsoldSearch.js
home.js
home.js
newSearch.jsoldSearch.js
response times <= 80ms
codex
http://codex.nflxext.com/web/v1/83af
<script/>
codex
here you go
http://codex.nflxext.com/web/v1/83af
<script/> <script/>
codex
cached! here you go
http://codex.nflxext.com/web/v1/83af
Recap• Build Time: build conditional graph (artifact)
• Run Time: apply truths to artifact
• Conditional bundling is transparent, universal, configuration free!
iii. Scaling For Netflix
web/v1 web/v2 tv/v5 tv/v7
Storage Metadata
Amazon S3Amazon DynamoDB
Storage Metadata
Build Time: Codex Artifact Management
web/v1
Build Time: Codex Artifact Management
web/v1web/v1
Build Time: Codex Artifact Management
saved!
SAVED!
web/v1web/v1
Build Time: Codex Artifact Management
saved!
Build Time: Codex Artifact Management
activate web/v1 web/v1
activated!
Build Time: Codex Artifact Management
Run Time: Codex Bundler
web/v1
here are the active build ids
Run Time: Codex Bundler
here are the artifacts
web/v1
here are the active build ids
Run Time: Codex Bundler
here are the artifacts
web/v1
here are the active build ids
Run Time: Codex Bundler
prod prod-new canary
16GB ought to be enough for us!__🙂
codex
codex
codex
400+ artifacts!
400+ artifacts!
…and we ran out of memory
32GB ought to be enough for us!__🤔
800+ artifacts!
800+ artifacts!
…and we ran out of memory
64GB ought to be enough for us…?__😭
1600+ artifacts!
1600+ artifacts!
…and we ran out of memory. again.
1600+ artifacts!Our teams will use as
much as we give them.
…and we ran out of memory. again.
Q: What's cheap, plentiful, and fast
enough?
Q: What's cheap, plentiful, and fast
enough?A: Disk.
codex
LevelDB
codex
LevelDB
100% CPU usage.
what's the problem?
~68%
v8::internal::Runtime_ParseJson
codex
LevelDB
codex
LevelDB
JSON.parse
JSON.parse is slow. And blocks the CPU!
codex
LevelDB
JSON.parse
LRU Cache: Saving the CPU
codex
LevelDB
JSON.parse
LRU Cache: Saving the CPU
codex
LevelDB
JSON.parse
LRU Cache: Saving the CPU
codex
LevelDB
JSON.parse
LRU Cache: Saving the CPU
codex
LevelDB
JSON.parse
LRU Cache: Saving the CPU
Breaking change to conditional graph
traversal algorithm!
http://codex.nflxext.com/web/v1/83af
http://codex.nflxext.com/web/v1/83af
old algorithm?
new algorithm?
old algorithm?
new algorithm?
http://codex.nflxext.com/1.0.0/web/v1/83afhttp://codex.nflxext.com/2.0.0/web/v1/83af
1.0.0
2.0.0
zuul
zuul 1.0.0
2.0.0
zuul 1.0.0
2.0.0
Good for now.
Good for now.
Continue to look for engineering wins.
What about operational resiliency?
eu-west-1
us-west-2
us-east-1
eu-west-1
us-west-2
us-east-1
eu-west-1
us-west-2
???
eu-west-1
us-west-2
???
eu-west-1
us-west-2
???
???
???
???
<script/> <script/>
codex
<script/>
codex
???
codex
<script/>
???
Recap• Management plane necessary at scale
• Performance is critical (TTI)
• Redundancy across 3 AWS zones
• Resilient against CDN failure
iv. Looking To The Future
why not {bundler}?
how do we support tree
shaking?
don't be afraid to challenge common
convention.
don't make assumptions about the upper limits.
don't optimize before you understand the system.
use the scientific method:
1. gather data 2. formulate hypothesis 3. test hypothesis 4. repeat
engineer for fault tolerance
Netflix scale is challenging.
https://www.flickr.com/clement127/ https://www.flickr.com/jose_antonio_hidalgo_jimenez/ https://www.flickr.com/reiterlied/
Lego Photo Credits
Image Credits
Image Credits
Image Credits
Artist: alecive (Alessandro Roncone) Iconset Homepage: https://github.com/alecive/FlatWoken
Alex Liu [email protected] @stinkydofu
fin