Living on the Edge

Adrian Cole@adrianfcole #netflixoss @denominatorOSS

http://www.linkedin.com/in/adrianforrestcole

How Netflix Streaming Works

Denominator

Cloud Prize

Netflix Member Web Site Home PagePersonalization Driven – What goes on to make this?

How Netflix Streaming Works

Customer Device (PC, PS3, TV…)

Web Site or Discovery API

User Data

Personalization

Streaming API

DRM

QoS Logging

OpenConnect CDN Boxes

CDN Management and

Steering

Content Encoding

Consumer Electronics

AWS Cloud Services

CDN Edge Locations

Content Delivery ServiceOpen Source Hardware Design + FreeBSD, bird, nginx

November 2012 Traffic

Real Web Server Dependencies Flow(Netflix Home page business transaction as seen by AppDynamics)

Start Here

memcached

Cassandra

Web service

S3 bucket

Three Personalization movie group choosers (for US, Canada and Latam)

Each icon is three to a few hundred instances across three AWS zones

Asgardhttp://techblog.netflix.com/2012/06/asgard-web-based-cloud-management-and.html

Portable DNS Control

Cassandra Replicas

Zone A

Cassandra Replicas

Zone B

Cassandra Replicas

Zone C

Regional Load Balancers

Cassandra Replicas

Zone A

Cassandra Replicas

Zone B

Cassandra Replicas

Zone C

Regional Load Balancers

UltraDNSAWS

Route53

A portable way to manage multiple DNS providers from Java

DenominatorDynECT

PORTABLE CONTROL OF DNS CLOUDS

Feature Set

• Do stuff in batches• Cleanly handle advanced records • Play nice with persistence• Don’t do too much

• … Use cool things

A Cloud Native Open Source Platform

Hello Denominator

Install get denominator from bintray or homebrew

Configure create config.ymlname: dynectprovider: dynectcredentials: customer: your_customer username: your_user password: your_password

Run denominator -n dynect –C config.yml zone list

Basic list

$ denominator -n dynect-test -C ~/.denominator/config.yml zone[Session#login] ---> POST https://api2.dynect.net/REST/Session HTTP/1.1[Session#login] <--- HTTP/1.1 200 OK (948ms)[DynECT#zones] ---> GET https://api2.dynect.net/REST/Zone HTTP/1.1[DynECT#zones] <--- HTTP/1.1 200 OK (514ms)denominator.io

Add Record

$ denominator -n dynect-test -C ~/.denominator/config.yml record -z denominator.io. replace -n www.denominator.io. -t A -d 192.0.2.1

;; in zone denominator.io. replacing rrset www.denominator.io. A values: [{address=192.0.2.1}][Session#login] ---> POST https://api2.dynect.net/REST/Session HTTP/1.1[Session#login] <--- HTTP/1.1 200 OK (914ms)[DynECT#recordsInZoneByNameAndType] ---> GET https://api2.dynect.net/REST/ARecord/denominator.io./www.denominator.io.?detail=Y HTTP/1.1[DynECT#recordsInZoneByNameAndType] <--- HTTP/1.1 200 OK (504ms)[DynECT#scheduleDeleteRecord] ---> DELETE https://api2.dynect.net/REST/ARecord/denominator.io./www.denominator.io/75626502 HTTP/1.1[DynECT#scheduleDeleteRecord] <--- HTTP/1.1 200 OK (157ms)[DynECT#scheduleCreateRecord] ---> POST https://api2.dynect.net/REST/ARecord/denominator.io./www.denominator.io. HTTP/1.1[DynECT#scheduleCreateRecord] <--- HTTP/1.1 200 OK (157ms)[DynECT#publish] ---> PUT https://api2.dynect.net/REST/Zone/denominator.io. HTTP/1.1[DynECT#publish] <--- HTTP/1.1 200 OK (209ms)

;; ok

From Java

mgr = Denominator.create(”dynect”, credentials(customer, username, password))

for (Zone zone : mgr.api().zones()) { processZone(zone);}

mgr.recordSetsInZone(“denominator.io.”) .put(a("www.denominator.io.", 300, "192.0.2.1"));

Availability by reshaping traffic

Netflix Cloud Prize

Boosting the @netflixOSSEcosystem

Best example application mash-up

Best new monkey

Best contribution to code quality

Best new feature

Best contribution to operational tools

Best portability enhancement

Best datastore integration

Best contribution to performance

Best usability enhancement

Judges choice award

Entrants

NetflixEngineering

Judges Winners

Nominations

Conforms to Rules

Working Code

Community Traction

Categories

Registration Opened

March 13Github

Apache Licensed

ContributionsGithub Close Entries

September 15GithubAward

Ceremony Dinner

November

AWS Re:Invent

Ten Prize Categories

$10K cash$5K AWS

AWS Re:Invent

TicketsTrophy

Functionality and scale now, portability coming

Moving from parts to a platform in 2013

Netflix is fostering an ecosystem

Rapid Evolution - Low MTBIAMSH(Mean Time Between Idea And Making Stuff Happen)

Takeaway

Denominator is a multi-cloud DNS abstraction built as a library and a cli.

Fixing DNS APIs, one vendor at a time

https://github.com/Netflix/denominatorhttps://groups.google.com/forum/#!forum/denominator-user

http://www.linkedin.com/in/adrianforrestcole@adrianfcole #netflixoss @denominatorOSS