ComputerNetworks:ArchitectureandProtocols

CS4450

Lecture23DNSandHTTP

WhatisDNS?

● Userhasnameofentityshe/hewantstoaccess● E.g.,www.cnn.com

● Content,host,etc.

● However,InternetroutesandforwardsrequestsbasedonIPaddresses● Needtoconvertname(e.g.,www.cnn.com)toanIPaddress

● DomainNameSystem(DNS)● ProvidesthemappingfromnametoIPaddress● UserasksDNS:whatistheIPaddressforwww.cnn.com● DNSresponds:157.166.255.18

CorrectnessRequirements

● Addressescanchangeunderneath● Movewww.cnn.comto4.125.91.21● Humans/Applicationsshouldbeunaffected

● NamecouldmaptomultipleIPaddresses● www.cnn.comtomultiplereplicastotheWebsite● Toenable“loadbalancing”orreducedlatency

● Replicasmayseedifferentload(eg,duetogeographiclocation)● Somereplicasmaybeclosertotheuser

● Multiplenamesforthesameaddress● E.g.,www.cnn.comandcnn.comshouldmaptosameIPaddresses

GoalsandApproach

● Goals● Correctness(frompreviousslide)● Scaling(names,users,updates,etc.)● Easeofmanagement(uniquenessofnames,etc.)● Availabilityandconsistency● Fastlookups

● Approach:Threeintertwinedhierarchies● HierarchicalNamespace:exploitstructureinnames● HierarchicalAdministration:hierarchyofauthorityovernames● HierarchicalInfrastructure:hierarchyofDNSservers

HierarchicalNamespace

● “TopLevelDomains”(TLDs)areatthetop● Domainsaresubtrees

● E.g..edu,cornell.edu,cs.cornell.edu● Nameisleaf-to-rootpath

● systems.cs.cornell.edu

root

edu com gov mil org net uk fr

cornell mit

cs ece

systems

…

HierarchicalAdministration

● Azonecorrespondstoanadministrativeauthorityresponsibleforcontiguousportionofhierarchy● Cornellcontrols*.cornell.edu● CScontrols*.cs.cornell.edu

● Namecollisionstriviallyavoided● Eachdomaincanensurethislocally

root

edu com gov mil org net uk fr

cornell mit

cs ece

systems

…

ICANN/IANA

HierarchicalInfrastructure

● Topofhierarchy:root● Locationhardwiredintootherservers

● Nextlevel:TopLevelDomain(TLD)servers● .com,.edu,etc.

● Bottomlevel:AuthoritativeDNSservers● Actuallydothemapping● Canbemaintainedlocallyorbyaserviceprovider

PerDomainAvailability

● DNSServersarereplicated● Primaryandsecondarynameserversarerequired● Nameserviceavailableifatleastonereplicaisup● Queriescanbeload-balancedamongreplicas

● Tryalternateserversontimeout● Exponentialbackoffwhenretryingthesameserver

WhoKnowsWhat?

● Everyserverknowsaddressofrootnameserver

● RootserversknowtheaddressofallTLDservers

● Everynodeknowstheaddressofallchildren

● AnauthoritativeDNSserverstoresname-to-addressmappings(“resourcerecords”)forallDNSnamesinthedomainthatithasauthorityfor

● Therefore,eachserver:● StoresonlyasubsetofthetotalDNSdatabase(scalable!)● Candiscoverserver(s)foranyportionofthehierarchy

BenefitsofThisApproach

● Scalableinnames,updates,lookups,users

● Highlyavailable:domainsreplicateindependently

● Extensible:canaddTLDsjustbychangingrootdb

● Autonomousadministration:● Eachdomainmanagesownnamesandservers● Andcanfurtherdelegate● Easilyensuresuniquenessofnames● Andconsistencyofdatabases

DNSRecords(details)

● DNSserversstoreresourcerecords(RRs)● RRis(name,value,type,TTL)

● Type=A:(->Address)● Name=hostname● Value=IPaddress

● Type=NS:(->NameServer)● Name=domain● Value=nameofdnsserverfordomain

● Type=MXL(->MaineXchanger)● name=domaininemailaddress● value=name(s)ofmainserver(s)

● Type=CNAME:(->CanonicalNAME)● Name=alias● Valueis“canonical”name

● Type=PTR:(->Pointer)● nameisreversedIP● valueiscorrespondinghostname

DNSRecords(detailscontinued)

InsertingResourceRecordsintoDNS

● Example:youjustcreatedcompany“FooBar”

● YougetablockofIPaddressesfromyourISP● E.g.,212.44.9.128/25

● Registerfoobar.comatregistrar(e.g.,GoDaddy)● ProvideregistrarwithnamesandIPaddressesofyourauthoritativenameserver(s)

● RegistrarinsertsRRpairsintothe.comTLDserver:● (foobar.com,dns1.foobar.com,NS)● (dns1.foobar.com,212.44.9.129,A)

● Storeresourcerecordsinyourserverdns1.foobar.com● e.g.,typeArecordforwww.foobar.com● e.g.,typeMXrecordforfoobar.com

DistributedHierarchicalDatabase

com edu org ac uk zw arpa

root

bar

west east

foo my

ac

cam

usr

in- addr

generic domains country domains

my.east.bar.edu usr.cam.ac.uk

Top-Level Domains (TLDs)

Everything scales but the root!

DNSRoot

● Located in Virginia, USA● How do we make the root scale?

Verisign, Dulles, VA

DNSRootServers

● 13 root servers (see http://www.root-servers.org/)● Labeled A through M

● How can we seamlessly scale this further?

B USC-ISI Marina del Rey, CA L ICANN Los Angeles, CA

E NASA Mt View, CA F Internet Software Consortium Palo Alto, CA

I Autonomica, Stockholm

K RIPE London

M WIDE Tokyo

A Verisign, Dulles, VA C Cogent, Herndon, VA D U Maryland College Park, MD G US DoD Vienna, VA H ARL Aberdeen, MD J Verisign

Anycast

● Routingfindsshortestpathstodestination

● Ifseverallocationsaregiventhesameaddress:

● Networkwilldeliverthepackettoclosestlocationwiththataddress

● Thisiscalled“anycast”● Nomodificationofroutingisneededforthis….

● Allowsforseamlessreplicationofresources● Anyproblemswiththisapproach?

DNSRootServers

● 13rootservers(seehttp://www.root-servers.org/)● LabeledAthroughM

● Replicationviaany-casting(localizedroutingforaddresses)

B USC-ISI Marina del Rey, CA L ICANN Los Angeles, CA

E NASA Mt View, CA F Internet Software Consortium Palo Alto, CA

I Autonomica, Stockholm

K RIPE London

M WIDE Tokyo

A Verisign, Dulles, VA C Cogent, Herndon, VA D U Maryland College Park, MD G US DoD Vienna, VA H ARL Aberdeen, MD J Verisign

UsingDNS

● Twocomponents● LocalDNSservers● Resolversoftwareonhosts

● LocalDNSserver(“defaultnameserver”)● Usuallyneartheendhoststhatuseit● Localhostsconfiguredwithlocalserver(e.g,/etc/resolv.conf)orlearnserverviaDHCP

● Clientapplication● ObtainDNSname(e.g.,fromtheURL)● Dogethostbyname()totriggerresolvercode● WhichthensendsrequesttolocalDNSserver

20

DNSclient (me.cs.cornell.edu)

rootservers

(mydns.cornell.edu) .eduservers

nyu.eduservers

LocalDNSserver

21

DNSclient (me.cs.cornell.edu)

rootservers

.eduservers

nyu.eduservers

www.nyu.edu?

(mydns.cornell.edu)

LocalDNSserver

22

root DNSserver

DNSclient (me.cs.cornell.edu)

.eduservers

nyu.eduservers

(mydns.cornell.edu)

www.nyu.edu?

LocalDNSserver

23

root DNSserver

DNSclient (me.cs.cornell.edu)

.eduservers

nyu.eduservers

(mydns.cornell.edu)

www.nyu.edu?

LocalDNSserver

24

root DNSserver

recursiveDNSquery

DNSclient (me.cs.cornell.edu)

(mydns.cornell.edu).eduservers

nyu.eduservers

www.nyu.edu?

LocalDNSserver

25

root DNSserver

DNSclient (me.cs.cornell.edu)

(mydns.cornell.edu).eduservers

nyu.eduservers

LocalDNSserver

root DNSserver

iterativeDNSquery

DNSclient (me.cs.cornell.edu)

(mydns.cornell.edu).eduservers

nyu.eduservers

Where is .edu?

Where is www.nyu.edu?

Where is nyu.edu?

LocalDNSserver

DNSProtocol

● QueryandReplymessages● Bothwiththesamemessageformat● seetextfordetails

● Client-ServerinteractiononUDPPort53● SpecsupportsTCPtoo,butnotalwaysimplemented● Reliabilityviarepeatingrequestsontimeout

● Resolutionisalmostalways“iterative”

Goals

● Scaling(names,users,updates,etc.)● Yes

● Easeofmanagement(uniquenessofnames,etc.)● Yes

● Availabilityandconsistency● Yes

● Fastlookups● ??

root DNSserver

iterativeDNSquery

DNSclient (me.cs.cornell.edu)

(mydns.cornell.edu).eduservers

nyu.eduservers

Where is .edu?

Where is www.nyu.edu?

Where is nyu.edu?

LocalDNSserver

DNSCaching

● HowDNScachingworks● DNSserverscacheresponsestoqueries● Responsesincludea“timetolive”(TTL)field● ServerdeletescachedentryafterTTLexpires

● Whycachingiseffective● Thetop-levelserversveryrarelychange● Popularsitesvisitedoften->localDNSserveroftenhastheinformationcached

Questions?

TheWeb

TheWeb–Precursor

● 1967,TedNelson,Xanadu:● Aworld-widepublishingnetworkthatwouldallowinformationtobestorednotasseparatefilesbutasconnectedliterature

● Ownersofdocumentswouldbeautomaticallypaidviaelectronicmeansforthevirtualcopyingoftheirdocuments

● Coinedtheterm“Hypertext”● Influencedresearchcommunity

● Whothenmissedtheweb…..

Ted Nelson

TheWeb–Precursor

● Physicisttryingtosolverealproblem● Distributedaccesstodata

● WorldWideWeb(WWW):adistributeddatabaseof“pages”linkedthroughHypertextTransportProtocol(HTTP)● FirstHTTPimplementation-1990

● TimBerners-LeeatCERN● HTTP/0.9–1991

● SimpleGETcommandfortheWeb● HTTP/1.0–1992

● Client/Serverinformation,simplecaching● HTTP/1.1-1996

Tim Berners-Lee

WebComponents

● Infrastructure:● Clients● Servers● Proxies

● Content:● Individualobjects(files,etc.)● Websites(coherentcollectionofobjects)

● Implementation● URL:namingcontent● HTTP:protocolforexchangingcontent

URLSyntax

protocol http, ftp, https, smtp, rtsp, etc.

hostname DNS name, IP address

port Defaults to protocol’s standard porte.g. http: 80 https: 443

directory path Hierarchical, reflecting file system

resource Identifies the desired resource

Can also extend to program executions: http://us.f413.mail.yahoo.com/ym/ShowLetter?box=%40B%40Bulk&MsgId=2604_1744106_29699_1123_1261_0_28917_3552_1289957100&Search=&Nhead=f&YY=31454&order=down&sort=date&pos=0&view=a&head=b

protocol://hostname[:port]/directorypath/resource

WebandDNS

● URLsusehostnames

● Thus,contentnamesaretiedtospecifichosts

● Whyisthisaproblem?

● Makespersistenceofnamesproblematic…

HyperTextTransferProtocol(HTTP)

● Client-serverarchitecture● Serveris“alwayson”and“wellknown”● Clientsinitiatecontacttoserver

● Synchronousrequest/replyprotocol● Runsontopoftransportlayer,Port80

● Stateless

● ASCIIformat

GET /somedir/page.html HTTP/1.1 Host: www.someschool.edu User-agent: Mozilla/4.0 Connection: close Accept-language: fr (blank line)

● Request line: method, resource, and protocol version ● Request headers: provide information or modify request ● Body: optional data (e.g., to “POST” data to the server)

request line

header lines

carriage return line feed indicates end of message

HTTPrequestmessage

● Status line: protocol version, status code, status phrase ● Response headers: provide information ● Body: optional data

HTTP/1.1 200 OK Connection close Date: Thu, 06 Aug 2006 12:00:15 GMT Server: Apache/1.3.0 (Unix) Last-Modified: Mon, 22 Jun 2006 ... Content-Length: 6821 Content-Type: text/html (blank line) data data data data data ...

status line (protocol, status code, status phrase)

header lines

data e.g., requested HTML file

HTTPresponsemessage

HTTPisStateless

● Eachrequest-responsetreatedindependently● ServersnotrequiredtoretainstateforHTTP

● Theapplicationmayhavelotsofstate,butnotHTTP

● Good:Improvesscalabilityontheserver-side● Failurehandlingiseasier● Canhandlehigherrateofrequests● Orderofrequestsdoesn’tmatter(toHTTP)

● Bad:Someapplicationsneedpersistentstate● Needtouniquelyidentifyuserorstoretemporaryinfo● e.g.,Shoppingcart,userprofiles,usagetracking,…

Question

● How does a stateless protocol keep state?

StateinaStatelessProtocol:Cookies

● Client-sidestatemaintenance● Clientstoressmallstateonbehalfofserver● Clientsendsstateinfuturerequeststotheserver

● Canprovideauthentication

Request

Response Set-Cookie: XYZ

Request Cookie: XYZ

HTTPPerformanceIssues

PerformanceGoals

● User● Fastdownloads● Highavailability

● Contentprovider● Happyusers(hence,above)● Cost-effectiveinfrastructure

● Network(secondary)● Avoidoverload

Cachingandreplicationresolvemostoftheseissues

HTTPPerformance

● MostWebpageshavemultipleobjects● e.g.,HTMLfileandabunchofembeddedimages

● Howdoyouretrievethoseobjects(naively)?● Oneitematatime

● Newconnectionper(small)object!

● Requires2RTTsworthoflatencyperobject

ImprovingHTTPPerformance

● PersistentConnections

● Maintainconnectionacrossmultiplerequests● Includingtransferssubsequenttocurrentpage● Clientorservercanteardownconnection

● Performanceadvantages:● Avoidoverheadofconnectionset-upandtear-down

● DefaultinHTTP/1.1

ImprovingHTTPPerformance

● PipelinedRequestsandResponses

● Batchrequestsandresponsestoreducethenumberofpackets

Client Server

Request 1Request 2Request 3

Transfer 1

Transfer 2

Transfer 3

Questions?

ImprovingHTTPPerformance:Caching

● Whydoescachingwork?● Exploitslocalityofreference

● Howwelldoescachingwork?● Verywell,uptoalimit

● Filepopularityhashighpeakbutlongtail● Largeoverlapinhighlypopularcontent● Butmanyuniquerequests

● Auniversalstory!● Hitrateofcachegrowslogarithmicallywithsize

ImprovingHTTPPerformance:Caching-How?

● ModifiertoGETrequests:● If-modified-since—returns“notmodified”ifresourcenotmodifiedsincespecifiedtime

● Clientspecifies“if-modified-since”timeinrequest

● Servercomparesthisagainst“lastmodified”timeofresource

● Serverreturns“NotModified”ifresourcehasnotchanged

● ….ora“OK”withthelatestversionotherwise

ImprovingHTTPPerformance:Caching-How?

● ModifiertoGETrequests:● If-modified-since—returns“notmodified”ifresourcenotmodifiedsincespecifiedtime

● Responseheader:● Expires—TTL:howlongit’ssafetocachetheresource● No-cache—ignoreallcaches;alwayshetresourcedirectlyfromserver

TypicalCachingInteraction

● Clientissuesrequestforobject

● Ifitisinlocalclientcache:● IfwithinTTL,respondtoclient● IfnotwithinTTL,sendif-modified-sincetoserver

● Ifserverhasupdatedcopy,itsendsit● Ifnot,serverrespondssayingthatitdoesn’t

● Ifnotinlocalclientcache:● Sendrequesttoserver● Thisrequestmaypassthroughothercaches,whichuseasimilaralgorithm

ImprovingHTTPPerformance:Caching-Where?

● Options● Client● Forwardproxies● Reverseproxies● ContentDistributionNetwork

ImprovingHTTPPerformance:Caching-Where?

● Baseline:Manyclientstransfersameinformation● Generateunnecessaryserverandnetworkload● Clientsexperienceunnecessarylatency

Server

Clients

Tier-1 ISP

ISP-1 ISP-2

CachingwithReverseProxies

● Cachedocumentsclosetoserver● Decreaseserverload

● Typicallydonebycontentprovider

Clients

Backbone ISP

ISP-1 ISP-2

Server

Reverse proxies

CachingwithForwardProxies

● Cachedocumentsclosetoclients● Reducenetworktrafficanddecreaselatency

● TypicallydonebyISPsorenterprises

Clients

Backbone ISP

ISP-1 ISP-2

Server

Reverse proxies

Forward proxies

ImprovingHTTPPerformance:Replication

● ReplicatepopularWebsiteacrossmanymachines● Spreadsloadonservers● Placescontentclosertoclients● Helpswhencontentisn’tcacheable

● Problem:Wanttodirectclienttoparticularreplica● Balanceloadacrossserverreplicas● Pairclientswithnearbyservers

● Commonsolution:● DNSreturnsdifferentaddressesbasedonclient’sgeolocation,serverload,etc.

ContentDistributionNetworks

● Cachingandreplicationasaservice

● Large-scaledistributedstorageinfrastructure(usually)administeredbyoneentity● e.g.,Akamaihasserversin20,000+locations

● Combinationof(pull)cachingand(push)replication● Pull:Directresultofclients’requests● Push:Expectationofhighaccessrate

● Alsodosomeprocessing● Handledynamicwebpages● Transcoding

CDNExample—Akamai

● Akamaicreatesnewdomainnamesforeachclient● e.g.,a128.g.akamai.netforcnn.com

● TheCDN’sDNSserversareauthoritativeforthenewdomains

● TheclientcontentprovidermodifiesitscontentsothatembeddedURLsreferencethenewdomains.● “Akamaize”content● e.g.:http://www.cnn.com/image-of-the-day.gifbecomeshttp://a128.g.akamai.net/image-of-the-day.gif

● RequestsnowsenttoCDN’sinfrastructure…

Cost-EffectiveContentDelivery

● Generaltheme:multiplesiteshostedonsharedphysicalinfrastructure● efficiencyofstatisticalmultiplexing● economiesofscale(volumepricing,etc.)● amortizationofhumanoperatorcosts

● Examples:● Webhostingcompanies● CDNs● Cloudinfrastructure

Questions?

Backupslide:HistoryofDNS

● Originally:per-hostfilehosts.txtin/etc/hosts● SRI(MenloPark)keptmastercopy● Downloadedregularly● Flatnamespaces

● AstheInternetgrewthissystembrokedown● SRIcouldn’thandletheload● Conflictsinselectingnames● Hostshadinaccuratecopiesofhosts.txt

● DomainNameSystem(DNS)inventedtofixthis● Firstserverimplementationdoneby4UCBstudents!

DNSMeasurements(MITdatafrom2000)

● What is being looked up?● ~60% requests for A records● ~25% for PTR records● ~5% for MX records● ~6% for ANY (wildcard) records

● How long does it take?● Median ~100msec (but 90th percentile ~500msec)● 80% have no referrals; 99.9% have fewer than four

● Query packets per lookup: ~2.4● But this is misleading….

DNSMeasurements(MITdatafrom2000)

● Does DNS give answers?● ~23% of lookups fail to elicit an answer!● ~13% of lookups result in NXDOMAIN (or similar)

● Mostly reverse lookups● Only ~64% of queries are successful!

● How come the web seems to work so well?

● ~ 63% of DNS packets in unanswered queries!● Failing queries are frequently retransmitted● 99.9% successful queries have ≤2 requests

MoraloftheStory

● The Internet was designed to be highly resilient.● No matter what goes wrong, it tries to recover

● In a highly resilient system, many things can be going wrong without you noticing it!

DNSMeasurements(MITdatafrom2000)

● Top 10% of names accounted for ~70% of lookups● Caching should really help!

● 9% of lookups are unique● Cache hit rate can never exceed 91%

● Cache hit rates ~ 75%● But caching for more than 10 hosts doesn’t add much

ACommonPattern…..

● Distributions of various metrics (file lengths, access patterns, etc.) often have two properties:● Large fraction of total metric in the top 10%● Sizable fraction (~10%) of total fraction in low values

● In an exponential distribution● Large fraction is in top 10%● But low values have very little of overall total

● Lesson: in networking, have to pay attention to both ends of distribution (high peak and long tail)● Here, caching helps, but not a panacea

Whynotnamecontentdirectly?

● How do you know where to send the request?

● How do you scale?

● How do you trust the response?● Requesting host● Network

● How would you design it?

Scorecard:GettingnSmallObjects

● Time dominated by latency

● One at a time: ~2n RTT

● M concurrent: ~2[n/m] RTT

● Persistent: ~(n+1) RTT

● Pipelined: ~2 RTT

● Pipelined/Persistent: ~2RTT first time, RTT later

Scorecard:GettingnLargeObjects

● Time dominated by bandwidth

● One at a time: ~nF/B

● M concurrent: it depends● If more flows get no additional bandwidth: ~nF/B● If shared with large population of users: ~[n/m] F/B

● Where each TCP connection gets the same bandwidth

● Pipelined and/or Persistent: ~nF/B● The only thing that helps is getting more bandwidth