Big Data Architectures - Inria · (9)
Transcript of Big Data Architectures - Inria · (9)
BigDataArchitectures
IoanaManolescuINRIASaclay&EcolePolytechnique
[email protected]://pages.saclay.inria.fr/ioana.manolescu/
M2DataandKnowledgeUniversitédeParisSaclay
BigDataArchitectures2019-2020 IoanaManolescu 1
Dimensionsofdistributeddatamanagementsystems
• Datamodel:– Relations,trees(XML,JSON),graphs(RDF,others…),nestedrelations
– Querylanguage• Heterogeneity(DM,QL):none,some,alot• Scale:small(~10-20sites)orlarge(~10.000sites)• ACIDproperties• Control:
– Singlemasterw/completecontroloverNslaves(Hadoop/HDFS)– Sitespublishindependentlyandprocessqueriesasdirectedbysinglemaster/mediator
– Many-mediatorsystems,orpeer-to-peer(P2P)withsuper-peers– Sitescompletelyindependent(P2P)
IoanaManolescu 2BigDataArchitectures2019-2020
Today'slecture
• P2Parchitectures:– highestdegreeofpeerautonomy– highdegreeofdistribution
• CloudBigDatamanagementarchitectures– Cloudcomputing– Structureddatamanagementontopofcloudservices
BigDataArchitectures2019-2020 IoanaManolescu 3
PEER-TO-PEERDATAMANAGEMENTARCHITECTURES
BigDataArchitectures2019-2020 IoanaManolescu 4
Peer-to-peerarchitectures• Idea:easy,large-scalesharingofdatawithnocentralpointof
control• Allthepeersplayidenticalroles• Peersmayjointhepeernetworkorleaveitatanytime
• Advantages:– Distributework;preservepeerindependence
• Disadvantages:– Lackofcontroloverpeerswhichmayleaveorfailàneedformechanismstocopewithpeersjoiningorleaving(churn)
– Schemaunknowninadvance;needfordatadiscovery
BigDataArchitectures2019-2020 IoanaManolescu 5
Peer-to-peerarchitectures• Large-scalesharingofdatawithnocentralpointofcontrol• TwomainfamiliesofP2Parchitectures:
– UnstructuredP2Pnetworks• Eachpeerisfreetoconnecttootherpeers;• Variant:super-peernetworks
– StructuredP2Pnetworks• Eachpeerisconnectedtoasetofotherpeersdeterminedbythesystem
• Also:hybridP2Parchitectures– A"central"subsetofthenetworkisstructured,therestisunstructured
BigDataArchitectures2019-2020 IoanaManolescu 6
UnstructuredP2PnetworksApeerjoinsthenetworkbyconnectingtoanotherpeer(«gettingintroduced»)Eachpeermayadvertisedatathatitpublishesàpeers«knowtheirneighbors»uptosomelevelBigDataArchitectures2019-2020 IoanaManolescu 7
UnstructuredP2Pnetworks
Apeerjoinsthenetworkbyconnectingtoanotherpeer(«gettingintroduced»)
BigDataArchitectures2019-2020 IoanaManolescu 8
I’vegot:«GoTseason1»«Mr.Robot»
(p1,«GoT»)(p1,«MR»)
(p1,«GoT»)(p1,«MR»)
(p1,«GoT»)(p1,«MR»)
(p1,«GoT»)(p1,«MR»)
(p1,«GoT»)(p1,«MR»)
(p1,«GoT»)(p1,«MR») (p1,«GoT»)
(p1,«MR»)(p1,«GoT»)(p1,«MR»)
(p1,«GoT»)(p1,«MR»)
(p1,«GoT»)(p1,«MR»)
UnstructuredP2Pnetworks
Apeerjoinsthenetworkbyconnectingtoanotherpeer(«gettingintroduced»)Eachpeermayadvertisedatathatitpublishesàpeers«knowtheirneighbors»uptosomelevelBigDataArchitectures2019-2020 IoanaManolescu 9
I’vegot:«GoTseason1»«Mr.Robot»
(p1,«GoT»)(p1,«MR»)
(p1,«GoT»)(p1,«MR»)
(p1,«GoT»)(p1,«MR»)
(p1,«GoT»)(p1,«MR»)
(p1,«GoT»)(p1,«MR»)
(p1,«GoT»)(p1,«MR») (p1,«GoT»)
(p1,«MR»)(p1,«GoT»)(p1,«MR»)
(p1,«GoT»)(p1,«MR»)
(p1,«GoT»)(p1,«MR»)
UnstructuredP2PnetworksQueriesareevaluatedbypropagationfromthequerypeertoitsneighborsandsoonrecursively(flooding)Toavoidsaturatingthenetwork,querieshaveTTL(time-to-live)Thismayleadtomissinganswersàa.replication;b.superpeers
BigDataArchitectures2019-2020 IoanaManolescu 10
Q?
Q?Q?
Q?
Q?Q?
Q?
HybridP2Pnetwork
• Smallsubsetofsuperpeersallconnectedtoeachother• Specializedbydatadomain,e.g.[Aa—Bw],[Ca—Dw],…orbyaddressspace• Eachpeerisconnectedatleasttoasuperpeer,whichroutesthepeer’squeries
BigDataArchitectures2019-2020 IoanaManolescu 11
Q?
Q?
Q?
Q?
StructuredP2Pnetwork• Peersformalogicaladdressspace0…2k-1
– Somepositionsmaybeempty– Thepeerpositionisobtainedwiththehelpofahashfunction
– e.g.,H(peerIPaddress)=n,0<=n<=2k-1– Thisalsoleadstothename:distributedhashtable,DHT
• Theglobaldatacatalogiscreatedanddistributedacrossthepeers,usingthesamehashfunction(seenext)
BigDataArchitectures2019-2020 IoanaManolescu 12
Catalogconstruction(indexing)instructuredP2Pnetworks
• Thecatalogisbuiltasasetofkey-valuepairs– Key:expectedtooccurinsearchqueries,e.g.«GoT»,«MrRobot»
– Value:theaddressofcontentinthenetworkmatchingthekey,e.g.«peer5/Users/a/movies/GoT»
• Ahashfunctionisusedtomapeverykeyintotheaddressspace;thisdistributes(key,value)pairs– H(key)=nàthe(key,value)pairissenttopeern– Ifnisempty,thenextpeerinlogicalorderischosen
BigDataArchitectures2019-2020 IoanaManolescu 13
Catalogconstruction(indexing)instructuredP2Pnetworks
BigDataArchitectures2019-2020 IoanaManolescu 14
12
Catalogconstruction(indexing)instructuredP2Pnetworks
BigDataArchitectures2019-2020 IoanaManolescu 15
12
SearchinginstructuredP2Pnetworks
Locateallitemscharacterizedby?Hash()=6Peer6knowsallthelocationsLocateallitemscharacterizedby?Hash()=14Peer15knowsallthelocationsHowdowefindpeers6and15?
BigDataArchitectures2019-2020 IoanaManolescu 16
ConnectionsbetweenpeersinstructuredP2Pnetworks
Apeer'sconnectionsaredictatedbythenetworkorganizationandthelogicaladdressofeachpeerinthespace0…2k-1Example:Chord(MIT,mostpopular)Eachpeernisconnectedto• n+1,n+2,…,n+2k-1,ortothefirstpeerfollowingthatposition
intheaddressspace;• ThepredecessorofnTheconnectionsarecalledfingers
BigDataArchitectures2019-2020 IoanaManolescu 17
ConnectionsbetweenpeersinChord
BigDataArchitectures2019-2020 IoanaManolescu 18
SearchinginstructuredP2PnetworksLocateallitemscharacterizedby?• Hash()=6• Peer6knowsallthelocationsHowdoespeer1findpeer6?• 6-1=5;2<=log2(5)<=3,thus6isafter1+22• Redirectthequestiontothe2ndfingerof1.Howdoespeer10findpeer3?• (3-modulo1610)=9;3<=log2(9)<=4,thus3isafter10+23
• Redirectthequestiontothe3rdfingerof10.
Quicktraversalsofthering(log2(N)hops)
BigDataArchitectures2019-2020 IoanaManolescu 19
PeersjoininginChordTojoin,apeernmustknow(any)peern'alreadyinthenetworkProceduren.join(n'):
s=n'.findSuccessor(n);buildFingers(s); successor=s;
BigDataArchitectures2019-2020 IoanaManolescu 20
PeersjoininginChordTojoin,apeernmustknow(any)peern'alreadyinthenetworkProceduren.join(n'):
s=n'.findSuccessor(n);buildFingers(s); successor=s;
BigDataArchitectures2019-2020 IoanaManolescu 21
PeersjoininginChordTojoin,apeernmustknow(any)peern'alreadyinthenetworkProceduren.join(n'):
s=n'.findSuccessor(n);buildFingers(s);successor=s;
If3hadsomekey-valuepairsforthekey2,3givesthemoverto2Thenetworkisnotstabilizedyet…
BigDataArchitectures2019-2020 IoanaManolescu 22
NetworkstabilizationinChordEachpeerperiodicallyrunsstabilize()n.stabilize():
x=n.succ().pred()if(n<x<succ)thensucc=x;
succ.notify(n)
n.notify(p):if(pred<p<n)thenpred=p
BigDataArchitectures2019-2020 IoanaManolescu 23
NetworkstabilizationinChordFirststabilize()of2:3learnsitsnewpredecessorn.stabilize():
x=n.succ().pred()if(n<x<succ)thensucc=x;
succ.notify(n)
n.notify(p):if(pred<p<n)thenpred=p
BigDataArchitectures2019-2020 IoanaManolescu 24
NetworkstabilizationinChordFirststabilize()of1:1and2connectn.stabilize():
x=n.succ().pred()if(n<x<succ)thensucc=x;
succ.notify(n)
n.notify(p):if(pred<p<n)thenpred=p
BigDataArchitectures2019-2020 IoanaManolescu 25
Peerleavingthenetwork
• Thepeerleaves(withsomeadvancenotice,«ingoodorder»)
• Networkadaptationtopeerleave:– (key,value)pairs:thoseoftheleavingpeeraremovedtoitssuccessor
– Routing:Pnotifiessuccessorandpredecessor,whichreconnect"overP"
BigDataArchitectures2019-2020 IoanaManolescu 26
Peerfailure• Withoutwarning• Intheabsenceofreplication,the(key,value)pairsheldonP
arelost– Peersmayalsore-publishperiodically
ExampleRunningstab(),6notices9isdown6replaces9withitsnextfinger10àallnodeshavecorrectsuccessors,butfingersarewrongRoutingstillworks,evenifalittlesloweddownFingersmustberecomputed
BigDataArchitectures2019-2020 IoanaManolescu 27
Peerfailure
Chordusessuccessorstoadjusttoanychange• Adjustmentmay«slowlypropagate»alongthering,sinceitisrelativelyrare
Topreventerroneousroutingduetosuccessorfailure,eachpeermaintainsalistofitsrdirectsuccessors(2log2N)• Whenthefirstonefails,thenextoneisused...• Allrsuccessorsmustfailsimultaneouslyinordertodisruptsearch
BigDataArchitectures2019-2020 IoanaManolescu 28
GossipinP2Parchitectures• Constant,«background»communicationbetweenpeers• Structuredorunstructurednetworks• Disseminatesinformationaboutpeernetwork,peerdata
BigDataArchitectures2019-2020 IoanaManolescu 29
Peer-to-peernetworks:wrap-up
• Datamodel:– Catalogandsearchatasimplekeylevel
• Querylanguage:keys• Heterogeneity:notthemainissue• Control:
– peersareautonomousinstoringandpublishing– queryprocessingthroughsymetricalgorithm(exceptforsuperpeers)
BigDataArchitectures2019-2020 IoanaManolescu 30
Peer-to-peerdatamanagement• Extractkey-valuepairsfromthedata&indexthem• Toprocessqueries:
– LookuprelevantdatafragmentsintheP2Pnetwork
– Rundistributedqueryplan
Peer1
PeeriLocaldataLocalqueryprocessorIndexing&look-up
Peern
PeermPeerp
…
BigDataArchitectures2019-2020 IoanaManolescu 31
Example:storingrelationaldatainP2Pdatamanagementplatform
• Eachpeerstoresahorizontalsliceofatable• Catalogatthegranularityofthetable:
– Keys:tablenames,e.g.Singer,Song– Value:fragmentdescription,e.g.,peer1:postgres:sch1/Singer&u=u1&p=p1,
– Query:selectSinger.birthdayfromSinger,SongwhereSong.title=«ComeAway»andSinger.sID=Song.singer
– Whatcanhappen?• Tryothergranularities
BigDataArchitectures2019-2020 IoanaManolescu 32
ModernP2Pdatamanagementsystem:Cassandra
– Partitionedrowstore,fullysymetricstructuredP2Parchitecture– BasedontheDynamoK-Vsystem[CHG+07]– Somenesting;indexes.Queries:select,project.Tablesongs:ALTERTABLEsongsADDtagsset<text>;UPDATEsongsSETtags=tags+{'2007'}WHEREid=8a172618…;UPDATEsongsSETtags=tags+{'covers'}WHEREid=8a172618…;UPDATEsongsSETtags=tags+{'1973'}WHEREid=a3e64f8f-…;SELECTid,tagsfromsongs;
33BigDataArchitectures2019-2020 IoanaManolescu
LargeCassandradeployments:• Apple:over75,000nodesstoringover10PBofdata• Netflix:2,500nodes,420TB,over1trillionrequestsperdayCAPtrade-off:timeoutfordecidingwhenanodeisdead«Duringgossipexchanges,everynodemaintainsaslidingwindowofinter-arrivaltimesofgossipmessagesfromothernodesinthecluster.Configuringthephi_convict_thresholdpropertyadjuststhesensitivityofthefailuredetector.Lowervaluesincreasethelikelihoodthatanunresponsivenodewillbemarkedasdown,whilehighervaluesdecreasethelikelihoodthattransientfailurescausingnodefailure.Usethedefaultvalueformostsituations,butincreaseitto10or12forAmazonEC2(duetofrequentlyencounterednetworkcongestion)tohelppreventfalsefailures.Valueshigherthan12andlowerthan5arenotrecommended.»
BigDataArchitectures2019-2020 IoanaManolescu 34
ModernP2Pdatamanagementsystem:Cassandra
STRUCTUREDDATAMANAGEMENTINCLOUDENVIRONMENTS
BigDataArchitectures2019-2020 IoanaManolescu 35
Cloudcomputing
• Idea:delegatelarge-scalestorageandlarge-scalecomputingtoremotecenters– Runbythe(only)enterpriseusingthem:"privateclouds"
• Largecompaniescanaffordthecosttoownandoperateacloudservice:LaPoste,Orange,...
– Runbyacompanywhorentsoutstorageandcomputingservices:"commercialclouds"
• Mainplayers:Amazon(hasbasicallycreatedtheindustry),Google,Microsoft
BigDataArchitectures2019-2020 IoanaManolescu 36
Advantagesofcloudcomputing• AllowcompaniestofocusontheirmainbusinessnotonIT
• Allowscalingtheresourceusageupanddownaccordingtotheneeds
• ComesatacostExamples:• Satelliteimagedataprocessingcompanywhichneedssignificantcomputingresources(only)whenithasanorderfromaclient
• ShopswithmoreclientsasChristmasapproaches
BigDataArchitectures2019-2020 IoanaManolescu 37
https://www.wired.com/2015/03/orbital-insight/
Howcloudserviceswork(1/3)• Storage
– Usershostfilesontrustedservers– TheserviceispaidbytheGBandday
• Totalcost=sum(filesizexfilestoragetime)• Computing
– Usersbuyvirtualcomputers("virtualmachines")– ServicepaidbythedurageofuseoftheVM– Eachvirtualcomputerishostedbysomephysicalcomputerinthecloudprovider'scluster
– Ifaphysicalmachinefails,thevirtualmachinewillberecreatedelsewhereandtheworkwillrestart
BigDataArchitectures2019-2020 IoanaManolescu 38
Howcloudserviceswork(2/3)• Computing(continued)
– Therearetypicallydifferentsizes(capacities)ofvirtualmachines• Small(S),Medium(M),Large(L),Extra-Large(XL)• Thedifferenceisinthecomputingspeed
• Faststorageofsmall-granularitydata,typicallyinmemoryinthecloud– For:metadata(catalog,usermanagement,...)– Key-valuestores,documentstores– Payperoperation(put,get)
• Otherservices– E.g.messagingqueuestosynchronizedifferentapplications
BigDataArchitectures2019-2020 IoanaManolescu 39
Howcloudserviceswork(3/3):cloudcomputingmodels
• Infrastructure-as-a-service– Thevendorprovidesaccesstocomputingresourcessuchasservers,storageandnetworking.
– Clientsusetheirownplatformsandapplicationswithinaserviceprovider’sinfrastructure.Theydonothostbuttheydevelop,deployandadministerinthecloud.
• Platform-as-a-service– Thevendorprovides:storageandothercomputingresources,prebuilttoolstodevelop,customizeandtesttheirownapplications.
– Clientsdonothostandmostlydonotadministereither.Theystilldevelopanddeployinthecloud.
BigDataArchitectures2019-2020 IoanaManolescu 40
Howcloudserviceswork(3/3):cloudcomputingmodels
• Software-as-a-service– Thevendorprovides:storageandothercomputingresources;softwareandapplicationsviaasubscriptionmodel(orpay-per-use...)
– Clientsaccesstheapplicationsremotely.Theydonotstore,host,developnoradminister.
BigDataArchitectures2019-2020 IoanaManolescu 41
Cloudservices
Fine-granularity data store
Virtual machines
Queue service
File storage service
Amazon Scalable
Storage Service (S3)
Google Cloud Storage
Windows Azure BLOB Storage
Amazon Elastic Compute Cloud
(EC2)
Google Compute Engine
Windows Azure Virtual Machines
Amazon DynamoDB
Google High Replication Datastore
Windows Azure Tables
Amazon Simple Queue Service
(SQS)
Google Task Queues
Windows Azure Queues
Cloud Platform
BigDataArchitectures2019-2020 42IoanaManolescu
Performanceinlarge-scaleclusters
• On-site(withinacompanyororganization)oroff-site(cloud)
• Theremaybevariablelatencyacrossthecluster,i.e.somemachine(s)maytemporarilybeslow,dueto– Sharedresources(CPU,cache,memory,network)– Maintenance,softwareupgrade– Globalresourcesharing,e.g.,networkswitches,distributedfilesystems
– Garbagecollection– Energymanagement
• Thevariabilitygetsamplifiedbyscale(seenext)
BigDataArchitectures2019-2020 IoanaManolescu 43
Latencyvariationsinlarge-scaleclusters
BigDataArchitectures2019-2020 IoanaManolescu 44
Considerasettingwhereeachserverresponds• in10ms,99%ofthetime• in1s,1%ofthetime(1in100,bluecurve)
Ifaclientneedstotalkto100servers,theprobabilityof>1slatencyis63%!
Source:DeanandBarroso,"TheTailatScale",CommunicationsofACM,2013
Structureddatamanagementincloudplatforms
• Thecloudprovides:– Distributedfilesystem;Virtualmachines;Fine-granularity(e.g.,key-value
ordocument)store;Distributedmessagequeues
• Basedonthis,needtoproposearchitecturesfor:– Storingverylargevolumesoffine-granularitydata(relational,XML,JSON,
graphs...)andqueryingit– Transactions– Concurrencycontrol
Cloud services provider
BigDataArchitectures2019-2020 45IoanaManolescu
AMADA:XMLdatamanagementwithintheAmazoncloud[CCM13]
Fine-granularity data store
Virtual machines
Queue service
File storage service
Amazon Scalable
Storage Service (S3)
Google Cloud Storage
Windows Azure BLOB Storage
Amazon Elastic Compute Cloud
(EC2)
Google Compute Engine
Windows Azure Virtual Machines
Amazon DynamoDB
Google High Replication Datastore
Windows Azure Tables
Amazon Simple Queue Service
(SQS)
Google Task Queues
Windows Azure Queues
Cloud Platform
BigDataArchitectures2019-2020 46IoanaManolescu
• Functionality:• XMLandRDFstorageandfine-grainindexinginthecloud
• Datastorage:1. Blobstorageincloudfilesystem(i.e.,S3)2. Fine-granularityindexingink-vstore
• Dataquerying:1. Consulttheindextodelimitthedocuments(graphs)
whichmustbeaccessed2. Evaluatequeryoverrelevantdocuments(graphs)
BigDataArchitectures2019-2020 IoanaManolescu 47
AMADA:XMLdatamanagementwithintheAmazoncloud[CCM13]
AMADA:XMLdatamanagementwithintheAmazoncloud[CCM13]
Front-end
①
⑤
②
④
Indexing module
③
Indexing service
Virtual machines
Queue service
File storage service
Cloud warehouse
BigDataArchitectures2019-2020 48IoanaManolescu
AMADAarchitecture
Front-end
Query processor
①
⑤
②
④
① ②
③ ④ ⑤
⑥ ⑦
⑧
Indexing module
③
Fine-granularity fast store
Virtual machines
Queue service
File storage service
Cloud warehouse
BigDataArchitectures2019-2020 49IoanaManolescu
Cloud warehouse
Indexingandstoragecosts
Front-end
①
⑤
②
④
Indexing module
③
Indexing cost depends on: - Documents set (D) - Indexing strategy (I)
Fine-granularity fast store
Virtual machines
Queue service
File storage service
BigDataArchitectures2019-2020 50IoanaManolescu
Cloud warehouse
Indexingandstoragecosts
Front-end
Indexing module
$r
$r
$putidx
$h
$put
$get
$r
Fine-granularity fast store
Virtual machines
Queue service
File storage service
Storage cost
BigDataArchitectures2019-2020 51IoanaManolescu
Cloud warehouse
Queryingcost
Front-end
Query processor
① ②
③ ④ ⑤
⑥ ⑦
⑧
Querying cost depends on: - Query (q) - Documents set (D) - Indexing strategy (I)
Fine-granularity fast store
Virtual machines
Queue service
File storage service
BigDataArchitectures2019-2020 52IoanaManolescu
table+ item+
key
attribute+
name
value
Fine-granularityfaststoreinAmazoncloud:DynamoDB
BigDataArchitectures2019-2020 53IoanaManolescu
Indexingfine-granularitydataintheAmazoncloud
Indexing strategy I: Function associating (key,(name,value)+)+ to a document
BigDataArchitectures2019-2020 54
DynamoDB data model
IoanaManolescu
table+ item+
key
attribute+
name
value
Indexingfine-granularitydataintheAmazoncloud
• Fourindexingstrategies- Label-URI(LU)- Label-URI-Path(LUP)- Label-URI-ID(LUI)- Label-URI-Path/Label-URI-ID(2LUPI)
BigDataArchitectures2019-2020 55
Indexing strategy I: Function associating (key,(name,value)+)+ to a document
IoanaManolescu
table+ item+
key
attribute+
name
value
XMLindexing:example
name
family
‘Smith’
name
brand
car
‘Ford’
‘Audi’
family
model
‘A3’
family
brand ‘Lee’
‘Ford’
model
’330d’
name car car
brand
‘BMW’
name
brand
car
‘Martin’
‘Ford’
family
model
‘Mustang’
doc1.xml doc2.xml doc3.xml doc4.xml
name
brand
car
‘Ford’
family
model
q
BigDataArchitectures2019-2020 56IoanaManolescu
XMLindexing:example
doc4.xml Martin Mustang
Result:
BigDataArchitectures2019-2020 57
name
brand
car
‘Ford’
family
model
q
name
family
‘Smith’
name
brand
car
‘Ford’
‘Audi’
family
model
‘A3’
family
brand ‘Lee’
‘Ford’
model
’330d’
name car car
brand
‘BMW’
name
brand
car
‘Martin’
‘Ford’
family
model
‘Mustang’
doc1.xml doc2.xml doc3.xml doc4.xml
IoanaManolescu
Label-URI(LU)strategy
efamily doc1.xml doc2.xml doc3.xml doc4.xml
Ø Ø Ø Ø
wFord doc2.xml doc3.xml doc4.xml
Ø Ø Ø
ename doc1.xml doc2.xml doc3.xml doc4.xml
Ø Ø Ø Ø
…
Look-up:IntersectionofURIsetsassociatedtoeachquerynode
Index: (key, (name, value))
BigDataArchitectures2019-2020 58
name
family
‘Smith’
name
brand
car
‘Ford’
‘Audi’
family
model
‘A3’
family
brand ‘Lee’
‘Ford’
model
’330d’
name car car
brand
‘BMW’
name
brand
car
‘Martin’
‘Ford’
family
model
‘Mustang’
doc1.xml doc2.xml doc3.xml doc4.xml ✔ ✖ ✔ ✔
name
brand
car
‘Ford’
family
model
q
IoanaManolescu
Label-URI-ID(LUI)strategy
efamily doc1.xml doc2.xml doc3.xml doc4.xml
[1 3 0] [1 8 0] [1 11 0] [1 8 0]
wFord doc2.xml doc3.xml doc4.xml
[3 1 2] [6 3 3] [6 3 2]
ename doc1.xml doc2.xml doc3.xml doc4.xml
[2 2 1] [2 2 1] [2 2 1] [2 2 1]
…
Look-up:StructuraljoinoverIDsassociatedtoeachquerynode
Index: (key, (name, value))
BigDataArchitectures2019-2020 59
name
brand
car
‘Ford’
family
model
q
name
family
‘Smith’
name
brand
car
‘Ford’
‘Audi’
family
model
‘A3’
family
brand ‘Lee’
‘Ford’
model
’330d’
name car car
brand
‘BMW’
name
brand
car
‘Martin’
‘Ford’
family
model
‘Mustang’
doc1.xml doc2.xml doc3.xml doc4.xml ✖ ✔ ✖ ✖
IoanaManolescu
Queryanswering(eightruns)
0
6000
12000
18000
24000
30000
36000
LU LUP LUI 2LUPI LU LUP LUI 2LUPIL XL
Res
pons
e Ti
me
(s)
1 instance8 instances
BigDataArchitectures2019-2020 IoanaManolescu 60
Queryansweringtime
0,1
1
10
100
1000
10000
L XL
Res
pons
e Ti
me
(s)
No Index LU LUP LUI 2LUPI No Index – 20000 docs LU – 3 docs LUP – 2 docs LUI – 1 doc 2LUPI – 1 doc
Q1 Log scale
BigDataArchitectures2019-2020 IoanaManolescu 61
Queryansweringcost
0
0,2
0,4
0,6
0,8
1
L XL
Cos
t ($)
No Index LU LUP LUI 2LUPI
BigDataArchitectures2019-2020 IoanaManolescu 62
Queryansweringcostdetail(XL)
BigDataArchitectures2019-2020 IoanaManolescu 63
Indexcostamortization
-100
-75
-50
-25
0
25
50
75
100
125
0 2 4 6 8 10 12 14 16 18 20
#run
s x
bene
fit(I,
W) -
bu
ildin
gCos
t(I)
# runs
LU LUP LUI 2LUPI
BigDataArchitectures2019-2020 IoanaManolescu 64
CloudDataWarehouseServices
• Software-as-a-Service(SaaS)solutions• SnowflakeintheAmazonCloud• GoogleBigQuery(https://cloud.google.com/bigquery/)
• AmazonRedshift(https://aws.amazon.com/redshift/)
• MicrosoftAzureSQLDataWarehouse(https://azure.microsoft.com/)
BigDataArchitectures2019-2020 IoanaManolescu 65
CloudDataWarehouseServicesTheneed:efficientdataprocessingatverylargescaleàdistributedsystemPrevioussolution:share-nothingarchitectures(MapReduceorSparkclusters)• Eachnodestoressomedataandcomputesonit• StorageandcomputationsaredistributedatthesametimeLimitations:1. Heterogeneousworkload,e.g.bulkloading(highI/O,littletono
CPU)...intensivecomputing(littletonoI/O,highCPU)àhardtochosemachines!
2. Membershipchangesarefrequentinthecloud,leadtodatashufflesandnegativelyimpactperformance
3. Hardtodoonlineupgradeinsymetric,homogeneousarchitecture
BigDataArchitectures2019-2020 IoanaManolescu 66
Snowflake:separatingstoragefromcomputationsinthecloud[DGZ+16]
• StoredataintheAmazonS3service.• Storemetadata(catalog,usermanagement,...)inhigh-performance,transactionalkey-valuestore
• Performcomputationsinvirtualwarehouses(Snowflakeproprietarysoftware)– Avirtualwarehouse(VW)hasasetofworkers.ItiscreatedondemandbyaSnowflakeclient.
– AclientcanhavemanyVWs.– EachqueryishandledwithinoneVW.
BigDataArchitectures2019-2020 IoanaManolescu 67
Snowflakearchitecture
BigDataArchitectures2019-2020 IoanaManolescu 68
ProprietarySQLengine(withafewmoregoodies)ineachVirtualWarehouse
SnowflakeVirtualWarehouses
• EveryVWhasaccesstothesamedata(fromS3)• AworkerbelongstoexactlyoneVW• VirtualWarehousescomein"T-shirtsizes"(XStoXXL)
– Hidescloudprovidernodes(eventheirnumber)fromSnowflakeclient
– AllowsindependentSnowflakepricingpolicy– Allows"smart"investmentofcloudservicescost:foraloadtask,bookaVWof(4nodesfora15h)vs.(32nodesfor2h).
BigDataArchitectures2019-2020 IoanaManolescu 69
SnowflakeVirtualWarehouses
• S3muchsloweraccessthanthelocaldiskofanodeàaworker'slocaldiskactsascache– LRUcachingofthedatalastneededonthisnode– Cachegranularity:tablecolumns
• ToavoidredundantcachingofthesamedatafragmentsacrossthenodesofasingleVW,theOptimizerassignsfiles(table)cachedatatonodesusinghashingonthethe(table,column)name– Ifthisdataiscached,itisonlycachedonaspecificVWnode
BigDataArchitectures2019-2020 IoanaManolescu 70
QueryevaluationinSnowflake1. Selectivedataaccess
– Eachtableisstoredasassetofshards– Insideeachshard,dataisstoredasasetof(compressed)columns
– Headersbuiltforeachcolumnwithintheshard• Minimumandmaximumvalues• Noneedtoreadashardifthequerypredicateisincompatiblewiththeheaderinformation
2.Queryoptimizer– Cost-andstatistic-based– Headerscomputedevenonintermediaryresults– Somedecisionstakenatruntime
3.Intermediaryqueryresultswritteninnodelocaldisks,then(ifneeded)toS3
BigDataArchitectures2019-2020 IoanaManolescu 71
ConcurrencycontrolinSnowflake
• Handledgloballyusingfine-granularitydatastore• Anupdatecreatesanewversionofatable(multiversionconcurrencycontrol):nofiner-granularityupdate
• Eachversionhasatimestamp• Possibletoexplicitlyquerytheversionatorafteracertaintimestamp
• Eachversionstaysavailable90daysafterdeletion
BigDataArchitectures2019-2020 IoanaManolescu 72
GLOBALCOURSEWRAP-UP
BigDataArchitectures2019-2020 IoanaManolescu 73
BigDataArchitectures• Moderndevelopmentofdistributedcomputingplatformsleadsto
unprecedentedexplosioninBigDataarchitecturesandsystems• Dimensionsofanalysis:
– Scaleofdistribution– Datamodel,querylanguagetheysupport– CAPcompromise(whichlevelofconsistency)– Performanceinfluencedby:
• Datastorage,indexing,queryevaluationalgorithms...• Queryoptimization• Synchronizationoperations
– Successinfluencedbyperformance,easeofuse,killerapplications
BigDataArchitectures2019-2020 IoanaManolescu 74
References• [CCM13]Webdataindexinginthecloud:efficiencyandcostreductions.JesusCamacho-Rodriguez,DarioColazzo,IoanaManolescu.EDBT,2013
• [CHC+07]G.DeCandia,D.Hastorun,M.Jampanietal.Dynamo:Amazon'shighlyavailablekey-valuestore,ACMSOSP2007
• [DGZ+16]TheSnowflakeElasticDataWarehouse.BenoîtDageville,ThierryCruanes,MarcinZukowskietal.,SIGMOD,2016
• [SMK+01]Chord:Ascalablepeer-to-peerlookupserviceforinternetapplications,I.Stoica,R.Morris,F.Kaashoeketal.ACMSIGCOMMComputerCommunicationReview,2001
BigDataArchitectures2019-2020 IoanaManolescu 75