Architecting Cloudy Applications

Architecting Cloudy Applications

Architecting Cloudy ApplicationsDavid [email protected]/dachou

Microsoft's Windows Azure platform is a virtualized and abstracted application platform that can be used to build highly scalable and reliable applications, with Java. The environment consists of a set of services such as NoSQL table storage, blob storage, queues, relational database service, internet service bus, access control, and more. Java applications can be built using these services via Web services APIs, and your own Java Virtual Machine, without worrying about the underlying server OS and infrastructure. Highlights of this session will include: An overview of the Windows Azure environment How to develop and deploy Java applications in Windows Azure How to architect horizontally scalable applications in Windows Azure

1Facebook (2009)+200B pageviews /month>3.9T feed actions /day+300M active users>1B chat mesgs /day100M search queries /day>6B minutes spent /day (ranked #2 on Internet)+20B photos, +2B/month growth600,000 photos served /sec25TB log data /day processed thru Scribe120M queries /sec on memcache

> IntroductionSize mattersTwitter (2009)600 requests /secavg 200-300 connections /sec; peak at 800MySQL handles 2,400 requests /sec30+ processes for handling odd jobsprocess a request in 200 milliseconds in Railsaverage time spent in the database is 50-100 milliseconds+16 GB of memcachedGoogle (2007)+20 petabytes of data processed /day by +100K MapReduce jobs 1 petabyte sort took ~6 hours on ~4K servers replicated onto ~48K disks+200 GFS clusters, each at 1-5K nodes, handling +5 petabytes of storage~40 GB /sec aggregate read/write throughput across the cluster+500 servers for each search query < 500ms>1B views / day on Youtube (2009)Myspace (2007)115B pageviews /month5M concurrent users @ peak+3B images, mp3, videos+10M new images/day160 Gbit/sec peak bandwidth

Flickr (2007)+4B queries /day+2B photos served~35M photos in squid cache~2M photos in squids RAM 38k req/sec to memcached (12M objects) 2 PB raw storage+400K photos added /daySource: multiple articles, High Scalabilityhttp://highscalability.com/2007founded by 6 people2008$29M funding from VC2009revenue - $270M$180M funding from Digital Sky Technologies20101,000+ employees$300M funding from Google and Softbank

Active unique players75M monthly60M daily1M daily 4 days after launch10M after 60 daysHosted in Amazon Web Services12,000 EC2 nodes3 Gigabits/sec of traffic between FarmVille and Facebook (at peak)caching cluster serves another 1.5 Gigabits/sec to the applicationCloud levels the playing fieldSource: How FarmVille Scales to Harvest 75 Million Players a Month, 2010.02.08, Tedd Hoffhttp://highscalability.com/blog/2010/2/8/how-farmville-scales-to-harvest-75-million-players-a-month.html

> IntroductionCharacteristicsOn-demand self-serviceBroad network accessResource poolingRapid elasticityMeasured serviceService modelsSoftware as a servicePlatform as a serviceInfrastructure as a serviceDeployment modelsPrivate cloudCommunity cloudPublic cloudHybrid cloudCloud computingSource: The NIST Definition of Cloud Computing, Version 15, 2009.10.07, Peter Mell and Tim Grance http://csrc.nist.gov/groups/SNS/cloud-computing/cloud-def-v15.docCloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. This cloud model promotes availability and is composed of five essential characteristics, three service models, and four deployment models.

> Introduction(On-Premise)Infrastructure(as a Service)Platform(as a Service)StorageServersNetworkingO/SMiddlewareVirtualizationDataApplicationsRuntimeStorageServersNetworkingO/SMiddlewareVirtualizationDataApplicationsRuntimeYou manageManaged by vendorManaged by vendorYou manageYou manageStorageServersNetworkingO/SMiddlewareVirtualizationApplicationsRuntimeDataSoftware(as a Service)Managed by vendorStorageServersNetworkingO/SMiddlewareVirtualizationApplicationsRuntimeDataService delivery models

> Introduction5app serverCommon characteristicssynchronous processessequential units of worktight couplingstatefulpessimistic concurrencyclustering for HAvertical scalingTraditional scale-up architecture

> Architecting for Scale > Vertical Scalingapp serverwebdata storeunits of workwebdata storeapp serverapp serverTraditional scale-up architecture

> Architecting for Scale > Vertical Scalingwebdata storewebTo scale, get bigger serversexpensivehas scaling limitsinefficient use of resources

Picture source: http://en.wikipedia.org/wiki/Amdahl%27s_law

7data storeapp serverTraditional scale-up architecture

> Architecting for Scale > Vertical Scalingapp serverwebwebWhen problems occurbigger failure impactTraditional scale-up architecture

> Architecting for Scale > Vertical Scalingapp serverwebdata storewebWhen problems occurbigger failure impactmore complex recoveryAt most two of these properties for any shared-data systemSource: Towards Robust Distributed Systems, Dr. Eric A. Brewer, UC BerkeleyCAPConsistency + Availability High data integritySingle site, cluster database, LDAP, xFS file system, etc.2-phase commit, data replication, etc.CAPConsistency + Partition Distributed database, distributed locking, etc.Pessimistic locking, minority partition unavailable, etc.CAPAvailability + Partition High scalabilityDistributed cache, DNS, etc.Optimistic locking, expiration/leases, etc.CAP (Consistency, Availability, Partition) Theorem

> Architecting for Scale > Fundamental Concepts10

Use more pieces, not bigger pieces

LEGO 10179 Ultimate Collector's Millennium Falcon33 x 22 x 8.3 inches (L/W/H)5,195 piecesLEGO 7778 Midi-scale Millennium Falcon9.3 x 6.7 x 3.2 inches (L/W/H) 356 pieces> Architecting for Scale > Horizontal scalingTo build for big scale use more of the same pieces, not bigger pieces; though a different approach may be needed11app serverScale-out architecture

> Architecting for Scale > Horizontal scalingapp serverwebdata storeCommon characteristicssmall logical units of workloosely-coupled processesstatelessevent-driven designoptimistic concurrencypartitioned dataredundancy fault-tolerancere-try-based recoverabilitywebdata storeapp serverapp serverapp serverapp serverapp serverScale-out architecture

> Architecting for Scale > Horizontal scalingapp serverwebdata storewebwebwebdata storewebwebTo scale, add more serversnot bigger serversdata storedata storedata storedata storeapp serverapp serverapp serverapp serverapp serverapp serverScale-out architecture

> Architecting for Scale > Horizontal scalingwebdata storewebwebwebdata storewebwebWhen problems occursmaller failure impacthigher perceived availabilitydata storedata storedata storedata storeapp serverapp serverapp serverScale-out architecture

> Architecting for Scale > Horizontal scalingapp serverwebdata storewebwebapp serverwebdata storewebwebWhen problems occursmaller failure impacthigher perceived availabilitysimpler recoverydata storedata storedata storedata storeapp serverapp serverScalable performance at extreme scaleasynchronous processesparallelizationsmaller footprintoptimized resource usagereduced response timeimproved throughputapp serverapp serverScale-out architecture + distributed computing

> Architecting for Scale > Horizontal scalingapp serverwebdata storewebwebapp serverwebdata storewebwebdata storedata storedata storedata storeparallel tasksasync tasksperceived response timeapp serverapp serverWhen problems occursmaller units of workdecoupling shields impactapp serverapp serverScale-out architecture + distributed computing

> Architecting for Scale > Horizontal scalingapp serverwebdata storewebwebapp serverwebdata storewebwebdata storedata storedata storedata storeapp serverWhen problems occursmaller units of workdecoupling shields impacteven simpler recoveryapp serverapp serverScale-out architecture + distributed computing

> Architecting for Scale > Horizontal scalingapp serverwebdata storewebwebapp serverwebdata storewebwebdata storedata storedata storedata storeLive Journal (from Brad Fitzpatrick, then Founder at Live Journal, 2007)

> Architecting for Scale > Cloud Architecture Patterns

Partitioned DataDistributedCacheWeb FrontendDistributed StorageApps & ServicesSource: http://danga.com/words/2007_06_usenix/usenix.pdf

19

Flickr (from Cal Henderson, then Director of Engineering at Yahoo, 2007)

> Architecting for Scale > Cloud Architecture PatternsPartitioned DataDistributedCacheWeb FrontendDistributed StorageApps & ServicesSource: http://highscalability.com/blog/2007/11/13/flickr-architecture.html

20SlideShare (from John Boutelle, CTO at Slideshare, 2008)


Partitioned DataDistributed CacheWebFrontendDistributed StorageApps &ServicesSource: http://www.slideshare.net/jboutelle/scalable-web-architectures-w-ruby-and-amazon-s3

21Twitter (from John Adams, Ops Engineer at Twitter, 2010)


PartitionedDataDistributedCacheWebFrontendDistributedStorageApps &ServicesQueuesAsyncProcessesSource: http://www.slideshare.net/netik/billions-of-hits-scaling-twitterSource: http://highscalability.com/blog/2009/6/27/scaling-twitter-making-twitter-10000-percent-faster.html

222010 stats (Source: http://www.facebook.com/press/info.php?statistics)People+500M active users50% of active users log on in any given daypeople spend +700B minutes /monthActivity on Facebook+900M objects that people interact with+30B pieces of content shared /monthGlobal Reach+70 translations available on the site~70% of users outside the US+300K users helped translate the site through the translations applicationPlatform+1M developers from +180 countries+70% of users engage with applications /month+550K active applications+1M websites have integrated with Facebook Platform +150M people engage with Facebook on external websites /month


Facebook(from Jeff Rothschild, VP Technology at Facebook, 2009)PartitionedDataDistributedCacheWebFrontendDistributedStorageApps &ServicesParallelProcessesAsyncProcessesSource: http://highscalability.com/blog/2009/10/12/high-performance-at-massive-scale-lessons-learned-at-faceboo-1.html

23Fundamental concepts

> Architecting for ScaleVertical scaling still works

Picture source: http://pdp.protopak.net/Belltheous90/DeathStarII.gif

24

Fundamental concepts

> Architecting for ScaleHorizontal scaling for cloud computingSmall pieces, loosely coupledDistributed computing best practicesasynchronous processes (event-driven design)parallelizationidempotent operations (handle duplicity)de-normalized, partitioned data (sharding)shared nothing architectureoptimistic concurrencyfault-tolerance by redundancy and replicationetc.Asynchronous processes & parallelization

> Architecting for Scale > Fundamental ConceptsDefer work as late as possiblereturn to user as quickly as possibleevent-driven design (instead of request-driven)Cloud computing friendlydistributes work to more servers (divide & conquer)smaller resource usage/footprintsmaller failure surfacedecouples process dependenciesWindows Azure platform servicesQueue ServiceAppFabric Service Businter-node communicationWorker RoleWeb RoleQueuesService BusWeb RoleWeb RoleWeb RoleWorker RoleWorker RoleWorker RolePartitioned data

> Architecting for Scale > Fundamental ConceptsShared nothing architecturetransaction locality (partition based on an entity that is the atomic target of majority of transactional processing)loosened referential integrity (avoid distributed transactions across shard and entity boundaries)design for dynamic redistribution and growth of data (elasticity)

Cloud computing friendlydivide & conquersize growth with virtually no limitssmaller failure surfaceWindows Azure platform servicesTable Storage ServiceSQL AzureWeb RoleQueuesWeb RoleWeb RoleWorker RoleRelational DatabaseRelational DatabaseRelational DatabaseWeb RolereadwriteIdempotent operations

> Architecting for Scale > Fundamental ConceptsRepeatable processesallow duplicates (additive)allow re-tries (overwrite)reject duplicates (optimistic locking)stateless designCloud computing friendlyresiliencyWindows Azure platform servicesQueue ServiceAppFabric Service Bus

Worker RoleService BusWorker RoleWorker RoleHybrid architectures

> Architecting for Scale > Fundamental ConceptsScale-out (horizontal)BASE: Basically Available, Soft state, Eventually consistentfocus on commitconservative (pessimistic)shared nothingfavor extreme sizee.g., user requests, data collection & processing, etc.

Scale-up (vertical)ACID: Atomicity, Consistency, Isolation, Durabilityavailability first; best effortaggressive (optimistic)transactionalfavor accuracy/consistencye.g., BI & analytics, financial processing, etc.

Most distributed systems employ both approaches 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.Thank you!

David [email protected]/dachou

Architecting Cloudy Applications

Technology

Transcript of Architecting Cloudy Applications