Best Practices In Building Scalable Cloud Ready Service Based
-
Upload
igor-moochnick -
Category
Technology
-
view
4.762 -
download
0
Transcript of Best Practices In Building Scalable Cloud Ready Service Based
![Page 1: Best Practices In Building Scalable Cloud Ready Service Based](https://reader036.fdocuments.in/reader036/viewer/2022070604/546e48c7af7959ed048b4aca/html5/thumbnails/1.jpg)
Best Practices in building scalable cloud-ready Service based systems
Discussion
Igor MoochnickIgorShare [email protected]
Blog: www.igorshare.com/blog
![Page 2: Best Practices In Building Scalable Cloud Ready Service Based](https://reader036.fdocuments.in/reader036/viewer/2022070604/546e48c7af7959ed048b4aca/html5/thumbnails/2.jpg)
Is this system scalable?
![Page 3: Best Practices In Building Scalable Cloud Ready Service Based](https://reader036.fdocuments.in/reader036/viewer/2022070604/546e48c7af7959ed048b4aca/html5/thumbnails/3.jpg)
What is scalability?
• Increase resources -> Increase performance proportionally to the amount of added resources
• Increase performance -> more units of works
![Page 4: Best Practices In Building Scalable Cloud Ready Service Based](https://reader036.fdocuments.in/reader036/viewer/2022070604/546e48c7af7959ed048b4aca/html5/thumbnails/4.jpg)
$10,000
machine
$1000machin
e
Scale-up And Scale-outVo
lum
e
$500machine
# MachinesScale Up
$500machin
e
$500machin
e
$500machin
e
Volu
me
$500machin
e
Scale Out
DNSWWW
![Page 5: Best Practices In Building Scalable Cloud Ready Service Based](https://reader036.fdocuments.in/reader036/viewer/2022070604/546e48c7af7959ed048b4aca/html5/thumbnails/5.jpg)
Is this system scalable?
![Page 6: Best Practices In Building Scalable Cloud Ready Service Based](https://reader036.fdocuments.in/reader036/viewer/2022070604/546e48c7af7959ed048b4aca/html5/thumbnails/6.jpg)
Is this system scalable?
![Page 7: Best Practices In Building Scalable Cloud Ready Service Based](https://reader036.fdocuments.in/reader036/viewer/2022070604/546e48c7af7959ed048b4aca/html5/thumbnails/7.jpg)
Is this system scalable?
![Page 8: Best Practices In Building Scalable Cloud Ready Service Based](https://reader036.fdocuments.in/reader036/viewer/2022070604/546e48c7af7959ed048b4aca/html5/thumbnails/8.jpg)
Is this system a scalable MADNESS?
![Page 9: Best Practices In Building Scalable Cloud Ready Service Based](https://reader036.fdocuments.in/reader036/viewer/2022070604/546e48c7af7959ed048b4aca/html5/thumbnails/9.jpg)
Here is the gun. Go kill yourself!
![Page 10: Best Practices In Building Scalable Cloud Ready Service Based](https://reader036.fdocuments.in/reader036/viewer/2022070604/546e48c7af7959ed048b4aca/html5/thumbnails/10.jpg)
Strong
Eventual
Optimistic
Missile Launch
Address Change
Stock Ticker
Now
In the Future
Consistency Level Changes are Visible Example
Maybe in the Future
Some Useful Definitions
Exactly Once
At Least Once
At Most Once
Bank Transfer
Streaming Video
No loss, no duplicates
No loss, duplicates
Assurance Message Delivery Example
Loss, no duplicates
Best Effort Stock TickerLoss, duplicates
Consistency Levels
Message Assurances
![Page 11: Best Practices In Building Scalable Cloud Ready Service Based](https://reader036.fdocuments.in/reader036/viewer/2022070604/546e48c7af7959ed048b4aca/html5/thumbnails/11.jpg)
Where did you start? Where did you end up?Shared State
ACID Transactions
Partitioned, Replicated State
Eventual Consistency
Exactly Once Messaging Best Effort Messaging
Machine Loss is a Catastrophe
Keep Processes Running
Machine Loss is Business As Usual
Recovery-Oriented Computing
The InfrastructureDeveloper's Experience
![Page 12: Best Practices In Building Scalable Cloud Ready Service Based](https://reader036.fdocuments.in/reader036/viewer/2022070604/546e48c7af7959ed048b4aca/html5/thumbnails/12.jpg)
The law
• The least scalable component of your system becomes a bottleneck for the whole system
![Page 13: Best Practices In Building Scalable Cloud Ready Service Based](https://reader036.fdocuments.in/reader036/viewer/2022070604/546e48c7af7959ed048b4aca/html5/thumbnails/13.jpg)
Recipe ingredients (Amazon guidelines)• Autonomy• Asynchrony• Controlled concurrency• Controlled parallelism• Decentralize• Decompose into small well-understood building blocks• Failure tolerant• Local responsibility• Recovery Built-in• Simplicity• Symmetry
![Page 14: Best Practices In Building Scalable Cloud Ready Service Based](https://reader036.fdocuments.in/reader036/viewer/2022070604/546e48c7af7959ed048b4aca/html5/thumbnails/14.jpg)
Key principals
• Things fail all the time!• Machines
– Disposable– Nameless– Self assembled
• State management– Caching– Loose consistency– Relax isolation
• Redundancy• Partitioning
• Loosely coupled messaging• Best effort• Message loss• Retries• Self monitors• Self heals• Designed to expect failures• Continue to work seamlessly
during the failure
![Page 15: Best Practices In Building Scalable Cloud Ready Service Based](https://reader036.fdocuments.in/reader036/viewer/2022070604/546e48c7af7959ed048b4aca/html5/thumbnails/15.jpg)
Application Development Patterns• Architecture
– Choose a high-level framework– Keep service and hosting code separate– Partition
• Design– Use loose coupling– Use caches and stale data– Have just a few simple recovery paths– Be topology-independent– Be hardware-indepedent
![Page 16: Best Practices In Building Scalable Cloud Ready Service Based](https://reader036.fdocuments.in/reader036/viewer/2022070604/546e48c7af7959ed048b4aca/html5/thumbnails/16.jpg)
Challenges Of Scalability
• How do I ensure incoming requests are processed at the right location?– Partition on service-specific input– Dynamically route to correct node– Fail over seamlessly
• How do I manage state inside my service?– Take a hard look at consistency requirements– Aggressively cache and use transient data– Partition the Storage Tier
![Page 17: Best Practices In Building Scalable Cloud Ready Service Based](https://reader036.fdocuments.in/reader036/viewer/2022070604/546e48c7af7959ed048b4aca/html5/thumbnails/17.jpg)
ACID vs. BASE
• ACID– Atomic– Consistent– Isolated– Durable
• Modern BASE-based systems– Basically Available– Soft-state (or scalable)– Eventually consistent
![Page 18: Best Practices In Building Scalable Cloud Ready Service Based](https://reader036.fdocuments.in/reader036/viewer/2022070604/546e48c7af7959ed048b4aca/html5/thumbnails/18.jpg)
What is the problem?
• Only two of three:– Strong Consistency
• All clients see the same view during updates– High Availability
• Some data replica is always available despite failures
– Partition tolerance• All the properties hold even if partitioned
![Page 19: Best Practices In Building Scalable Cloud Ready Service Based](https://reader036.fdocuments.in/reader036/viewer/2022070604/546e48c7af7959ed048b4aca/html5/thumbnails/19.jpg)
Techniques
• Expiration based caching: AP• Quorum / majority algorithms: PC• Two-phase commit: AC
![Page 20: Best Practices In Building Scalable Cloud Ready Service Based](https://reader036.fdocuments.in/reader036/viewer/2022070604/546e48c7af7959ed048b4aca/html5/thumbnails/20.jpg)
Scaling data in 3 steps
• Partitioning• Routing• State management
![Page 21: Best Practices In Building Scalable Cloud Ready Service Based](https://reader036.fdocuments.in/reader036/viewer/2022070604/546e48c7af7959ed048b4aca/html5/thumbnails/21.jpg)
Solving the data congestion
• Throttling (especially on startup after failure)• Denormalization• Scale vs. Performance• Fault Tolerance and recoverability• Geo-distribution• Content distribution providers (like Akamai)
![Page 22: Best Practices In Building Scalable Cloud Ready Service Based](https://reader036.fdocuments.in/reader036/viewer/2022070604/546e48c7af7959ed048b4aca/html5/thumbnails/22.jpg)
Fault tolerance
• Throttling incoming traffic• Limit retries• Server failover• Data center failover• Consider using queues
![Page 23: Best Practices In Building Scalable Cloud Ready Service Based](https://reader036.fdocuments.in/reader036/viewer/2022070604/546e48c7af7959ed048b4aca/html5/thumbnails/23.jpg)
Monitoring
• Monitor data about what the user sees – this is what is most important
• Make sure not to overdo – kills the components you rely on
• Be frugal– Built in counters and monitor the trends - can help you to
predict the spikes and allocate on demand extra resources
![Page 24: Best Practices In Building Scalable Cloud Ready Service Based](https://reader036.fdocuments.in/reader036/viewer/2022070604/546e48c7af7959ed048b4aca/html5/thumbnails/24.jpg)
Monitoring
• Availability• Performance• Alerts• Auto throttling• Capacity thresholds• Load• Transactions• Should measure
realistic/relevant actions and behavior!
Importance
![Page 25: Best Practices In Building Scalable Cloud Ready Service Based](https://reader036.fdocuments.in/reader036/viewer/2022070604/546e48c7af7959ed048b4aca/html5/thumbnails/25.jpg)
Diagnosing & Logging
• Non-blocking• Asynchronously• Size – can be too big (there is “too much of a good thing”)
– Have control over “what” and “how much”• Performance hit (“do no harm”)• Should not become a bottleneck• Be careful what you log
– Horizontally– Vertically
• Should be able to replay logs and correlate the requests– <time><correlate-id><node-id><action><data><result>
![Page 26: Best Practices In Building Scalable Cloud Ready Service Based](https://reader036.fdocuments.in/reader036/viewer/2022070604/546e48c7af7959ed048b4aca/html5/thumbnails/26.jpg)
Troubleshooting the distributed systems• Decoupling• Role isolation• Single box
• Allow to separate the functionality from the rest of the system
• Allow to run all from a single box• Have stubs and simulators• Be able to “replay” the logs
![Page 27: Best Practices In Building Scalable Cloud Ready Service Based](https://reader036.fdocuments.in/reader036/viewer/2022070604/546e48c7af7959ed048b4aca/html5/thumbnails/27.jpg)
Deployments
• Deployment packaging• Rolling out gradually or
atomically• Automatic deployments• Staging environment• Building confidence with
real customer data• Rolling back• Security trumps feature• Load balancing
• Consider linear scale• Keep IT in mind• Upgradability
![Page 28: Best Practices In Building Scalable Cloud Ready Service Based](https://reader036.fdocuments.in/reader036/viewer/2022070604/546e48c7af7959ed048b4aca/html5/thumbnails/28.jpg)
Deployment
• It’s hard• It’s hard to make it right• Automate everything – simplifies the repeatability• Version Forward/Backward compatible• Rolling upgrade and rollback• Be nice to your friends• Know and manage your environments
• Compensate for gradual system recovery• Clean queues
![Page 29: Best Practices In Building Scalable Cloud Ready Service Based](https://reader036.fdocuments.in/reader036/viewer/2022070604/546e48c7af7959ed048b4aca/html5/thumbnails/29.jpg)
Resources
• Availability & Consistency presentation of Amazon CTO Dr. Werner Vogel http://www.infoq.com/presentations/availability-consistency
• Microsoft PDC’08 Presentationshttps://sessions.microsoftpdc.com/timeline.aspx
![Page 30: Best Practices In Building Scalable Cloud Ready Service Based](https://reader036.fdocuments.in/reader036/viewer/2022070604/546e48c7af7959ed048b4aca/html5/thumbnails/30.jpg)
Q&A
![Page 31: Best Practices In Building Scalable Cloud Ready Service Based](https://reader036.fdocuments.in/reader036/viewer/2022070604/546e48c7af7959ed048b4aca/html5/thumbnails/31.jpg)
Thank you!