Scalability Design Principles - Internal Session
-
date post
18-Oct-2014 -
Category
Design
-
view
2.659 -
download
0
description
Transcript of Scalability Design Principles - Internal Session
![Page 1: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/1.jpg)
Scalable Web Design – Principles and Patterns
Speaker : Sachin Prakash Sancheti Principal Architect – Cloud (Windows Azure)
1
![Page 2: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/2.jpg)
Context
2
![Page 3: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/3.jpg)
3
Server Busy!!
Your Request can not be processed,
please try after some time
I am trying to book a
ticket for 1 hour now
Please wait!
![Page 4: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/4.jpg)
Any Real life Examples?
4
Survey
![Page 5: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/5.jpg)
What is Scalability?
• It is NOT – Only Performance
– High Availability
– Business Continuity Planning
• It Is – Traffic, User Growth
– Dataset, Database Size Growth
5
![Page 6: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/6.jpg)
What is Scalability?
• Scalability – “The Scalability is measure of number of users it can effectively
support at the same time without degrading the defined performance”
– Has limits – E.g. “With two load balanced capacity it should support 1000 concurrent users with average response time of 3 seconds”
• “Performance is what an individual user experiences; Scalability is how many users get to experience it TOGETHER”
6
![Page 7: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/7.jpg)
What is the Concern?
• Scalability is a business concern
– Google observed 500-milisecond delay to page response caused 20% decrease in traffic
– Amazon.com observed 100-milisecond delay caused a 1% decrease in retail revenue
– Remember “Performance is what an individual user experiences; Scalability is how many users get to experience it TOGETHER”
7
![Page 8: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/8.jpg)
Handling Scalability – Degraded Application
• Degraded Application – Doing nothing and loosing business
8
![Page 9: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/9.jpg)
Handling Scalability - Throttling
• Throttling – Throttling the requests to temporarily stop accepting new requests
and serve better to existing or important users
9
![Page 10: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/10.jpg)
Handling Scalability – Adding Resources
• Adding Resources – Scaling up – Vertical Scaling
• Get Bigger
• Widening the roads
– Scaling out – Horizontal Scaling
• Get More
• Routing the traffic (Partitioning)
10
![Page 11: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/11.jpg)
Typical Web Application Resources
• Web Server, Application Server (Middle Tier) and Database Tier
11
Web Server
Database Server
Application Server
![Page 12: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/12.jpg)
Scaling Solutions
• Vertical Scaling OR Scaling Up – Increasing resource power
– Remember widening the roads!!
• Horizontal Scaling OR Scaling Out – Adding additional machines/nodes
– Remember routing the traffic
12
![Page 13: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/13.jpg)
Vertical Vs. Horizontal Scaling
13
Vertical Scaling Horizontal Scaling
Higher Capital Investment On Demand Investment
Utilization concerns Utilization can be optimized
Relatively Quicker and works with the current design
Relatively more time consuming and needs redesigning
Limiting Scale Internet Scale
Not Cloud Native Design Cloud Native Design
![Page 14: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/14.jpg)
Web/Application Server Scalability
14
![Page 15: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/15.jpg)
Scaling Out Web Server – Load Balancing
15
Web Server
Web Server
Web Server
• Design for Fault Tolerance – Intent : Enables system to continue its
intended operation, possibly at a reduced level, rather than failing completely, when some part of the system fails
– Drivers: Degraded services are better than no service at all. Compare cost effectiveness
– Solution: • Load Balancing
• Monitoring, Self Healing, Restart
![Page 16: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/16.jpg)
Pattern - Bi-directional Scaling
• Design for Scaling Out (Bidirectional) – Intent: Deployment built using commodity of hardware working
together for economies of scale. Optimization is easier with scaling out and in, rather than scaling up and down. Driven for Elasticity
– Driver: Optimized utilization, cost saving
– Solution:
• Stateless Application Design
• Nothing is shared except Database
• Scaling every tier is possible – Web/Service/Database etc.
16
![Page 17: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/17.jpg)
Scaling Out / Horizontally: Adding Removing Boxes
![Page 18: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/18.jpg)
Design Principle - Stateless Design
• Stateless designs increases scalability – Don’t store anything locally on Web Server
• Session State – Local Sessions – Avoid – Not Scalable
• Load Balancer Sticky sessions can create hot spot load
– Central Session – Good – Distributed Cache, Database
– Client Session – Better – Client Cookie
– No Session – Awesome
18
![Page 19: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/19.jpg)
Design Principle – Loosely Coupled
• Components and layers should be loosely coupled to be able to scale each layer separately
19
Database Server
Web Servers
Application Servers
![Page 20: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/20.jpg)
Caching in Scalability
• Caching helps in avoiding scale
• In-memory distributed cache offers an excellent solution to data storage bottlenecks
• Distributed caching clusters can keep growing horizontally, just like the application servers. This reduces pressure on data storage so that it is no longer a scalability bottleneck.
20
![Page 21: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/21.jpg)
Design Pattern - Cache Aside Pattern
• Prefer Cache to Database for Reading – Intent : Increase read throughput and
reduce database bottleneck
– Drivers: Distributed cache are faster and shared across web/application servers
– Solution:
• Update cache and database both for synchronization
• Read from Cache
• Decorator Design Pattern
21
Distributed Cache
Write
Read
![Page 22: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/22.jpg)
Design Pattern - Cache Read-through/Write-through (RT/WT)
• Prefer Cache to Database – Intent: Increase read throughput and reduce database bottleneck. Use
Cache for read write both
– Drivers: Distributed cache are faster and shared across web/application servers
– Solution:
• Application treats cache as the main data store and reads data from it and writes data to it.
• The cache is responsible for reading and writing this data to the database, thereby relieving the application of this responsibility, asynchronously
22
![Page 23: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/23.jpg)
23
Design Pattern - Cache Read-through/Write-through (RT/WT)
![Page 24: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/24.jpg)
Database Scalability
24
![Page 25: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/25.jpg)
CAP Theorem
• CAP theorem, also known as Brewer's theorem, states that it is impossible for a distributed computer system to simultaneously provide all three of the following guarantees: Consistency, Availability and Partition tolerance.
• Consistency: All clients always have the same view of the data
• Availability: Each client can always read and write
• Partition Tolerance: The system works well despite physical network partition
25
![Page 26: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/26.jpg)
CAP Theorem – Database Placements
26
![Page 27: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/27.jpg)
Database Scaling – Replication - Read Mostly Pattern
• Intent: Increase database scalability by separating write and read operations – Generally most of the applications have around 80% read and 20%
write
• Drivers: Separate read write responsibilities, High availability benefits
• Solution: – Read Write Separation
– Master Slave Pattern
27
![Page 28: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/28.jpg)
Database Scaling – Read Write Separation
28
Reads and Writes
Reads
![Page 29: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/29.jpg)
Design Pattern – Partitioning / Sharding
• Design for Database Sharding – Intent: Increasing data size might rise throttling. Database scale and
performance is more important than reliability. CAP Theorem
– Drivers: Scaling database layer, increasing database throughput
– Solution:
• Database Sharding / Horizontal Partitioning
• Database Federation
29
![Page 30: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/30.jpg)
Shard Resolver
Shard = User ID % 4
Database Sharding Example
30
Shard 0 25%
Shard 1 25%
Shard 2 25%
User ID=3
Shard 3 25%
![Page 31: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/31.jpg)
Design Principles – Eventually Consistent
• BASE Opposite to ACID – Intent: Real internet scale model. Postpone the consistency.
• Basically Available, Soft state, Eventual consistency
– Solution:
• Queue Based processing Model
• Change in behavior
– Order Placed successfully TO Order Received Successfully
31
![Page 32: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/32.jpg)
Design Principles – Asynchronous Processing
• Blocking is bane for Scalability – Intent:
• Avoid blocking calls, reduce contention
– Solution:
• Queue Based processing Model
• Fire and Forget Calls
• 1000 users blocked for 5 seconds = 5000 users per second
32
![Page 33: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/33.jpg)
Design Principles – Parallel Design
• Design for Parallel and Reliable Work – Intent: Increasing resources should results in a proportional increase
in performance. Dependent services might not be available. Blocking is bane of scalability
– Drivers: Higher reliability, Proportional distribution
– Solution:
• Concern Independent Scaling
• Reliability through Queue
• Queue driven worker tasks - more messages more workers faster work
33
![Page 34: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/34.jpg)
Queue Based Pattern
34
![Page 35: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/35.jpg)
Queue - Load Leveling, Load Balancing, Loose Coupling
35
![Page 36: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/36.jpg)
Design Principles – Queue Based Pattern
• Idempotent – Design the operation to be idempotent; that is, if it's carried out more
than once, it's as if it was carried out just once
– Implement the receiver in such a way that it can receive a message multiple times safely, either through a filter that removes already received messages or by adjustment of message semantics
36
![Page 37: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/37.jpg)
Design Principles – Capacity Planning
• Everything has a limit: Compose a Scale – Intent: Design Around Provider SLAs and Capacity
– Solution:
• Know the limits, measure the scalability and increase the scale
• E.g. Storage supports up to 10000 transactions/sec
– Add storage for higher scale
• E.g. Queue supports 5000 messages per seconds
– Add additional Queues (Partitioning) for additional scale
37
![Page 38: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/38.jpg)
Design Pattern – Multi Site Deployment Pattern
38
Database Server
Web Servers
Application Servers
Database Server
Web Servers
Application Servers
Sync
Routing • Performance Based • Round Robin • Failover
Asia United States
![Page 39: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/39.jpg)
Summary
39
![Page 40: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/40.jpg)
Scalability Principles
40
Scalability
Stateless
Parallelization
Asynchronous
Partitioning
Idempotent
Fault Tolerance
![Page 41: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/41.jpg)
Vertical Vs. Horizontal Scaling
41
Vertical Scaling Horizontal Scaling
ACID BASE
Availability First Focus on Commit
Pessimistic Locking Optimistic Locking
Transactional Shared nothing
Favor Consistency Maximum Scalability
Most Distributed Systems Realize Both
![Page 42: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/42.jpg)
Thank You !
42
![Page 43: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/43.jpg)
43
![Page 44: Scalability Design Principles - Internal Session](https://reader034.fdocuments.in/reader034/viewer/2022051608/5441e386b1af9f720a8b456a/html5/thumbnails/44.jpg)
44
Some of the images are taken by utilizing Google search and due credit to the source. Author do not claim any creation or originality of the contents. It is used only for learning purposes