Cloud architectural patters and Microsoft Azure tools
-
Upload
pushkar-chivate -
Category
Technology
-
view
261 -
download
2
description
Transcript of Cloud architectural patters and Microsoft Azure tools
Welcome
Cloud Architectural Patterns (using Microsoft Azure tools)
Pushkar Chivate
Agenda• Quick Look: Microsoft Azure as a platform
• Overview of patterns for cloud applications
• Specifically Data-Management, Resiliency and Messaging patterns
• Demo • Azure Tables,
• DocumentDB,
• Azure Service bus (Queues and Pub-Sub architecture)
What is the Cloud?
IT
Capacity
Time
Load
Forecast
Allocated
IT-capacity
Under capacity
Over
capacity
Actual
Load
How the Cloud Helps?
IT
Capacity
Time
Load
Forecast
Actual
Load
Capacity on
Demand
Cloud ModelsIaaS Infrastructure as a service(e.g. Azure VMs)
PaaS Platform as a service(e.g. Azure Storage, Service Bus etc.)
SaaS Software as a Service(e.g. Office 365)
Microsoft Azure is a cloud platform
● On-Demand
● Self-service
● Pay per use
● Highly automated
● Designed for Failure
● Designed for Scale
Microsoft AzureDesigned for Failure
Network load balancer
VM1 VM2 VM3 VM4 VM5 VM6
VM7 VM8
Fault Domain
1
Fault Domain
2Fault Domain
3UD1
UD2
Microsoft AzureDesigned for Scale
Scale out
Network load balancer
VM1 VM2 VM3 VM4
Azure VMs, Websites&
Rethinking AppsA look at various features
Azure offers…
Architectural Pattern
An architectural pattern is a general, reusable solution to a commonly occurring problem within a given
context.
Cloud PatternsData-Management Patterns
● Index Table & Sharding PatternCreate indexes over the fields in data stores that are frequently referenced by query criteria. This pattern can improve query performance by allowing applications to more quickly retrieve data from a data store. Divide a data store into a set of horizontal partitions shards
● Materialized View PatternGenerate prepopulated views over the data in one or more data stores when the data is formatted in a way that does not favor the required query operations. This pattern can help to support efficient querying and data extraction, and improve performance
● Command and Query Responsibility Segregation (CQRS) PatternSegregate operations that read data from operations that update data by using separate interfaces.
Cloud PatternsMessaging Patterns
● Pipes and Filter PatternDecompose a task that performs complex processing into a series of discrete elements that can be reused.
● Priority Queue PatternPrioritize requests sent to services so that requests with a higher priority are
received and processed more quickly than those of a lower priority
.
● Queue-based Load Leveling PatternUse a queue that acts as a buffer between a task and a service that it invokes in order to smooth intermittent heavy loads that may otherwise cause the service to fail or the task to timeout.
Data StorageVarious data storage techniques
● SQL Azure
● Azure Table
● Blob storage
● Document DB
Materialized View PatternThis pattern can be used to support efficient querying and data extraction, and improve
application performance
What is a materialized view?
Materialized view is a database object that contains results of a query
Materialized View
OrderID Account
1 A
2 B
ItemID Name Stock
100 Shirts 100
101 Pants 125
OrderID ItemID Qty Ordered
1 100 2
1 101 3
2 100 2
Materialized View
Database
tables
ItemID Name Total Qty Ordered
100 Shirts 4
101 Pants 3
Denormalized data
Data refreshes
periodically
Materialized Views Vs Regular Views
Materialized View● Stored as a physical
database object
● Indexes on the base tables are not used , can create indexes on materialized view
● Faster performance
● Needs to be updated
Regular View● Stored as a query
against the base object
● Indexes on the base tables are used
● No performance improvement
● Need not be updated
Materialized view
When underlying data is complex and difficult to query
directly
Simplify queries - expose data in a way that doesn’t
require knowledge of underlying data structure
Provide access to specific subset of data (security
reasons)
Bridging the disjoint when using different data stores
When to use materialized
views?
Materialized viewSnapshot
s
Database
1
Table
1
Materialized view
1
Database
2
Table
1
Materialized view
2
Snapsho
t
Materialized
view
Databases supporting Materialized views
Oracle
Called Indexed Views
IBM DB2SQL
Server
MySQL
No Native support
Can be implemented
using triggers
Called Materialized Query
Table
Called Materialized View
Materialized view
When not to use materialized views?● The source data is simple and easy to query.
● The source data changes very quickly, or can be accessed without using a view. The processing overhead of creating views may be avoidable in these cases.
● Consistency is a high priority. The views may not always be fully consistent with the original data
Azure Tables
What are Azure Tables?
● Non relational entities (not RDBMS).
● Tables are nothing but collection of entities
● Entities are comprised of properties
● Properties are name value pairs
Azure TablesAzure Storage
Account
Employees
(Table)
Entity
Property
Entity
Property
Property PropertyProperty
Azure Table EntitiesAzure Storage
Account
Employees
(Table) Entity
Property
<Name,Value
>
Property
<Name,Value
>
Entities● Partition Key
● Row Key
● Timestamp
● … other properties
● Each entity can have different number and type of properties
Partition Key + Row Key = Unique Identifier
(No support for secondary index in Azure Table)
Property
<Name,Value
>
Azure Tables Partitions
Employees (Table) Certifications (Table)
(Entity)
EmployeeName=…CertName=…BirthDate=…
CertNumber=…
EmployeeName=…CertName=…BirthDate=…
CertNumber=…
FavoriteTeam=…
(Entity)
CertName=…EmployeeName=…
CertNumber=…BirthDate=…
CertName=…EmployeeName=…
CertNumber=…BirthDate=…
PK = EmployeeName
RK = CertName
PK = CertName
RK = EmployeeName
How do you Partition the data?• EmployeeName or CertName?
Azure Tables Partitions
Employees (Table) Certifications (Table)
(PK) EmployeeName(E1), (RK) CertName (C1)(PK) EmployeeName(E1), (RK) CertName
(C2)
(PK) EmployeeName(E2), (RK) CertName
(C1)
(PK) EmployeeName(E2), (RK) CertName
(C3)
(PK) EmployeeName(E3), (RK) CertName
(C2)
(PK) EmployeeName(E3), (RK) CertName
(C3)
How do you Partition the data?• EmployeeName or CertName?
(PK) CertName (C1), (RK) EmployeeName(E1)
(PK) CertName (C1), (RK)
EmployeeName(E2)
(PK) CertName (C2), (RK)
EmployeeName(E1)
(PK) CertName (C2), (RK)
EmployeeName(E3)
(PK) CertName (C3), (RK)
EmployeeName(E2)
(PK) CertName (C3), (RK)
EmployeeName(E3)
O
R
Azure Tables
EmployeeName BirthDate FavoriteTeam
David Anderson 1/1/1970
Nancy Wilson 4/15/1965 Atlanta Falcons
John Doe April 1, 1989
No Fixed Schema for
entities
Employee table
Azure Table
Demo
Azure Tables
Summary
● No nice relational schema model that we get in RDBMS
● Easy to work with
● Built to scale
● Great pricing
DocumentDB• Azure Document DB - relatively new service
• It’s fully managed Document database as a service
• It stores the data in JSON
• The Azure Tables are extremely scalable and cheap, but if you start querying on any other attributes then you start running into problems.
• It’s massively scalable but fully query-able by all parts of JSON tree.
• Capacity units are 10GB in size and predictable reads and writes / second.
DocumentDB
DocumentDB
Demo
Azure Service Bus
● Secure messaging capabilities
● Enable loosely coupled solutions
● Achieve Publish-Subscribe scenarios
Tightly Coupled
Store Front
End
Store Back
End
Order
processing
Shipping
Loosely Coupled
Store Front
End
Store Back
End
Order
processing
Shipping
Order Queue
Loosely Coupled
Store Front
End
Store Back
End
Order
processingOrder Queue
Shipping
Service Bus – Queue
Demo
Publish - Subscribe
Store Front
End
Store Back
End
Order
processingOrder Queue
Order Printers
Subscription
1
Subscription
2
Shipping
Order
Papers
Service Bus – AMQP• Service Bus supports the Advanced Message Queueing Protocol (AMQP) 1.0.
• AMQP enables you to build cross-platform, hybrid applications using an open standard protocol
• Reliable: The AMQP 1.0 protocol allows messages to be exchanged with a range of reliability guarantees, from fire-and-forget to reliable, exactly-once acknowledged delivery.
• Using AMQP from .Net:Endpoint=sb://[namespace].servicebus.windows.net;SharedSecretIssuer=[issuer
name];SharedSecretValue=[issuer key];TransportType=Amqp
• Using AMQP from Java:# servicebus.properties - sample JNDI configuration# Register a ConnectionFactory in JNDI using the form:# connectionfactory.[jndi_name] = [ConnectionURL]connectionfactory.SBCF = amqps://[username]:[password]@[namespace].servicebus.windows.net
Service Bus – AMQP
Cloud PatternsResiliency Patterns
● Circuit Breaker PatternHandle faults that may take a variable amount of time to rectify when connecting to a remote service or resource. This pattern can improve the stability and resiliency of an application.
● Retry PatternEnable an application to handle temporary failures when connecting to a service or
network resource by transparently retrying the operation in the expectation that the failure is transient. This pattern can improve the stability of the application.
● Compensating Transaction PatternUndo the work performed by a series of steps, which together define an eventually consistent operation, if one or more of the operations fails. Operations that follow the eventual consistency model are commonly found in cloud-hosted applications that implement complex business processes and workflows.
SummaryWe Looked at…
- Azure as Infrastructure (VMs)
- Azure as Platform -Websites- Storage - Azure Tables, DocumentDB- Messaging – Service Bus, Queues, Topics, AMQP- Azure CDN
Summary
Network load balancer
VM1 VM2 VM3 VM4
SQL DatabaseAzure
TablesDocumentDB
Service Bus Queue
Subscription 1 Subscription 2
Worker Role 1 Worker Role 2
Cach
e
References
- MSDN (cloud architectural patterns)
- azure.microsoft.com (Learning videos and documentation)
- some images from www