Windows AzureData Services
Scott Klueppel
Chief Cloud Evangelist
SOAlutions, Inc.#gwab
@kloopdogg
Agenda - Overview - Management Portal / Tools - Windows Azure Storage - Tables, Blobs, Queues - Windows Azure SQL Database - Putting them together with Cloud Design Pattern Examples
Windows Azure StorageOverviewManagement PortalStorage Emulator / Tools
What is Windows Azure Storage? - Storage as a service – Blobs, Tables, and Queues - Lots of it (up to 200 TB) – pay as you go
- Highly Available - Redundancy
- Highly Scalable - Partitions
- Blazing Speed - Performance “Targets”
Getting Started with Storage - Create a storage account in the portal - Connection string Account Name Account Key
Getting Started with Storage - Use storage emulator for development - Connection string UseDevelopmentStorage=true
Windows Azure SDK
Windows AzureSDK
Visual Studio2013
Visual Studio2012
Visual Studio2010
2.2 Oct2013 Supported Supported Not Supported
2.1 Jul2013 Not Supported Supported Supported
2.0 Apr2013 Not Supported Supported Supported
- SDK contains client libraries - Committed to backwards compatibility
Storage Client Library - SCL 2.1 included with SDK 2.2 - SCL 3.0.x upgrade available with NuGet
Tools to interact with storage Visual Studio Server Explorer CloudXplorer/TableXplorer
AzureXplorer (VS Extension)
DemoManagement Portal
Storage
Tools
Windows Azure StorageTablesBlobsQueues
Tables Concept OverviewAccount Table Entity
gwabjax
racers
results
Name=RickyStatus=ActiveName=DaleStatus=ActiveRace=Jax 500Date=3/29/14
- Structured storage in the form of tables-1MB max size-252 properties
- Quick queries by clustered index (only index)- 2,000 entities per second per partition- Data-defined partition scheme
-Table name + PK
Table Details
- Tables store data more like key-value pairs - No schema
- De-normalized structure optimized for performance
Table Details
First Name Last Name Date of Birth Last Win
Ricky Bobby 1/1/1971
Dale Earnhardt 2/2/1972 Daytona 500
Cal Naughton March 3, 1973
Table Entities - Property Types byte[]
boolDateTimedoubleGuidInt32Int64String
Table Entities - Required properties PartitionKey, RowKey, Timestamp
- Operations on entities Delete Insert Upsert (Insert + Merge/Replace) Update (Merge or Replace) Retrieve Query
Using Tables1. Get table reference
2. Execute operations on entities
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("DefaultConnection"));
CloudTableClient tableClient = storageAccount.CreateCloudTableClient();CloudTable table = tableClient.GetTableReference("racers");
TableOperation upsertOperation = TableOperation.InsertOrReplace(racer);table.Execute(upsertOperation);
Using Tables with DTOs/POCOs1. Get table reference (like before)2. Execute operations on entities via
EntityAdapterWith TableEntity – RacerEntity
var operation = TableOperation.InsertOrReplace(racerEntity);table.Execute(operation);
With DTO – Racer
var adapter = new RacerAdapter(racer);var operation = TableOperation.InsertOrReplace(adapter);table.Execute(operation);
Table Best Practices1. Avoid querying across partitions2. Batch inserts (within partition)
- Order and details in same table/partition
3. Storing aggregate copies - Ensures 1 query 1 partition - Format data up front
4. Use intelligent PKs
DemoTables
Blobs
Queues
Blobs Concept Overview
Account Container Blob
gwabjax
photos
videos
DSC000256.jpg
teamlogo.png
1stPlace.avi
http://<account>.blob.core.windows.net/<container>/<blobname>
Blobs - Types of Blobs - Page - Block
- Operations on blobs Exists List Upload from Stream, ByteArray, Text, File Download to Stream, ByteArray, Text, File Delete
Using Blobs1. Get blob container reference
2. Perform operations with blobs
CloudStorageAccount account = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("DefaultConnection"));
CloudBlobClient client = account.CreateCloudBlobClient();CloudBlobContainer container = client.GetContainerReference("photos");
CloudBlockBlob blockBlob = container.GetBlockBlobReference(refId);blockBlob.UploadFromStream(stream);blockBlob.Metadata["FileName"] = fileName;blockBlob.SetMetadata();
DemoTables
Blobs
Queues
Queues Concept OverviewAccount Queue Messages
gwabjax
orders
emails-to-send
Bulkhttp://...
Smallhttp://...
http://...
Queues - Operations on queues Add Peek Get Update Delete
- Queue naming - All lowercase - Alpha-numeric and hyphen “-” - 3-63 characters
Client Web Role
Storage
Queues in the wildWorker Role
Browser
Web App
Get Survey
CompleteSurvey
Post Results
SurveysQueue
Process
Task
Thank you
Add
Get
Delete Process*
Client Web Role
Storage
Queues in the wildWorker Role
BlobStore
Browser
Web App
Get Survey
CompleteSurvey
Post Results
SmallSurveys
LargeSurveys
SmallTask
BigTask
Small Survey
Get
Delete Process*
AddThank you
Client Web Role
Storage
Queues in the wildWorker Role
Blob/TableStore
Browser
Web App
Get Survey
CompleteSurvey
Post Results
SmallSurveysQueue
LargeSurveysQueue
SmallTask
LargeTask
LargeSurvey
Get
Delete Process*
AddThank you
Upload
Download
Using Queues1. Get queue reference
2. Perform operations with messages
CloudStorageAccount account = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("DefaultConnection"));
CloudQueueClient client = account.CreateCloudQueueClient();CloudQueue queue = client.GetQueueReference("orders");
CloudQueueMessage message = new CloudQueueMessage("ticket-order-17"); queue.AddMessage(message);
DemoTables
Blobs
Queues
Windows Azure SQL Database(formerly SQL Azure)Management PortalData Access
Windows Azure SQL DatabaseTools - SSMS- Visual Studio/SSDT- Modeling toolsCode- ADO.NET- Enterprise Library- ORMs (e.g. Entity Framework, nHibernate)
Key Features- High Availability/Durability
- Three Replicas (1 Primary – 2 Secondary)- 99.9% SLA
- Manageability- Portal, PowerShell, SSMS, REST API
- Predictable Performance- Scale-out*
What’s Different – Maybe Bad?Missing- Extended stored procedures- SQL-CLR- Service Broker- Table partitioning
Important: All tables must have a clustered index
Data Types?Data Type Category Windows Azure SQL Database Support
Exact numerics bigint, bit, decimal, int, money, numeric, smallint, smallmoney, tinyint
Approximate numerics
float, real
Date and time date, datetime2, datetime, datetimeoffset, smalldatetime, time
Character strings char, varchar, text
Unicode character strings
nchar, nvarchar, ntext
Binary strings binary, varbinary, image
Spatial data types geography, geometry
Other data types cursor, hierarchyid, sql_variant, table, timestamp, uniqueidentifier, xml
Development Story- Local SQL 2012/Express database - SSDT Database projects
- Deploy directly to Azure from Visual Studio- Create deployment package- Use Release Management
- Azure Connection String<add name="DefaultConnection"
connectionString="Server=tcp:[server].database.windows.net,1433;Database=[database]; User ID=[database]-host@[server]; Password=[password];Encrypt=true;"
providerName="System.Data.SqlClient"/>
DemoManagement Portal
Create Server
Create Database
Firewall Rules
Patterns at Work• Cache-aside Pattern• Materialized View Pattern• Competing Consumers Pattern• Compensating Transaction Pattern
Cache-Aside Pattern- Build cache on-demand
- Read cache first- If not there
- Get from data store- Store in cache
Materialized View Pattern- Improve performance in systems with
difficult queries- Pre-populated views - Not updated by app, can be entirely rebuilt- Inherent delays- Doesn’t query well
Competing Consumers Pattern- Protect against a large influx (burst) of
requests- Balanced workload- Scalable
Compensating Transaction Pattern- Eventual consistency
- Series of autonomous steps- Intermediate steps appear inconsistent
- Compensating Transaction- Intelligent process to undo succeeded steps
Compensating Transaction Pattern- Example: Itinerary Creation
- Book 3 flights and 2 hotel rooms- If one or more are unavailable, we start
over
Compensating Transaction Pattern- Example:
Itinerary Creation
What did we learn?- Azure data storage options/uses- Each has pros/cons/best fit- Management Portal and Tools- Design patterns
Questions?
Scott Klueppel
Chief Cloud Evangelist
SOAlutions, Inc.#gwab
@kloopdogg
Thank you!
Scott Klueppel
Chief Cloud Evangelist
SOAlutions, Inc.#gwab
@kloopdogg
References• Cloud Design Patterns (P&P)
http://msdn.microsoft.com/en-us/library/dn568099.aspx
• Data Services – Storagehttp://msdn.microsoft.com/en-us/library/windowsazure/gg433040.aspx
• Storage Differences – Emulator vs Cloudhttp://msdn.microsoft.com/en-us/gg433135
• Data Services – SQL Databasehttp://msdn.microsoft.com/en-us/library/windowsazure/ee336279.aspx