Deep thoughts from the real world of azure
-
Upload
michele-bustamante -
Category
Technology
-
view
108 -
download
0
description
Transcript of Deep thoughts from the real world of azure
Deep Thoughtsfrom the
Real World of Azure
Michele Leroux Bustamante
CIO, SollianceCofounder, Snapboard
Windows Azure FeaturesCompute
VirtualNetwork
TrafficManager
CDN
Storage
Queues
Storage
Tables
Storage
Blobs
SQL Database
SQL Data Sync
SQL Reporting
HDInsight (Hadoop)
Caching Recovery Services
Windows Azure Active Directory
Media Services
Service Bus
Store
Web Site Cloud Service
Virtual Machine
ApplicationServices
Marketplace
DataServices
Networking
MobileServices
BizTalkServices
Azure FeaturesCompute
Web Site Cloud Service
Virtual Machine
Your Datacenter
Virtualization
O/S
Hardware
Network
Data
Applications
Firewall
Web Sites
Applications
Data
Cloud Services
Applications
Firewall Rules
Data
Virtual Network
Virtual Machines
Virtual Network
Data
Applications
Firewall Rules
O/S
Focus on the Application
Microsoft Azure
IAASVIRTUAL MACHINES
Virtual Machine ProvisioningChoose a VM
Blob Storage
Generate
Provision
Select your VHD
Blob Storage
Upload
Provision
PAASCLOUD SERVICES & WEB SITES
WEB SITES
Deep Thoughts on Web Sites
• New portal • Publishing process• Setting overrides• Log streaming• SWAP (but…)
CLOUD SERVICES
Cloud Services Are…
Web Role Worker Role
Upload to Windows Azure
Windows Azure Compute Controller
LB
ServiceService
Packaging & DeploymentServiceArtifacts
Model
Config*.cscfg
*.cspkgMicrosoft
Visual StudioWeb Role
Worker Role
ROLE
VIP Swap
VM1 VM2
VM3 VM4
VM1 VM2
VM3 VM4
Deep Thoughts on Cloud Services
• Packaging settings outside of web.config
• Proper swap, separate VM• Application insights agent• Mostly for worker roles, reading
queues, async processing
Deep Thoughts on Worker Roles
• Set up configuration settings to shut off processing
• Implement graceful shutdown
Azure Features
Storage
Tables
SQL Database
SQL Data Sync
SQL Reporting
HDInsight (Hadoop)
DataServices
SQL DB and SQL Server
• SQL DB (formerly SQL Azure)– 150GB limit– No backup logs / replay– IO performance – Other SQL features
• SQL DB Premium– 500GB limit– 30 days of backups– Restore on demand
• SQL Server on Azure VM– Popular alternative– 1TB limit (blob storage)
What do YOU want in a DB?
• No limits manage yourself– SQL Server VM
• Managed and affordable– SQL DB
• Managed and $$$–MySQL / ClearDB cluster– SQL DB Premium– Other similar options for MySQL/SQL
Server outside of Azure
Deep Thoughts on Relational Data
• All platforms require retry mechanism
• Mind your connection pool settings• Practice backup and restore• Always use data masking to dev, test• PII, encryption• Review queries, profile, tune• Use transactions for Pete’s sake
noSQL
• It’s either obvious…– Very large quantities of data– Easily compartmentalized data, key value lookup or document-
centric– Logs, audit trails, notifications, profile data from social networks,
associative information about users / “things” in the relational store
• Or, it requires planning…– Could be large quantities– Could require noSQL scale, parallel access performance
• Many people return to relational• Can start relational and figure your model out first• Or, combine the two
Understanding noSQL
• http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis
Big Data/Map Reduce from 10K Feet
Company B Company CCompany A
AggregateData
(all Companies)
HDInsight
Company A
Server
Company C
Company B
HDInsight(local)
HDInsight(local)
HDInsight(local)
Don’t Drink the KoolaidnoSQL
Hadoop
Big Data
Don’t Drink the Koolaid
• noSQL requires planning• BigData is not for everyone
• Get an expert, like: Lynn Langit– www.lynnlangit.com – Youtube.com/socaldevgal Shout Out!
Azure Features
Service Bus
ApplicationServices
Storage
Queues
DataServices
Service Bus as DMZ
Service Service Service
Web API Web API
Browser BrowserRich
ClientsMVC /JQuery AJAX
Service
MobileClients
DMZ
Corporate Domain
MVCSite
Web Forms
Site
Service Bus to Data On Premise / Migration
Service
Client
Web Application
Service Bus
Corporate Domain
Windows Azure
Queues
Queue
Queue
Topics
TopicSubSubSub
Queue ComparisonService Bus
QueuesStorage Queues
Unlimited message lifetime
7 days expiry
Max 64K message size Max 256K message size
Max 5GB total storage Max 100TB total storageDuplicate detection
Order guarantees
Dead letter queue
Storage metrics
Purge capability
Long polling Manual back-off polling
Queue ComparisonService Bus
QueuesStorage Queues
Unlimited message lifetime
7 days expiry
Max 64K message size Max 256K message size
Max 5GB total storage Max 100TB total storageDuplicate detection
Order guarantees
Dead letter queue
Storage metrics
Purge capability
Long polling Manual back-off polling
Queue ComparisonService Bus
QueuesStorage Queues
EASYCOMPLEX
Deep Thoughts on Async Work
• Queues and worker roles• Use error queues• Monitor queues for expiring items• Use service bus for parallel
processing• Use web jobs to kick off queue
processing
Azure Features
CDN
Storage
Blobs
Media Services
ApplicationServices
DataServices
Networking
Common Blob Storage Usage
• Web site content, javascript, css, images• Logs (setting)• App content
– Can separate by user for key isolation via Azure Explorer
• Media– Can use with media services, encoding services
• Replace file storage– Retain folder structure– Request by path
• Very large files– Set up chunking process for upload– Use hash to check integrity
Blob Storage Integrity
Windows Azure Storage
Blob Container
Service
validatesignature
WindowsAzure
MD5Hash +
MD5Hash +
Shared Access Signatures
Service
Public Blob Access
createupdatedelete
read
Service
Public Container
Access
createupdatedeleteread
list
Service
Private Container
createupdatedeleteread
list
list
Service
Private Container
readaccessfor limited timewithsharedaccesskey
sharedaccesspolicy
Browser Client
Shared AccessSignature (SAS)
>1 hour requiresauthenticationheader in request(no browser)
Deep Thoughts on Blobs
• Doh, protect your content!• Separate containers for management
roles and delegation• Write some reusable tools for saving,
chunking, shared access, reading, writing to streams, etc.
• Backup, backup, backup (AZCopy)• Snapshots, maybe
Azure Features
Azure Active Directory
Access Control
ApplicationServices
Single Sign On
Access Control (THE OLD WAY)
Your App
AccessControl
Yahoo! WindowsLive
Browser
1 25
3
4
User Provisioning
Apps
AzureActive
Directory
Browser
AppsApps
AppsApps
Apps
DirSync with AAD
Apps
AzureActive
Directory
Browser
On PremiseAD
AppsApps
DirSync
AppsApps
Apps
Single Sign-On (sort of)
46
Deep Thoughts on IAM
• Don’t write your own identity server• Use one that can support social and
corporate identity• Provide a smooth sign up/
registration• Choose social login options carefully• You are more secure with a service• Have a back door for login (yep)
Deep Thoughts on Security Audits
• Have separate security logs• Log failed access • Retain logs 1 yr min• No production keys in open (encrypt)• Encrypt PII in database• Mask data to dev, test• Explore every web threat• Protect your content, APIs
WORST PASSWORDS TOP 3 FOR 2013
123456password12345678
Deep Thoughts on Deployment
• Wouldn’t automated deployment / upgrade be nice?
• Be very very careful– Know the location of all parts prod,
staging, dev, and test– Review configurations, again, and again– Beware running jobs (workers)
• Pray• Just kidding
53
Deep Thoughts on Logs
• Is built in better?• Know where your logs are (yep)–WAD, table storage events, security
audits, IIS logs
• Table storage or blobs (csv)?• Real time info• Historical queries• Logging can (should) evolve
Deep Thoughts on Troubleshooting
• Log everything possible• Use an agent if possible for
monitoring• Configure error alerts• Use google analytics, really• Be ready with real time queries
58
59
60
Backup and Recovery
• Back up your data base, test recovery
• Back up blob storage nightly, keep a rolling archive
• Know how to spin up a complete architecture, in emergency – State of code– Data, storage–Moving parts
Azure FeaturesCompute
VirtualNetwork
TrafficManager
CDN
Storage
Queues
Storage
Tables
Storage
Blobs
SQL Database
SQL Data Sync
SQL Reporting
HDInsight (Hadoop)
Caching Recovery Services
Windows Azure Active Directory
Access Control
Media Services
Service Bus
Store
Web Site Cloud Service
Virtual Machine
ApplicationServices
Marketplace
DataServices
Networking
MobileServices
He taught Chuck Norris martial arts
THANK YOU!